Six independent palliative care doctors grade Palli Sahayak's RAG outputs on 40 difficult Indian palliative care cases — 20 oncology and 20 pulmonology — using a six-dimension rubric anchored to WHO, IAPC, Pallium India, NICE and GOLD guidelines.
Open Review Console → Read the rubric specPalli Sahayak is a voice-first AI clinical decision-support tool for palliative care, deployed in 22 Indian languages and grounded in evidence-based palliative-care guidelines. This console is a small, independent study: six palliative care doctors rate the tool's answers on a set of difficult standardized cases that span pain management, end-of-life care, end-stage COPD, post-TB fibrosis and Indian occupational lung disease.
20 oncology (pain, non-pain symptoms, end-of-life, psychosocial) + 20 pulmonology (COPD, post-TB fibrosis / IPF, lung cancer, occupational disease). 10 easy + 10 moderate + 20 difficult tiers.
60%+ of cases include India-specific elements: NDPS opioid access, family-based decision-making, rural distance, regional formulary, occupational exposures (silicosis, byssinosis).
Two reviewers independently grade each (vignette, RAG-output) pair. Inter-rater reliability assessed via weighted Cohen's κ; target κ ≥ 0.70 per dimension.
Six dimensions × 1-5 Likert + 25 binary sub-items + 0-10 overall score + free-text comments + controlled-vocabulary tags.
| # | Dimension | What it measures |
|---|---|---|
| 1 | Clinical Accuracy | Concordance with WHO / IAPC / Pallium India / NICE / GOLD guidelines |
| 2 | Safety | Contraindications, dose limits, drug interactions, emergency triggers |
| 3 | Empathy | Acknowledgment of distress; non-judgmental tone; cultural validation |
| 4 | Actionability | Can the ASHA / caregiver actually act on the advice in their setting? |
| 5 | Completeness | Symptom + side effects + follow-up + caregiver education + escalation |
| 6 | Indian Context Fit | NDPS opioid access, language register, family decision-making, regional formulary |
Go to palli-sahayak-review-production.up.railway.app on any modern browser (desktop recommended for the 3-column layout).
Pick your name from the dropdown and enter the PIN distributed separately by the KCDH-A team. Sessions last 8 hours; you can come back across multiple sessions.
The dashboard shows all 40 cases with status (pending / yours / others'). Open any case to see the vignette on the left, the RAG output in the middle, and the rubric on the right. Allow ~10-15 minutes per case for the calibration round, ≤ 10 min thereafter.
For any case where you assign a safety score of 1-2, or where the response reveals a previously undocumented failure mode, tick the "Flag for committee review" box at the bottom. The KCDH-A clinical team will adjudicate flagged cases.