Skip to content

shreyaskommuri/ClinicAct

BranchesTags

Repository files navigation

ClinicAct - AI-Powered Clinical Sidecar

Next.js TypeScript FHIR R4 AI

An intelligent clinical sidecar application that seamlessly integrates with Medplum EMR to automate clinical documentation. ClinicAct uses AI-powered voice transcription to listen to patient consultations, extract clinical intents, and generate draft FHIR orders for physician review—complete with medication safety validation and automated patient aftercare.

🌟 What is ClinicAct?

ClinicAct acts as an "AI scribe" for healthcare providers. During a patient consultation, it:

  1. Listens - Records and transcribes doctor-patient conversations using medical-grade speech recognition
  2. Understands - Extracts clinical intents (prescriptions, labs, referrals) using AI
  3. Validates - Checks medication orders for safety concerns (dangerous dosages, interactions)
  4. Documents - Generates compliant FHIR R4 resources ready for EMR submission
  5. Follows Up - Sends personalized aftercare emails and enables voice-based patient recap calls

🏗️ Tech Stack

Core Framework

Technology Version Purpose
Next.js 15.0.3 React framework with App Router
TypeScript 5.x Type-safe development
Tailwind CSS 3.4.14 Utility-first styling
React 18.3.1 UI components

Healthcare & FHIR

Technology Purpose
Medplum Headless FHIR R4 server for patient data and clinical resources
@medplum/core Medplum SDK for FHIR operations
@medplum/fhirtypes TypeScript types for FHIR R4 resources

AI & Machine Learning

Technology Purpose
Google Gemini (gemini-2.0-flash-exp) Clinical action extraction & aftercare generation
Anthropic Claude Transcript diarization & safety validation (fallback)
Deepgram (nova-2-medical) Medical-grade speech-to-text transcription

Communications

Technology Purpose
SendGrid Transactional email delivery for aftercare summaries
Twilio Voice webhooks for patient recap phone calls

Data Storage

Technology Purpose
MongoDB Session storage for voice recap PIN authentication

✨ Features

📋 4-Step Clinical Workflow

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│  1. SELECT      │───▶│  2. TRANSCRIPT  │───▶│  3. ACTIONS     │───▶│  4. AFTERCARE   │
│     PATIENT     │    │                 │    │                 │    │                 │
├─────────────────┤    ├─────────────────┤    ├─────────────────┤    ├─────────────────┤
│ • Search EMR    │    │ • Voice record  │    │ • Review orders │    │ • Email summary │
│ • View history  │    │ • Live transcr. │    │ • Safety alerts │    │ • Voice recap   │
│ • Select patient│    │ • AI extraction │    │ • Approve/edit  │    │ • Send to patient│
└─────────────────┘    └─────────────────┘    └─────────────────┘    └─────────────────┘

🎙️ Voice Transcription

  • Real-time recording with browser MediaRecorder API
  • Medical-optimized ASR using Deepgram Nova-2-Medical model
  • Speaker diarization to distinguish doctor vs patient speech
  • Smart formatting with punctuation and paragraph detection

🤖 AI Action Generation

  • Clinical intent extraction from natural conversation
  • FHIR R4 resource generation for:
    • MedicationRequest - Prescriptions
    • ServiceRequest - Labs, imaging, referrals
    • Appointment - Follow-up scheduling
  • Questionnaire matching to auto-fill EMR forms
  • Contextual patient data integration

⚠️ Medication Safety Validation

  • Dangerous dosage detection (e.g., "1 pound of Tylenol")
  • Unit validation for weight-based dosing (mg, g, mcg)
  • Warning banner system with Accept/Dismiss doctor approval
  • Local fallback when AI quota exceeded

📝 Dynamic Form Auto-Fill

  • Questionnaire-driven forms fetched from Medplum
  • AI-populated fields from transcript data
  • Completion tracking with visual progress bar
  • Real-time validation before submission

📧 Aftercare Communication

  • AI-generated email summaries personalized for patients
  • Includes: Medications, instructions, follow-up dates, insurance info
  • SendGrid delivery with delivery status tracking

📞 Voice Recap System

  • PIN-authenticated phone calls for patients to review visit
  • Text-to-speech summary of consultation highlights
  • AI Q&A agent to answer patient questions about their visit
  • Twilio-powered voice webhooks

📁 Project Structure

DorityWebsite/
├── app/
│   ├── layout.tsx              # Root layout with metadata
│   ├── page.tsx                # Main workflow page
│   ├── globals.css             # Global Tailwind styles
│   └── api/
│       ├── analyze/            # AI action extraction endpoint
│       ├── deepgram/           # Voice transcription
│       │   └── transcribe/     # Deepgram Nova-2-Medical
│       ├── medplum/            # FHIR patient operations
│       │   ├── patient/        # Single patient fetch
│       │   └── patients/       # Patient search
│       ├── questionnaire/      # Form definitions
│       ├── recap/              # Voice recap system
│       │   ├── create-session/ # Generate PIN session
│       │   ├── update-session/ # Update with aftercare
│       │   └── agent/          # AI Q&A for voice calls
│       ├── session/            # Workflow session management
│       │   ├── start/          # Initialize session
│       │   ├── aftercare/      # Generate aftercare content
│       │   ├── send-aftercare/ # Email via SendGrid
│       │   └── apply-actions/  # Submit to EMR
│       ├── twilio/             # Voice webhook handlers
│       │   └── voice-webhook/  # Incoming call handler
│       └── validate-recommendations/  # Safety checks
│
├── components/
│   ├── WorkflowContext.tsx     # Global state provider
│   ├── WorkflowStepper.tsx     # 4-step progress UI
│   ├── PatientSearchPanel.tsx  # Patient selection
│   ├── PatientDashboard.tsx    # Patient overview
│   ├── TranscriptInputPanel.tsx # Transcript entry
│   ├── VoiceTranscription.tsx  # Live voice recording
│   ├── ActionList.tsx          # Draft orders list
│   ├── ActionCard.tsx          # Individual order card
│   ├── QuestionnaireForm.tsx   # Dynamic FHIR forms
│   ├── AftercarePanel.tsx      # Email composer
│   ├── Sidebar.tsx             # Navigation
│   └── TopNav.tsx              # Header
│
├── lib/
│   ├── medplum-client.ts       # Medplum SDK wrapper
│   ├── mongodb.ts              # MongoDB connection
│   ├── completion-utils.ts     # Form completion calc
│   └── smart-dummy-data.ts     # Test data generation
│
├── contexts/
│   └── PatientContext.tsx      # Patient state
│
└── package.json

🚀 Getting Started

Prerequisites

  • Node.js 18.x or higher
  • npm or yarn
  • Medplum account (for FHIR backend)
  • API keys for AI services (see Environment Variables)

Installation

  1. Clone the repository

    git clone https://github.com/your-org/ClinicAct.git
cd ClinicAct/DorityWebsite

  2. Install dependencies

    npm install

  3. Configure environment variables

    Create a .env.local file in the DorityWebsite directory:

    # ═══════════════════════════════════════════════════════════
# MEDPLUM - FHIR Backend (Required)
# ═══════════════════════════════════════════════════════════
MEDPLUM_BASE_URL=https://api.medplum.com
MEDPLUM_CLIENT_ID=your_medplum_client_id
MEDPLUM_CLIENT_SECRET=your_medplum_client_secret

# ═══════════════════════════════════════════════════════════
# AI SERVICES (At least one required)
# ═══════════════════════════════════════════════════════════
# Google Gemini - Primary AI for action generation
GEMINI_API_KEY=your_gemini_api_key

# Anthropic Claude - Transcript diarization & safety validation
ANTHROPIC_API_KEY=your_anthropic_api_key

# ═══════════════════════════════════════════════════════════
# VOICE TRANSCRIPTION (Required for voice features)
# ═══════════════════════════════════════════════════════════
DEEPGRAM_API_KEY=your_deepgram_api_key

# ═══════════════════════════════════════════════════════════
# EMAIL SERVICE (Required for aftercare emails)
# ═══════════════════════════════════════════════════════════
SENDGRID_API_KEY=your_sendgrid_api_key
SENDGRID_FROM_EMAIL=noreply@yourclinic.com

# ═══════════════════════════════════════════════════════════
# VOICE RECAP (Optional - for phone call feature)
# ═══════════════════════════════════════════════════════════
# MongoDB - Session storage for PIN authentication
MONGODB_URI=mongodb+srv://user:pass@cluster.mongodb.net/clinicact

# Twilio - Voice call handling
TWILIO_ACCOUNT_SID=your_twilio_account_sid
TWILIO_AUTH_TOKEN=your_twilio_auth_token
TWILIO_PHONE_NUMBER=+1234567890

  4. Start the development server

    npm run dev

  5. Open in browser

    Navigate to http://localhost:3000

🔌 API Endpoints

Core Workflow APIs

Endpoint Method Description
/api/medplum/patients GET Search patients by name
/api/medplum/patient GET Get patient by ID with full history
/api/deepgram/transcribe POST Transcribe audio file
/api/analyze POST Extract clinical actions from transcript
/api/questionnaire GET Fetch Medplum questionnaire definitions
/api/session/apply-actions POST Submit approved actions to EMR

Aftercare APIs

Endpoint Method Description
/api/session/aftercare POST Generate aftercare email content
/api/session/send-aftercare POST Send email via SendGrid

Voice Recap APIs

Endpoint Method Description
/api/recap/create-session POST Create PIN-authenticated recap session
/api/recap/update-session POST Update session with aftercare data
/api/recap/agent POST AI agent for answering patient questions
/api/twilio/voice-webhook POST Handle incoming Twilio voice calls

🔧 Configuration

Medplum Setup

  1. Create a Medplum project at medplum.com
  2. Create a Client Application in your project
  3. Copy the Client ID and Client Secret to your .env.local
  4. Ensure your Medplum project has:
    • Patient resources with test data
    • Questionnaire resources for form templates

Twilio Setup (Optional - for voice recap)

  1. Create a Twilio account at twilio.com
  2. Purchase a phone number with voice capabilities
  3. Configure the webhook URL: 
    https://your-domain.com/api/twilio/voice-webhook
  4. For local development, use ngrok: 
    ngrok http 3000
# Then update Twilio webhook to: https://xxx.ngrok.io/api/twilio/voice-webhook

MongoDB Setup (Optional - for voice recap)

  1. Create a MongoDB Atlas cluster at mongodb.com
  2. Create a database named clinicact
  3. Copy the connection string to MONGODB_URI

🧪 Testing the Workflow

  1. Start the app: npm run dev
  2. Select a patient: Search by name in the patient panel
  3. Record or paste transcript: Use the voice recorder or paste consultation text
  4. Generate actions: Click "Generate Actions" to extract clinical intents
  5. Review safety warnings: Accept or dismiss any medication warnings
  6. Fill questionnaire forms: Complete auto-populated forms
  7. Approve actions: Review and approve each draft order
  8. Send aftercare: Generate and send patient summary email

📖 Development Guidelines

Code Standards

  • TypeScript for all files with strict type checking
  • Functional React components with hooks
  • "use client" directive for client-side components
  • FHIR types from @medplum/fhirtypes

Styling

  • Tailwind CSS utility classes only
  • Clinical theme: slate/blue color palette
  • Mobile-first responsive design

State Management

  • WorkflowContext for global workflow state
  • PatientContext for selected patient data
  • Local state for component-specific data

🤝 Contributing

  1. Fork the repository
  2. Create a feature branch: git checkout -b feature/amazing-feature
  3. Commit changes: git commit -m 'Add amazing feature'
  4. Push to branch: git push origin feature/amazing-feature
  5. Open a Pull Request

📄 License

Private - Internal Use Only

🙏 Acknowledgments

About

AI-powered clinical workflow layer that turns doctor conversations into draft EMR actions using FHIR.

Topics

ai fhir emr-management medical-ai healthtech healthcare-ai

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages