DispatchQA
Client project, June to November 2026
- 127commits
- ~450backend tests
- 57API endpoints
- 8services in Docker Compose
A freight dispatch team makes far more calls than any manager can review. DispatchQA reviews them first, so the manager only has to judge.
What it does
- Scores each call on six criteria plus an 11-item checklist, alongside the transcript
- Gives reviewers a workflow to agree with or correct the AI, with locked, audited reviews
- Shows a dispatcher leaderboard, agent profiles and AI-versus-reviewer agreement
- Emails managers a scheduled PDF report
Why it is built this way
- Call audio and transcripts never leave the client's server, so transcription and scoring run on local models
- Three user roles with mandatory two-step verification and a security log
- A scorer evaluation harness measures agreement with human reviewers before any model or prompt change ships
Built with Next.js, TypeScript, FastAPI, PostgreSQL, faster-whisper, Ollama, Docker, nginx, GitHub Actions
Engineering decisions and trade-offs
- A fair-share queue on CPU-only hardware
- The server cannot score every call, so calls are shared round-robin across dispatchers, least-served first. Every dispatcher gets coverage, and calls not reached within 48 hours are marked as such rather than silently dropped.
- Deterministic, auditable scores
- The model runs at temperature 0 with its output shape enforced at generation time. Python computes the totals rather than trusting the model's arithmetic, and every score records which model, prompt and settings produced it.
- The API never runs the models
- A single worker process owns all heavy work and is protected by database locks, so the dashboard stays responsive while calls are processed.
- Capped retries
- A failed analysis is retried at most three times, an hour apart, and each call rolls back on its own. One outage cannot exhaust the whole queue.