For judges & reviewers
How this actually works
No login required. This is the same technical breakdown the team uses to answer engineering questions — what happens at each stage, which model or piece of code does it, and which numbers are measured rather than guessed.
Full Architecture
Radio in, evidence out — in depth
Not just STT. Four separate hackathon-brainstorm concepts — memory, meaning, evidence, and restraint — merged into one pipeline instead of shipping as four disconnected demos.
A radio call comes in
“Rear is moving again when I get back on throttle.” Raw audio, nothing structured yet.
Speech becomes text
faster-whisper ASR transcribes the call. This alone is what most teams building this brief will ship.
The voice is scored for tone
Calm, elevated/stressed, or fatigued, with a confidence number.
This is the bare-minimum output the hackathon brief actually required. Everything below is where the real product starts.
The complaint is sorted, not diagnosed
One of 5 fixed categories — rear grip, front/braking, tyre wear, visibility/track, other mechanical. Never a fabricated diagnosis, just “this is the kind of thing being reported.”
FEEL2PHYSICS concept: free-form driver language mapped to a frozen, defensible taxonomy — not reduced to a sentiment score, not invented on the fly.
Checked against the driver's own baseline
Not “how a car should behave” in the abstract — this specific driver, this specific session, this specific corner, against their own last 5 laps.
The single biggest differentiator from a generic anomaly detector: no population average, no invented “correct” driving model. A scrappy driver isn't flagged for being themselves.
Memory kicks in
Past incidents are searched for anything that sounds similar and looks similar in telemetry. Two separate scores — sounds alike, looks alike — never mashed into one fake number.
ECHO LAP concept: nothing is judged in isolation. Every new report is checked against everything reported before — real semantic retrieval + real telemetry fingerprinting, gated together, not blended.
Lead time gets measured
Literally: how many seconds passed between the driver's warning and the moment the car's data actually got worse. Reported as null, honestly, when the data doesn't support a number — never forced.
The actual product thesis: the driver is a sensor arriving before the graph does. This number is what proves that, per incident, with real seconds — not a claim, a measurement.
One incident card, not four tabs
Transcript, tone, category, evidence, historical match, lead time, plain-English summary — one screen, one narrative.
RacePulse concept: every claim decomposed into inspectable, separately-labeled components — built to survive a skeptical engineer, or judge, asking “why?”
What actually makes this different
Most teams tackling this brief ship steps 02–03 and call it done — a transcript and a mood badge next to a lap chart. The thesis here is that a single opaque “risk score” is a demo crowd-pleaser and a production liability: a confident number nobody can decompose is what gets a real incident-review committee to switch a system off. Every number in this pipeline traces back to a named piece of evidence, and it says “I don’t know” out loud (INSUFFICIENT_DATA, a null lead time, no match) rather than manufacturing a positive to look impressive. We’re not shipping a smarter transcript — we’re shipping a plug-in layer that sits on top of the telemetry and radio systems a team already runs, and turns them into evidence an engineer would actually trust enough to act on mid-session.
System Design
Full stack, layer by layer
Every box below is a real, running piece of this system — not a planned architecture. Reproduce any layer directly: model IDs and pinned revisions are in services/radio_ai/app/config.py.
Audio ingest
- ·Radio clip (.mp3/.wav)
- ·16kHz mono resample (soundfile + scipy or torchaudio)
Speech-to-text
- ·distil-whisper/distil-large-v3.5-ct2 via faster-whisper (CTranslate2)
- ·CPU fallback: Systran/faster-whisper-small.en
- ·Pinned to a specific commit SHA — no silent model upgrades
Acoustic tone model
- ·laion/voiceclap-commercial encoder + attribute heads
- ·Outputs raw Arousal/Fatigue/Quality/Noise scores
- ·Mapped to CALM / ELEVATED_AROUSAL / FATIGUED by a threshold calibrated on 20 real human-labeled clips — not the model's raw default
Complaint classifier
- ·Production: sentence-transformers/all-MiniLM-L6-v2 embeddings + per-category prototype cosine similarity
- ·Fallback option: zero-shot NLI (DeBERTa v3)
- ·No fine-tuning — the taxonomy description text IS the model input
Evidence engine (classical, not ML)
- ·Own-baseline: median + MAD over the driver's own last 5 laps at that segment
- ·Retrieval: brute-force cosine similarity, in-memory (corpus is 15–30 incidents — FAISS is four orders of magnitude of complexity this doesn't need)
- ·Retrieval gate: category match AND cosine ≥ 0.40 — two independent conditions, empirically threshold-set over 247 hand-written phrase pairs
- ·Lead time: first_observable_change_time − radio_event_time, requires 2 consecutive deviating laps or reports null
Storage
- ·SQLite (storage/incidents.db) — incident metadata, recurrence flags
- ·Embeddings — in-memory Python list at process runtime, not persisted to SQLite (corpus is tiny; an index would be overhead, not optimization)
- ·Contracts — frozen JSON schemas + fixtures in contracts/, the interface every service is validated against
- ·Telemetry — Parquet windows per incident, read by pandas
API layer
- ·radio_ai (FastAPI) — POST /v1/radio/analyze
- ·core_api (FastAPI) — /v1/incidents/*, /v1/replay/*, /v1/live-pipeline
- ·mock_server (FastAPI) — same paths, fixture-backed, for frontend-first development
- ·No service imports another's code directly — every boundary is the JSON contract, not a shared library
Frontend
- ·Next.js 16 (Turbopack), React 18, TypeScript, Tailwind
- ·Fetches over HTTP with an embedded fixture fallback baked into the client — stays interactive if the backend is cold-starting
Direct answer: do we use a generative LLM? Tool-calling? MCP?
No, deliberately, on all three. No GPT/Claude/Gemini-style generative model is in this pipeline, no agent framework, no tool-calling loop, no MCP server. Every model above is a small, pinned, task-specific pretrained model (an ASR transcriber, a speech-emotion encoder, a sentence-embedding model) — not a chat model reasoning in free text. That’s a product decision, not a limitation: a generative model summarizing “what happened” can hallucinate a plausible-sounding but wrong diagnosis, and there is no way to audit that after the fact. Every number this system produces instead traces to a named, deterministic computation — a threshold, a cosine similarity, a median — specifically so an engineer (or a judge) can ask “why did it say that” and get a real answer, not a plausible-sounding one.