Who Decides Which Model Runs? An On-Device Audio Pipeline Routed by Jev 🎙️🔒
Every on-device AI project hits the same awkward question: you've got a handful of small local models (language detection, transcription, redaction), and something has to decide which one runs, on what, and in what order. The lazy answer is ""ask an LLM."" The problem is that you've just put a slow, expensive, network-bound text generator in charge of a pipeline that was supposed to be fast and private. 😬
A new build on the Jev showcase site, On-Device Audio Pipeline from Desert Ant Labs (posted September 22, 2026), takes another route. It uses TypeSafe AI's Jev, the typed-decision ""System One"" model, as the traffic controller. Per the page: Jev ""makes about 20 decisions in one call and picks which local model runs"" in milliseconds, with no LLM in the inference loop.
The build page is short, so this post covers what it says, then goes further into how I'd design something similar and the privacy question you should ask first.
What the Build Does
According to the write-up, the demo processes audio files through on-device models coordinated by Jev:
- 👂 Ear — on-device language detection
- 🗣️ Voz — on-device speech transcription
- 🫥 Redact — on-device PII removal
- 🧠Jev — the decision layer, about 20 decisions per call, choosing which local model runs
Three use cases are demonstrated: turning voice memos into to-do lists, transcribing meetings with sensitive details redacted, and splitting podcasts into clips. The page reports routing in milliseconds and end-to-end results in seconds, but it doesn't publish a detailed benchmark, hardware spec or code, so I can't tell you more than that. The tags are on-device, audio, model routing and PII redaction.
Why Routing Is the Right Job for a Decision Model
Notice what Jev isn't doing here. It isn't transcribing, summarizing or writing. It's answering small typed questions about the situation and letting code act on the answers. That is exactly the shape Jev is built for:
- Choice: which transcription model fits this clip?
- Noul (yes/no): does this contain personal data that needs redacting?
- Score: how noisy is the audio, from clean to unusable?
Twenty of those in a single call is plausible because extra questions cost a few input tokens and almost no extra wall-clock time. That's the fan-out pattern in action.
A Design Sketch You Can Adapt (Python)
Important: the code below is my own illustration of the pattern, not Desert Ant Labs' implementation. detect_language, transcribe, redact and the model registry are placeholders for whatever local models you use.
from typesafe_sdk import Choice, Noul, Score, TypeSafeClient
client = TypeSafeClient(model=""jev-1.13.0"")
def route(features: dict):
# features = cheap, non-sensitive facts about the clip
r = client.system_one(
state=features,
questions={
""language"": Choice(
instructions=""Best language model for this audio"",
criteria={""en"": ""English"", ""es"": ""Spanish"", ""hi"": ""Hindi"", ""other"": ""Anything else""},
),
""noise"": Score(
instructions=""How noisy is the recording"",
criteria=[""Clean"", ""Some noise"", ""Very noisy""],
),
""multi_speaker"": Noul(instructions=""More than one person is speaking""),
""needs_redaction"": Noul(instructions=""Sensitive personal details are likely present""),
},
)
a = r.answers
return {
""lang"": a[""language""].choice,
""lang_confident"": a[""language""].confidence >= 0.7,
""heavy_model"": a[""noise""].score > 1.2,
""diarize"": a[""multi_speaker""].noul > 0.6,
""redact"": a[""needs_redaction""].noul > 0.3, # low bar: redacting extra is cheap
}
def process(audio_path):
feats = extract_cheap_features(audio_path) # duration, SNR, first-seconds guess
plan = route(feats)
text = transcribe(audio_path, lang=plan[""lang""], heavy=plan[""heavy_model""])
return redact(text) if plan[""redact""] else textThe design choice that matters: the decision layer sees cheap derived features, and code (not the model) enforces the policy. Note the asymmetric thresholds. Falsely redacting is harmless, so the bar is low. Falsely skipping redaction leaks data, so I lean toward redacting whenever in doubt.
The Privacy Question You Should Ask First ⚠️
""On-device"" is the headline, but Jev is a hosted API (documented endpoint: api.typesafe.ai). So a fair question for any build like this is: what exactly gets sent to the decision model? The build page says the audio models run on-device; it doesn't spell out what the routing call receives. Before you copy this pattern for real user data, decide:
- Do you send only metadata and features (duration, noise level, detected language), never the transcript?
- If you do send text, is it redacted before it leaves the device?
- Does your privacy policy and any regulation you fall under (GDPR, HIPAA, call-recording laws) allow this data flow?
- What happens when the network is down? You may need a local fallback router.
If the whole point of your app is that nothing leaves the phone, a hosted router undermines it. In that case a local logprob-based approach like choosekit, or plain rules, may fit better.
""On-device"" only means something if you can say exactly what crosses the network boundary.
Other Places This Pattern Fits
- 📹 Video pipelines: choose which local vision model runs per clip.
- 📄 Document scanners: OCR, handwriting or table extractor?
- 🤖 Robotics and IoT: route sensor events to the right handler without a cloud LLM.
- 🎧 Call centers: decide per call whether to transcribe, translate or escalate.
Limits and Honest Unknowns
- The page gives no numbers beyond ""about 20 decisions per call"" and ""milliseconds,"" and no hardware or dataset details. Don't quote it as a benchmark.
- Routing quality depends on the questions you write. Jev reads literally, so phrase criteria as clear positives.
- Jev takes text only. Audio must be reduced to text or features first, which means your feature extraction is now part of your accuracy story.
- Untrusted text in the state (for example a transcript) can argue for its own classification. Keep policy checks in code.
Frequently Asked Questions about Jev-Routed Audio Pipelines
Q: What is the On-Device Audio Pipeline build?
A demo from Desert Ant Labs, listed on the Jev showcase site, that processes audio with on-device models for language detection, transcription and PII redaction, with Jev deciding which local model runs.
Q: Does Jev process the audio itself?
No. Jev takes text state and returns typed decisions. The audio work is done by separate on-device models, and Jev only routes between them.
Q: Is the whole thing really on-device?
The audio models are described as on-device. Jev itself is a hosted API, so the routing call involves the network unless you use a local alternative. The page doesn't detail what data that call carries.
Q: How many decisions can Jev make in one call?
The build cites about 20 decisions per call. Documented limits are 64k tokens for state plus all questions, so you can batch many questions as long as they fit.
Q: Why not use an LLM to route between local models?
An LLM adds latency, cost and a generation step that can return malformed output. A typed decision model returns a fixed-shape answer with probabilities in one pass.
Q: How do I handle PII safely?
Redact before any text leaves the device, send derived features instead of transcripts where possible, and set a low threshold for triggering redaction since over-redacting is cheap.
Q: What confidence threshold should I use for routing?
Scale it to the cost of a wrong route. Choosing a lighter transcription model on a clean clip can tolerate lower confidence than deciding whether to skip redaction.
Q: Can I do this fully offline?
Not with the hosted Jev API. For fully offline routing, consider a local model with logprob scoring (such as choosekit) or deterministic rules.
Q: What use cases did the demo show?
Voice memos to to-do lists, meeting transcripts with redaction, and podcast segmentation into clips.
Q: Is there source code for the build?
The page I reviewed doesn't link source code or a benchmark. The Python in this post is my own sketch of the pattern, not the authors' code.
Your Turn 💬
Here's what I'm genuinely unsure about: would you let a hosted decision model route your on-device pipeline, or does that break the promise of ""on-device""? If you're building something with local models, how do you decide which one runs today? Rules, a small classifier, an LLM? Tell me what works and what fell over in production. 👇