Open source · macOS menu bar · v0.9.1

Talk to your Mac.
Jev decides. The app verifies.

Hold ⌥ Space, say what you want — "open a new session in Devin", "type in the prompt box check CPU usage", "quit Spotify" — and release. Speech stays on-device, every decision is a typed answer from TypeSafe's Jev, and nothing is called done until the screen proves it.

on-device speechno LLM in the loop by defaultMIT licensed
listening · hold ⌥Space

    Release the key and the command fires. No silence timer, nothing cut off.

    01 / The flow

    One utterance, end to end

    Pick a command to light up the path it takes. Solid boxes are local code, indigo boxes are single Jev requests, green is verification, dashed amber is a bounded fallback that only runs when the step before it failed.

    command

    Jev Voice pipeline: hear, decide, act, verify you hear · on-device decide act verify not matched matched destructive / browser verified stuck Hold ⌥Space · speak · release "Done." (spoken)or the exact reason it stopped SpeechAnalyzer50–300 ms Normalize · splitcontent is never split Confirm: yes / noheld utterance · 15 s Local parserdeterministic · 0 ms Jev router7 questions · 1 request Safety policyblock · confirm Executoropen · quit · URL · keys Fast pathapp-actions · ⌘↩ Observe AX tree≤1.5 s · partial-aware Jev Choicepick a real element Act oncepress · click · type full re-observe → wake → OCR → DeepSeekeach once · bounded · then a clear failure Goal metverified change Re-observe · postconditionno "done" on confidence
    local codeJev requestverificationbounded fallback

    what the log says

    02 / How it decides

    Decide, don't generate

    Jev is a System One model: it answers typed questions with a choice or a probability, in one round trip. It never writes the text that gets typed, and it can only pick a control that is actually on screen — so it cannot invent a button, a coordinate or a URL.

    One request routes the clause

    Every clause the local parser can't settle becomes a single systemOne call with the clause, the full transcript, the frontmost app, installed apps and aliases as state — and seven questions.

    questiontypeanswer space
    actionchoiceopenApp · closeApp · openURL · webSearch · dictate · uiTask · system · none
    target_appchoiceinstalled app names + none
    system_actionchoicevolumeSet · mute · lockScreen · screenshot · …
    mentions_urlnoulp(the clause names a website)
    refers_to_frontmostnoulp("quit it" means the frontmost app)
    composesnoulp(the user wants wording written)
    destructivenoulp(hard to undo)

    One Choice picks the next step

    For UI work the app reads the target window's Accessibility tree, ranks the buttons, links, rows, tabs and text fields by overlap with the goal words, and asks Jev one next_action Choice over those real candidates plus done and stuck. Alongside it: goal_reached, wrong_surface, needs_text as probabilities.

    candidates:
      [1] AXButton  "New session"
      [2] AXTextArea "Ask Devin…"     ← dictation goals see only text inputs
      [3] AXRow     "Check RAM and CPU consuming resources"
      [4] OCR       "Send"  (x=1412,y=884)  ← when the tree is thin
      done · stuck
    
    next_action → 1   (p=0.91)
    goal_reached → 0.08

    Arguments — URLs, queries, the words to type — are extracted deterministically by SlotExtractor: "type in the prompt box to check CPU" types check CPU. Words that are content never influence which control gets picked.

    0 mslocal parser for open/quit/switch/volume/URL/search
    1 requestper routed clause; one more per UI step
    ~10×faster per step than the generative planner it replaced
    ≤ 1 in 10commands ever reach the DeepSeek fallback

    03 / Reliability

    Observe · act once · verify

    Computer use fails in boring ways: a sparse Accessibility tree, a click that lands on the wrong row, a "Done" that wasn't. Each step of the loop is designed around those failures.

    OBSERVE

    A complete, fresh tree

    The in-process walker spends up to 1.5 s (4 s on a full re-observe) and marks snapshots partial when it hits the deadline. Electron apps expose a thin tree until touched, so a wake-up pass focuses the window and re-walks. A partial tree is never evidence that the UI is stuck.

    TARGET

    Text goes into text fields

    When the goal contains text to type, only AXTextField, AXTextArea, AXSearchField and AXComboBox are candidates. Create/send controls fire at most once per task; if the screen already changed, the step is done regardless of Jev's confidence.

    ACT

    Real events, grounded geometry

    Clicks use AXPress; if the element refuses and its frame is known, one real CGEvent click at the frame centre. Typing is in-process Unicode key events into the focused field. Nothing is ever aimed at a coordinate a model made up.

    VERIFY

    Postconditions, not vibes

    Each action has its own check: the field contains the text, the element count changed, the prompt box emptied, a window appeared. Only a verified change yields "Done". If the step budget runs out but the goal is verifiably met, that's success — not a false failure.

    HEAR

    Push-to-talk, no guessing

    Key-down starts the on-device engine, key-up finalizes. Toggle mode keeps a silence timer with a Jev complete? judgment on top (≥0.75 finalize now, ≤0.35 keep listening). Only one analyzer may own the audio stream; restarts are serialized so each utterance finalizes exactly once.

    FAIL

    Fail clearly, fast

    Every run has a 90 s wall clock; fallbacks have 6 steps / 30 s. When the ladder is exhausted the app says why — "I can't see a text box in Devin", "Allow Jev Voice to control System Events" — instead of guessing or pretending.

    when Jev says stuck

    1
    Full re-observe

    4 s walk, no interactive-count requirement. Most "stuck" cases end here.

    2
    Wake the tree

    Focus the window and re-walk — Electron trees grow from ~30 to ~600 elements.

    3
    OCR

    Apple Vision reads visible labels with positions; Jev picks among them like any element.

    4
    Chrome DevTools / DeepSeek

    DOM via CDP for Chromium windows; a tool-calling planner with a scaled screenshot as last resort, capped.

    5
    Clear failure

    A spoken reason and the log line that explains it. Never a silent "Done".

    04 / Safety

    Policy is data, confirmation is context-aware

    policy.json ships with the app and can be overridden in Application Support. Blocked patterns never run. Confirmation words ask a spoken yes/no — and know whether you're inside an app or on the public web.

    Always

    • shell / terminal commands, credential and payment phrases, irreversible file operations — refused
    • delete · remove · empty trash · publish · pay — ask first
    • controls labelled delete, uninstall, shut down, pay… — ask before pressing, even if you didn't say the word
    • generated scripts (learned tools) must compile, pass the policy scan and a Jev review, and are previewed aloud

    Only in a browser

    • send · submit · post · tweet · reply — ask when the target is a website or a browser
    • the same words inside Devin, cmux or Notes run immediately — "send the prompt" needs no yes/no
    • in hold mode the answer is your next held utterance or the popover's Yes/No; the mic never opens unasked
    • what leaves the Mac: transcript + routing context to api.typesafe.ai; a scaled screenshot to DeepSeek only if the fallback is entered

    05 / Install

    Homebrew, then a TypeSafe key

    Install

    brew tap chris-wozniczek/jev-voice https://github.com/chris-wozniczek/jev-voice-control
    brew install --cask jev-voice
    open "/Applications/Jev Voice.app"

    Homebrew 7+ may ask you to brew trust chris-wozniczek/jev-voice first. Then paste your TypeSafe API key in the popover's Settings (gear icon). macOS 26 is recommended for the streaming speech engine; older macOS falls back to the classic Apple recognizer or Whisper.

    Build from source

    make app    # build/Jev Voice.app (ad-hoc signed)
    make test   # 200+ unit tests on the pure logic
    make dist   # zip + sha256

    Permissions it asks for

    • Microphonehear you
    • Speech Recognitionon-device transcription
    • Accessibilityread controls, click, type
    • Screen RecordingOCR fallback only
    • Automation · System Eventsvolume, dark mode, scripts

    Debug

    log show --last 5m --info \
      --predicate 'subsystem == "com.chriswozniczek.jevvoice"'

    Look for stage=axtree elements= partial=, stage=ocr, stage=click, outcome=.