Coding agents that can listen

AI coding agents can now write the code that renders a product demo, a launch film or a game. They can't hear any of it. An agent won't notice that a hit lands 80 ms after the cut, or that a key press sounds cheap. earshot fixes that. It turns sound into text an agent can reason about: what a file sounds like, how well it fits a description, where the best moment sits in a long recording, and whether a clip is a real recording or synthesized in code. It runs locally on CPU as a CLI, a Python API and an MCP server.

How it works

Under the hood are two open models, CLAP for text-audio embeddings and an AudioSet tagger, plus a set of plain measurements like LUFS, true peak, spectrum and tempo. Commands like describe, match and find return a plain-language summary first and full JSON if you ask for it. timeline produces a second-by-second listening transcript of a film, including sync to picture cuts. sheet renders it as an annotated PNG for agents that read images.

The part I like most is forge. The agent writes a sound as a recipe of physical synthesis: modal resonators, filtered noise, a computed room. A critic built from earshot scores the renders, an evolutionary search keeps the best ones, and the agent reads the feedback ("bumpy tail") and rewrites the recipe. fit goes the other way. It turns a recording into a recipe module you can re-render and vary. Version 0.3 also ships a bank of 201 sound effects stored as code. They render by id in seconds, with no models and no network.

Try it

git clone https://github.com/Leonxlnx/earshot && cd earshot
python -m venv .venv
.venv/bin/python -m pip install torch --index-url https://download.pytorch.org/whl/cpu
.venv/bin/python -m pip install -e ".[mcp,llm,dev]"
.venv/bin/python -m earshot doctor --load
claude mcp add earshot --scope user -e EARSHOT_THREADS=3 -- /path/to/earshot/.venv/bin/python -m earshot mcp

You need Python 3.10+ and ffmpeg on PATH, and the first model use downloads about 1.1 GB.

My take

The README is unusually honest. Every claim links to the command that measured it, and the limits get their own section. It says the forge can game its own models (Goodhart) and that the realness model is easy to fool. The blind tests had one listener, and the README says so. I trust a project more when it tells me where it breaks.

There are real limitations. Each CLI call loads the models, which takes 20 to 40 seconds, so for anything beyond a few calls you'll want the MCP server or a Python script. Tags inside a busy mix are only hints. The models can't judge taste, emotion, speech content or anything above 14 kHz. The launch scorecard reflects the style of 7 reference films, not good films in general.

You should care if you build videos, demos or games in code with an agent and you're tired of being the only one in the loop who can hear the result.