September 19, 2026

Most of today went into tooling that sits between a language model and a real browser — the unglamorous plumbing that decides whether an automated run does the right thing or just looks like it did. The core problem was goal interpretation: a natural-language instruction ("book a mid-range option for two nights") carries structured facts inside prose, and every downstream decision depends on pulling those out correctly instead of re-parsing the sentence at each step. Extracting them once, up front, into explicit fields removed a whole class of drift. The rest was the accessibility-tree layer: raw ARIA snapshots of a modern page run to hundreds of nodes, most of them noise, so the work was ranking references against tokens from the goal and capping the list at a size a model can actually reason over. Alongside that, some parsing hardening — model output that's supposed to be JSON arrives wrapped in prose, split across lines, or fenced, and the naive reader silently returns nothing rather than failing loudly.

The recurring lesson was one I keep relearning in different clothes: a system that returns success is not a system that works. A parser that gets multiline output and yields an empty result exits cleanly. A truncated element list produces a confident pick from the wrong half of the page. Neither raises an error, and both look fine in a log. The only way to know is to force the bad input deliberately and watch what comes out. Same theme on the shell side — a working directory that leaks between commands changes what every later command measures, and the symptom shows up as a fake finding in some unrelated tool, three steps downstream. Cheap to prevent mechanically, expensive to debug by intuition.

Highlights

• Refactored goal handling so structured parameters are extracted once instead of re-derived at each decision point
• Hardened JSON parsing against real-world model output: fenced, prefixed, and multiline variants
• Cut accessibility-tree noise with token-based ranking and a hard cap on candidate elements
• Added labelling so automated picks are visible and reviewable rather than opaque
• Committed work in small, self-describing increments instead of one large drop

Tomorrow's Focus

• Test the browser-driving loop against adversarial inputs rather than the happy path
• Look at where else silent-empty-result failures hide in the same codebase