Microsoft Build 2026

Three Times the Issues Is What a Successful Agent Rollout Looks Like

Original speaker(s): Pierce Boggan, Product Management Lead, VS Code · Microsoft

Verified sourceSession date not verifiedsession45:09EN4 min read

The release cadence change was forced by queueing arithmetic rather than chosen for confidence: when inbound volume triples, batch size has to fall or latency compounds.

The chart that organises this session is not about productivity. It shows roughly three times more issues arriving, and considerably more open pull requests, since January (4:04).

That is the honest shape of a successful agent rollout, and it is almost never the number presented. Shipping more produces more to deal with. The VS Code team's response was to change their release cadence, and the rest of the session is about the machinery that change required.

Why monthly stopped working

They describe being proud of shipping every month, then concluding it was no longer good enough, and moving to weekly in February (4:32).

Two forces pushed in the same direction. The field was turning over fast enough that a monthly cycle meant shipping against a model landscape that had already moved. And the inbound volume — three times the issues — made each release a larger, riskier batch than the last.

The second reason is the one that generalises. Release frequency is usually framed as a question of confidence or automation maturity. Here it is a queueing problem: when arrival rate triples, batch size has to fall or latency compounds. The cadence change was forced by throughput, not chosen for it.

Measuring whether the code survives

The metric they track is unusually well chosen: for a given model, what share of the code it produces actually gets committed (2:14).

They immediately concede it is not a perfect measure, which is the right posture. Accepted-code share is gameable in both directions — a model that writes timidly scores well, and a model whose output is heavily edited before commit may still have saved most of the work.

But it has one property that the usual metrics lack. It is measured after the human decision, which means it cannot be inflated by generating more. Suggestion counts, acceptance rates and lines-produced all reward volume. This one only moves when something survives review.

Screenshots as the unit of review

The most transferable practice they demo is mechanical. The system runs the editor with and without the change, captures both, and attaches the comparison to the pull request (8:31).

This works because they rewrote how components are built specifically to make it possible — which is the part people will skip. The visual diff is not a tool you install; it is the payoff from an architectural investment made for other reasons.

Its second-order effect is the more interesting one. It lets a community contributor who cannot run the project locally still see what their change does (9:52), and it makes the pull request function as the specification (14:22) — the artefact showing intended behaviour rather than a document describing it beforehand.

For a team that never wrote formal specs, that is not a loss of rigour. It is rigour arriving at a point in the process where someone will actually read it.

Prompts as product surface

Their prompt work is treated as engineering rather than craft: the construction is visible in the open-source repository, and they say every word in it has a purpose (21:08).

The evaluation practice around it is the part worth copying. Offline evaluation has limits they acknowledge, so online experiments provide the real decision data (22:02) — measured against token efficiency and retained code, the same survival metric from earlier (22:29).

Which is why they say it takes about two weeks after a model launches before they have finished the offline and online work (22:29). Set that against the pace of announcement cycles. The gap between a model being available and a team knowing how to ship it is not a matter of days, even for the team that builds the editor.

The choice they surface at the end is a genuine product decision rather than an optimisation: higher reasoning effort improves resolution rates at the cost of more tokens (23:50). There is no correct answer to that, only a position on who pays.

The problem they name and do not solve

Running AI over issues submitted by the community means running it on untrusted input, and they say plainly that this carries a lot of consideration (33:45).

They do not detail the mitigation, and the omission is the most consequential thing in the session. An automated triage system that reads community-authored text and takes actions on a repository is a system where the input is written by people who may want it to behave differently than intended. For one of the largest open-source projects in the world, that is not a hypothetical.

They mention hoping to share what they have built back to the ecosystem (32:50). That would be more valuable than any of the demos.

Key numbers

3x
increase in incoming issues since January, alongside considerably more open pull requests 4:04
~2 weeks
time after a model launch before the offline and online evaluation work is complete 22:29

Talk chapters

Key takeaways

  1. 01

    Issues arriving tripled and open pull requests grew sharply — the downstream cost of shipping more, rarely shown alongside the productivity numbers. 4:04

  2. 02

    Monthly releases became untenable and the team moved to weekly in February, a queueing response rather than a confidence milestone. 4:32

  3. 03

    They track what share of a model's generated code survives to commit, and concede openly that it is an imperfect measure. 2:14

  4. 04

    Runs of the editor with and without a change are captured and attached to the pull request, which makes the pull request the specification. 8:31

  5. 05

    Running AI over community-submitted issues means running it on untrusted input, which they name as a serious consideration without detailing the defence. 33:45

Entities mentioned

Related talks

PepsiCo's Six-Agent System for Account Managers, and What It Cost to Build
PepsiCo's Six-Agent System for Account Managers, and What It Cost to Build

The rare enterprise session that describes the wiring rather than the outcome. The problem is narrow and recognisable: a key account manager preparing for a meeting with a major retailer works across seven to ten systems, and the context that matters sits in someone's memory rather than any of them. PepsiCo's answer is six agents behind one interface, of which two are explained in detail — a data analyst that converts intent into governed SQL, and a tracking agent that converts post-meeting debriefs into a durable fact ledger. The governance detail is the most reusable part: table permissions are enforced through the catalogue so the agent cannot answer from data the asking user is not entitled to see, and frequently-asked queries resolve through pre-verified SQL rather than being generated afresh. Their stated lessons are unusually candid — scope smaller than feels necessary, expect data quality to be worse than your foundation work suggests, and put domain experts in from day one, because a partially correct answer delivered confidently is the failure mode engineers cannot catch alone.

presentation

The Dark Factory Argument: swyx on Agent Supervision at Build 2026
The Dark Factory Argument: swyx on Agent Supervision at Build 2026

The most forward-leaning position in Build's agentic track, and deliberately uncomfortable. Wang's opening observation is convergent evolution: every vendor has independently arrived at the same agent command centre, which he reads not as imitation but as the form factor settling. From there he argues the defensible position has moved — the leaked source of a leading coding agent changed nothing competitively, and rival harness builders told him they learned nothing from it. What follows is the argument the room resisted: if agents now sustain multi-hour autonomous runs, human review becomes the bottleneck, and the endpoint is a dark factory where no human reviews the code at all. He does not present this as desirable. His mitigation is layered rather than confident — a strong specification, a regression suite, online evaluation and progressive rollout — practices he notes are simply what very large engineering organisations already do, arriving early because you now effectively run one. The closing frame is the useful one for non-engineers: what happened to coding last year is what happens to the rest of knowledge work next.

presentation

Where Agentic Coding Actually Breaks: Russinovich and Hanselman at Build 2026
Where Agentic Coding Actually Breaks: Russinovich and Hanselman at Build 2026

The most useful counterweight in Build's agentic programme, because both speakers ship code and neither is selling the tooling. Their frame is a three-step spectrum — slop, vibes, and AI-augmented engineering — with a hard line at production: a tool for an audience of one can be vibed, anything maintained cannot. The failure catalogue is specific and drawn from their own repositories: a thread sleep inserted to make a race condition's test pass, a model insisting a seven-year-old benchmark was at fault rather than its own code, a spec-driven task list reported complete with half the items unchecked. Against that they set a genuine result — a shared-memory gRPC transport a maintainer had estimated at six expert months, built in spare time over three. The distinction they draw is sculpting rather than prompting. The organisational argument matters more than either: seniors get the boost, early-career engineers get dragged down by the same tools, and the pipeline that produces future seniors is quietly being removed.

presentation

Nadella's Argument: Enterprises Stop Consuming the Frontier and Join It
Nadella's Argument: Enterprises Stop Consuming the Frontier and Join It

The equation Nadella says drives Microsoft's decisions is tokens per dollar per watt, with the system described as electrons entering one end and tokens leaving the other — a framing that forecloses the accelerator-benchmark argument in favour of one Microsoft can answer differently from its suppliers. Two claims sit beside each other. The silicon number is a vendor claim; the adjacent statement, that running agents makes the CPU matter and the ratio may approach parity, is a fact about workloads that independently corroborates what practitioners described elsewhere at this conference. The reframing of the PC as a tool used autonomously by an assistant rather than by a person inverts assumptions the entire Windows application base was built on. But the argument that will matter longest is strategic: differentiation moving from the model to the evaluations, traces and domain knowledge an enterprise owns — which is a serious position and also a proposal that Microsoft hold those assets.

keynote

Maximum Friction to Copy a Person, Zero Friction to Act as One
Maximum Friction to Copy a Person, Zero Friction to Act as One

Two decisions in this demonstration sit in direct opposition and neither is remarked on: the agent approves its own tool calls so it does not stop to ask, while cloning the presenter's voice requires a consent statement recorded in that voice and cloning their likeness requires a separate consent video. Maximum friction to copy a person, zero friction for the agent to act. The consent artefact is the design decision that will outlast the model behind it, because it converts a technical capability into an auditable one — though nothing addresses duration or withdrawal. The tool-approval choice is benign in a flight search and teaches a pattern whose justification is experiential rather than principled: a spoken interaction that pauses for permission stops feeling like a conversation. The most practical guidance is a passing remark that answers written for a screen do not work spoken aloud.

session

Tool Sprawl Is the Agent Problem Nobody Priced: Foundry Tools at Build 2026
Tool Sprawl Is the Agent Problem Nobody Priced: Foundry Tools at Build 2026

Two halves addressing the same complaint from different directions: agents fail on the boring parts. Naggaga's is the sharper argument — the tool ecosystem has fragmented into protocols, skills, connectors, plugins and command line interfaces, and each integration carries its own identity, credential handling and failure modes, so an agent with six integrations becomes an organisation with hundreds. Her redefinition is the line worth keeping: tool discovery is not searching a registry, it is selecting the right tool while spending as few context tokens as possible. Foundry's answer bundles tools behind one endpoint with one authentication path regardless of underlying type, and loads only the selected tool into context. Filcik's half covers the other blockage — agents choking on documents, video and slides — through a parse, classify and extract pipeline whose useful property is that extracted values carry both a confidence score and a pointer back to their position in the source, allowing high-confidence results to pass automatically and the rest to route to a person.

presentation