Why Sonnet 5.5 scored lower at Max than Xhigh—and how to choose effort
Include scope, stopping rules, and rework when calibrating an agent, not just reasoning depth.
On Anthropic's reported FrontierCode 1.1 Main results, Claude Sonnet 5.5 scored 52.1% at Xhigh effort and 46.2% at Max: a 5.9-percentage-point gap in the unexpected direction. The useful question is not whether Max is universally worse. It is whether extra autonomous work still meets the contract for an accepted deliverable.
What the benchmark actually rewards
The System Card describes FrontierCode as 150 software-engineering tasks from Cognition, derived from real open-source pull requests. An agent receives a repository, issue, and project guidelines, then works without human intervention. Grading combines blocking functional conditions with a weighted code-quality rubric.
The target is a mergeable patch requiring no human edits. Out-of-scope changes can be penalized even when useful. Anthropic's release footnote says Max more often launched Claude Code's code-review skill with multiple subagents; in two cases Cognition examined, that led to a timeout or extra edits outside the task.
This explains a failure mechanism in this harness. It does not establish a universal ranking of effort levels or predict a team's acceptance rate.
Effort changes execution behavior too
Anthropic's prompting guide notes that lower effort can stop to check in before a long task is complete, and Low can skip meaningful verification. Higher effort can initiate additional reviews, documentation, or related changes.
One failure returns unfinished work to a person; the other expands an assignment without permission. Speed alone cannot distinguish efficient completion from omitted checks. More tokens cannot distinguish necessary diligence from unwanted work.
Specify three things before testing: required artifacts, required checks, and the boundary at which to stop or ask. A bug-fix task might require a targeted patch and a test exercising the behavior, while forbidding unrelated refactors. Acceptance must cover all three.
Choose a starting point, not a universal default
Effort is recalibrated in Sonnet 5.5, so an old Sonnet 5 setting does not imply the same thinking budget. Anthropic recommends High for general Claude API work; Medium for well-specified agentic coding and multistep tool tasks, raising to High for harder or longer tasks; and Medium or Low for latency-sensitive chat.
Reserve Xhigh and Max for workloads with a measured quality benefit. They require adaptive thinking and do not accept between_tools. These are upstream recommendations, not claims about Buda's picker defaults or exposed controls.
Maintain a workload-to-setting table rather than fixing every job at Max. A researched quotation, a repository patch, and an interactive reply have different failure costs.
Make stopping part of the prompt
A useful example is:
Finish the requested change and run a real check that exercises it. When required checks pass, stop and report. Do not launch extra reviewer subagents, expand features, or modify unrelated files unless requested. Suggest deeper review separately. If a check cannot run, name it and explain why; do not report verified completion.
This does not ban necessary tests. It separates required verification from optional expansion. For high-risk work, independent review can be an explicit requirement.
Anthropic also reports a Max-effort coding test where a scope-limiting instruction stopped reviewer subagents and reduced session cost by about a third without changing quality. That is a vendor experiment, not a universal discount or Buda measurement. Copy the controlled-test method, not the promised saving.
Measure five outcomes across comparable runs
Keep inputs, permissions, prompts, time limits, and acceptance rules fixed. Evaluate representative tasks at two or three neighboring levels, starting from identical state so earlier files do not contaminate later runs.
Record accepted completion, scope violations, end-to-end time, execution consumption, and human rework. A cheap failure is not cheap delivery; a passing artifact with unauthorized writes is not an unqualified success.
Within a single billing unit, total attempt cost divided by accepted tasks can summarize execution cost. A group with zero accepted tasks has no finite cost-per-success estimate. Keep human review time separate, and never add dollars, Credits, and minutes without a defined conversion.
The guide also notes that changing top-level effort invalidates prompt cache; beta per-message effort can preserve it under adaptive thinking. Record cache policy so a cache difference does not masquerade as an effort effect.
A cheap attempt can produce an expensive delivery
The following numbers are hypothetical, not Anthropic or Buda measurements. Run the same ten independent tasks once at each of two settings, with identical inputs, tools and acceptance rules.
| Experiment | Lower effort | Higher effort |
|---|---|---|
| Consumption across all attempts | 20 units | 30 units |
| Tasks accepted without rework or scope violations | 5 | 9 |
| Mean consumption per attempt | 2 units | 3 units |
| Consumption per directly accepted delivery | 4 units | About 3.33 units |
The lower setting uses one-third less per attempt but yields substantially fewer usable outputs. This measures the batch's consumption divided by directly accepted outputs, not unrun retries or the eventual cost of delivering all ten tasks. That requires tracking later attempts and human repairs. Without those records, the ratio is not a complete project budget.
Higher effort can also lose through scope expansion: required tests pass, then the agent changes shared modules, rewrites documentation or launches more reviewers. Consumption rises and new risks appear. Fix permissions, scope and stopping conditions before interpreting every extra step as greater care—or lowering effort across the board.
Let failure categories determine the next experiment
The same failed acceptance check can call for different remedies:
- Missing input: the right interface description or reporting period was absent. Supply it; higher effort cannot reliably recover an unknown fact.
- Missing validation: code changed but required tests did not run. Establish tool availability and completion criteria before comparing settings.
- Insufficient reasoning: inputs and checks were available, yet an important constraint was missed. Try higher effort against the same acceptance criteria.
- Out-of-scope action: unrelated files, fields or business rules changed. Tighten scope and authorization; more changes are not inherently better.
- Early escalation: a long task paused to ask a question. Distinguish a legitimate permission boundary from an unfinished, clearly authorized task; only the latter is a completion failure.
Averages can conceal a critical minority. Nine valid summaries do not cancel out one contract summary that misses a liability clause. High-risk deliveries may require independent human review even when aggregate acceptance is strong.
Turn calibration into a conditional operating rule
A reusable rule might be: start at Medium for complete inputs, read-only tools and a clear delivery checklist; stop if a critical field is missing; deliver only after source and field checks; escalate according to an agreed rule when repeated attempts still fail. This is a proposed workflow, not a capability guarantee.
When effort, prompts or tool permissions change, rerun retained regression samples. Record model ID, platform, setting, prompt/tool versions, new-session status and cache behavior. If conditions cannot be held comparable, use the experiment to find problems, not to advertise a precise savings percentage.
Keep the default contexts separate: Claude Platform/API defaults must not be conflated with Claude application or Claude Code defaults. High here refers to the API documentation, not every client's default or an effort control exposed by Buda. Record the actual configuration rather than filling the experiment table with an assumed default.
Use the lowest sufficient setting
Begin with the lowest level that reliably clears acceptance, escalating when the failure is actually reasoning-related. Missing evidence needs better inputs; missing permission needs authorization; premature stopping needs a clearer completion contract.
Buda can carry the execution context and reusable workflow. Capture inputs, tool boundaries, and review criteria in that workflow rather than repeatedly asking an agent to be more careful. This is a design recommendation, not a claim that Buda reproduces the benchmark or exposes these effort parameters.
FAQ
Should Max be disabled?
No. Measure its benefit for complex tasks in your own harness.
Is 52.1% a predicted team success rate?
No. It belongs to a particular coding benchmark and grading setup.
Does lower effort always save money?
A single attempt may cost less, but retries and rework can reverse the result.
Does limiting extra review remove necessary tests?
Not if the contract requires necessary checks before stopping and treats additional review separately.