The score went up. The trust did not.

Here is the contradiction I keep running into. Where a task has a checkable success signal, you can raise a coding agent’s benchmark accuracy by paying for more evaluation attempts, scoring the task solved if any attempt passes the hidden tests. Repeatable behavior does not follow. So agent accuracy and reliability are different axes: the extra attempts that buy the first do nothing for the second, and each must be measured separately.

TL;DR

  • Accuracy can be bought at evaluation time. Where a task has a checkable success signal, benchmark accuracy (the fraction of tasks scored as eventually solved) can be raised by spending more at evaluation time under a pass-if-any protocol: run more candidate attempts, each with some chance of passing, and count the task solved if any passes the hidden tests.
  • Pass-if-any climbs “with seemingly no limit.” An added candidate cannot lower the chance that at least one passes, and raises it to the extent the attempt has some fresh chance of succeeding, so that pass-if-any score can climb with sampling budget with, in Kapoor et al.’s exact phrase, “seemingly no limit” (by less the more the failures are correlated), whether or not the agent behaves the same way twice.
  • Reliability is a separate axis. Reliability is Rabanser et al.’s (2026) four dimensions; its consistency dimension asks whether repeated runs of the same task reach the same outcome, and the extra evaluation attempts that buy accuracy do not make behavior repeat.
  • Capability outran reliability. Over about 24 months, capability rose fast while overall reliability improved slowly and unevenly, so reliability has to be measured separately (Rabanser et al., 2026); with no accuracy-per-year trend to set against the reliability slopes, that temporal gap is the authors’ qualitative reading.
  • High accuracy is not trustworthy behavior. So a high accuracy score signals capability of the evaluated system under its protocol, not trustworthy repeatable behavior, and consistency can even be high for an agent that is consistently wrong. Reliability has to be measured on its own.

On this page

Accuracy is something you can buy

The cleanest illustration is the one that first made me distrust the word accuracy. On competitive programming, AlphaCode scores near 0% with zero-shot generation, over 15% at 1,000 samples, and over 30% at about 1,000,000 samples (from Li et al. (2022), reported by Kapoor et al. (2024), Section 2.1). Read the metric before you read the trend, because the trend is not what it looks like: it credits the task as solved if one of a small set of submitted attempts passes the hidden tests, a pass@k-style score, not single-answer accuracy. Nothing about the model itself got better across those columns. The weights are not changing between 1,000 and 1,000,000 samples; the inference system around them, model plus sampling plus selection, spends far more computation to buy more chances at a passing attempt. And AlphaCode did not just draw at random: it filtered, clustered, and ranked the samples to a small submission set without access to the hidden tests; the benchmark then credited the task if a submitted candidate passed those tests. That is the split I keep coming back to. Selection and scoring are separate steps, and production usually lacks a verifier cheap and accurate enough to do the selection, so turning a high pass-if-any score into deployable success is a separate, harder problem.

So state the condition plainly, because it is the whole game: this accuracy is purchasable where a verifiable success signal exists (competitive programming has one, does the code pass the tests) and you score pass-if-any. Kapoor et al. (2024) put the ceiling bluntly, writing that accuracy can be increased with “seemingly no limit,” and calling out “scientifically meaningless methods such as retrying.” I read “meaningless” carefully the first time, because it lands harder than it means: it means meaningless as evidence of architectural progress, not operationally useless. Retrying can be a fine deployment tactic, but a score inflated by uncontrolled retries cannot tell you the design got better.

The cost side is where this got uncomfortable for me as a reader of leaderboards. On HumanEval (164 tasks), Kapoor et al. (2024, Sections 2.2 to 2.3, Figure 1) found agent strategies spanning almost two orders of magnitude in cost for similar accuracy: the LATS strategy cost more than 50x the simple “warming” baseline, Reflexion and LDB cost more than 50% more, and none showed a significant accuracy gain over the best baseline. And it runs the other way too, which surprised me more: with joint cost-accuracy optimization, Kapoor et al. (2024, Section 3.2, Figure 2) held accuracy at no significant loss for 53% lower variable cost on GPT-3.5 and 41% lower cost on Llama-3-70B for HotPotQA. Sit with that for a second. Accuracy that is not significantly different sits on top of wildly different amounts of spend, so without cost control it cannot even establish that one architecture beats another (Kapoor et al., 2024). It measures capability and budget at once, and says nothing about whether behavior repeats.

Reliability did not keep pace

Now the other quantity, and here I had to slow down and define the term before I could measure anything with it. Rabanser et al. (2026), “Towards a Science of AI Agent Reliability” (ICML 2026), define reliability as four dimensions, all “independent of raw capability”: consistency, robustness, predictability, and safety. Read “independent” as a distinct construct measured on its own, not statistically uncorrelated: the same paper reports a reliability-versus-accuracy correlation. They aggregate the first three into an overall reliability score and treat safety separately. The one that matters most here is consistency: “Does the system behave the same way when run multiple times under identical conditions.” And “the same way” is narrower than it first sounds. Read it as the same outcome, passed or failed, not identical wording or reasoning path: different correct solutions are not a failure, unstable outcomes are. Consistency is also orthogonal to correctness, which is the part I see people miss: an agent can be perfectly consistent and consistently wrong.

Then they tracked it, and this is the measurement I most wanted to see. Plotting overall reliability against model release date across 15 models from 3 providers, running each task five times (K=5) at temperature 0, they measured a slope of 0.03 per year on GAIA (a 165-task split, correlation r=0.46) and 0.09 per year on tau-bench (a 26-task clean subset, r=0.86) (Rabanser et al., 2026, Figure 1). Read together, the result is mixed: GAIA barely climbed, tau-bench gained more visibly over roughly two years. Both are overall reliability (the three aggregated dimensions, safety separate), not the consistency sub-dimension alone. Two caveats: the protocol is narrow (two benchmarks, one agent setup each), and Rabanser reports no accuracy-per-year trend to set against these reliability-per-year slopes, so “capability outran reliability” is a qualitative reading, not a fitted year-on-year comparison.

I want to be careful, because it is easy to overstate. Overall reliability did not stay frozen, and on tau-bench it moved a fair amount. But it rose slowly, and that lag is what this article is about.

Why accuracy and reliability come apart for AI agents

Once you separate the two questions each metric is actually asking, the mechanism stops being mysterious. It helped me to write them side by side and refuse to let them blur into each other.

Accuracy under a pass-if-any protocol asks: across the candidate attempts, did at least one pass? Its answer improves as you add candidates, the maximum over more of them. You do not need stable behavior for that number to climb. One candidate passing, confirmed at evaluation time by an oracle, lifts the number whether or not the attempts agreed.

Consistency asks the opposite-shaped question: across repeated runs of the same task under identical conditions, did the agent reach the same outcome? That is about whether the runs agree with each other, and here is the part that trips people up: more runs do not make them agree. A wide, scattered distribution can post a high pass-if-any score (some candidate passes) while showing low consistency (the runs disagree); a tight one can sit around a wrong answer. This is the sibling point to our piece on why a single run is a sample of size one: “did any candidate pass” and “do repeated runs agree” measure the same hidden distribution differently. More runs do not change its spread; they only sharpen our estimate of it.

So the real distinction is one of unit, and it is worth saying slowly. More candidate attempts scored pass-if-any raise the benchmark and change nothing about the single-attempt policy they score. Changing the deployed system within a run (retries, voting, verification) is a different unit, a compound system whose consistency must be measured on that system, not assumed.

The steelman: the score is fine, stop overthinking it

Let me give the objection its due, because it is half right.

The case goes: benchmark accuracy is a real, comparable signal that correlates with capability, and teams have shipped plenty of useful software on it. Reliability obsession, the argument runs, is analysis paralysis: if an agent solves the task, who cares whether it does so the same way twice? Ship it, add a retry, move on. For workloads with a cheap checkable verifier and no cost sensitivity, retry-until-pass is a legitimate pattern, and the score is fine there. I will not pretend otherwise.

Here is where it stops being fine. An extra attempt never lowers pass-if-any success and raises it only when it brings a fresh chance, by less the more the failures are correlated: when a fixed policy reproduces the same wrong path, extra attempts add little, down to nothing when the runs fail together, and a fallible verifier can also pass a wrong answer or reject a right one. So retries help, just less than a pass-if-any score implies. And in production, where a cheap accurate verifier is often missing, a retry is expensive, or a human is calibrating trust to the last few runs, consistency becomes the property you rely on, and the accuracy number never measured it.

Here is the strongest form of the objection. If the retries and verification ship as part of the product, the unit is the complete retrying system, and repeated runs of it can be more successful and more repeatable. Granted, and that is the point: reliability must be measured on the deployed system as it will run, not assumed from a pass-if-any score and not assumed unchanged. The steelman is right that accuracy signals capability; it is wrong that it is the only signal you need, where getting it wrong is expensive.

What raises a score, what tightens consistency

It helped me to lay the two levers out in one table, because in a hurry I catch myself treating them as the same column. They are not.

What raises a pass-if-any evaluation scoreWhat can tighten outcome consistency (within-run engineering)
More candidate attempts under pass-if-any (the max over draws rises)Voting or aggregating candidates inside one deployed response
A larger sampling and compute budget at evaluation timeVerification or a validator applied within the run
Filtering and ranking many samples down to a small submission setA more deterministic policy (lower temperature, fixed tool order), often at a latency or cost penalty

Those levers can tighten consistency but not always; they can also move accuracy, and even temperature 0 (Rabanser’s setting) leaves some variation. The asymmetry is the whole point I keep returning to: extra attempts that raise the score do nothing for the underlying policy’s consistency.

How we think about it at NativeFold

This is where NativeFold comes in, and only here. We exist for that second question: the repeatable-outcome behavior a pass-if-any score never captures. Task success and outcome stability answer different questions, and neither alone establishes trustworthy use: a stable-but-wrong agent is not trustworthy, and an accurate-but-unstable one is not either. What we measure is whether an agent reaches the same outcome across repeated runs, reported as stability with its uncertainty, one part of reliability, not the whole of it or of trust. We measure that gap; we do not sell or perform the interventions that tighten consistency. That a headline accuracy score is not, on its own, evidence of repeatable behavior, and that both questions must be measured, is the hypothesis our measurement exists to test.

People ask me how many runs that takes, and I think it is the wrong question. A study is sized for the precision its claim requires, and convergence diagnostics show whether that precision was reached. If you want the full method, it is written up on our methodology page.

Frequently asked questions

Does a higher benchmark score mean a better AI agent?

It signals capability of the evaluated system under its sampling and budget, not trustworthiness. A pass-if-any benchmark counts a task solved if any attempt passes, so more evaluation attempts raise the number without making any single production run repeat. Without cost control it also cannot show that one design beats another (Kapoor et al., 2024). Whether behavior repeats is a separate question.

Can you improve agent accuracy by retrying?

Where a checkable success signal exists and you score pass-if-any, yes: an extra attempt cannot lower the chance at least one passes, and raises it to the extent it brings a fresh chance of succeeding. Two limits matter. The gain shrinks as failures become correlated, down to nothing when the runs fail together. And retries cost latency and money at deployment, where a verifier cheap and accurate enough to pick the winning attempt is often missing.

Is accuracy the same as reliability for AI agents?

No. A pass-if-any (pass@k) score asks whether at least one candidate passes; that is the scoring protocol, not accuracy in general. Reliability’s consistency dimension asks whether repeated runs reach the same outcome, which more candidates do not fix. Over about 24 months, Rabanser et al. (2026) found overall reliability improving slowly and unevenly while capability rose fast. It also varies by workload: some tasks an agent handles consistently, others it does not, so a single headline number hides task-to-task heterogeneity.

What is agent reliability?

Rabanser et al. (2026) define it as four dimensions distinct from raw capability (measured on their own, not statistically uncorrelated with it): consistency, robustness, and predictability (aggregated into an overall score) plus safety (separate). Consistency, the one most people mean, is whether repeated runs under identical conditions reach the same outcome, task passed or failed, not identical wording or reasoning path. Several different correct answers are fine; unstable outcomes are the failure. An agent can also be consistent and consistently wrong.

References