Your analytics can record that an SDK was installed. What they cannot recover from that signal is agent tool selection: whether a coding agent chose your vendor for a job the developer did not name, and which vendors it installed instead on the builds you lost. That decision is made in the agent’s context, and no ordinary downstream event identifies it as the agent’s rather than the person’s.

TL;DR

  • Agents, not just people, choose tools. When a developer names a job but not a vendor, a coding agent can select which tool to install, subject to the developer’s review of the diff.
  • Analytics see the install, not the decision. Your analytics can record that an install happened, but not that an agent chose it, or which alternatives it installed instead on other runs.
  • More event types won’t close the gap. Product analytics can ingest server-side and machine-generated events, not just human clicks, but none of those signals identify the selection as the agent’s decision.
  • It’s a structural blind spot, not a tracking gap. This is a structural blind spot in attribution, not a tracking gap you can patch by adding one more event.
  • Seeing it takes a different instrument. Observing which tool agents select across repeated controlled runs, and reporting the uncertainty, because estimating a selection rate across stochastic runs requires repetition, not a single draw.

On this page

The selection can happen inside the agent

Picture your growth dashboard on a good week: signups up, activation healthy, the funnel green. Now picture a build that never shows up as a decision on it. A developer tells a coding agent to add transactional email, and the agent proposes a vendor, runs the install, and writes the integration. The developer reads the diff, keeps it, and moves on. A vendor was just chosen, and the agent did most of the choosing.

That is the shift. AI-assisted development is now common, and some of it routes the tool choice through the agent. The selection is not always the agent’s alone. An agent may ask the developer which library to use, offer a short list, reuse a dependency already in the repo, follow a constraint in the project’s config, fail an install and fall back, or have its choice thrown out at diff review. The distinction that matters is who chose the vendor. When a developer names the job but not the vendor, the agent’s pick is the selection, even if the developer skims and approves the diff afterward. How often an agent’s default actually decides the vendor across the real market is the open question this article is built around. To our knowledge, no published study has estimated category-level vendor selection share across repeated, end-to-end runs of production coding agents. Adjacent work points at the mechanism without closing that gap: LLMs have been shown to favor well-established libraries when they generate code (Twist et al., 2026), and a pre-registered 2026 protocol sets out to measure build-versus-buy and library selection inside Claude Code and Codex (Lulla et al., 2026).

Your analytics can see the install, not the selection

Product analytics are not limited to human clicks. Modern tools ingest server-side events, machine-generated and anonymous events, API and CLI calls, package downloads, and webhook payloads. So an install is often visible to you: a package fetch shows in registry stats, an API key gets created, first calls hit your API, SDK telemetry may report a new integration.

What none of those signals carry is the selection behind the install: that an agent chose you for a job no one named, and which vendors it installed on the runs where you were not. The selection is a decision taken in the agent’s context, shaped by its training-data priors and defaults: what it reaches for when no vendor is named. How strongly those priors weigh against docs reachability, install ergonomics, or an existing repo dependency is not something public research has decomposed, so treat the mechanism as real and its weights as unknown. The install may reach your servers. What your ordinary downstream telemetry cannot infer is which vendor the agent chose over its alternatives, or the builds you lost outside your systems entirely.

This is an attribution blind spot, not a gap you can patch

Here is the distinction I want you to hold, because it changes what you do about it. A tracking gap is an event you forgot to fire and can add tomorrow. An attribution blind spot is structural: you can log that something happened and still have no instrument that tells you who decided it or what the alternatives were.

Agent selection is the second kind. You can watch installs accumulate, from human sessions and server-side events alike, and still not know whether the agent surfaced you and chose you, surfaced you and passed, or never surfaced you at all. Those three cases can produce the same download count. Adding another event does not separate them, because what you are missing is not an event on your side. It is the agent’s decision, made in its own context, that your instruments have no view into. A companion piece lays out the four positions this creates, from being the category default to the mirage of appearing in answers while remaining absent from actual builds: mentions are not selection. The narrower point here is that your downstream telemetry records the installs that reached you, not the builds in which a competitor was selected instead.

Seeing the selection takes a different instrument

If your analytics cannot recover the selection, what can? Observing the agent’s behavior directly: running coding agents on tasks in a defined category, under controlled conditions, and reading which tool each run selected and installed from the execution trace rather than from a downstream signal or a survey answer. What the trace discloses is the selection the agent made and which tool won across runs, not a latent list of everything it weighed or a read-off of the agent’s reasons.

Agent-Native Market Share is NativeFold’s controlled-run selection share: across repeated controlled runs on a category’s tasks, how often coding agents select each vendor, reported with the uncertainty around that estimate. It depends on the agents, models, task distribution, and category definition chosen, so it is a property of a defined evaluation protocol, not a market-weighted estimate of real-world adoption.

Measuring behavior directly carries a cost that mention-counting avoids: agent runs are stochastic, so one run is a single draw and a second can differ. Repeated task-completion evaluations of the same agent vary run to run (Bjarnason et al., 2026, Table 1). That is task-completion variance, not selection variance, and its magnitude does not transfer here. The principle does: a single trace can establish what happened in one run, but estimating a selection rate, or comparing vendors reliably, requires repeated runs and reported uncertainty. The full statistical treatment of why one run tells you so little is in why a single agent run can mislead.

It is tempting to reach for something cheaper: a static agent-readiness score, a self-assessed checklist of whether your docs and SDK look agent-friendly. That is a proxy, not a measurement: it predicts what an agent should do, it does not observe what an agent did. A readiness score can look encouraging while your measured selection share is low, and the gap between the two is the whole problem.

The strongest version of “our analytics are already fine”

Let me give the objection its best form, because most of it is right. Conventional analytics are not broken, and they are not confined to human clicks: a good growth team runs them well across human sessions and server-side signals alike, from signups and retention to the machine events that mark an install. The best version presses further, and it deserves a real answer.

A sharp growth engineer will say: I can read package-manager telemetry and download counts, I can look at install-source metadata, I can watch for agent and CLI user-agents on my endpoints, I can capture the prompt some CLIs pass along, I can tag referral parameters, I can run post-install surveys. All of that is true, and worth doing. Here is why it still falls short of selection. Those signals confirm that an install happened, and some flag that an agent was involved. What ordinary downstream telemetry cannot give you is the comparison: which vendors the agent installed on the runs where you were not. User-agents and install-source metadata identify a channel, not a choice among competitors, and a post-install survey reaches the human, who often did not make the call. Even richer instrumentation, agent runtime traces or deep IDE integration, would show the selection an agent made on a run, not why one vendor beat another, which a single trace does not reveal.

So the honest version of “our analytics are already fine” is this: they tell you a great deal about your human funnel and your product usage, and they can tell you an install happened. They do not tell you your share of the selections in a category, measured against the competitors you lost to. That is the gap, and it is not one your existing instruments were built to close.

The hypothesis our measurement exists to test

So I will state the claim underneath this article as what it is, a hypothesis and not a measured finding. A controlled-run selection share is something we can measure. What it implies for the open market, how much of real-world adoption is settled by an agent’s default rather than a person’s deliberate choice, is something that, to our knowledge, no published study has estimated: category-level vendor selection share across repeated, end-to-end runs of production coding agents. That is not a weakness in the argument. It is the reason the argument matters.

This is the hypothesis our measurement exists to test: that across developer-tool categories, a share of installs worth measuring is shaped by the agent’s default, upstream of the deliberation your funnel can record. The way NativeFold frames the bet is this. Coding-agent installs can be won and lost before the agent reads your docs. We measure which vendor agents reach for by default in your category, and what makes them switch. And the further step, the one the measurement is built toward: We are building the tool that closes the loop: from measuring what agents do, to knowing what to change.

Until that loop exists, the useful move today is smaller and free. Stop reading a green human funnel as evidence that the agent channel is healthy. It is close to silent about that channel, because it was never built to observe it. Once you accept that, the next question stops being “why is our activation flat” and becomes “who does the agent install when nobody names us,” which is answerable, just not with the instrument you already have.

Our methodology page sets out how we measure agent behavior directly and what we allow ourselves to claim from it.

Frequently asked questions

Why can’t my product analytics track AI agent installs?

You often can see the install itself. Product analytics ingest server-side and machine events too, so a package download, a new API key, first API calls, or SDK telemetry can register. What they cannot recover is the selection behind it: that a coding agent chose your vendor for a job no one named, and which competitors it installed on the builds you lost. That decision is made in the agent’s context, so it is not identifiable from the install alone.

Do coding agents choose which SDK to install?

Sometimes. When a developer names a job and not a vendor, a coding agent can select the tool, run the install, and write the integration, subject to the developer approving the diff. It may instead ask which library to use, offer a short list, reuse a dependency already in the repo, or have its choice rejected on review. How often an agent’s default decides the vendor across the real market has not been publicly measured under controlled conditions, and the choice is shaped by the agent’s training-data priors and defaults, whose relative weight public research has not decomposed.

How do you measure agent tool selection if analytics cannot see it?

By observing agent behavior directly: running coding agents on a category’s tasks under controlled conditions and reading which tool each run selected and installed from the execution trace, rather than from a downstream signal or a survey answer. Because agent runs are stochastic, the measurement repeats runs and reports its uncertainty. NativeFold publishes the result as Agent-Native Market Share, a controlled-run selection share, category by category.

Does measuring agent selection replace my product analytics?

No. They answer different questions. Product analytics remain the right tool for your human funnel and product usage: signups, activation, retention, and the installs that show up in your own telemetry. They are not built to tell you which vendor an agent chose for a job no one named, or which competitor it installed instead. Selection measurement is an addition for the agent channel, not a replacement for the funnel you already run.

References

  • Bjarnason, Silva and Monperrus, “On Randomness in Agentic Evals”, ICLR 2026 Workshop on Agents in the Wild. Cited for run-to-run variance in repeated task-completion evaluations (task-completion variance, not selection variance; magnitude not transferred). arXiv:2602.07150
  • NativeFold, “GEO for developer tools: mentions are not market share”. /blog/geo-for-developer-tools
  • NativeFold, “Agent evaluation variance: why a single agent run can be misleading”. /blog/agent-evaluation-variance
  • Twist, Harman, Syme, Noppen, Yannakoudakis, Nauck and Zhang, “A Study of LLMs’ Preferences for Libraries and Programming Languages”, Findings of the Association for Computational Linguistics: ACL 2026, pp. 331-351. Cited for the finding that LLMs favor widely adopted libraries when generating code (a preference in single-shot code generation, not a category-level selection share across agent runs). arXiv:2503.17181
  • Lulla, Galster, Zhang, Baltes and Treude, “The Impact of Configuring Agentic AI Coding Tools on Build-vs-Buy Decisions: A Study Protocol”, ESEM 2026, Registered Reports track. Cited as a pre-registered protocol to measure build-versus-buy and library selection in Claude Code and OpenAI Codex (a study design; results not yet reported; not a category-level selection-share estimate). arXiv:2606.03907