Stars Are Not Adoption: Monitoring Open-Source MCP Growth and Search Quality

Stars, npm downloads, repository clones, search rank, and real queries describe different stages. Combining them into one growth score hides the question that matters most: which layer worked, and which layer was temporarily unobservable?

On August 3, 2026, the weekly Agent Search MCP report reached a revealing state. npm weekly downloads and GitHub clone data were available, while the GitHub star field was ?. Five days later, a separate daily search-quality probe completed normally and emitted no alert.

A single green-or-red dashboard would make both runs easy to misread. ? could become zero, silence could look like a job that never ran, and one more star could be described as one more user.

I eventually split the system into two pipelines. One observes how the project is discovered. The other checks whether the search product can still complete a minimum task. They share timestamps and version context, but they do not share conclusions.

Separate five numbers that look deceptively similar

Common open-source metrics belong to at least five layers:

The signals can guide an investigation, but they cannot be added into a conversion funnel. CI installs, cache misses, and upgrades can repeat npm downloads. Clones may be automated. Even when GitHub, npm, and search signals rise together, the result is a hypothesis to inspect, not proof of adoption.

The weekly report therefore keeps the source, observation window, and error state for every field. Missing data remains unknown; it does not become zero. An inaccessible directory endpoint is not automatically reported as “not listed.”

The growth collector reports facts, not a score

The collector reads three groups each week: GitHub repository and 14-day clone data, npm weekly downloads, and ecosystem directories that expose a public status. It emits structured JSON before a reporting layer turns the snapshot into prose.

That separation prevents the collector from guessing merely to make a report look complete. Its core behavior is closer to this:

snapshot = {
    "github": read_github_repo_and_clones(),
    "npm": read_npm_weekly_downloads(),
    "directories": read_public_directory_status(),
}
 
for source, result in snapshot.items():
    if result.failed:
        snapshot[source] = {"status": "unknown", "reason": result.error_class}

In the August 3 run, the GitHub CLI path timed out and left stars unknown. A later read-only check could inspect the repository page, but that supplementary fact must not rewrite the original collector as successful. Doing so would hide the next failure of the same path.

This changed how I read a weekly report. It is not a scoreboard. It is a diagnostic map showing which distribution path has evidence, which path is blind, and whether the next investigation belongs to the product or the collector.

Keep the quality probe independent from growth

Growth does not mean the product still works. Agent Search MCP depends on several upstream sources. A repository can keep attracting attention while one language path times out or every provider returns an empty result.

The daily quality probe therefore performs only three fixed tasks: a timely English query, a Chinese query, and an English technical query. Each run records:

• whether the command completed successfully;
• how many results were admitted;
• whether total latency exceeded 30 seconds;
• which fixed query degraded.

The current minimum is merely “at least one result within 30 seconds.” It is a liveness probe, not a relevance benchmark, citation-quality result, or long-term SLA. A deeper claim still needs a versioned query set, complete traces, and blinded review. I describe those boundaries separately in the agent-search evaluation framework.

The probe is silent on success and emits details on degradation. Healthy cron runs do not generate a daily stream of noise. If a query returns no result, times out, or the command fails, the alert layer receives the query, latency, and error class. Silence counts as success only when the cron exit state and run record agree; stdout alone is insufficient.

Three conclusions that are easy to get wrong

1. unknown is not zero

An API timeout, expired authentication path, or upstream challenge can make a field unreadable. Zero is a product fact. unknown is an observability fact. They need different types and displays.

2. A search challenge is not “no rank”

An automated DuckDuckGo request can encounter a human challenge. That proves only that this check was blocked. It does not prove that the page is absent from the index. Real GSC demand, fixed-engine rank checks, and AI-citation observations should also remain separate datasets.

3. Attention is not adoption

In a read-only GitHub snapshot on August 8, 2026, Agent Search MCP had 91 stars and 7 forks, and its latest stable release was v3.2.0. Those facts describe public attention and distribution. The project has no install telemetry, user accounts, or downstream deployment evidence that would justify calling them 91 users.

The same boundary applies to the npm and clone snapshots in the August 3 report. They describe requests and clone activity in their own windows. A responsible adoption claim would need identifiable downstream integrations, maintainer feedback, reproduction records, or consented product signals.

The monitoring contract I keep now

The practice converged on five rules:

1. Retain source, window, version context, and collection status for every number.
2. Keep collection failures as unknown; never substitute zero or let prose guess.
3. Store growth, runtime quality, real search demand, and active rank checks separately.
4. Use silent success only for low-noise probes that also retain exit state and run history.
5. Report what was observed; do not derive user counts from stars, downloads, or clones.

The system has clear limits. Three fixed queries detect only coarse regressions. Public APIs are affected by authentication, rate limits, and network conditions. Directory and rank checks can be blocked by challenge pages. This is a cheap, continuous early-warning layer, not a complete product-analytics platform.

If you are adding observability to a search agent or MCP server, begin with two independent jobs: one that collects distribution facts and one that runs a few bilingual liveness queries. Resist a unified score until every failure can retain its original meaning.

You can inspect the open-source sample in the Agent Search MCP repository, including its releases and evaluation material, or read when it is and is not a free-first Tavily alternative.