The assistant traffic missing from our analytics

The browser dashboard couldn’t explain where some signups came from, so I checked the server logs. Assistants were requesting product information and documentation without generating corresponding browser events. In a later sampled week, the logs showed over 3,000 verified ChatGPT-User requests and hundreds from another assistant.

The requests gave me a way to see which pages assistants were reading. I still couldn’t connect a fetch to a person who signed up. That distinction mattered when I looked at pages that appeared to get little attention in the browser dashboard.

How assistant fetches go uncounted

Browser analytics commonly depend on JavaScript executing in a visitor's browser. A server-side assistant fetch can retrieve HTML without executing that script. Some analytics systems also filter data-centre traffic. The load balancer can therefore record a successful response while the browser analytics record no event.

An assistant fetch and a later human visit can also lack a shared identifier. Someone might ask an assistant about a product and then search for the company or type its address. That visit may carry no assistant referrer. The assistant could also fetch the page without recommending the product. The request log alone cannot establish which happened.

The counts include repeat fetches. A request can tell me that an assistant retrieved a page, while the number of people asking questions remains unknown.

These were later observations, so they do not reconstruct the traffic when the signup pattern first became noticeable. They came from the logging already available at work. I tested an experimental collector separately on sites I control; it was not collecting this workplace traffic.

Which content received requests

The early investigation drew attention to retention terms, security information and sub-processor lists. These pages answer questions that can determine whether a prospective customer can use a service. Low time on page or scroll depth may say little about whether the page supplied the fact the reader needed.

A query of frequently requested pages also showed assistants reading model release information, integration articles and pricing. These examples did not describe the complete distribution of traffic.

The paths identify content worth reviewing. They suggest questions about current model information, integration details and commercial terms, although the requests do not disclose the questions themselves. To establish whether an assistant answered correctly, we would need to examine its answer.

When reviewing a page with low recorded human engagement, I would check what product questions it answers before removing it. I would also check that the answer is present in the HTML delivered to the requester. In a separate workflow I examined, Markdown versions helped Claude Code, while ChatGPT did not request them. I would make the ordinary HTML complete first, then measure the benefit of alternative formats for the clients that use them.

Separating different kinds of automated traffic

The logs contained indexing and training crawlers as well as live fetchers. These categories need separate reporting because their requests have different purposes. Combining them obscures the activity the investigation was trying to understand.

Even the live-fetch category has limits. A user-agent string is supplied by the sender. Vendor verification can support a connection to vendor infrastructure, while other agents use ordinary browser identities and remain outside this coverage. The count needs to be presented with both its verification state and the scope of collection.

In a separate collector experiment, I checked request coverage against an independent log baseline. That matters when comparing periods. If the collector can see more routes in one period than another, a higher count can reflect the change in coverage.

Read the page before cutting it

If a page about retention or security looks quiet in browser analytics, I’d check the server logs and read what the assistant actually receives. A single fetch may be enough to retrieve a fact that matters to a buyer. The logs can tell me whether identifiable agents requested that page and whether the server returned it. To assess a recommendation or credit a signup, I’d need evidence linking the fetch to the answer or the person’s visit.