Insights

How to identify and verify AI agent requests in your logs

By Sam HallPublished

To identify an AI agent request, start with the record your systems received. Inspect the declared user-agent, check the applicable provider verification evidence, and record what the request did on your store. Keep those three questions separate.

A familiar name in a user-agent is a useful lead. It is not enough to establish identity. Google explicitly warns that user-agent strings can be spoofed, and publishes verification guidance for its crawlers and fetchers. Google’s user-triggered fetcher documentation

The workflow below is for reviewing your own authorized logs. It does not require changing crawler permissions or weakening your checkout controls.

1. Establish what your logs cover

Choose a short time window and name the surfaces in scope: for example, the storefront host, a product-feed host and the cart API. Ask your hosting or security team which of those requests appear at the CDN, at the origin and in application events.

Check sampling, retention, missing time periods, cache hits and requests stopped at the edge. An origin log alone cannot tell you about a request that never reached that origin. Cloudflare’s HTTP request dataset, for example, distinguishes edge and origin responses and includes request-source and security-action fields. Cloudflare HTTP request fields

A sampled dashboard is useful for finding a candidate request. It is a weaker basis for saying that no request occurred. Cloudflare documents sampling in Bot Analytics; confirm the coverage of the view you actually use. Cloudflare Bot Analytics

2. Collect a small, safe evidence record

For each candidate, ask for:

  • Timestamp and time zone
  • Host, path and HTTP method
  • Request identifier and the system that produced it
  • Declared user-agent and any recorded referral source
  • Network or authentication verification result, with its source and check date
  • Edge response, application response and any recorded challenge or error
  • Coverage notes, such as a sampled export or an unavailable application log

Keep source addresses and other restricted fields inside your authorized logging environment. Use an allowlist of fields for exports. Remove cookies, authorization values, personal details and sensitive URL parameters. A useful review should not require copying raw request bodies or payment information into a spreadsheet.

3. Check the provider’s documented role

The same provider can operate several services with different purposes.

  • OpenAI documents OAI-SearchBot for search, GPTBot for model development and ChatGPT-User for certain user-triggered actions. A ChatGPT-User fetch is not an automatic crawl. OpenAI crawler documentation
  • Anthropic distinguishes ClaudeBot, Claude-SearchBot and Claude-User. Its documentation links a source-IP list for checking whether a crawler comes from Anthropic. Anthropic crawler guidance
  • Perplexity distinguishes its search crawler from Perplexity-User and publishes separate IP lists. Perplexity crawler documentation

Use the provider directory to find the relevant source. Provider documentation describes what a service is designed to do; the merchant record establishes which request you received. Neither tells you what a private shopping conversation contained.

4. Verify the request with the applicable evidence

When a provider publishes IP ranges, check the trustworthy client address against the list for that specific service. Do not substitute a broad cloud-hosting network or the address of your own reverse proxy. Ask the team responsible for your edge configuration to establish which address field is reliable. Do not trust a client-supplied forwarding header merely because it contains an IP address.

For Google’s documented DNS procedure, reverse-resolve the trustworthy source address, check the allowed hostname pattern for that crawler or fetcher, then forward-resolve the hostname and confirm it returns the original address. A Google-looking substring is insufficient. Google request-verification procedure

Follow the complete signature procedure where that is the applicable method. The presence of a signature header alone does not establish a successful check.

Record the method, result and reference used. For an old request, a current IP list may not establish which addresses belonged to a service at the time. If contemporaneous evidence is unavailable, preserve that limitation.

A failed or incomplete check also needs context. It may reflect an incorrect claim, a missing field, stale reference data or a collection problem. Do not silently promote it to verified.

5. Write the finding at the right level

Use the distinction in Cartograph’s evidence definitions: verified, claimed or unknown.

For example, if a request declares ChatGPT-User but the source address was not retained, the defensible finding is that the request claimed that identity and verification was unavailable. A 200 response on a product URL establishes the recorded HTTP result. It does not establish that the product was recommended, understood or purchased.

Verification answers a question about request identity. It does not establish purchase authorization, a shopper’s intentions or permission to bypass security controls. Unknown traffic should remain unknown; it should not be counted as human by default.

Questions to give your website team

Can we produce one request with its original source record? Can we explain how its identity was checked? Do we know which surfaces and time periods are missing? Can we show the response without exposing customer or authentication data?

Start there before turning a list of user-agent names into a traffic or revenue claim. For the broader merchant context, read Would you recognize an AI shopper?

Where Cartograph fits

Cartograph is being built to organize merchant-side request evidence and observed commerce milestones. We are preparing for pilots. If this is a problem your team is working on, contact Cartograph to register your interest. An inquiry registers interest only. Product access, setup and pilot terms would be agreed separately.

Workflow sources checked October 16, 2026. Provider behavior and verification guidance can change; check the linked documentation before operational use.

Sources

Related reading