Foglift Tracker: Measure AI Referrals and Crawler Access

The missing first-party layer

AI Visibility monitoring answers a question about the answer layer: when a user asks an engine about your category, does the engine name or cite your brand? That signal is useful, but the engine does not send a prompt-level conversion report to your analytics stack. Your site sees only the requests and referrals that reach it.

Foglift Tracker adds that first-party layer. Server and edge adapters classify matching crawler requests and recognized referrals. A hosted browser pixel gives static and no-code sites a simpler referral-only path. Both surfaces aggregate the matched engine, landing path, and time period without cookies or fingerprinting.

Two signals, one install

A server or edge adapter can inspect both the User-Agent and Referer headers. A documented crawler match becomes a crawler event. A request from a recognized AI assistant host becomes a referral event. The browser pixel sees only the page view and document referrer, so it can report recognized referrals but cannot detect crawler requests.

The signals answer different questions. A crawler event shows that a request carrying a documented agent identity reached the installed adapter for a specific path. It does not establish successful access, indexing, training, citation, or recommendation. A recognized referral shows that a visitor arrived from a supported AI assistant host. It identifies the landing page, but it does not expose the prompt or the exact answer the visitor saw.

Landing-page context without inventing prompts

Each recognized referral includes the normalized landing path. That gives you useful context about what the visitor chose to open, while keeping the limits of the evidence visible.

Consider a concrete example. Imagine your B2B SaaS product is a project management tool. Over a week your tracker logs the following referral visits:

• A visitor from ChatGPT lands on /pricing. A pricing-shaped prompt is one hypothesis to compare against your monitored prompt set and the timing of recent answer checks.
• A visitor from Perplexity lands on /integrations/asana. The path suggests integration intent, but the original prompt and answer remain unknown.
• A visitor from Claude lands on the homepage. The event confirms the referring host and landing path. It cannot distinguish a brand link, a cited source, or another link inside the assistant experience.
• A visitor from Gemini lands on /blog/agile-sprint-planning. The content topic gives you a candidate intent cluster to test in your prompt monitoring panel.

Landing paths support hypotheses about intent clusters. They do not reconstruct prompts. Compare repeated path patterns with a fixed prompt-monitoring panel, on-site conversions, and the timing of content changes. That combined view can show whether pricing, feature, comparison, or educational pages attract recognized AI referrals.

Treat the result as directional attribution. A referral event is strong evidence about origin and destination. It is weak evidence about the unseen prompt and answer. Keeping those grades separate prevents a useful first-party signal from becoming an unsupported citation claim.

Install

Choose the path that matches what your stack can observe. The package ships adapters for Next.js, Express, plain Node.js HTTP servers, and Cloudflare Workers. Version 1.4.0 uses a publishable workspace identifier for new installs on every plan. These server and edge paths can classify both crawler requests and recognized referrals.

npm install @foglift/tracker

The Next.js install is a four-line middleware:

// middleware.ts
import { trackAITraffic } from "@foglift/tracker/nextjs";

export const middleware = trackAITraffic({
  siteToken: "YOUR_WORKSPACE_ID",
});

export const config = {
  matcher: ["/((?!_next/static|_next/image|favicon.ico).*)"],
};

Express:

import express from "express";
import { trackAITraffic } from "@foglift/tracker/express";

const app = express();
app.use(trackAITraffic({ siteToken: "YOUR_WORKSPACE_ID" }));

Plain Node.js HTTP servers and Cloudflare Workers each have their own export with the same shape. Use the Next.js installation guide for middleware composition and request-lifetime reporting. The Cloudflare Workers guide covers route assignment, ctx.waitUntil, and edge verification. The full adapter reference remains on the Foglift Tracker product page.

Copy the workspace ID from the AI Traffic setup panel. The ID is publishable and can appear in site configuration. Each server adapter derives the request hostname, and Foglift accepts the event only when that hostname belongs to the workspace. Existing API-key installs remain supported for backward compatibility.

Static and no-code sites can use the hosted browser pixel with the public workspace identifier shown in the AI Traffic setup panel. This path records recognized referrals only. It cannot observe crawler requests because most crawlers do not execute page JavaScript. WordPress operators can follow the WordPress referral tracking guide for plugin and theme placement options.

<script async
  src="https://foglift.io/pixel.js"
  data-foglift-brand="WORKSPACE_UUID"></script>

Sites behind nginx, including cache-fronted WordPress installs, can mirror matching crawler requests before the application cache answers. The nginx Tracker guide provides a generated, public-token configuration and requires an nginx -t validation before reload.

What changes when you have this data

Three things, in increasing order of leverage.

First, the crawler feed shows which documented agents requested which paths. A new page may receive OAI-SearchBot, GPTBot, or Claude-SearchBot requests at different times. That history helps you diagnose crawler access and coverage. A request does not prove successful indexing or a later citation, and an absent request is inconclusive until you inspect the observation window, robots policy, CDN, and WAF logs.

Second, the referral feed shows which pages receive recognized AI-assistant traffic. Three Perplexity referrals to one page establish three visits from that supported host during the measured period. They do not reveal whether the link appeared as a formal citation, a product link, or another assistant surface. Zero referrals means only that the tracker observed no matching click in that window.

Third, landing-page patterns give prompt monitoring a better place to look. If three of ten ChatGPT referrals land on /pricing, add or inspect pricing-shaped prompts in your fixed monitoring panel. If integration pages dominate, test the relevant comparison and migration prompts. The prompt panel supplies the answer-layer evidence that referral logs cannot provide.

Why this ships now

AI Visibility and website analytics measure different parts of the journey. Prompt monitoring measures answer presence, citations, and competitors. Request and referral analytics measure what reaches your site. Joining those layers with conversions gives operators a more complete view without pretending that any single event proves the whole path.

Foglift Tracker classifies the request data already available at the browser, server, or edge layer. Foglift AI Visibility monitoring covers the answer layer. Use both with your conversion analytics to separate crawler access, answer visibility, recognized referral traffic, and business outcomes.

Sources & Further Reading

• Foglift Tracker product page: current adapters, install snippets, pricing boundary, and supported signals.
• Foglift Next.js Tracker guide: middleware composition, matcher scope, and request-lifetime reporting.
• Foglift Cloudflare Workers Tracker guide: edge-route assignment, public-token setup, and server-signal verification.
• Foglift WordPress Tracker guide: browser-pixel placement and the referral-only measurement boundary.
• Foglift nginx Tracker guide: public-token request mirroring for nginx and cache-fronted sites.
• Foglift AI crawler field guide: provider roles, request verification, and access-versus-citation evidence.
• OpenAI crawler documentation: OAI-SearchBot, GPTBot, ChatGPT-User, robots controls, and IP ranges.
• Anthropic crawler documentation: Claude-SearchBot, ClaudeBot, Claude-User, and robots behavior.
• Perplexity crawler documentation: PerplexityBot, Perplexity-User, robots behavior, and published IP files.

Get started

Open the AI Traffic setup panel and choose a server adapter when you need crawler plus referral measurement, or the hosted pixel when you need referral-only measurement on a static site. The Portal page at foglift.io/app/data/crawlers renders both feeds side by side, with referrals on the primary tab and crawlers on the secondary. The product page at foglift.io/tracker covers the full adapter list, the FAQ, and the install snippets in one place.

New server, edge, nginx, and browser installs use the publishable workspace identifier shown inside the Portal on every plan. Server, edge, and nginx paths can record crawler requests. The browser pixel reports recognized referrals only. Start with the install path your stack supports, then compare the resulting traffic with a fixed AI Visibility prompt panel and your conversion data.

Fundamentals: Learn about GEO (Generative Engine Optimization) and AEO (Answer Engine Optimization) (the two frameworks for optimizing your content for AI search engines).