Tracker docs

Track AI crawlers with nginx

nginx sees requests even when WordPress or another application serves a full-page cache. Its mirror module can report matching AI crawler requests asynchronously using a public workspace identifier. Secret API keys stay out of the nginx configuration.

1. Copy your generated block

Open AI Traffic while signed in to get this block with your workspace ID filled in. The generated mirror forwards the path and crawler user agent; query strings are not sent.

# Foglift AI crawler map v2026-07-18
# Re-copy this block when Foglift adds crawler patterns.
map $http_user_agent $foglift_ai_crawler {
    default 0;
    ~*(GPTBot|OAI-SearchBot|ChatGPT-User|ClaudeBot|Claude-SearchBot|Claude-User|Claude-Web|anthropic-ai|PerplexityBot|Perplexity-User|Googlebot|Google-CloudVertexBot|Google-NotebookLM|Google-Read-Aloud|Google-Agent|Amazonbot|Amzn-SearchBot|Amzn-User|Applebot|bingbot|DuckAssistBot|MistralAI-User|MistralAI-Index|meta-externalagent|meta-externalfetcher|cohere-ai|Bytespider|CCBot) 1;
}

server {
    # Add this beside your existing location blocks.
    location / {
        mirror /_foglift_ai_crawler;
        mirror_request_body off;
        # Keep your existing proxy_pass / try_files directives here.
    }

    location = /_foglift_ai_crawler {
        internal;
        if ($foglift_ai_crawler = 0) { return 204; }
        proxy_pass https://foglift.io/api/v1/crawler-analytics;
        proxy_method POST;
        proxy_pass_request_body off;
        proxy_set_header Content-Length "";
        proxy_set_header X-Foglift-Mirror 1;
        proxy_set_header X-Foglift-Site-Token YOUR_WORKSPACE_ID;
        proxy_set_header X-Foglift-Original-Host $host;
        proxy_set_header X-Foglift-Original-URI $uri;
        proxy_set_header X-Foglift-Original-User-Agent $http_user_agent;
    }
}

2. Validate, reload, and verify

Merge the locations into the existing server block without removing current proxy or cache directives.
Run nginx -t. Reload nginx only after validation passes.
Keep AI Traffic open; Foglift checks every five seconds and confirms the first server-observed event.

The bot pattern list is frozen into your config at copy time. Re-copy this block when Foglift adds crawler patterns; the dated version comment shows which list is installed.

Open AI Traffic