Manual crawler logs

Use manual setup when your CDN or host can POST normalized JSON directly to WatchLLM.

Endpoint

POST /api/events

curl -X POST "https://your-watchllm-host.com/api/events" \
  -H "Authorization: Bearer $WATCHLLM_API_KEY" \
  -H "content-type: application/json" \
  --data '[{"timestamp":"2026-05-01T12:00:00Z","status_code":200,"request_method":"GET","request_path":"/","query_string":null,"content_type":"text/html","client_ip":"198.51.100.10","hostname":"example.com","user_agent":"Mozilla/5.0 GPTBot/1.3","referrer":"https://chatgpt.com/","country":"US"}]'

Required fields

Send timestamp, request_path, hostname, and user_agent. Status, method, content type, query string, referrer, client IP, and country improve filtering and diagnostics.

Response

The endpoint returns { "accepted": n, "skipped": n }. Skipped rows are usually non-HTML assets or traffic that cannot be classified as an AI crawler request.

Verify

Open /workspace/settings/crawler-logs?provider=generic to view the Manual setup card, then confirm accepted rows in /workspace/crawlers.