Google Cloud CDN crawler logs

Route Google Cloud load balancer or Cloud CDN logs through Pub/Sub or a small shipper before posting them to WatchLLM.

Endpoint

POST /api/events/gcp

curl -X POST "https://your-watchllm-host.com/api/events/gcp" \
  -H "x-api-key: $WATCHLLM_API_KEY" \
  -H "content-type: application/json" \
  --data '{"message":{"data":"eyJodHRwUmVxdWVzdCI6eyJyZXF1ZXN0TWV0aG9kIjoiR0VUIiwicmVxdWVzdFVybCI6Imh0dHBzOi8vZXhhbXBsZS5jb20vIiwic3RhdHVzIjoyMDAsInVzZXJBZ2VudCI6Ik1vemlsbGEvNS4wIENsYXVkZUJvdCJ9fQ=="}}'

Optional bearer guard

Set GCP_CRAWLER_INGEST_BEARER_TOKEN when using Pub/Sub push in production. When configured, /api/events/gcp requires both the WatchLLM project API key and Authorization: Bearer <gcp-token>.

Use x-api-key for the WatchLLM project key whenever the GCP bearer guard is enabled, because the Authorization header is reserved for the GCP push token.

Verify

Open /workspace/settings/crawler-logs?provider=gcp, then confirm country, hostname, path, and crawler identity in /workspace/crawlers.