First, if you are interested in using live log monitoring, make sure you have activated the log monitoring feature for your project and followed the setup steps here.

With Log Monitoring, you can identify and analyze hits from OpenAI, Perplexity, Claude, and Mistral LLMs directly within your projects.
​

Detection needs to be activated by your dedicated (and amazing) Oncrawl CSM. Feel free to reach out to them to request activation or deactivation for specific bots based on your needs. You can also submit your request directly via the in-app chat, a member of our technical support team will be happy to assist you.
​

Currently, we support detection for the following bots :

Open AI :

Perplexity

Claude

Mistral

To identify these hits in your logs, we analyzes user-agent strings and public IP addresses used by each bot platform.
​

For this detection to work, make sure these elements (user-agent and IP address) are clearly present in your log files. Without them, Oncrawl won’t be able to surface LLM bot activity.

Below is a list of publicly documented user-agents and IP address ranges for the LLM bots we currently support.
​

OpenAI :

Perplexity :

Claude :

Mistral :

⚠️ Keep in mind that enabling detection for multiple LLMs bots can significantly impact your average daily log lines quota. Make sure to take this into account before proceeding with the setup. ⚠️