All Collections
Log Monitoring
Log monitoring
How to get the most out of live log monitoring
How to get the most out of live log monitoring

Here's how to use the live log monitoring function in OnCrawl to see in near real time how Google explores your site

Updated over a week ago

If you are interested in using live log monitoring, make sure that you have activated the log monitoring feature for your project and followed the setup steps here.

What is live log monitoring?

Live log monitoring is near-instantaneous information about your log files, displayed as soon as it is obtained, for the period over the course of the last few days.

This is different to other log monitoring features in OnCrawl. Outside of live monitoring, we aggregate log monitoring data per day so that you can easily cross-analyze it with other data about the URLs on your website. This means that other log monitoring metrics are not available until a day after they have occurred.

Consequently, this means that information in live log monitoring is available in crawl analyses, and won't be until the end of the day.

How to enable live log monitoring

Live log monitoring is available to all OnCrawl users with the log monitoring feature.

If log files pertaining to the last few days are received during that period, the information from these files will automatically be visible in the live log monitoring charts.

Depending on how you provide your log files to OnCrawl, you may need to increase the frequency at which you provide them to OnCrawl in order to see regular updates in the live log monitoring interface.

For example:

  • If you provide files twice per day, for example, the live log monitoring will update twice per day.

  • If you need updates at 30-minute intervals, you should provide log files at that rate.

If you use Amazon S3 to store your log files, you can set this up automatically.

Available data

OnCrawl provides the most recent information, in number of events, for:

- Hits from

  • Google Desktop

  • Google mobile

  • AdsBot Google

  • AdsBot Google mobile

  • Google Adsense

  • Google Adsense mobile

  • Google images

  • Google news

  • Google video

  • Adsbot Google mobile

- Hits with a Google SERP referrer (SEO visits)

- Status codes encountered by bots

- Break down of bot hits and SEO visits by segmentation page groups

You can also find the time of the latest recorded event in the most recent file, and hourly totals for any sub-period you are interested in.

Did this answer your question?