Log Monitoring

[Tutorial] How to upload a log file to OnCrawl

Need help getting started with log file analysis? Here's one place to start!

Written by Rebecca Berbel
Updated over a week ago

[Tutorial] How to combine crawl and log data

Use cross-data analysis on crawl and log data to find active orphan pages and other interesting information.

Written by Rebecca Berbel
Updated over a week ago

How to set up log monitoring if you use GCP

Do you use a Google Cloud Storage bucket to store your log data? Here's how to connect your logs to OnCrawl.

Written by Rebecca Berbel
Updated over a week ago

How to get the most out of live log monitoring

Here's how to use the live log monitoring function in OnCrawl to see in near real time how Google explores your site

M

Written by Magalie Rousseaux
Updated over a week ago

How often should I upload new log files to OnCrawl?

Log monitoring only works if it is constantly and continuously updated. How often should you upload new logs?

Written by Rebecca Berbel
Updated over a week ago

How to calculate the number of resources needed for log analysis

OnCrawl log analysis levels are based on the number of lines in your logs. Here's how to estimate how many lines you need.

Written by Rebecca Berbel
Updated over a week ago

Log Monitoring Setup Step-by-step

Prerequisite: An OnCrawl Log Monitoring Subscription

Written by Rebecca Berbel
Updated over a week ago

Server side log format configuration

Written by Philippe David
Updated over a week ago

How to secure a FTP connection when uploading log files to OnCrawl?

Written by Philippe David
Updated over a week ago

How many days of logs should I upload?

Find out how many days of logs to upload for different uses in OnCrawl's analyses.

Written by Tanguy
Updated over a week ago

How to filter log lines

Don't want to share everything? Want faster processing? Remove unnecessary lines from your logs before uploading to OnCrawl.

Written by Rebecca Berbel
Updated over a week ago

How to reduce the size of log files

The size of access log files can be easily reduced by 95% with pre-filtering and compression

Written by Philippe David
Updated over a week ago

How to automate the daily upload of log files on the FTP?

This is a sample script for Apache server on Linux

Written by Philippe David
Updated over a week ago

Splunk integration

If you're a Splunk user, the OnCrawl Splunk integration automates the uploading of indexed logs to your private, secure FTP space.

Written by Rebecca Berbel
Updated over a week ago

How to set up log monitoring if you use Amazon S3

Do you use an Amazon S3 bucket to store your log data? Here's how to connect your logs to OnCrawl.

Written by Rebecca Berbel
Updated over a week ago

How to check the quality and status of uploaded log files

The OnCrawl Log Manager Tool is an interface that can help you understand and monitor your logs uploading and parsing processes.

Written by Rebecca Berbel
Updated over a week ago

Log monitoring date ranges

What dates are covered in the log monitoring report? How can you change the period or the level of detail?

Written by Rebecca Berbel
Updated over a week ago

SEA & vertical bots

The SEA & Vertical bots dashboard provides information about hits by additional Google bots that is extracted from an analysis of your logs.

Written by Rebecca Berbel
Updated over a week ago