No matter how often you upload log files to OnCrawl, you should do it regularly — that is, at even intervals throughout the day.

If you're interested in getting the full benefits of OnCrawl's Live Log monitoring feature, you should upload new log data at least every hour, but you can also upload data more often if you want a faster, more granular view.

If you aren't interested in live log monitoring, you should upload new log data about once per day.

OnCrawl provides Live Log Monitoring features, which will give you a minute-by-minute look at bot activity on your site over the last few minutes and up to the last several days. This feature requires up-to-the-minute data to be useful.

You can use the Log Manager Tools to track and report on how often you upload log files to OnCrawl.

From the project home page, click on "Log Manager Tools" :

Error — No data to analyze

We can't find any log files in the past 24 hours.

Make sure your log file upload process is working correctly.

Red — Not optimized

You provided only one file in the last 24 hours. Live Log monitoring can't provide pertinent information fast enough.

To fix this issue, you should automate your uploading method if you haven't already, and increase the frequency at which the script or process runs.

Orange — Optimizable

You provided at least four files for different hours over the last 24 hours. Live Log monitoring might give you a few insights, but it won't be fast enough to be helpful — or "live"!

To optimize this point, increase the frequency at which your script or process for uploading log files runs. You should aim to upload new log data at least once per hour.

Green — All good

You provided files at nearly 1 file for each hour (or more frequently!) over the past 24 hours. Your Live Log monitoring shows you near-instantaneous data about how bots and visitors interact with your website.