Skip to main content

Crawler logs

Feature Request

Provide a centralized dashboard to visualize and analyze search engine and AI bot crawling activity on the website. The dashboard should make it easy to identify which bots are visiting, how often they come, what pages they focus on, and how they interact with the site from a technical standpoint (status codes, crawl frequency, crawl distribution).

Posted by Rankshift ·
Status: Completed1 comment

Log in to comment and vote

Comments1

  • Rick Tousseyn

    •

    Dec 8, 2025

    This is a very nice feature, Profound & PeecAI started doing this. Would make it a lot easier to identify bot traffic via logfiles. Please upvote!