Crawler logs

Provide a centralized dashboard to visualize and analyze search engine and AI bot crawling activity on the website. The dashboard should make it easy to identify which bots are visiting, how often they come, what pages they focus on, and how they interact with the site from a technical standpoint (status codes, crawl frequency, crawl distribution).