Pages and crawl activity

Two views that answer different questions: Pages is the current state of your site, Crawl activity is what Monitor has been doing.

Pages

Every page Monitor knows about, with its status, when it was last checked, and what is currently wrong with it.

Sort and filter to ask the useful questions:

• Which pages are not returning 200?
• Which are excluded from indexing?
• Which have open issues, and at what severity?
• Which have changed recently?

A single page

Open one and you get everything recorded for it:

• Status and headers — what the server actually returned.
• Metadata — title, description, canonical, robots directives.
• Structured data — the schema on the page, and whether it is valid.
• Links — what it links to, what links to it, and what is broken.
• Rendered output — what a crawler sees after JavaScript runs, which is not always what the source says.
• History — every previous check, so you can see when something changed.

That last one is the one people underuse. When a page’s traffic drops, its history usually says why and when.

Crawl activity

What Monitor did, and when: which crawls ran, how many pages each covered, how long they took, and anything that stopped one — timeouts, rate limiting, a server refusing requests.

Worth checking when results look wrong. A crawl that covered 40 of 400 pages produces a report that looks reassuring and means nothing, and this view is where that is visible.

When pages go missing

A page disappearing from the list is a finding, not a gap. It usually means nothing links to it any more and it is not in the sitemap — which is exactly the situation where a page quietly stops being found.

Related

• Issues and severity
• Changes and diffs
• Platform and spec checks