How to add a site to Monitor

Add a site to Monitor so it starts finding your pages and checking them. You only enter the site’s address: Monitor finds the pages itself and decides how often to check each one.

Before you start: Monitor must be enabled for your workspace. It is set up by arrangement, so if you do not see the Monitor icon in the top bar, contact us. You also need:

• To be a member of the workspace you are adding the site to.

• A site anyone can reach without signing in. Monitor does not sign in or solve bot challenges.

• If your firewall or CDN challenges unknown bots, an allow rule for the WorkoverMonitor user agent, so Monitor gets the real page.

Add the site

1. In the top bar, select the Monitor icon.
2. Select Monitor a site.
3. In Site address, enter the site’s address, for example https://example.com. If a site connected in this workspace appears under Connected in this workspace, you can select it instead of typing.
4. Select Start monitoring.

Monitor then opens the new site. It saves the site by its scheme and host only, so example.com, https://example.com/ and https://example.com/pricing all add the same site. If the site is already in this workspace, Monitor opens it instead of adding a second copy.

What happens next

1. Monitor finds your pages. It reads the site’s robots.txt and sitemaps, then follows internal links from the pages it finds. The Overview tab shows Pages tracked against the site’s page limit, for example “of 5,000”.
2. It checks each page once. While this first pass runs, Checked shows “crawl in progress”. To start checks straight away rather than waiting for their scheduled turn, select Check now on Overview.
3. It stays quiet about pages until the first pass finishes. Page problems found before every page has been checked once are treated as the site’s existing condition and are not emailed. You can still read them on the Issues tab. Whole-site problems, such as an expired certificate or robots.txt blocking search engines, are the exception: if one is still open when the first pass finishes, it is emailed then.
4. Then it keeps watching. Each page is re-checked on its own interval, from every 15 minutes for your most important pages to every 24 hours for the rest.

By default, alerts go to everyone in the workspace. To change who is told, see Alerts and incidents.

Troubleshooting

“Enter a site address, for example https://example.com“ Monitor could not read what you typed as a web address. Enter the site’s domain, such as example.com or https://example.com.

“That address is on a private or internal network, so it cannot be monitored.” Monitor only watches public sites. Enter the address people use to reach the site from the internet.

No pages are found. Monitor finds pages from sitemaps first. Make sure your robots.txt has a Sitemap: line, or that /sitemap.xml exists, then select Find pages on Overview to read the sitemaps again.

Check now says monitoring is paused. The site is paused. Turn off Pause crawling under Settings > Crawling & extraction.

Check now says the site asked us to slow down. The site answered with a rate-limit response. Monitor pauses checks for a short while and resumes on its own.

Related

• What Monitor does
• Monitor site settings
• Alerts and incidents
• Issues and severity