1. Home
  2. Concepts
  3. Content Sources
  4. Crawling Webpages
  5. Errors in Web Crawling

Errors in Web Crawling

Web Crawling feature can fail in two stages: 

  1. Fails to start  due to URL validation failure either due to connectivity issues or a misspelt  URL, click Retry or Edit Configuration to Edit URL 
  2. Fails after starting successful URL Validation for the given website and during web crawling.

Note: If you are attempting to crawl the same site without enabling automatic Frequent Scheduling toggle, a Duplication warning message pops up, reading “web crawling cannot be duplicated” instead try using the Crawl by schedule.