1
0
Fork 0

Update crawler-overview.md

This commit is contained in:
mblandineau 2019-08-01 14:40:33 +02:00 committed by GitHub
parent daaa88f361
commit bd3b68ebf1
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23

View file

@ -1,11 +1,11 @@
---
layout: two-columns
title: Crawler Overview
title: Scraper Overview
---
## How?
The DocSearch crawler is written in python and heavily based on the [Scrapy][1]
The DocSearch scraper is written in python and heavily based on the [Scrapy][1]
framework. It will crawl all pages of your website and extract content from the
HTML structure to populate an Algolia index.