Update crawler-overview.md
This commit is contained in:
parent
daaa88f361
commit
bd3b68ebf1
1 changed files with 2 additions and 2 deletions
|
|
@ -1,11 +1,11 @@
|
|||
---
|
||||
layout: two-columns
|
||||
title: Crawler Overview
|
||||
title: Scraper Overview
|
||||
---
|
||||
|
||||
## How?
|
||||
|
||||
The DocSearch crawler is written in python and heavily based on the [Scrapy][1]
|
||||
The DocSearch scraper is written in python and heavily based on the [Scrapy][1]
|
||||
framework. It will crawl all pages of your website and extract content from the
|
||||
HTML structure to populate an Algolia index.
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue