nikon/docsearch
1
0
Fork 0
Code Activity Actions

update documentation as algolia/docsearch-scraper#431

Browse source
This commit is contained in:
s-pace 2018-12-19 08:31:54 +01:00 committed by Tim Carry
parent e5552fac38
commit 938b533636
2 changed files with 11 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

12
docs/src/config-file.md
View file

@ -606,11 +606,19 @@ DocSearch to index all your content.
You can override the user agent used to crawl your website. By default, this
value is:
`'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_1) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/70.0.3538.110 Safari/537.36'`
`'Algolia DocSearch Crawler'`
However, if the crawl of your website requires a browser emulation (i.e.
`js_render=true`), our `user_agent` is:
`Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/71.0.3578.98 Safari/537.36`
To override it, from the configuration:
```json
{
"user_agent": "DocSearch Bot"
"user_agent": "Custom Bot"
}
```

2
docs/src/run-your-own.md
View file

@ -85,7 +85,7 @@ Now that you have your environment variables set, you can run the crawler
according to your config.
Running `pipenv shell` will enable your virtual environment. From there, you can
run one crawl with the following command:
run one crawl with the following command:
```sh
$ ./docsearch run /path/to/your/config.json