diff --git a/docs/src/behavior.md b/docs/src/behavior.md index e5ac9ae5..9cd60bf9 100644 --- a/docs/src/behavior.md +++ b/docs/src/behavior.md @@ -4,15 +4,15 @@ title: Dropdown Behavior --- Our JS library `docsearch.js` is a wrapper of the [autocomplete.js][1] library. -This library will listen to keystrokes in the search input, query Algolia, and -display the results in a dropdown. Everything is already configured for you to -work with DocSearch, but it also exposes configuration options you can use to go -even further. +This library will listen to every keystrokes in the search input, query Algolia, +and display the results in a dropdown. Everything is already configured for you +to work with DocSearch. Our UI library also exposes configuration options you +can use to go even further. You will discover Algolia. ## `appId` If you're [running the DocSearch crawler on your own][2], you'll need to define -your application ID using the `appId` key. If you're using the free hosted +your own application ID using the `appId` key. If you're using the free hosted version, you don't need to add anything. ```javascript @@ -24,9 +24,10 @@ docsearch({ ## `handleSelected` -This method is called when a suggestion is selected (either with a click or -keyboard). By default, DocSearch will redirect the browser to the results page -at the related anchor, but you can override it to add your own behavior. +This method is called when a suggestion is selected (either from a click or a +keystroke). By default, DocSearch will display links redirecting to the results +page, at the related anchor. You can override it to add your own behavior. +Please note that you can already open a new tab thanks to the `CMD/CTRL + Click` The method is called with the following arguments: @@ -38,11 +39,11 @@ The method is called with the following arguments: - `suggestion`, the object representing the current selection. Contains a `.url` key representing the destination. -- `datasetNumber`, this should always equal `1` as DocSearch is searching - into one dataset at a time. You can ignore this attribute. +- `datasetNumber`, this should always equal `1` as DocSearch is searching into + one dataset at a time. You can ignore this attribute. -- `context`, additional information about the selection. Contains - a `.selectionMethod` key that can be either `click`, `enterKey`, `tabKey` or +- `context`, additional information about the selection. Contains a + `.selectionMethod` key that can be either `click`, `enterKey`, `tabKey` or `blur`, depending how the suggestion was selected. ```javascript @@ -59,7 +60,7 @@ docsearch({ ## `queryHook` This method will be called on every keystroke to transform the typed keywords -before sending them to Algolia. By default, it does not do anything, but we +before querying Algolia. By default, it does not do anything, but we provide this hook for you to add your own logic if needed. ```javascript diff --git a/docs/src/config-file.md b/docs/src/config-file.md index e6675005..44ef8377 100644 --- a/docs/src/config-file.md +++ b/docs/src/config-file.md @@ -3,9 +3,9 @@ layout: two-columns title: Config Files --- -For each DocSearch request we receive, we'll create a custom JSON config file -that will define how the crawler should behave. You can find all the configs in -[this repository][1]. +For each DocSearch request we receive, we create a custom JSON config file that +will define how the crawler should behave. You can find all the configs in [this +repository][1]. A DocSearch looks like this: @@ -28,11 +28,11 @@ A DocSearch looks like this: ## `index_name` This is the name of the Algolia index where your records will be pushed. The -`apiKey` we will share with you will be restricted to work on this index. +`apiKey` we will share with you is restricted to work with this index. When using the free DocSearch crawler, the `indexName` will always be the name -of the config. If you're running DocSearch yourself, you can use any name you'd -like. +of the configuration file. If you're running DocSearch yourself, you can use any +name you'd like. ```json { @@ -43,9 +43,9 @@ like. ## `start_urls` This array contains the list of URLs that will be used to start crawling your -website. The crawler will recursively follow any links on those pages. It will -not follow links that are on another domain and never follow links defined in -`stop_urls`. +website. The crawler will recursively follow any links (`` tags) from those +pages. It will not follow links that are on another domain and never follow +matching links from `stop_urls`. ```json { @@ -56,7 +56,7 @@ not follow links that are on another domain and never follow links defined in ### `selectors_key`, tailor your selectors You can define finer sets of selectors depending on the URL. This is done thanks -to the parameter `selectors_key` from your `start_urls` item. +to the parameter `selectors_key` from your `start_urls`. ```json { @@ -92,7 +92,7 @@ to the parameter `selectors_key` from your `start_urls` item. ``` To find the right subset to use based on the URL, we iterate over these -`start_urls` items. The first one matching is applied. +`start_urls` items. Only the first one matching is applied. Considering the URL `http://www.example.com/en/api/` with the configuration: @@ -113,20 +113,20 @@ Considering the URL `http://www.example.com/en/api/` with the configuration: ``` Only the set of selector related to `doc` will be applied to the URL. The -correct configuration should be build the other way around (as primarily +correct configuration should be built the other way around (as primarily described). If one `start_urls` item has no `selectors_key` defined, the `default` set will -be used. Do not forget to precise this fallback. +be used. Do not forget to set this fallback set of selectors. ### Using regular expressions -The `start_urls` option also allows for passing an object in place of a string, -to express more complex patterns. This object must contain a `url` key -containing the regular expression matching the url, as well as a `variables` key -that will be used to replace the named matches. +The `start_urls` and `stop_urls` options also enable you to provide regular +expression to express more complex patterns. This object must at least contain a +`url` key targeting a reachable page. -The following example will make it clearer: +You can also define `variables` key that will be injected into your specific URL +pattern. The following example makes this variable feature clearer: ```json { @@ -142,10 +142,10 @@ The following example will make it clearer: } ``` -The beneficial side effect of using this syntax is that all records that will -then be extracted from crawling `http://www.example.com/docs/en/latest` will -have `lang: en` and `version: latest` added to it, allowing you to then filter -based on those values. +The beneficial side effect of using this syntax is that every records extracted +from pages matching `http://www.example.com/docs/en/latest` will have attributes +`lang: en` and `version: latest`. It enables you to filter on [these +`facetFilters`][3]. The following example shows how you can filter results matching specifics language and version from the frontend @@ -616,3 +616,4 @@ value is: [1]: https://github.com/algolia/docsearch-configs/tree/master/configs [2]: https://www.algolia.com/doc/api-reference/settings-api-parameters/ +[3]: https://www.algolia.com/doc/api-reference/api-parameters/facetFilters/ diff --git a/docs/src/run-your-own.md b/docs/src/run-your-own.md index 25d5ea31..38d8bad2 100644 --- a/docs/src/run-your-own.md +++ b/docs/src/run-your-own.md @@ -4,25 +4,32 @@ title: Run your own --- The version of DocSearch we provide for free is one hosted on our own servers, -running every 24 hours. To update your results more often than that, or to index -content sitting behind a firewall, you might want to run the crawler yourself. +running every 24 hours. To update your results with more control, if you are not +compliant with our checklist, or to index content sitting behind a firewall, +you might want to run the crawler yourself. -The code of DocSearch is open source, and we packaged it as a Docker image to -make this even easier for you to use. +The whole code of DocSearch is open source, and we packaged it as a Docker image +to make this even easier for you to use. ## Installation -Start by cloning [the repository][1] and then installing the python 2 -dependencies using `pip install -r requirements.txt`. This will give you access -to the `./docsearch` command-line tool. +The scraper is a python tool [based on scrapy][1]. Start by cloning [the open +source repository][2]. We do recommend [pipenv][3] to install the whole python +environment -We then recommend using DocSearch from inside a Docker image. You can setup one -by running `./docsearch docker:build`. +- [Install pipenv][4] +- `pipenv install` +- `pipenv shell` + +You should be ready to go. + +You can use DocSearch from inside a Docker image. You can setup one by running +`./docsearch docker:build`. ## Configuration You'll need to set your Algolia application ID and admin API key as environment -variables. If you don't have an Algolia account, you should [create one][2]. +variables. If you don't have an Algolia account, you should [create one][5]. - `APPLICATION_ID` should be set to your Application ID @@ -37,17 +44,15 @@ APPLICATION_ID=YOUR_APP_ID API_KEY=YOUR_API_KEY ``` -## Creating a new config +## Create a new config -To create your config, run `./docsearch bootstrap`. A prompt will ask you for a +To create a config, run `./docsearch bootstrap`. A prompt will ask you for some information and will then output a JSON config you can use as a base. ```sh $ ./docsearch bootstrap # Enter your documentation url start url: http://www.example.com/docs/ -# You most probably don't need variables -Does the start_urls require variables ? [y/n]: n # Pick another name, or press enter index_name is example [enter to confirm]: @@ -70,29 +75,37 @@ index_name is example [enter to confirm]: ================= ``` -Copy-paste the content into a filename `example.json`, we'll use it later to -start the crawling. You can find the complete list of available options in [our -documentation][3], or browse the [list of live configs][4]. +Create a file from this text into a filename `example.json`, we'll use it later +on to start the crawl. You can find the complete list of available options in +[the configuration documentation][6], or browse the [list of live configs][7]. ## Running your config Now that you have your environment variables set, you can run the crawler according to your config. +Running `pipenv shell` will enable your virtual environment. From there, you can run one crawl with the following command: +code source + +```sh +$ ./docsearch run /path/to/your/config.json +``` + +Or from your built docker image: + ```sh $ ./docsearch docker:run /path/to/your/config.json ``` -This will crawl all pages, extract content from them and then push it to -Algolia. +This will start the crawl. It extracts content from parsed pages and push the +built records to Algolia. ## Testing your results You can test your results by running `./docsearch playground`. This will open a -web page with a search input where you can do live tests against the indexed -results. +web page with a search input. You can do live tests against the indexed results. -![Playground][6] {mt-2} +![Playground][9] {mt-2} _Note that if the command fails (it can happen on non-Mac machines), you can get the same result by running a live server in the `./playground` subdirectory.\`_ @@ -100,7 +113,7 @@ the same result by running a live server in the `./playground` subdirectory.\`_ ## Integration Once you're satisfied with your config, you can integrate the dropdown menu in -your website by following the [instructions here][5]. +your website by following the [instructions here][8]. The difference is that you'll also have to add the `appId` key to your `docsearch()` instance. Also don't forget to use a **search** API key here (in @@ -122,9 +135,12 @@ commands. Note that we use this command-line tool internally at Algolia to run the free hosted version, so you might not need all the listed commands. -[1]: https://github.com/algolia/docsearch-scraper -[2]: https://www.algolia.com/pricing#community -[3]: ./config-file.html -[4]: https://github.com/algolia/docsearch-configs/tree/master/configs -[5]: ./dropdown.html -[6]: ./assets/playground.png +[1]: https://scrapy.org/ +[2]: https://github.com/algolia/docsearch-scraper +[3]: https://github.com/pypa/pipenv +[4]: https://pipenv.readthedocs.io/en/latest/install/#installing-pipenv +[5]: https://www.algolia.com/pricing#community +[6]: ./config-file.html +[7]: https://github.com/algolia/docsearch-configs/tree/master/configs +[8]: ./dropdown.html +[9]: ./assets/playground.png