172 lines
7.8 KiB
Text
172 lines
7.8 KiB
Text
---
|
|
title: Migrating from legacy
|
|
---
|
|
|
|
import Tabs from '@theme/Tabs';
|
|
import TabItem from '@theme/TabItem';
|
|
|
|
## Introduction
|
|
|
|
With this new version of the [DocSearch UI][1], we wanted to go further and provide better tooling for you to create and maintain your config file, and some extra Algolia features that you all have been requesting for a long time!
|
|
|
|
## What's new?
|
|
|
|
### Scraper
|
|
|
|
The DocSearch infrastructure now leverages the [Algolia Crawler][2]. We've teamed up with our friends and created a new [DocSearch helper][4], that extracts records as we were previously doing with our beloved [DocSearch scraper][3]!
|
|
|
|
The best part, is that you no longer need to install any tooling on your side if you want to maintain or update your index!
|
|
|
|
We now provide [a web interface][7] that will allow you to:
|
|
|
|
- Start, schedule and monitor your crawls
|
|
- Edit your config file from our live editor
|
|
- Test your results directly with [DocSearch v3][1]
|
|
|
|
### Algolia application and credentials
|
|
|
|
We've received a lot of requests asking for:
|
|
|
|
- A way to manage team members
|
|
- Browse and see how Algolia records are indexed
|
|
- See and subscribe to other Algolia features
|
|
|
|
They are now all available, in **your own Algolia application**, for free :D
|
|
|
|
## Config file key mapping
|
|
|
|
Below are the keys that can be found in the [`legacy` DocSearch configs][14] and their translation to an [Algolia Crawler config][16]. More detailed documentation of the Algolia Crawler can be found on the [the official documentation][15]
|
|
|
|
| `legacy` | `current` | description |
|
|
| --- | --- | --- |
|
|
| `start_urls` | [`startUrls`][20] | Now accepts URLs only, see [`helpers.docsearch`][30] to handle custom variables |
|
|
| `page_rank` | [`pageRank`][31] | Can be added to the `recordProps` in [`helpers.docsearch`][30], should be passed as a **string** |
|
|
| `js_render` | [`renderJavaScript`][21] | Unchanged |
|
|
| `js_wait` | [`renderJavascript.waitTime`][22] | See documentation of [`renderJavaScript`][21] |
|
|
| `index_name` | **removed**, see [`actions`][23] | Handled directly in the [`actions`][23] |
|
|
| `sitemap_urls` | [`sitemaps`][24] | Unchanged |
|
|
| `stop_urls` | [`exclusionPatterns`][25] | Supports [`micromatch`][27] |
|
|
| `selectors_exclude` | **removed** | Should be handled in the [`recordExtractor`][28] and [`helpers.docsearch`][29] |
|
|
| `custom_settings` | [`initialIndexSettings`][26] | Unchanged |
|
|
| `scrape_start_urls` | **removed** | Can be handled with [`exclusionPatterns`][25] |
|
|
| `strip_chars` | **removed** | `#` are removed automatically from anchor links, edge cases should be handled in the [`recordExtractor`][28] and [`helpers.docsearch`][29] |
|
|
| `conversation_id` | **removed** | Not needed anymore |
|
|
| `nb_hits` | **removed** | Not needed anymore |
|
|
| `sitemap_alternate_links` | **removed** | Not needed anymore |
|
|
| `stop_content` | **removed** | Should be handled in the [`recordExtractor`][28] and [`helpers.docsearch`][29] |
|
|
|
|
## FAQ
|
|
|
|
### Migration seems to have started, but I don't have received any emails
|
|
|
|
Due to the large number of indices DocSearch has, we need to migrate configs in small incremental batches.
|
|
|
|
If you have not received a migration mail yet, don't worry, your turn will come!
|
|
|
|
### What do I need to do to migrate?
|
|
|
|
We've tried to make the migration as seamless as possible for you and took care of all the pain part:
|
|
|
|
- [Your existing config file][11] will be migrated to an [Algolia Crawler config][12]
|
|
- Crawls will be started and scheduled
|
|
- Your Algolia application will be ready to go with a populated index!
|
|
|
|
All you need to do is **update your frontend integration with the credentials you'll receive by email** like below:
|
|
|
|
<Tabs
|
|
groupId="language"
|
|
defaultValue="js"
|
|
values={[
|
|
{ label: 'JavaScript', value: 'js', },
|
|
{ label: 'React', value: 'react', }
|
|
]
|
|
}>
|
|
<TabItem value="js">
|
|
|
|
```js app.js
|
|
docsearch({
|
|
container: '#docsearch',
|
|
appId: 'YOUR_NEW_ALGOLIA_APP_ID',
|
|
apiKey: 'YOUR_NEW_ALGOLIA_SEARCH_API_KEY',
|
|
indexName: 'YOUR_INDEX_NAME', // it does not change
|
|
});
|
|
```
|
|
|
|
</TabItem>
|
|
|
|
<TabItem value="react">
|
|
|
|
```jsx App.js
|
|
<DocSearch
|
|
appId="YOUR_NEW_ALGOLIA_APP_ID"
|
|
apiKey="YOUR_NEW_ALGOLIA_SEARCH_API_KEY"
|
|
indexName="YOUR_INDEX_NAME" // it does not change
|
|
/>
|
|
```
|
|
|
|
</TabItem>
|
|
|
|
</Tabs>
|
|
|
|
### Why is the API key different in the dashboard?
|
|
|
|
Algolia apps come with a default search API key, which also allow you to list indices, settings and search on **every** indices of your app. In the email, we provide a search **ONLY** API key, scoped to your production index, so you don't have to worry disclosing it in the frontend.
|
|
|
|
### What should I do with my legacy config and credentials?
|
|
|
|
Your [legacy config][11] will be parsed to a [Crawler config][12], please use [the dedicated web interface][7] to make any changes if you already received your access!
|
|
|
|
Your credentials will remain available, but **once all the existing configs have been migrated, we will stop the daily crawl jobs**.
|
|
|
|
### Why do I see two Algolia apps in my dashboard?
|
|
|
|
We did not remove access to the legacy DocSearch application (`BH4D9OD16A`) to give you the time to get familiar with our new infrastructure. `BH4D9OD16A` will remain available until the migration has been completed for all the DocSearch users.
|
|
|
|
_Please only refer to your new Algolia application if you already have access_
|
|
|
|
### Are the [`docsearch-scraper`][8] and [`docsearch-configs`][9] repository still maintained?
|
|
|
|
At the time you are reading this, the migration hasn't been completed, so yes they are still maintained.
|
|
|
|
**Once the migration has been completed:**
|
|
|
|
- The [`docsearch-scraper`][8] will be archived and not maintained in favor of our [Algolia Crawler][2], you'll still be able to use our [run your own][3] solution if you want!
|
|
- The [`docsearch-configs`][9] repository will be archived and and host **all** of [the existing and active **legacy** DocSearch config file][11], and [their parsed version][12]. You can get a preview [on this branch][10].
|
|
|
|
### I just applied, can I join the new infra?
|
|
|
|
We are still at an early stage of the migration, so our focus is on live configuration first.
|
|
|
|
We plan to also include new indices (< 30 days activity) to the migration batch during December 2021.
|
|
|
|
## Useful links
|
|
|
|
- [Docusaurus blog post](https://docusaurus.io/blog/2021/11/21/algolia-docsearch-migration)
|
|
- [Algolia Dev chat 11-23-2021](https://www.youtube.com/watch?v=htsjpojpKtc&t=2404s)
|
|
|
|
[1]: /docs/DocSearch-v3
|
|
[2]: https://www.algolia.com/products/search-and-discovery/crawler/
|
|
[3]: /docs/legacy/run-your-own
|
|
[4]: /docs/record-extractor
|
|
[7]: https://crawler.algolia.com/
|
|
[8]: https://github.com/algolia/docsearch-scraper
|
|
[9]: https://github.com/algolia/docsearch-configs
|
|
[10]: https://github.com/algolia/docsearch-configs/tree/feat/crawler
|
|
[11]: https://github.com/algolia/docsearch-configs
|
|
[12]: https://github.com/algolia/docsearch-configs/tree/feat/crawler/crawler-configs
|
|
[13]: https://www.algolia.com/dashboard
|
|
[14]: /docs/legacy/config-file
|
|
[15]: https://www.algolia.com/doc/tools/crawler/getting-started/overview/
|
|
[16]: https://www.algolia.com/doc/tools/crawler/apis/configuration/
|
|
[20]: https://www.algolia.com/doc/tools/crawler/apis/configuration/start-urls/
|
|
[21]: https://www.algolia.com/doc/tools/crawler/apis/configuration/render-java-script/
|
|
[22]: https://www.algolia.com/doc/tools/crawler/apis/configuration/render-java-script/#parameter-param-waittime
|
|
[23]: https://www.algolia.com/doc/tools/crawler/apis/configuration/actions/#parameter-param-indexname
|
|
[24]: https://www.algolia.com/doc/tools/crawler/apis/configuration/sitemaps/
|
|
[25]: https://www.algolia.com/doc/tools/crawler/apis/configuration/exclusion-patterns/
|
|
[26]: https://www.algolia.com/doc/tools/crawler/apis/configuration/initial-index-settings/
|
|
[27]: https://github.com/micromatch/micromatch
|
|
[28]: https://www.algolia.com/doc/tools/crawler/apis/configuration/actions/#parameter-param-recordextractor
|
|
[29]: /docs/record-extractor
|
|
[30]: /docs/record-extractor#with-custom-variables
|
|
[31]: /docs/record-extractor#pagerank
|