1
0
Fork 0

docs(migration): update content for FAQs (#1291)

This commit is contained in:
Clément Vannicatte 2022-02-04 11:53:57 +01:00 committed by GitHub
parent 3475c9c8bb
commit a82b2efe04
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
11 changed files with 725 additions and 538 deletions

View file

@ -0,0 +1,68 @@
---
title: DocSearch program
---
If you're not finding the answer to your question on this website, this page will help you. If you're still unsure, don't hesitate to send [your question to us][1] directly.
For questions related to the DocSearch x Algolia Crawler, please see our [Crawler FAQ](/docs/crawler).
## What do I need to install on my side?
You just need to [implement DocSearch in your frontend](/docs/DocSearch-v3) with the credentials received by email when your application has been deployed.
DocSearch leverages the [Algolia Crawler](https://www.algolia.com/products/search-and-discovery/crawler/), which offers a web [interface](https://crawler.algolia.com/) to create, monitor, edit, start your Crawlers. If you have any questions regarding it, please see our [Crawler FAQ](/docs/crawler).
## How much does it cost?
It's free!
We know that paying for search infrastructure is a cost not all open source projects can afford. That's why we decided to keep DocSearch free for everyone. All we ask in exchange is that you keep the "Search by [Algolia][2]" logo displayed next to the search results.
If this is not possible for you, you're free to [open your own Algolia account](https://www.algolia.com/pricing) and run [DocSearch on your own][3] without this limitation. In that case, though, depending on the size of your documentation, you might need a paid account (free accounts can hold as much as 10k records).
## What data are you collecting?
We save the data we extract from your website markup, which we put in a custom JSON format instead of HTML. This is the data we put in the Algolia DocSearch index. The selectors in your config define what data to scrape.
As the website owner, we also give you access to your own Algolia application. This will let you see how your website is indexed in Algolia, detailed analytics about the anonymized searches in your website, team managements, and more!
## Where is my data hosted?
We host the DocSearch data on Algolia's servers, with replications around the globe. You can find more details about the actual [server specs here](https://www.algolia.com/doc/guides/infrastructure/servers/), and more complete information in our [privacy policy](https://www.algolia.com/policies/privacy).
## Can I use DocSearch on non-doc pages?
The free DocSearch we provide will **only** crawl open-source projects documentation pages or technical blogs. To use it on other parts of your website, you'll need to create your own Algolia account and either:
- Run the [DocSearch crawler][3] on your own
- Use one of our other [framework integrations or API clients](https://www.algolia.com/doc/api-client/getting-started/install/javascript/?client=javascript)
## Can you index code samples?
Yes, but we do not recommend it.
Code samples are a great way for humans to understand how people use a specific method. It often requires boilerplate code though, repeated across examples, which adds noise to the results.
## A documentation website I like does not use DocSearch. What can I do?
We'd love to help!
If one of your favorite tool documentation websites is missing DocSearch, we encourage you to file an issue in their repository explaining how DocSearch could help. Feel free to [send us an email][1] as well, and we'll provide all the help we can.
## How did we build this website?
We build this website with [Docusaurus v2](https://docusaurus.io/). We were helped by a great man who inspired us a lot, Endi. We want [to pay a tribute to this exceptional human being that will be always part of the DocSearch project](https://docusaurus.io/blog/2020/01/07/tribute-to-endi). Rest in peace mate!
## Can I share the `apiKey` in my repo?
The `apiKey` the DocSearch team provides is [a search-only key](https://www.algolia.com/doc/guides/security/api-keys/#search-only-api-key) and can be safely shared publicly. You can track it in your version control system (e.g. git). If you are running the scraper on your own, please make sure to create a search-only key and [do not share your Admin key](https://www.algolia.com/doc/guides/security/api-keys/#admin-api-key).
## Why is the email API key different in the dashboard?
Algolia apps come with a default search API key, which also allow you to list indices, settings and search on **every** indices of your app.
When your application is deployed, we provide a search **ONLY** API key, scoped to your production index, so you don't have to worry disclosing it in the frontend.
[1]: mailto:docsearch@algolia.com
[2]: https://www.algolia.com/
[3]: /docs/legacy/run-your-own

View file

@ -1,5 +1,5 @@
---
title: DocSearch v3
title: Getting Started
---
import Tabs from '@theme/Tabs';
@ -94,7 +94,7 @@ And the JavaScript at the end of your `<body>`:
You can hint the browser to improve the speed of the first query by adding a `preconnect`, see [#preconnect](#preconnect)
## Get started
## Implementation
<Tabs
groupId="language"

View file

@ -0,0 +1,97 @@
---
title: DocSearch x Algolia Crawler
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
If you're not finding the answer to your question on this website, this page will help you. If you're still unsure, don't hesitate to send [your question to us][1] directly.
You can also read our [Crawler FAQ](https://www.algolia.com/doc/tools/crawler/troubleshooting/faq), to understand how it behaves:
- [One of my pages was not crawled](https://www.algolia.com/doc/tools/crawler/troubleshooting/faq/#one-of-my-pages-was-not-crawled)
- [Why are my pages skipped?](https://www.algolia.com/doc/tools/crawler/troubleshooting/faq/#when-are-pages-skipped-or-ignored)
For questions related to the DocSearch program, please see our [DocSearch program FAQ](/docs/DocSearch-program).
## How often will you crawl my website?
Crawls are scheduled at a random time once a week. You can [configure this schedule from the config file](https://www.algolia.com/doc/tools/crawler/apis/configuration/schedule/) or trigger one manually from [the Crawler interface][2].
## Why do I have duplicate content in my results?
This can happen when you have more than one URL pointing to the same content, for example with `./docs`, `./docs/` and `./docs/index.html`.
We recommend configuring canonical URLs on your website, you can read more on the ["Consolidate duplicate URLs" guide by Google](https://developers.google.com/search/docs/advanced/crawling/consolidate-duplicate-urls).
Ultimately, it is possible to set set the [`exclusionPatterns`](https://www.algolia.com/doc/tools/crawler/apis/configuration/exclusion-patterns/) to all the patterns you want to exclude.
## Are the [`docsearch-scraper`](https://github.com/algolia/docsearch-scraper) and [`docsearch-configs`](https://github.com/algolia/docsearch-configs) repository still maintained?
We've deprecated our legacy infrastructure, but you can still use it to [run your own instance](/docs/legacy/run-your-own) and plug it to [DocSearch v3](/docs/DocSearch-v3)!
## Migration seems to have started, but I don't have received any emails
All of the indices from the [Legacy infrastructure](/docs/legacy/required-configuration) have been migrated, please search for emails from `docsearch@algolia.com`.
It is also possible that you were not part of the members of the previous index, don't hesitate to [contact our support team](mailto:support@algolia.com)
## What do I need to do to migrate?
All the steps are detailed in the email you've received, but in order to use the new infrastructure you need to:
- Join the Algolia application with the invite included in the email
- Update your frontend integration with the credentials received in the email.
<Tabs
groupId="language"
defaultValue="js"
values={[
{ label: 'JavaScript', value: 'js', },
{ label: 'React', value: 'react', }
]
}>
<TabItem value="js">
```js app.js
docsearch({
container: '#docsearch',
appId: 'YOUR_NEW_ALGOLIA_APP_ID',
apiKey: 'YOUR_NEW_ALGOLIA_SEARCH_API_KEY',
indexName: 'YOUR_INDEX_NAME', // it does not change
});
```
</TabItem>
<TabItem value="react">
```jsx App.js
<DocSearch
appId="YOUR_NEW_ALGOLIA_APP_ID"
apiKey="YOUR_NEW_ALGOLIA_SEARCH_API_KEY"
indexName="YOUR_INDEX_NAME" // it does not change
/>
```
</TabItem>
</Tabs>
## What should I do with my legacy config and credentials?
You can forget about them, we will do the cleaning once all of our users have migrated to the new infrastructure!
You should use [the dedicated web interface][2] to make any changes to your index.
## Why do I see two Algolia apps in my dashboard?
We did not remove access to the legacy DocSearch application (`BH4D9OD16A`) to give you the time to get familiar with our new infrastructure. `BH4D9OD16A` will remain available until the migration has been completed for all the DocSearch users.
## Links related to the migration
- [Docusaurus blog post](https://docusaurus.io/blog/2021/11/21/algolia-docsearch-migration)
- [Algolia Dev chat 11-23-2021](https://www.youtube.com/watch?v=htsjpojpKtc&t=2404s)
[1]: mailto:docsearch@algolia.com
[2]: https://crawler.algolia.com/

View file

@ -1,91 +0,0 @@
---
title: FAQ
---
If you're not finding the answer to your question on this website, this page will help you. If you're still unsure, don't hesitate to send [your question to us][1] directly.
You can also read our [Crawler FAQ][17], to understand how it behaves:
- [One of my pages was not crawled][15]
- [Why are my pages skipped?][16]
## How often will you crawl my website?
Crawls are scheduled on a random time to happen once a week, you are also able to trigger new ones directly from [the Crawler interface][2]
## What do I need to install on my side?
Nothing.
DocSearch leverages the [Algolia Crawler][3], which offers an [interface][2] to create, monitor, edit, start your Crawlers.
## How much does it cost?
Nothing.
We know that paying for search infrastructure is a cost not all open source projects can afford. That's why we decided to keep DocSearch free for everyone. All we ask in exchange is that you keep the "Search by [Algolia][4]" logo displayed next to the search results.
If this is not possible for you, you're free to [open your own Algolia account][5] and run [DocSearch on your own][6] without this limitation. In that case, though, depending on the size of your documentation, you might need a paid account (free accounts can hold as much as 10k records).
## What data are you collecting?
We save the data we extract from your website markup, which we put in a custom JSON format instead of HTML. This is the data we put in the Algolia DocSearch index. The selectors in your config define what data to scrape.
As the website owner, we also give you access to your own Algolia application. This will let you see how your website is indexed in Algolia, detailed analytics about the anonymized searches in your website, team managements, and more!
## Where is my data hosted?
We host the DocSearch data on Algolia's servers, with replications around the globe. You can find more details about the actual [server specs here][7], and more complete information in our [privacy policy][8].
## Can I use DocSearch on non-doc pages?
The free DocSearch we provide will **only** crawl open-source projects documentation pages or technical blogs. To use it on other parts of your website, you'll need to create your own Algolia account and either:
- Run the [DocSearch crawler][6] on your own
- Use one of our other [framework integrations or API clients][9]
## Can you index code samples?
Yes, but we do not recommend it.
Code samples are a great way for humans to understand how people use a specific method. It often requires boilerplate code though, repeated across examples, which adds noise to the results.
## Why do I have duplicate content in my results?
This can happen when you have more than one URL pointing to the same content, for example with `./docs`, `./docs/` and `./docs/index.html`.
We recommend configuring canonical URLs on your website, you can read more on the ["Consolidate duplicate URLs" guide by Google](https://developers.google.com/search/docs/advanced/crawling/consolidate-duplicate-urls) and use our [`ignoreCanonicalTo`](https://www.algolia.com/doc/tools/crawler/apis/configuration/ignore-canonical-to/) option directly in your crawler config.
Ultimately, it is possible to set set the [`exclusionPatterns`][10] to all the patterns you want to exclude.
## A documentation website I like does not use DocSearch. What can I do?
We'd love to help!
If one of your favorite tool documentation websites is missing DocSearch, we encourage you to file an issue in their repository explaining how DocSearch could help. Feel free to [send us an email][1] as well, and we'll provide all the help we can.
## How did we build this website?
We build this website with [Docusaurus v2][11]. We were helped by a great man who inspired us a lot, Endi. We want [to pay a tribute to this exceptional human being that will be always part of the DocSearch project][12]. Rest in peace mate!
## Can I share the `apiKey` in my repo?
The `apiKey` the DocSearch team provides is [a search-only key][13] and can be safely shared publicly. You can track it in your version control system (e.g. git). If you are running the scraper on your own, please make sure to create a search-only key and [do not share your Admin key][14].
[1]: mailto:docsearch@algolia.com
[2]: https://crawler.algolia.com/
[3]: https://www.algolia.com/products/search-and-discovery/crawler/
[4]: https://www.algolia.com/
[5]: https://www.algolia.com/pricing
[6]: /docs/legacy/run-your-own
[7]: https://www.algolia.com/doc/guides/infrastructure/servers/
[8]: https://www.algolia.com/policies/privacy
[9]: https://www.algolia.com/doc/api-client/getting-started/install/javascript/?client=javascript
[10]: https://www.algolia.com/doc/tools/crawler/apis/configuration/exclusion-patterns/
[11]: https://docusaurus.io/
[12]: https://docusaurus.io/blog/2020/01/07/tribute-to-endi
[13]: https://www.algolia.com/doc/guides/security/api-keys/#search-only-api-key
[14]: https://www.algolia.com/doc/guides/security/api-keys/#admin-api-key
[15]: https://www.algolia.com/doc/tools/crawler/troubleshooting/faq/#one-of-my-pages-was-not-crawled
[16]: https://www.algolia.com/doc/tools/crawler/troubleshooting/faq/#when-are-pages-skipped-or-ignored
[17]: https://www.algolia.com/doc/tools/crawler/troubleshooting/faq

View file

@ -33,6 +33,15 @@ We've received a lot of requests asking for:
They are now all available, in **your own Algolia application**, for free :D
## FAQ
You can find answers related to the DocSearch migration in our [Crawler FAQ page](/docs/crawler).
### Useful links
- [Docusaurus blog post](https://docusaurus.io/blog/2021/11/21/algolia-docsearch-migration)
- [Algolia Dev chat 11-23-2021](https://www.youtube.com/watch?v=htsjpojpKtc&t=2404s)
## Config file key mapping
Below are the keys that can be found in the [`legacy` DocSearch configs][14] and their translation to an [Algolia Crawler config][16]. More detailed documentation of the Algolia Crawler can be found on the [the official documentation][15]
@ -55,106 +64,11 @@ Below are the keys that can be found in the [`legacy` DocSearch configs][14] and
| `sitemap_alternate_links` | **removed** | Not needed anymore |
| `stop_content` | **removed** | Should be handled in the [`recordExtractor`][28] and [`helpers.docsearch`][29] |
## FAQ
### Migration seems to have started, but I don't have received any emails
Due to the large number of indices DocSearch has, we need to migrate configs in small incremental batches.
If you have not received a migration mail yet, don't worry, your turn will come!
### What do I need to do to migrate?
We've tried to make the migration as seamless as possible for you and took care of all the pain part:
- [Your existing config file][11] will be migrated to an [Algolia Crawler config][12]
- Crawls will be started and scheduled
- Your Algolia application will be ready to go with a populated index!
All you need to do is **update your frontend integration with the credentials you'll receive by email** like below:
<Tabs
groupId="language"
defaultValue="js"
values={[
{ label: 'JavaScript', value: 'js', },
{ label: 'React', value: 'react', }
]
}>
<TabItem value="js">
```js app.js
docsearch({
container: '#docsearch',
appId: 'YOUR_NEW_ALGOLIA_APP_ID',
apiKey: 'YOUR_NEW_ALGOLIA_SEARCH_API_KEY',
indexName: 'YOUR_INDEX_NAME', // it does not change
});
```
</TabItem>
<TabItem value="react">
```jsx App.js
<DocSearch
appId="YOUR_NEW_ALGOLIA_APP_ID"
apiKey="YOUR_NEW_ALGOLIA_SEARCH_API_KEY"
indexName="YOUR_INDEX_NAME" // it does not change
/>
```
</TabItem>
</Tabs>
### Why is the API key different in the dashboard?
Algolia apps come with a default search API key, which also allow you to list indices, settings and search on **every** indices of your app. In the email, we provide a search **ONLY** API key, scoped to your production index, so you don't have to worry disclosing it in the frontend.
### What should I do with my legacy config and credentials?
Your [legacy config][11] will be parsed to a [Crawler config][12], please use [the dedicated web interface][7] to make any changes if you already received your access!
Your credentials will remain available, but **once all the existing configs have been migrated, we will stop the daily crawl jobs**.
### Why do I see two Algolia apps in my dashboard?
We did not remove access to the legacy DocSearch application (`BH4D9OD16A`) to give you the time to get familiar with our new infrastructure. `BH4D9OD16A` will remain available until the migration has been completed for all the DocSearch users.
_Please only refer to your new Algolia application if you already have access_
### Are the [`docsearch-scraper`][8] and [`docsearch-configs`][9] repository still maintained?
At the time you are reading this, the migration hasn't been completed, so yes they are still maintained.
**Once the migration has been completed:**
- The [`docsearch-scraper`][8] will be archived and not maintained in favor of our [Algolia Crawler][2], you'll still be able to use our [run your own][3] solution if you want!
- The [`docsearch-configs`][9] repository will be archived and and host **all** of [the existing and active **legacy** DocSearch config file][11], and [their parsed version][12]. You can get a preview [on this branch][10].
### I just applied, can I join the new infra?
We are still at an early stage of the migration, so our focus is on live configuration first.
We plan to also include new indices (< 30 days activity) to the migration batch during December 2021.
## Useful links
- [Docusaurus blog post](https://docusaurus.io/blog/2021/11/21/algolia-docsearch-migration)
- [Algolia Dev chat 11-23-2021](https://www.youtube.com/watch?v=htsjpojpKtc&t=2404s)
[1]: /docs/DocSearch-v3
[2]: https://www.algolia.com/products/search-and-discovery/crawler/
[3]: /docs/legacy/run-your-own
[4]: /docs/record-extractor
[7]: https://crawler.algolia.com/
[8]: https://github.com/algolia/docsearch-scraper
[9]: https://github.com/algolia/docsearch-configs
[10]: https://github.com/algolia/docsearch-configs/tree/feat/crawler
[11]: https://github.com/algolia/docsearch-configs
[12]: https://github.com/algolia/docsearch-configs/tree/feat/crawler/crawler-configs
[13]: https://www.algolia.com/dashboard
[14]: /docs/legacy/config-file
[15]: https://www.algolia.com/doc/tools/crawler/getting-started/overview/
[16]: https://www.algolia.com/doc/tools/crawler/apis/configuration/

View file

@ -36,13 +36,13 @@ new Crawler({
selectors: '',
defaultValue: 'Documentation',
},
lvl1: 'header h1',
lvl2: 'article h2',
lvl3: 'article h3',
lvl4: 'article h4',
lvl5: 'article h5',
lvl6: 'article h6',
content: 'article p, article li',
lvl1: ['header h1', 'article h1', 'main h1', 'h1', 'head > title'],
lvl2: ['article h2', 'main h2', 'h2'],
lvl3: ['article h3', 'main h3', 'h3'],
lvl4: ['article h4', 'main h4', 'h4'],
lvl5: ['article h5', 'main h5', 'h5'],
lvl6: ['article h6', 'main h6', 'h6'],
content: ['article p, article li', 'main p, main li', 'p, li'],
},
});
},
@ -50,7 +50,7 @@ new Crawler({
],
initialIndexSettings: {
YOUR_INDEX_NAME: {
attributesForFaceting: ['type'],
attributesForFaceting: ['type', 'lang'],
attributesToRetrieve: [
'hierarchy',
'content',

View file

@ -15,7 +15,7 @@ const lightCodeTheme = require('prism-react-renderer/themes/github');
organizationName: 'Algolia',
projectName: 'DocSearch',
onBrokenLinks: 'throw',
onBrokenMarkdownLinks: 'warn',
onBrokenMarkdownLinks: 'throw',
presets: [
[
'@docusaurus/preset-classic',
@ -97,7 +97,7 @@ const lightCodeTheme = require('prism-react-renderer/themes/github');
},
{
label: 'FAQ',
to: 'docs/faq',
to: 'docs/docsearch-program',
},
{
label: 'DocSearch v3',

View file

@ -18,8 +18,8 @@
"dependencies": {
"@algolia/ui-library": "4.0.0-beta.71",
"@docsearch/react": "3.0.0-alpha.50",
"@docusaurus/core": "2.0.0-beta.14",
"@docusaurus/preset-classic": "2.0.0-beta.14",
"@docusaurus/core": "2.0.0-beta.15",
"@docusaurus/preset-classic": "2.0.0-beta.15",
"@mdx-js/react": "1.6.22",
"clsx": "1.1.1",
"file-loader": "6.2.0",

View file

@ -10,15 +10,45 @@
*/
module.exports = {
docs: {
'Getting started': [
'what-is-docsearch',
'who-can-apply',
'migrating-from-legacy',
],
'Adding DocSearch to your UI': ['DocSearch-v3', 'api', 'styling'],
'Algolia Crawler': ['record-extractor', 'templates', 'manage-your-crawls'],
'Requirements, tips, FAQ': ['faq', 'tips', 'integrations'],
'Under the hood': ['how-does-it-work', 'required-configuration'],
},
docs: [
{
type: 'category',
label: 'Introduction',
items: ['what-is-docsearch', 'who-can-apply', 'migrating-from-legacy'],
},
{
type: 'category',
label: 'DocSearch v3',
items: ['DocSearch-v3', 'api', 'styling'],
},
{
type: 'category',
label: 'Algolia Crawler',
items: ['record-extractor', 'templates', 'manage-your-crawls'],
},
{
type: 'category',
label: 'Requirements, tips, FAQ',
items: [
{
type: 'category',
label: 'FAQ',
items: ['crawler', 'DocSearch-program'],
},
{
type: 'doc',
id: 'tips',
},
{
type: 'doc',
id: 'integrations',
},
],
},
{
type: 'category',
label: 'Under the hood',
items: ['how-does-it-work', 'required-configuration'],
},
],
};

View file

@ -1,6 +1,6 @@
import { Hero, Button, InlineLink } from '@algolia/ui-library';
import { useColorMode } from '@docusaurus/theme-common';
import { useBaseUrlUtils } from '@docusaurus/useBaseUrl';
import useThemeContext from '@theme/hooks/useThemeContext';
import React from 'react';
import DocSearchLogo from './DocSearchLogo';
@ -8,7 +8,7 @@ import showcaseProjects from './showcase-projects.json';
function Home() {
const { withBaseUrl } = useBaseUrlUtils();
const { isDarkTheme } = useThemeContext();
const { isDarkTheme } = useColorMode();
React.useEffect(() => {
if (isDarkTheme) {

819
yarn.lock

File diff suppressed because it is too large Load diff