nikon/docsearch
1
0
Fork 0
Code Activity Actions

adding how to build an index and updating content

Browse source
This commit is contained in:
Sylvain Pace 2018-08-31 09:50:26 +02:00
parent c09a1c3796
commit 06203939b0
14 changed files with 134 additions and 27 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
docs/config.json
View file

@ -74,6 +74,9 @@
{
"title": "Config files",
"url": "config-file.html"
},{
"title": "Build an index",
"url": "how-do-we-build-an-index.html"
},
{
"title": "Run your own",

BIN
docs/src/assets/default-colorscheme.png
View file

Binary file not shown.
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 81 KiB

8
docs/src/behavior.md
View file

@ -3,7 +3,7 @@ layout: two-columns
title: Dropdown Behavior
---
`docsearch.js` is a wrapper around the [autocomplete.js][1] library. This
Our [JS library `docsearch.js`] is a wrapper of the [autocomplete.js][1] library. This
library will listen to keystrokes in the search input, query Algolia and display
the results in a dropdown. Everything is already configured for you to work with
DocSearch, but it also exposes configuration options you can use to go even
@ -11,7 +11,7 @@ further.
## `appId`
If you're running the DocSearch crawler yourself, you'll need to define your
If you're running the DocSearch crawler on your own, you'll need to define your
application ID using the `appId` key. If you're using the free hosted version,
you don't need to add anything.
@ -25,7 +25,7 @@ docsearch({
## `handleSelected`
This method is called when a suggestion is selected. By default, DocSearch will
redirect the browser to the page matching part of the page, but you can override
redirect the browser to the result's page at the related anchor, but you can override
it to add your own behavior.
The method is called with three arguments:
@ -50,7 +50,7 @@ docsearch({
This method will be called on every keystroke to transform the typed keywords
before sending them to Algolia. By default, it does not do anything, but we
provide this hook for you to add your own logic if you want.
provide this hook for you to add your own logic if needed.
```javascript
docsearch({

9
docs/src/config-file.md
View file

@ -240,8 +240,8 @@ Other special keys can be set, as documented below.
The default way of extracting content through selectors is to read the HTML
markup from top to bottom. This works well with semi-structured content, like a
hierarchy of headers. This breaks when relevant information is not part of the
same node flow. For example when the title is in a header or a sidebar.
hierarchy of headers. This breaks when the relevant information is not part of the
same flow. For example when the title is apart in a header or a sidebar.
For that reason, you can set a selector as global, meaning that it will match on
the whole page, and will be the same for all records extracted on this page.
@ -308,8 +308,7 @@ CSS selectors are a clear and concise way to target elements of a page, but they
have a limitations. For example, you cannot go _up_ the cascade with CSS.
If you need a more powerful selector mecanism, you can write your selectors
using XPath by setting `type: xpath`. You should also set `global: true` on the
same selector.
using XPath by setting `type: xpath`.
The following example will look for a `li.chapter.active.done` and then go up
two levels in the DOM until it found a `a`. The content of this `a` will then be
@ -472,7 +471,7 @@ With the above config and the `sitemap.xml` below, both
`http://www.example.com/docs/` and `http://www.example.com/docs/de/` will be
crawled.
```html
```xml
<url>
<loc>http://www.example.com/docs/</loc>
<xhtml:link rel="alternate" hreflang="de" href="http://www.example.com/de/"/>

6
docs/src/crawler-overview.md
View file

@ -3,19 +3,23 @@ layout: two-columns
title: Crawler Overview
---
### How?
The DocSearch crawler is written in python and heavily based on the [Scrapy][1]
framework. It will crawl all pages of your website and extract content from the
HTML structure to populate an Algolia index.
It will automatically follow every internal link to make sure we are not missing
any content, and will use the semantics of your HTML structure to construct its
records. This means that `h1`...`h6` titles will be used for the hierarchy, and
records. This means that `h1`...`h6` (`selectors`) titles will be used for the hierarchy, and
each `p` of text will be used as a potential result.
Those CSS selectors can be overwritten, and each website actually has its own
JSON configuration file that describe in more details how the crawling should
behave. You can find the complete list of options in [our documentation][2].
### When?
We automatically run each config every 24h. This is done from our own
infrastructure, meaning that you don't need to install anything on your side. We
run this service entirely free of charge, but we're asking that you keep the

9
docs/src/dropdown.md
View file

@ -3,8 +3,9 @@ layout: two-columns
title: Dropdown
---
To add the dropdown of results next to your search input, you'll have to include
the `docsearch.js` library into your website as per the following example. Your
Once your algolia DocSearch index is ready, correctly set up and filled with the right data, you will need to integrate our dedicated Search-UI.
To add the dropdown of results below your search input, you'll have to include
the docsearch library into your website as per the following example. Your
`apiKey` and `indexName` credentials will be given to you as soon as we've
created your config.
@ -26,6 +27,8 @@ created your config.
// Replace inputSelector with a CSS selector
// matching your search input
inputSelector: '<YOUR_CSS_SELECTOR>',
// Set debug to true if you want to inspect the dropdown
debug: false
});
</script>
```
@ -33,7 +36,7 @@ created your config.
## Testing
If you're eager to test DocSearch but don't have credentials of your own yet,
you can use the one we use on this own website:
you can use the one we use on this website:
```javascript
docsearch({

8
docs/src/faq.md
View file

@ -19,9 +19,9 @@ updated.
Nothing.
The DocSearch crawler is running on our own infra. It will read HTML content
The DocSearch crawler is running on our own infra. It will read the HTML content
from your website and populate an Algolia index with it every day. All you need
to do is keep your website online, and we take care of the rest.
to do is keep your website online, and we take care of the rest. If you wante to edit your configuration; please submit a pull request.
### How much does it cost?
@ -50,9 +50,7 @@ results.
You will understand what people do not.
You will understand what people do not.
_If you don't have Analytics access, send us an email and we'll enable it._
_If you don't have Analytics access, [send us an email][1] and we'll enable it._
### Where is my data hosted?

49
docs/src/how-do-we-build-an-index.md Normal file
View file

@ -0,0 +1,49 @@
---
layout: two-columns
title: How do we build a DocSearch index?
---
In this section you will learn how do we build a DocSearch index from your page.
### Everything start from your page
<img src="./assets/how_do_we_build_docsearch_index_1.png" alt="1st step" class="mt-2"/>
_Note: We would rather avoid useless js rendering. Every used information should be avaible without any client side rendering. You can use `curl` in order to have a better idea of how does it look._
### We extract the payload thank to your set of `selectors `
<img src="./assets/how_do_we_build_docsearch_index_2.png" alt="2nd step" class="mt-2"/>
We will only focus on the information highlted thank to your selectors.
### We iterate though the HTML flow and build the payload
<img src="./assets/how_do_we_build_docsearch_index_3.png" alt="3rd step" class="mt-2"/>
This payload will be the only considered data from your page.
### We iterate though the payload and start pushing record
<img src="./assets/how_do_we_build_docsearch_index_4.png" alt="4th step" class="mt-2"/>
We index the temporary record at every time we agregate an element int it (if `min_indexed_level` is set to `0` ).
### We pile them up based on the current temporary record
<img src="./assets/how_do_we_build_docsearch_index_5.png" alt="5th step" class="mt-2"/>
Base on the position withitn the flow, we nest elements as much as possble to keep the contextual relevancy.
### We iterate until we match a `text` element
<img src="./assets/how_do_we_build_docsearch_index_6.png" alt="6th step" class="mt-2"/>
### We override text element when we find a newer one
<img src="./assets/how_do_we_build_docsearch_index_7.png" alt="7th step" class="mt-2"/>
### We flush deeper element than the newest one
<img src="./assets/how_do_we_build_docsearch_index_7.png" alt="7th step" class="mt-2"/>
Contextual information and hierarchy must be updated once we encouter a new level since it highlights a new sub section not related to the previous one.
If you need any further onformation, please [do not hesitate and send us your feedback][1].
[1]: mailto:docsearch@algolia.com

5
docs/src/how-does-it-work.md
View file

@ -27,7 +27,7 @@ or ignore, as well as the specific CSS selectors to be used for selecting
headers, subheaders, etc. All configs are publicly available in our [config
repository][1].
This step still requires some manual work, but thanks to the 900+ configs we
This step still requires some manual work, but thanks to the 1 000+ configs we
already created, we're able to automate most of it. Once done, we'll run a first
indexing of your website and have it run automatically every 24h.
@ -39,7 +39,7 @@ your Algolia index on each keystroke in a dropdown menu.
The default styling of the dropdown uses a grey theme to fit in most designs.
The dropdown itself is made of HTML with custom CSS classes and we recommend
that you overwrite those classes to provide a theming more inline with the rest
that [you overwrite those classes][3] to provide a theming more inline with the rest
of your website.
Now that DocSearch is set, you don't have anything else to do. We'll keep
@ -49,3 +49,4 @@ results.
[1]: https://github.com/algolia/docsearch-configs/tree/master/configs
[2]: apply.html
[3]: styling.html

4
docs/src/integrations.md
View file

@ -13,8 +13,8 @@ DocSearch on your website:
- [GitBook][5]
- [pkgdown][6] - [DocSearch indexing][7]
If you're a maintaining such a tool and would like to get added to the list, get
[in touch with us][8]
If you're a maintaining a similar tool and would like to be added to the list, get
[in touch with us][8], we would be happy to help.
[1]: https://docusaurus.io/
[2]: https://docusaurus.io/docs/en/search#docsNav

6
docs/src/styling.md
View file

@ -55,9 +55,9 @@ and you're encouraged to style it to fit your own theming. All we ask is that
you keep the `search by Algolia` logo and link next to your search results.
The logo is automatically added in the dropdown with the default styling. It's
OK to hide it through CSS, as long as you re-add it somewhere else on your page
fine if you hide it through CSS, as long as you re-add it somewhere else on your page
close to the search input or search results. It's our way to let more people
know about what do, and how they could also have from fast and relevant search
know about what we do, and how they could also have a fast and relevant search
on their website.
If you're using your own [paid Algolia account][2] and [run the crawler
@ -88,7 +88,7 @@ other display.
## Advanced styling
To more heavily style the results, feel free to have a look at the [SCSS source
If you want to do more heavy styling, feel free to have a look at the [SCSS source
code][4]. `_variables.scss` contains all the default theming, sizing and
breakpoints.

46
docs/src/tips.md
View file

@ -76,5 +76,49 @@ class to the `Installation` and `Troubleshooting` links in your sidebar.
The name of the CSS class does not matter, as long as it's something that can be
used as part of a CSS selector.
## Consistency of your content
Consistency is a pillar of a meaningful documentation. In addition to increasing
the **intelligibility** of a document it also shortens the time required for a user
to find the coveted information. The document's **topic** should be easily **identifiable**
and its **outline** sharply demarcated.
The hierarchy should always have the same size. Try to **avoid orphan records**
such like the implicit introduction/conclusion or asides. The selectors must be
efficient for **every document** and highlight the proper hierarchy.
They need to match only the coveted elements depending on their level.
Be careful to avoid the **edge effect** by matching unexpected **superfluous elements**.
Selectors should only match information from **real document webpage**
and remain ineffective for others ones (e.g., landing page, table of content, etc.).
We urge the maintainer to define a **dedicated class** for the **main DOM container**
that includes the actual document content such as `.docSearch-content`
Since documentation should be **interactive**, it is a key point to
**verbalize concepts with standardized words**.
This **redundancy**, empowered with the **search experience** (dropdown),
will even enable the **learn-as-you-type experience**.
The **way to find the information** plays a key role in **leading** the user to the
**retrieved knowledge itself**. You can also use the **synonym feature**.
## Avoid duplicates by promoting unicity
The more time-consuming reading documentation is, the more painful and reluctant its use will be.
You must avoid hazy points or catch-all. In addition to it being unhelpful,
the catch-all document may be **confusing** and **counterproductive**.
Last but not least duplicates introduce noise and mislead users. This is why
you should always focus on the relevant content and avoid duplicating content
within your site (e.g. landing page which contains all of the information, summing up, etc.).
In cases where the duplicates's existence is expected since it belongs to another
dataset (e.g. a different version), you should use [facets](https://www.algolia.com/doc/guides/searching/faceting/).
## Conciseness
What is clearly thought out is clearly and concisely expressed.
We really recommend you to read this blog post about [how to build a helpful search for techniacal documentation][2].
[1]: https://www.sitemaps.org/index.html
https://blog.algolia.com/how-to-build-a-helpful-search-for-technical-documentation-the-laravel-example/
[2]: https://blog.algolia.com/how-to-build-a-helpful-search-for-technical-documentation-the-laravel-example/

6
docs/src/what-is-docsearch.md
View file

@ -3,6 +3,8 @@ layout: two-columns
title: What is DocSearch?
---
### Why?
DocSearch is born out of the need to scratch our own itch. As developers, we
spent a lot of time reading documentation, and it can be hard to find relevant
information in large documentations. We're not blaming anyone here; building a
@ -13,11 +15,15 @@ building search interfaces. We wanted to use those skills to help others. That's
why we created a way to automatically extract content from tech documentation
and make it available to everyone from the first keystroke.
### Quick description
DocSearch itself is made of a crawler and a frontend library. We run the crawler
on our end every 24h to extract content from your website and push it to an
Algolia index. You'll then have to add the frontend library to your website to
redirect all the search requests to this index.
### How to feature DocSearch?
DocSearch is entirely free and automated. The one thing we'll need from you is
to read [our checklist][2] and apply! After that, we'll share with you the
JavaScript snippet needed to add DocSearch to your website. We ask that you keep

2
docs/src/who-can-apply.md
View file

@ -50,7 +50,7 @@ following criteria:
- ☹️ If your website is rendered in the browser through JavaScript, it means
that we'll have to crawl it through a much slower browser emulation. We highly
recommend that you put in place server-side rendering if you can.
recommend that you put in place server-side rendering for the useful textual content.
[1]: ./apply.html
[2]: ./run-your-own.html