doc proof reading part 1

docs/src/how-does-it-work.md

@@ -4,8 +4,8 @@ title: How does it work?
 ---
 
 Getting up and ready with DocSearch is a straightforward process that requires a
-three steps: you apply, we configure the crawler for you, and you update your
-frontend.
+three steps: you apply, we configure the crawler for you, and you integrate our Search-UI in your
+frontend. It is as simple as copying and pasting a snippet.
 
 <img src="./assets/docsearch-how-it-works.png" alt="How it works" class="mt-2"/>
 
@@ -21,7 +21,7 @@ applications, so please give us a couple of days to get back to you :)
 
 ### We create a configuration
 
-Once we receive your application, we'll have a look at your website and create a
+Once we receive [your application][2], we'll have a look at your website and create a
 custom configuration file for it. This file defines which URLs we should crawl
 or ignore, as well as the specific CSS selectors to be used for selecting
 headers, subheaders, etc. All configs are publicly available in our [config
@@ -48,3 +48,4 @@ All we ask is that you keep the "search by Algolia" logo next to your search
 results.
 
 [1]: https://github.com/algolia/docsearch-configs/tree/master/configs
+[2]: apply.html

docs/src/inside-the-engine.md

@@ -4,58 +4,57 @@ title: Inside the engine
 ---
 
 This page will explain in more details how the crawler extracts content from
-your page, and how it ranks it in the results.
+your page every 24h, and how it ranks it in the results.
 
 ## Crawling
 
 Each crawl will begin its journey by the value of the `start_urls` you have in
-your config. It will read those pages and recursively follow every link in those
-pages until it has crawled all your website.
+your config. It will read those pages and recursively extract and follow every
+links in those pages until it has browsed every compliant pages.
 
-If it detects a `sitemap.xml`, it will use this link list instead of crawling
-all pages.
+If you have explictly defined a `sitemap.xml`, our crawler will scrap every 
+provided and compliant pages. We do recommend to use [a sitemap][1] since it 
+clearly exposes URLs to crawl and avoid missing page that aren't linked from 
+another one.
 
 ## Extracting content
 
-Then, for each page, it will read the HTML markup from top to bottom. It will
-look for HTML elements matching your CSS `selectors`. It will look for elements
-matching your `text` selector (`<p>` by default). Each of those matches will be
-later transformed into an Algolia record.
+Building records using the scraper is pretty intuitive. According to your settings,
+we extract the payload of your webpage and index it, preserving your data's structure.
+This is achieved in a simple way:
+* We **read top down** your web page following your HTML flow and pick out your
+matching elements according their **levels** based on the `selectors_level` defined.
+* We create a record for each paragraph along with its hierarchical path.
+This construction is based on their **time of appearance** along the flow.
+* We **index** these records with the appropriate global settings (e.g. metadata, tags, etc.)
 
-For each matching `text` element, the crawler will also keep in memory the
-current hierarchy of headers (identified by the `lvl0` to `lvl5` selectors) that
-it had to traverse to get to this text. This hierarchical information, as well
-as some generic page metadata (such has the page url) are then pushed to
-Algolia.
-
-Note that the crawler performs sanity checks before pushing data to Algolia. For
-example if you changed the markup of your website, the selectors might not match
-anything. If we detect that something is wrong with your current crawl, we don't
-overwrite your previous index.
+_**Note:** The above process performs sanity tests as it scrapes, in order to detect errors.
+If indeed there are any serious warnings, it will abort and therefore not overwrite your current index.
+These checks ensure that your dedicated index isn't flushed._
 
 ## Ranking records
 
 Algolia always returns the most relevant results first, using a [tie-breaking
-approach][1]. DocSearch will first search for exact matches in your keywords
+approach][2]. DocSearch will first search for exact matches in your keywords
 then fallback to partial matches. Those results will then be ordered based, once
 again, on the page hierarchy, as extracted from the `selectors`.
 
-The default strategy is to first look at the closest header of the matching
-text. For example, a paragraph under `Settings / API / verySpecificMethod()`,
-will be ranked higher than one under `Settings / API`. The idea here is that if
-you have a match under a deep hierarchy, chances are that this match is specific
-and might be more interesting that something found in a broad topic.
+The default strategy is to promote records having matching words in the highetst level fist.
+Thus if two results have the same matching words, the one having them in the highest level
+(lvl0) will be reanked higher. We also use the position of the matching words. The sooner 
+they appear within the HTML flow, the higher the record will be ranked.
 
-But this does not work in all cases as some documentations don't have deep
-hierarchy. In that case, we use the paragraph position. The first paragraph of
-the page will be ranked higher than the last one.
+The relevancy is based on several factors and can be customised according to the
+algolia tie-breaking method.
 
-You also have a way to boost some pages directly in your config by using the
-`page_rank` option. This accepts a numeric value, and all pages with a
-`page_rank` of 5 will be returned before pages with a `page_rank` of 1.
+You can boost pages depending ont their URLs. This is done from the `start_urls` and its
+`page_rank` attributes. It is a numeric value, default to 0. The bigger it is, the higher results from the matching pages will be ranked.
+For example all pages with a `page_rank` of 5 will be returned before pages with a `page_rank` of 1.
 
-You could even overwrite the default `customRanking` used by the index by using
+You could even change the relevancy strategy by [overwriting the default `customRanking`][3] used by the index by using
 the `custom_settings` option of your config.
 
-[1]:
+[1]:https://www.sitemaps.org/
+[2]:
   https://www.algolia.com/doc/guides/ranking/ranking-formula/#tie-breaking-approach
+[3]:[https://www.algolia.com/doc/guides/ranking/custom-ranking/]

docs/src/tips.md

@@ -9,8 +9,8 @@ share some tips on how you can make the most out of DocSearch.
 
 ### Use a `sitemap.xml`
 
-If your website has a sitemap, DocSearch will use it to get the list of pages to
-index. If it can't find one, it will follow every link of every page instead.
+If you provide a sitemap from your configuration, DocSearch will use it to diretly browse the pages to
+index. Pages are still crawled which means we extract every compliant link.
 
 We highly recommend you add a `sitemap.xml` to your website if you don't have
 one already. This will make the indexing faster, but will also give you more
@@ -24,20 +24,23 @@ Sitemaps are also considered good practice for other aspects, including SEO
 DocSearch works better on structured documentation. Relevance of results is
 based on the structural hierarchy of content. In simpler terms it means that we
 read the `<h1>`, ..., `<h6>` headings of your page to guess the hierarchy of
-information.
+information. This hierarchy brings contextual information to your records.
 
-Documentation starts by explains generic concepts first and then goes deeper
+Documentation starts by explaining generic concepts first and then goes deeper
 into specifics. This is represented in your HTML markup by the hierarchy of
 headings you're using. For example, concepts discussed under a `<h4>` are more
-specific than concepts discussed under a `<h2>` in the same page.
+specific than concepts discussed under a `<h2>` in the same page. The sooner the information comes up within the page, the higher it is ranked. 
 
 DocSearch uses this structure to fine-tune the relevance of results as well as
 to provide potential filtering. Documentation that follow this pattern often
 have better relevance of search results.
 
+Finding the right depth of your documentation tree and how to split-up your content is one of the most complex tasks.
+For large documents, we usually recommend having 4 levels (from lvl0 until lvl3).
+Three different levels should be the minimum.
+
 _Note that you don't have to use `<hX>` tags and can use
-`<span class="title-X">` for example instead. Your crawling configuration file
-will need to mirror those changes, though._
+`<span class="title-X">` for example instead. Your will need to update your set of `selectors`
 
 ### Set a unique class to the element holding the content
 
@@ -74,3 +77,4 @@ The name of the CSS class does not matter, as long as it's something that can be
 used as part of a CSS selector.
 
 [1]: https://www.sitemaps.org/index.html
+https://blog.algolia.com/how-to-build-a-helpful-search-for-technical-documentation-the-laravel-example/

docs/src/who-can-apply.md

@@ -8,7 +8,7 @@ large technical documentation. For that reason, we offering our free hosting
 version to all technical documentations.
 
 We're always sad to have to turn down applications, but with the number of
-requests we received everyday, we had to focus on technical documentations. We
+requests we received everyday, we had to focus on several criteria. We
 hope you understand.
 
 ### The checklist
@@ -19,22 +19,22 @@ points.
 - You must be the **owner** of the website, or at least have the power to update
   its content. You'll have to include a JavaScript snippet to enable DocSearch.
 
-- Your website must be **publicly available**. We cannot index websites that are
-  sitting behind an authentication or available on your machine.
+- Your website must be **publicly available**. We do not host index for websites
+  that are available behind an authentication or a private network.
 
 - Your website must be a **documentation website**. We do not index blogs and
-  commercial one-pagers.
+  commercial content.
 
-- Your website must **have some content**. We won't index empty websites nor
+- Your website must **have some final content**. We won't index empty websites nor
   those filled with lorem ipsum placeholder content. Please, wait until you have
-  written some documentation before applying.
+  written some documentation before applying. We would be happy to help you as soon as you have a steady design.
 
 If in doubt, don't hesitate to [apply][1] and we'll figure it out together.
 
 Even if we cannot accept your request, this does not mean that you cannot enjoy
-great search on your website. DocSearch is entirely open source and you can run
-it yourself, or use any of our other API clients to take advantage of the
-features of Algolia.
+great search on your website. DocSearch is entirely open source and [you can run
+it yourself][2], or use any of [our other API clients][3] to take advantage of the
+Algolia's features.
 
 ### Priority
 
@@ -45,7 +45,7 @@ following criteria:
 - 🙂 If your project is open source, We'll handle it before any other
   close-source product. We love open source and want to help as much as we can.
 
-- 🙂 If you're using one of our [official integrations][2], creating your config
+- 🙂 If you're using one of our [official integrations][4], creating your config
   will be much faster for us.
 
 - ☹️ If your website is rendered in the browser through JavaScript, it means
@@ -53,4 +53,8 @@ following criteria:
   recommend that you put in place server-side rendering if you can.
 
 [1]: ./apply.html
-[2]: ./integrations.html
+[2]: ./run-your-own.html
+[3]: https://www.algolia.com/doc/api-reference/
+[4]: ./integrations.html
+
+