Auto-Indexing

What is IndexNow and How Does it Help You?

IndexNow is a new protocol developed by Microsoft and Yandex which allows website owners to instantly notify search engines (Bing & Yandex, for now) whenever website contents on any URL is created or updated.

In turn, this makes it possible for IndexNow-enabled search engines to quickly crawl and discover new URLs, as well as reflect website content changes in their search results sooner.

So, one of the benefits of using the IndexNow feature from Squirrly is that it can help speed up the regular process of waiting for a search engine to discover and index a new page.

With IndexNow, search engines know immediately the “URLs that have changed, helping them prioritize crawl for these URLs and thereby limiting organic crawling to discover new content.”

In order to ensure the authenticity of the submitted URLs (match the ownership of the domain along with submitted URLs), search engines require websites to generate a unique API key.

To make this process easier, Squirrly will automatically generate an API key for your website, as well as serve it to the search engines.

Keep reading to learn how to easily configure automatic URL submission for your website using the IndexNow feature from Squirrly.

Configure Automatic URL Submission using IndexNow

To enable IndexNow, go to your WordPress Dashboard > Squirrly SEO > All Features, and make sure the Auto-Indexing option is ACTIVE for your website (as shown below).

The rest can be configured at Squirrly SEO > Auto-Indexing > Settings. 

Submit URLs to IndexNow

• To reach this option, go to: Squirrly SEO > Auto-Indexing > Submit URLs

This option allows you to manually send any URL to the IndexNow API.

You can add multiple URLs (up to 10,000), and the URLs can be of blog articles, your home page, or any other web content.

All you have to do is insert all the URLs (one per line) in the dedicated text field shown below.

Click on Submit URLs, and all the added URLs will be submitted for indexing without you having to use any additional tool or take any extra step.

Squirrly SEO will also display a log of the most recent 20 URLs sent to the IndexNow API, to help you easily check the URLs you have submitted.

The URLs will be listed under the IndexNow History section.

For every URL, Squirrly will show:

• the date when the URL was submitted
• the response code associated to your submission (a 200 response code indicates that the submission has been successful). This information is available in the Message column.

Another thing worth noting is that a cloud icon indicates that the submission of that URL to the IndexNow API has been made automatically. Items that DON’T have a cloud icon are the URLs you’ve added manually.

Response Codes You May Encounter in the Auto-Indexing History Section

We mentioned the 200 response code and how it indicates that the submission has been successful. 

But that is NOT the only response code you may encounter in the Auto-Indexing History section, in the Message column.

Here is the full list of response codes and what each one of them means:  

Submit URLs from the Posts Page (Individually, or in Bulk)

Using the Submit URLs feature is NOT the only way you can send URLs to the IndexNow API.

Squirrly also enables you to submit a post (or page) from the Posts page in WordPress Editor.

Follow these steps if you want to submit individual posts/pages:

• Go to the Posts page;
• Hover over the post you want to send to the IndexNow API;
• You’ll see multiple options appear. Click on the one that reads: Submit to IndexNow.

Once the post gets submitted successfully, a notification will appear at the top of the Posts page.

In case you want to submit multiple URLs from the Posts page in BULK:

• Select the published posts that you want to submit to IndexNow;
• Choose the Submit to IndexNow option from the Bulk Actions;
• Finally, click on Apply.

Submit Individual URLs via Bulk SEO

• To reach this section, go to: Squirrly SEO > Optimize > Bulk SEO

Squirrly also enables you to send any published URL to the IndexNow API via the Bulk SEO section. For this, all you have to do is click on the Arrow icon shown in the screenshot below.

Auto-Indexing Settings

• To reach this option, go to Squirrly SEO > Auto-Indexing > Settings

In the Auto-Indexing Settings section, you will see the following 4 OPTIONS:

1. IndexNow Key
2. IndexNow Key URL
3. Automatically Submit Post Types
4. IndexNow Endpoints

1. IndexNow Key

To submit URLs, you must “prove” ownership of the site for which URLs are being submitted. This requires an API key. Once you submit your URLs to search engines, search engines will crawl the key file to verify ownership.

Squirrly automatically generates an API key for your website and dynamically hosts it on your website.

Once generated, the IndexNow API Key will be displayed in the IndexNow Key field.

Only you and the search engines should know the key and your file key location. Although this is unlikely, in case the API key becomes known to third parties, you can always reset the key using the “Reset Key“ button.

Squirrly would then generate a new API Key for your site in just a few seconds.

2. IndexNow Key URL

This indicates where the IndexNow API Key has been placed (the API Key location) on your website. You also have the option to check if the API key is accessible for search engines.

You can do this by clicking on the “Check Key” yellow button.

Clicking on this button should open a new Tab which displays your API key and contains the exact name of the key file.

3. Automatically Submit Post Types

You have the option to choose which content you want to automatically submit to IndexNow, by selecting or deselecting the desired post types.

4. IndexNow Endpoints

Select the IndexNow endpoints you want to submit the URLs to. Once you’ve completed this step, click on “Save Settings” to save your preferences.

Squirrly will now automatically submit URLs to the IndexNow API for the configured post types, whenever a new post is published, modified or deleted on your website.

What about the Google Indexing API?

Squirrly’s got that one covered, too!

In addition to automatically submitting URLs to Bing and the IndexNow API, Squirrly now also automatically submits your website’s URLs to the Google Indexing API.

This allows you to directly notify Google when pages are added or removed. 

In order to set this up for your site, you need to make sure that you’ve enabled the connection with Google Search Console by following the steps from this tutorial: Connect Tools – Google Search Console section.

More about how Squirrly’s integration with the Google Indexing API works:

• Any post/page you’ve optimized using the Squirrly Live Assistant (no matter the keyword you’ve optimized for) is sent to the Google Indexing API as part of the Save Post process.
• Whenever an URL is submitted to the IndexNow API, whether the submission has been made manually or automatically, that URL also gets submitted to the Google Indexing API.

🚨 Requirements! Squirrly does all that automatically, provided that the connection with Google Search Console which can be set up from: Squirrly SEO > Technical SEO > Connect Tools is ENABLED.

That’s it!

We hope this guide has made configuring the IndexNow feature from Squirrly SEO a smooth process.

If you have any other questions about using the IndexNow feature on your website, don’t hesitate to reach out to our support team.

We’re always happy to help!