Google Search Console and Indexing API in 2026

Google Search Console is the single most powerful free tool Google gives site owners, yet most people treat it like a read-only dashboard. They glance at impressions, maybe check a few queries, and close the tab. In 2026, that approach leaves massive amounts of organic traffic on the table.

The real power of GSC lies deeper: URL inspection, the Page indexing report, and the APIs that let you submit sitemaps, inspect URLs and pull performance data in code. At wppoland.com the whole publishing pipeline reports to Search Console and IndexNow after every deploy. This guide walks you through every layer of that stack, from proper setup through CI/CD automation, and marks where Google’s documentation draws the line on what the Indexing API is for.

#What can Google Search Console and its API do?

Most site owners check Search Console once a week, scan the performance chart, and leave. That is like buying a Swiss Army knife and only using the bottle opener.

GSC gives you direct signals from Google about how your site is perceived. The URL Inspection tool tells you exactly what Googlebot sees when it renders your page, including whether JavaScript is blocking critical content. The Page indexing report (formerly Index Coverage) shows how many of your pages Google indexed and, more importantly, the reason for every URL it did not. The Core Web Vitals section shows real-user performance data that directly impacts rankings.

The Search Console API takes this further by letting you pull performance data programmatically, build custom dashboards, and automate monitoring. Combined with sitemap submission through the API and, for job postings and livestreams, the Indexing API, you stop hoping Google finds your content and start telling it what changed. For agencies managing dozens of WordPress properties, that monitoring is what catches a broken template or a stray noindex before it costs weeks of traffic.

#How to set up Google Search Console for WordPress

Before you touch any advanced tooling, configure the Search Console property the way Google expects. Google offers two property types: Domain and URL prefix.

Domain properties cover all subdomains and protocols (http, https, www, non-www) under a single umbrella. This is the recommended approach for most sites. It requires DNS verification, which means adding a TXT record to your domain’s DNS configuration. For WordPress sites on managed hosting, this typically takes under five minutes through your hosting panel.

URL prefix properties are useful when you need to track a specific subdirectory (like example.com/blog/) or when you lack DNS access. Verification options include HTML file upload, meta tag, Google Analytics, or Google Tag Manager.

After verification, take these setup steps:

• Link to GA4: In GSC Settings, associate your Google Analytics 4 property. This lets you see GSC data directly in GA4 reports and enables richer attribution analysis.
• Set user permissions: Add team members with appropriate access levels. “Full” users can see all data and take actions; “Restricted” users can only view data. For agency setups, use “Owner” delegation carefully since it grants the ability to remove other users.
• Submit your primary sitemap: Navigate to Sitemaps and submit your XML sitemap URL. For WordPress sites using Yoast or RankMath, this is typically https://yourdomain.com/sitemap_index.xml.
• Verify international targeting: If you run a multilingual site, check hreflang with a crawler and spot-check pages in URL Inspection. Google retired the International Targeting report in 2022, so Search Console no longer lists hreflang errors.

If someone on your team is new to Search Console, Google Search Central published two curated YouTube playlists on 6 October 2026 (Daniel Waisberg’s announcement). Search fundamentals for beginners covers crawling, indexing, serving and the performance reports. Technical SEO fundamentals covers robots.txt, crawl budget, the URL inspection tool, and the Search Console bulk data exports and APIs, which is the ground the rest of this guide stands on.

#Why pages are excluded in Google Search Console

The Page indexing report is where you discover whether Google is actually indexing the pages you care about. It splits every URL Google knows into Indexed and Not indexed, and lists a reason for each not-indexed group. Google’s help page stresses that “not indexed” is not necessarily bad: redirects, alternates and deliberate noindex pages belong there.

#Understanding exclusion reasons

The most common reasons, with Google’s own definition first and our reading second:

Discovered - currently not indexed: Google found the URL (usually through a sitemap or internal link) but has not crawled it yet. Google’s documentation says that typically it wanted to crawl the URL but expected that to overload the site, so it rescheduled the crawl, which is why the last crawl date is empty. On sites with thousands of thin or near-duplicate URLs this status piles up, so strengthen internal linking to the pages that matter and remove the URLs that should never have existed.

Crawled - currently not indexed: Google fetched the page but did not add it to the index. Google says it “may or may not be indexed in the future” and that there is no need to resubmit the URL. Because Google actually saw the content, our first check is uniqueness and depth: thin pages and duplicates of other pages on the same site are the usual pattern we find.

Soft 404: The page returns a 200 status code but Google thinks it behaves like a 404. This happens with empty category pages, search result pages with zero results, or pages with minimal unique content. Either add meaningful content, return a proper 404 status, or use a noindex tag.

Blocked by robots.txt: Your robots.txt is preventing Googlebot from crawling the URL. Check whether this is intentional. In WordPress, common culprits include blocking /wp-admin/admin-ajax.php (which breaks AJAX-rendered content) or overly broad disallow rules.

Alternate page with proper canonical tag: The page points its canonical to another URL. Google is respecting that signal. Google’s help page says the canonical is indexed and there is nothing to do. You see it on parameter URLs and on mobile or AMP alternates.

#How to use the URL Inspection tool

For any specific URL, the URL Inspection tool shows the indexed version: the canonical Google selected, the referring sitemap, and when it was last crawled. This is not a live test, it is the version Google last stored. Use “Test live URL” to see what Google renders now, which is how you debug JavaScript-heavy WordPress themes. Two limits from the same help page: the live test cannot predict which canonical Google will choose, and “URL is on Google” means eligible to appear, not guaranteed to appear.

“Request indexing” in the same tool is the official route for a handful of regular pages. Google’s recrawl guide says you need owner or full user rights, that there is a quota for individual URLs, and that requesting a recrawl of the same URL several times does not speed it up. It also says crawling “can take anywhere from a few days to a few weeks”.

#How to use the Search Console performance report

The Performance report contains four core metrics: impressions, clicks, average CTR, and average position. The real value comes from filtering and cross-referencing these dimensions.

#Finding keywords in positions 5 to 20

Filter the Queries report to show keywords where your average position is between 5 and 20. These are terms where you are already visible but not yet capturing significant clicks. Sort by impressions to find the highest-volume opportunities. For each of these:

1. Check the landing page Google associates with the query (use the Pages filter).
2. Evaluate whether the page’s title tag and meta description are optimized for that specific query.
3. Look at the content depth. Position 8-15 often means Google considers your content relevant but not comprehensive enough.
4. Add missing subtopics, update outdated information, and improve internal linking from other topically related pages.

#Device and country analysis

Filter by Device to identify pages that rank well on desktop but poorly on mobile, or vice versa. This often indicates rendering or UX issues specific to one platform. Filter by Country to discover unexpected international traffic that you could capitalize on with targeted content or hreflang implementation.

Since October 2026 the Country filter accepts more than one country at a time. Google announced it in a LinkedIn post: “If you operate in multiple countries, you can now view their combined Google Search traffic.” Barry Schwartz reported it on Search Engine Roundtable on 6 October 2026. For an English site that sells into the UK and Ireland, tick both and you get one combined line for clicks and impressions, instead of two exports summed in a spreadsheet. Then run Ireland on its own against that total: if it barely moves the line, a separate en-IE version is not worth the maintenance yet.

#Comparing periods before and after a change

Use the Compare feature to measure periods before and after changes. After a site migration, content update, or technical fix, compare the two-week period before and after to quantify impact. Export this data via the API for custom reporting dashboards that stakeholders actually look at.

#The Google Indexing API

The Indexing API lets you notify Google when a page is added, updated, or removed. Its scope is narrow. Google’s quickstart says it “can only be used to crawl pages with either JobPosting or BroadcastEvent embedded in a VideoObject”: job postings and livestream videos. The same page says all submissions “undergo rigorous spam detection” and that attempts to abuse the API may lead to revoked access. Plugins that push every blog post through it are working outside that documentation, and we do not do it on client sites. For regular pages, use sitemaps and the URL Inspection tool.

#Quota and limits

The quota page sets defaults per Google Cloud project, not per Search Console property:

URL_UPDATED tells Google a URL is new or changed, URL_DELETED tells it the page was removed. Google describes the 200 as a default for onboarding and testing: production use needs additional approval through a quota request form, and the quota “may increase or decrease based on the document quality”. You can also batch requests and read the notification status of URLs you already sent.

#Setting up authentication

1. Create a Google Cloud project at console.cloud.google.com.
2. Enable the Indexing API in the API Library.
3. Create a Service Account under IAM & Admin. Download the JSON key file.
4. Add the Service Account email as a delegated owner of your Search Console property. Google’s prerequisites describe it: open the property, click “Add an owner” in the Verified owner list, and paste the address from the client_email field of the JSON key (it looks like [email protected]).

#Node.js implementation

Here is a minimal implementation for notifying Google about a job posting URL:

const { google } = require('googleapis');

const auth = new google.auth.GoogleAuth({
  credentials: JSON.parse(process.env.GOOGLE_INDEXING_CREDENTIALS),
  scopes: ['https://www.googleapis.com/auth/indexing'],
});

const indexing = google.indexing({ version: 'v3', auth });

async function submitUrl(url) {
  const res = await indexing.urlNotifications.publish({
    requestBody: { url, type: 'URL_UPDATED' },
  });
  console.log(`Submitted: ${url} - ${res.data.urlNotificationMetadata.latestUpdate.type}`);
}

Store your service account credentials as an environment variable (GOOGLE_INDEXING_CREDENTIALS) containing the JSON key file contents. Never commit credentials to version control.

#Batch submission with rate limiting

When you need to submit multiple URLs, respect the daily quota and add delays between requests to avoid hitting rate limits:

const DAILY_QUOTA = 200;
const DELAY_BETWEEN_REQUESTS_MS = 1000;

async function submitBatch(urls) {
  const batch = urls.slice(0, DAILY_QUOTA);
  for (const url of batch) {
    try {
      await submitUrl(url);
      await new Promise(resolve => setTimeout(resolve, DELAY_BETWEEN_REQUESTS_MS));
    } catch (error) {
      console.error(`Failed to submit ${url}: ${error.message}`);
    }
  }
  console.log(`Submitted ${batch.length} of ${urls.length} URLs`);
}

#IndexNow vs Google Indexing API

While the Google Indexing API targets Google specifically, IndexNow is an open protocol that notifies several search engines at once. The participant list published at indexnow.org names Bing, Yandex, Seznam, Naver, Yep, the Internet Archive and Amazonbot. Google is not on that list.

IndexNow works by sending a simple HTTP POST or GET request to the search engine’s IndexNow endpoint with your site’s API key. The key is a text file hosted at the root of your domain that proves ownership.

curl "https://api.indexnow.org/indexnow?url=https://wppoland.com/en/new-post/&key=YOUR_API_KEY"

For batch submissions, the IndexNow documentation allows up to 10,000 URLs per POST. A 200 response only means the engine received the list, not that it will crawl it:

const axios = require('axios');

async function submitIndexNow(urls, apiKey, host) {
  const response = await axios.post('https://api.indexnow.org/indexnow', {
    host,
    key: apiKey,
    keyLocation: `https://${host}/${apiKey}.txt`,
    urlList: urls,
  });
  console.log(`IndexNow response: ${response.status}`);
}

IndexNow is not limited to job postings, which makes it the push channel for regular content on the engines that support it. Google still finds those pages through your sitemap.

#Submit sitemaps with the Search Console API

Beyond the Indexing API, the Search Console API lets you manage sitemaps programmatically. This is useful for sites with multiple sitemap files or dynamic sitemap generation.

#Submitting sitemaps programmatically

The sitemap methods need a client authorized with the https://www.googleapis.com/auth/webmasters scope, not the indexing scope from the previous section, and the service account must have access to the property.

const { google } = require('googleapis');

const auth = new google.auth.GoogleAuth({
  credentials: JSON.parse(process.env.GOOGLE_SC_CREDENTIALS),
  scopes: ['https://www.googleapis.com/auth/webmasters'],
});
const searchconsole = google.searchconsole({ version: 'v1', auth });

async function submitSitemap(siteUrl, sitemapUrl) {
  await searchconsole.sitemaps.submit({
    siteUrl,
    feedpath: sitemapUrl,
  });
  console.log(`Sitemap submitted: ${sitemapUrl}`);
}

#Monitoring sitemap status

Query the sitemap list endpoint to check errors, warnings, submitted URL counts and when Google last downloaded each file. The sitemaps resource marks contents[].indexed as “Deprecated; do not use”, so take indexed counts from the Page indexing report instead:

async function checkSitemaps(siteUrl) {
  const res = await searchconsole.sitemaps.list({ siteUrl });
  for (const sitemap of res.data.sitemap || []) {
    const submitted = (sitemap.contents || []).reduce((n, c) => n + Number(c.submitted || 0), 0);
    console.log(`${sitemap.path}: ${submitted} submitted, last downloaded ${sitemap.lastDownloaded}`);
    if (Number(sitemap.errors) > 0 || Number(sitemap.warnings) > 0) {
      console.warn(`  Errors: ${sitemap.errors}, warnings: ${sitemap.warnings}`);
    }
  }
}

lastDownloaded is the field worth alerting on: it is how we caught the stale fetch described further down.

Do not rely on the old ping endpoint. Google announced its deprecation on 26 June 2023: it stopped working six months later and requests to it return 404. Sitemaps go through the Sitemap: line in robots.txt or through Search Console. The same post says Google uses lastmod to schedule crawls only when it “consistently match[es] reality”.

#Automate Google indexing in a CI/CD pipeline

For static sites built with Astro, Next.js, or similar frameworks (like the wppoland.com stack), integrating indexing into the build pipeline ensures every deployment triggers the right notifications.

#Post-build hook example

Create a post-build script that reads your generated sitemap, identifies new or changed URLs, and sends them to IndexNow. Only job posting and livestream URLs go to the Google Indexing API:

const { readFileSync } = require('fs');
const { XMLParser } = require('fast-xml-parser');

async function postBuildIndex(sitemapPath) {
  const xml = readFileSync(sitemapPath, 'utf-8');
  const parser = new XMLParser();
  const sitemap = parser.parse(xml);

  const urls = sitemap.urlset.url
    .filter(entry => {
      const lastmod = new Date(entry.lastmod);
      const oneDayAgo = new Date(Date.now() - 86400000);
      return lastmod > oneDayAgo;
    })
    .map(entry => entry.loc);

  console.log(`Found ${urls.length} recently modified URLs`);
  await submitIndexNow(urls, process.env.INDEXNOW_KEY, 'wppoland.com');  // IndexNow
  const jobUrls = urls.filter(u => u.includes('/jobs/'));  // only JobPosting pages
  await submitBatch(jobUrls);  // Google Indexing API
}

#CI/CD integration

In your GitHub Actions or GitLab CI pipeline, add a step after the deployment job:

- name: Notify search engines
  env:
    GOOGLE_INDEXING_CREDENTIALS: ${{ secrets.GOOGLE_INDEXING_CREDENTIALS }}
    INDEXNOW_KEY: ${{ secrets.INDEXNOW_KEY }}
  run: node scripts/post-build-index.js

#Quota management strategies

With a default of 200 publish requests per day for the whole Cloud project, a job board needs to prioritize:

• New postings first: A posting that Google has never seen gets top priority.
• Expired postings second: Send URL_DELETED when a job is filled, so it leaves results instead of sending applicants to a dead page.
• Skip unchanged content: Compare lastmod dates or content hashes to avoid wasting quota on pages that have not changed.
• Queue overflow for the next day: If a deployment has more than the quota, persist the remaining URLs and process them the following day via a scheduled task.

#How long crawling, indexing and serving actually take

Most “why is it not indexed yet” tickets are a clock problem, not a bug. Gary Illyes showed Google’s internal timelines at Search Central Live in Barcelona, and Search Engine Roundtable published the table on 5 October 2026. Every row has a typical time and a slowest time:

Two caveats from the source. The processes are linked, so a page cannot be indexed before it is crawled and delays stack up. And Gary himself told Barry Schwartz:

“mind that this was an exercise to see if the audience can relate to the numbers we pulled internally and put in those slides.”

Gary Illyes, Google, Search Engine Roundtable

Read the table as orders of magnitude, not an SLA.

Pin one more limit next to it. The Search Console API limits page allows the URL Inspection API 2,000 queries per day and 600 per minute per site, so a site with 6,000 URLs needs three days to inspect all of them once.

#Real examples of indexing delays in Search Console

• A stale sitemap fetch hid 2,034 URLs. GSC last fetched our six location sitemaps on 1 September at 10:08, when they held 2,037 URLs. The corpus then grew to 4,071 and Google did not refetch for five days, although it read the blog sitemaps daily. Resubmitting through the API on 6 September got the fetch the same day. Pages with impressions went from 223 to 343 and impressions from 1,570 to 4,382 week on week, but those pages produced 2 clicks in 7 days. Discovery moved, conversion did not.
• Verdicts are snapshots. On 17 September, 3,609 of 6,747 sitemap URLs (53.5%) had no impression in 90 days. In a sample of 19 inspected URLs, 6 showed “Excluded by noindex” while the live page served index, follow. A recheck on 27 September of 20 random URLs found 3 still excluded, last crawled between 7 May and 20 August. The noindex was gone, Google just had not come back.
• Submitting a guessed URL list wastes the quota. Our IndexNow script built 2,328 URLs from filenames and none of them was in the sitemap, all answered 301. Build the list from the sitemap, not from the filesystem.

The practical rule: before you call a page weak, compare the date of the verdict with the date of your last change. If the verdict is older, you are measuring the past.

#Why pages are not indexed after submission

When pages stay out of the index after a sitemap submission or an indexing request, check these five problems before you look at content quality.

#robots.txt blocking critical resources

A common WordPress misconfiguration is blocking CSS or JavaScript files that Googlebot needs to render the page. Use the URL Inspection tool’s “Test Live URL” to see what Googlebot actually renders. If the page looks broken, check whether your robots.txt blocks any required assets. Your robots.txt should always allow:

User-agent: Googlebot
Allow: /wp-content/themes/
Allow: /wp-content/plugins/
Allow: /wp-includes/

Also block AI scrapers that consume crawl budget without sending traffic:

User-agent: GPTBot
Disallow: /

User-agent: CCBot
Disallow: /

#Canonical conflicts

If your page has a <link rel="canonical"> pointing to a different URL, Google will index the canonical target instead. This happens frequently with trailing slash mismatches, HTTP/HTTPS conflicts, or www/non-www variations. Check that your WordPress settings, CDN, and server configuration all agree on the canonical URL format. In Yoast or RankMath, verify the canonical is not being overridden per page.

#Noindex tags and headers

Search for <meta name="robots" content="noindex"> in your page source. In WordPress, this can be set at the page level (via SEO plugins), at the category level, or even globally through Settings > Reading > “Discourage search engines from indexing this site.” Also check HTTP response headers for X-Robots-Tag: noindex, which some security plugins or CDN configurations add inadvertently.

#JavaScript rendering problems

If your WordPress theme relies on JavaScript to load content (React-based themes, heavy AJAX pagination), Googlebot may not see the full content. The URL Inspection tool shows you the rendered HTML. Compare it with your source HTML to identify what is being loaded client-side only. Server-side rendering or pre-rendering is the permanent fix.

#Mobile-first indexing gaps

Google uses the mobile version of your page for indexing and ranking. If your mobile version hides content, uses different markup, or loads different structured data than desktop, the mobile version is what gets indexed. The URL Inspection tool shows which crawler fetched the page under Page availability > Crawled > Crawled as, so confirm it says smartphone and check the rendered HTML for missing content. Keep content parity between mobile and desktop.

#How to build an image sitemap for AVIF and WebP

Yoast and RankMath add image entries automatically, but a custom or headless stack has to do it itself. Google’s image sitemap documentation describes the tags as a way of telling Google about images “that we might not otherwise find (such as images your site reaches with JavaScript code)”. A plain <img> in server-rendered HTML is found without them.

<url>
  <loc>https://wppoland.com/en/my-post/</loc>
  <image:image>
    <image:loc>https://wppoland.com/images/my-post.avif</image:loc>
  </image:image>
</url>

Each <url> can hold up to 1,000 <image:image> tags. Google removed <image:caption>, <image:title>, <image:geo_location> and <image:license> from its documentation, so generators that still emit them waste bytes. Images on a CDN domain are fine as long as that domain is verified in Search Console too and robots.txt does not block it.

#How to get a new page indexed, step by step

Here is the complete workflow we use at wppoland.com for every piece of content we publish or update:

1. Publish or update the page in WordPress or your CMS. Ensure all structured data, meta tags, and internal links are in place before going live.
2. Sitemap auto-update: Your XML sitemap regenerates automatically (Yoast, RankMath, or your static-site generator handles this). Verify the lastmod date reflects the current timestamp.
3. IndexNow: The post-build script or publish hook sends the URL to IndexNow, notifying Bing, Yandex, and the other participating engines.
4. Google Indexing API, job postings and livestreams only: For a page with JobPosting or a BroadcastEvent in a VideoObject, the same script sends URL_UPDATED. Regular pages skip this step.
5. Sitemap through the API: If the sitemap file itself changed, resubmit it with sitemaps.submit and check lastDownloaded the next day. For a handful of important regular pages, use “Request indexing” in the URL Inspection tool.
6. Monitor in GSC: Check the URL Inspection tool, or the URL Inspection API within its 2,000-a-day limit, to confirm the page was crawled and indexed. Google’s own guidance is that crawling can take days to weeks.
7. Iterate: If the page is still not indexed, read the reason in the Page indexing report, fix the underlying issue, and only then request a recrawl.

This workflow keeps every change visible to Google without abusing an API outside its documented scope. Whether you run a job board, an e-commerce store with seasonal products, or an agency managing client properties, the order is the same: accurate sitemaps first, push channels where they are allowed, and Search Console data to check the result.

For the delivery side of this topic we keep the work under GEO and LLMO.