Reads your sitemap daily, submits the pages that are new or changed for indexing, and records every submission in a sheet.
Uses
Google Sheets
Tell search engines about your new pages
Run every day at 07:00.Run every day at 07:00.
Set sitemapUrl to blank.Set sitemapUrl to blank.Set indexNowKey to blank.Set indexNowKey to blank.Set spreadsheetUrl to blank.Set spreadsheetUrl to blank.Set sheetTab to Index submissions.Set sheetTab to Index submissions.Set maxPagesPerRun to 25.Set maxPagesPerRun to 25.Set keyFileUrl to blank.Set keyFileUrl to blank.Set indexNowEndpoint to https://api.indexnow.org/indexnow.Set indexNowEndpoint to https://api.indexnow.org/indexnow.
Set pageLimit to maxPagesPerRun multiplied by 1.Set pageLimit to maxPagesPerRun multiplied by 1.
If pageLimit is greater than or equal to 1 is noIf pageLimit is greater than or equal to 1 is noYes
Stop and report a problem: Set the 'maxPagesPerRun' step to how many pages may be submitted in one run, as a whole number - for example 25. It is currently: maxPagesPerRun.Stop and report a problem: Set the 'maxPagesPerRun' step to how many pages may be submitted in one run, as a whole number - for example 25. It is currently: maxPagesPerRun.No
No steps
If pageLimit is greater than 500If pageLimit is greater than 500Yes
Stop and report a problem: The 'maxPagesPerRun' step is set higher than 500. Search engines throttle and then ignore bulk submissions, so keep it at 500 or fewer. It is currently: maxPagesPerRun.Stop and report a problem: The 'maxPagesPerRun' step is set higher than 500. Search engines throttle and then ignore bulk submissions, so keep it at 500 or fewer. It is currently: maxPagesPerRun.No
No steps
Trim indexNowKey and save it as submissionKey.Trim indexNowKey and save it as submissionKey.
Get the first item of the last item of sitemapUrl split by :// split by / and save it as siteHost.Get the first item of the last item of sitemapUrl split by :// split by / and save it as siteHost.If siteHost is emptyIf siteHost is emptyYes
Stop and report a problem: Could not work out which site the 'sitemapUrl' step belongs to. It should look like https://example.com/sitemap.xml, not: sitemapUrl.Stop and report a problem: Could not work out which site the 'sitemapUrl' step belongs to. It should look like https://example.com/sitemap.xml, not: sitemapUrl.No
No steps
Trim keyFileUrl and save it as keyLocation.Trim keyFileUrl and save it as keyLocation.If keyLocation is emptyIf keyLocation is emptyYes
Change keyLocation to https://@siteHost/@submissionKey.txt.Change keyLocation to https://@siteHost/@submissionKey.txt.No
No steps
Set sitemapText to blank.Set sitemapText to blank.
If a step below fails, save the error as sitemapReadErrorIf a step below fails, save the error as sitemapReadErrorTry
Fetch sitemapUrl and save it as sitemapResponse.Fetch sitemapUrl and save it as sitemapResponse.Trim sitemapResponse.body and save it as sitemapText.Trim sitemapResponse.body and save it as sitemapText.On error
Stop and report a problem: Could not read the sitemap in the 'sitemapUrl' step. sitemapReadError.message with sitemapUrl replaced by the sitemap address.Stop and report a problem: Could not read the sitemap in the 'sitemapUrl' step. sitemapReadError.message with sitemapUrl replaced by the sitemap address.If sitemapText contains <sitemapindexIf sitemapText contains <sitemapindexYes
Stop and report a problem: That address is a sitemap index - a list of other sitemaps - rather than a sitemap of pages. Point the 'sitemapUrl' step at one of the sitemaps it lists, or add this workflow once per sitemap.Stop and report a problem: That address is a sitemap index - a list of other sitemaps - rather than a sitemap of pages. Point the 'sitemapUrl' step at one of the sitemaps it lists, or add this workflow once per sitemap.No
No steps
If sitemapText doesn't contain <loc>If sitemapText doesn't contain <loc>Yes
Stop and report a problem: Nothing at the address in the 'sitemapUrl' step looks like a sitemap: it contains no <loc> entries. Check the address opens a sitemap in a browser.Stop and report a problem: Nothing at the address in the 'sitemapUrl' step looks like a sitemap: it contains no <loc> entries. Check the address opens a sitemap in a browser.No
No steps
Replace & with & in sitemapText and save it as sitemapText.Replace & with & in sitemapText and save it as sitemapText.Take sitemapText split by <loc> from item 2 on and save it as sitemapEntries.Take sitemapText split by <loc> from item 2 on and save it as sitemapEntries.Set tabIsNew to false.Set tabIsNew to false.If a step below fails, save the error as createTabErrorIf a step below fails, save the error as createTabErrorTry
Google SheetsCreate the tab sheetTab in spreadsheetUrl.Create the tab sheetTab in spreadsheetUrl.Change tabIsNew to true.Change tabIsNew to true.On error
Show in the output: Recording this run on the existing '@sheetTab' tab.Show in the output: Recording this run on the existing '@sheetTab' tab.If tabIsNew is yesIf tabIsNew is yesYes
Google SheetsAdd a row to sheetTab in spreadsheetUrl.Add a row to sheetTab in spreadsheetUrl.No
No steps
Start an empty list and save it as alreadySubmitted.Start an empty list and save it as alreadySubmitted.If a step below fails, save the error as historyReadErrorIf a step below fails, save the error as historyReadErrorTry
Google SheetsRead the rows of spreadsheetUrl. Save it as history.Read the rows of spreadsheetUrl. Save it as history.
For each loggedRow in history.rows:For each loggedRow in history.rows: Set loggedKey to blank. Set loggedKey to blank. If a step below fails, save the error as loggedRowError If a step below fails, save the error as loggedRowErrorTry
If loggedRow["Page URL"] isn't blankIf loggedRow["Page URL"] isn't blankYes
Change loggedKey to loggedRow["Page URL"] @ loggedRow["Last modified"].Change loggedKey to loggedRow["Page URL"] @ loggedRow["Last modified"].No
No steps
On error
Show in the output: Skipping a record row that has no 'Page URL' and 'Last modified' value..Show in the output: Skipping a record row that has no 'Page URL' and 'Last modified' value.. If loggedKey isn't empty If loggedKey isn't emptyYes
Add loggedKey to the end of alreadySubmitted and save it as alreadySubmitted.Add loggedKey to the end of alreadySubmitted and save it as alreadySubmitted.No
No steps
On error
Show in the output: Could not read what has been submitted before, so every page in the sitemap is being treated as new. historyReadError.message.Show in the output: Could not read what has been submitted before, so every page in the sitemap is being treated as new. historyReadError.message.Start an empty list and save it as pendingPages.Start an empty list and save it as pendingPages.Start an empty list and save it as pendingKeys.Start an empty list and save it as pendingKeys.Set unchangedCount to 0.Set unchangedCount to 0.For each sitemapEntry in sitemapEntries:For each sitemapEntry in sitemapEntries:If the length of pendingPages is greater than or equal to pageLimitIf the length of pendingPages is greater than or equal to pageLimitYes
Stop repeating.Stop repeating.No
No steps
If sitemapEntry doesn't contain </loc>If sitemapEntry doesn't contain </loc>Yes
Skip to the next item.Skip to the next item.No
No steps
Extract the first web address from the first item of sitemapEntry split by </loc> and save it as pageUrl.Extract the first web address from the first item of sitemapEntry split by </loc> and save it as pageUrl.If pageUrl is emptyIf pageUrl is emptyYes
Skip to the next item.Skip to the next item.No
No steps
Set lastModified to blank.Set lastModified to blank.If sitemapEntry contains <lastmod>If sitemapEntry contains <lastmod>Yes
Trim the first item of the last item of sitemapEntry split by <lastmod> split by </lastmod> and save it as lastModified.Trim the first item of the last item of sitemapEntry split by <lastmod> split by </lastmod> and save it as lastModified.No
No steps
Set pageKey to pageUrl @ lastModified.Set pageKey to pageUrl @ lastModified.If alreadySubmitted contains pageKeyIf alreadySubmitted contains pageKeyYes
Add 1 to unchangedCount.Add 1 to unchangedCount.Skip to the next item.Skip to the next item.No
No steps
If pendingKeys contains pageKeyIf pendingKeys contains pageKeyYes
Skip to the next item.Skip to the next item.No
No steps
Add a record of url and lastModified to the end of pendingPages and save it as pendingPages.Add a record of url and lastModified to the end of pendingPages and save it as pendingPages.Add pageKey to the end of pendingKeys and save it as pendingKeys.Add pageKey to the end of pendingKeys and save it as pendingKeys.If pendingPages is emptyIf pendingPages is emptyYes
Finish the run: Nothing to submit: all unchangedCount page(s) in the sitemap have already been submitted at their current last-modified date.Finish the run: Nothing to submit: all unchangedCount page(s) in the sitemap have already been submitted at their current last-modified date.No
No steps
Start an empty list and save it as submissionUrls.Start an empty list and save it as submissionUrls.For each pendingPage in pendingPages:For each pendingPage in pendingPages:Add pendingPage.url to the end of submissionUrls and save it as submissionUrls.Add pendingPage.url to the end of submissionUrls and save it as submissionUrls.Set submissionStatus to 0.Set submissionStatus to 0.Set submissionAccepted to false.Set submissionAccepted to false.If a step below fails, save the error as submissionErrorIf a step below fails, save the error as submissionErrorTry
Send data to indexNowEndpoint and save it as submission.Send data to indexNowEndpoint and save it as submission.Change submissionStatus to submission.status.Change submissionStatus to submission.status.Change submissionAccepted to submission.ok.Change submissionAccepted to submission.ok.On error
Stop and report a problem: Could not reach the search engine indexing endpoint. submissionError.message with indexNowEndpoint replaced by the indexing endpoint Nothing was recorded, so the next run will offer these pages again.Stop and report a problem: Could not reach the search engine indexing endpoint. submissionError.message with indexNowEndpoint replaced by the indexing endpoint Nothing was recorded, so the next run will offer these pages again.If submissionAccepted is noIf submissionAccepted is noYes
Stop and report a problem: The indexing endpoint refused the submission (HTTP submissionStatus). Check that keyLocation opens in a browser and contains exactly the key from the 'indexNowKey' step. Nothing was recorded, so the next run will offer these pages again.Stop and report a problem: The indexing endpoint refused the submission (HTTP submissionStatus). Check that keyLocation opens in a browser and contains exactly the key from the 'indexNowKey' step. Nothing was recorded, so the next run will offer these pages again.No
No steps
Get today's date and save it as submittedOn.Get today's date and save it as submittedOn.Set submissionResult to Accepted (HTTP submissionStatus).Set submissionResult to Accepted (HTTP submissionStatus).For each submittedPage in pendingPages:For each submittedPage in pendingPages:If a step below fails, save the error as recordErrorIf a step below fails, save the error as recordErrorTry
Google SheetsAdd a row to sheetTab in spreadsheetUrl.Add a row to sheetTab in spreadsheetUrl.On error
Show in the output: Submitted but could not record submittedPage.url, so the next run will offer it again. recordError.message.Show in the output: Submitted but could not record submittedPage.url, so the next run will offer it again. recordError.message.Show in the output: Submitted the length of pendingPages new or changed page(s) for indexing and left unchangedCount unchanged page(s) alone.Show in the output: Submitted the length of pendingPages new or changed page(s) for indexing and left unchangedCount unchanged page(s) alone.