I had asked Google to index one page three times. Each time Search Console accepted the request, and each time the page came back as "URL is not on Google." It wasn't blocked. It answered 200, had a self-referencing canonical tag, sat in the sitemap, and had links pointing at it from the home page and three other pages. On October 8, 2026, I found out it had been on Google the whole time, at a different address: the same path without .html. Google had indexed /page and was skipping /page.html as a duplicate of it.
The canonical tag said .html. The sitemap said .html. The hreflang tags said .html. My templates were inconsistent, so I fixed every internal link on that site to use .html, about 1,900 of them, and deployed a preview. Then I opened view source on the preview. Every link said /page. Not one .html left.
The setting that did that is called Pretty URLs. It's on by default for Netlify sites, and it rewrites the links inside the HTML Netlify serves, after you upload it. It doesn't touch your canonical tags, your hreflang tags or your sitemap. So a site that declares .html addresses everywhere ends up telling Google two different things, and Google picks one. On that site it picked the wrong one for eleven pages, and 42 other pages sat under "Discovered, currently not indexed" with no crawl date at all.
This post is the general version: what the setting actually does, when it's the right choice, when it quietly splits a site, how Cloudflare and Vercel handle the same question, and what Google, Bing and AI crawlers each see.
Fix guides, one per host. Each is a plain Markdown file you can work through yourself or hand to an AI coding agent: diagnose, decide, fix, verify. Netlify (save), Cloudflare Pages and Workers (save), Vercel (save). Free, no signup.
What Pretty URLs actually does
Netlify's docs describe it two ways. The post processing overview says that with it on, "Netlify rewrites /about to /about/ or /about.html to /about/" (Netlify). The redirect options page says it is "enabled by default for sites" and that it "forwards paths like /about to /about/" (Netlify). The checkbox in the Netlify UI, under Project configuration > Developer settings > Post processing > Pretty URLs, describes the link rewriting differently again: /about.html becomes /about, and /about/index.html becomes /about/.
None of those sentences tells you the part that matters for search: which addresses answer, and what your served HTML says. So I measured it on live Netlify sites between October 8 and 10, 2026.
With the setting on:
- A root relative link written as
<a href="/about.html">in the file I uploaded was served as<a href='/about'>. The quotes change from double to single because Netlify re-serializes the tag. That makes it easy to spot: if your templates use double quotes and view source shows single ones, Netlify rewrote the link. <link rel="canonical">and<link rel="alternate" hreflang>tags were not touched. They keep whatever form you wrote.- On a flat site, where the file is
about.html, both/aboutand/about.htmlanswered 200 with the same page./about/answered a 301 to/about. - On a folder site, where the file is
about/index.html,/aboutanswered a 301 to/about/, and/about.htmlanswered a 301 to/about/as well. /about/index.htmland/index.htmlanswered 200.
With the setting off:
- Links were served exactly as written.
/aboutstill answered 200 fromabout.html. Turning the setting off does not remove the extensionless address. I hadn't expected that, and it's why redirect rules are part of the fix, not an optional extra.- A folder page answered 200 at both
/blogand/blog/, because nothing was adding the slash anymore.
Two more facts trip people up. The setting only changes what's live at your next deploy: "Enabling post processing features may require a redeploy before changes take effect." And it survives settings you'd expect to stop it. My netlify.toml had skip_processing = true under [build.processing], and Pretty URLs still rewrote every link on a CLI deploy. That key comes from Netlify's old asset optimization feature, which bundled and minified CSS and JavaScript and compressed images. It reached end of service on October 17, 2023, and Netlify said any related netlify.toml settings "will be ignored" (Netlify). The setting that does control link rewriting today is [build.processing.html] with pretty_urls, and Netlify's file based configuration page says it overrides the UI (Netlify).
One last trap: Netlify's list of deployed files shows the bytes you uploaded, before post processing. If you compare those files against your build output, everything matches. Compare the served HTML instead.
Why Google ends up with the wrong copy
Google lists its canonical signals "in order of how strongly they can influence canonicalization": redirects are "a strong signal", rel="canonical" is "a strong signal", and sitemap inclusion is "a weak signal." On the same page it asks site owners to "link to the canonical URL rather than a duplicate URL," because "Linking consistently to the URL that you consider to be canonical helps Google understand your preference," and it says not to "specify different URLs as canonical for the same page using different canonicalization techniques" (Google Search Central).
Put a flat site with Pretty URLs on through that list. There's no redirect between /about and /about.html, since both answer 200, so the strongest signal is missing. The canonical tag says .html. The sitemap, the weak one, says .html. Every internal link Google crawls says /about. And Google is clear that the canonical tag is "a hint, not a rule" (Google Search Central). It can and did go the other way.
The damage isn't just one wrong URL in the index. Google says "The canonical page will be crawled most regularly; duplicates are crawled less frequently." When Google picks the extensionless copy, the address you put in your sitemap becomes the duplicate, and the pages you most wanted crawled get crawled least. On my site, the Pages report showed exactly that shape: eleven extensionless copies indexed, and a long tail of .html pages "found by Google, but not crawled yet," which is how Search Console describes Discovered, currently not indexed (Search Console Help).
hreflang makes it worse. Google wants alternates as fully qualified URLs, and "If two pages don't both point to each other, the tags will be ignored" (Google Search Central). Since Pretty URLs leaves hreflang tags alone and rewrites the language switcher links in the page body, your language versions point at each other in one form while every visible link uses the other.
When Pretty URLs is the right setting
Leave it on when your site is built from folder pages, about/index.html, and every canonical, sitemap entry and hreflang tag already ends in a slash. That's the common case for generators. Eleventy writes subdir/template.md to _site/subdir/template/index.html by default (Eleventy). Hugo generates "pretty URLs" by default (Hugo). Astro's build.format defaults to 'directory', which "will generate a directory with a nested index.html file for each page" (Astro). Jekyll's permalink: pretty and Next.js's trailingSlash: true with a static export produce the same shape (Jekyll, Next.js).
On a folder site Pretty URLs does the one job you can't do yourself. Netlify matches redirect rules "regardless of whether or not they contain a trailing slash," and "You cannot use a redirect rule to add or remove a trailing slash" (Netlify). A rule from /about to /about/ loops; I wrote about that trap in an older post on force = true. Pretty URLs answers /about with a single 301 to /about/, and the link rewriting does nothing harmful because your links were already in that form. This site works that way and keeps the setting on, and so does the one folder style site among the ones I fixed.
The only loose end on a folder site is /about/index.html, which answered 200 on every Netlify site I checked, with the setting on or off. Its canonical tag points at /about/, nothing links to it, and Google has no reason to prefer it. Leave it alone.
When it splits a site
Turn it off, or at least check, when your site is made of flat .html files and your canonical tags say so. That covers hand written HTML, most static sites exported from design tools, Jekyll pages (about.md becomes /about.html (Jekyll)) and Jekyll posts under the default permalink: date style (Jekyll), Hugo with uglyURLs, Astro with build.format: 'file', and a Next.js static export with default settings, which writes the /me route as /me.html (Next.js).
The warning signs, in the order I'd check them:
- View source shows
href='/about'in single quotes where your templates wrotehref="/about.html". curl -sI https://example.com/aboutandcurl -sI https://example.com/about.htmlboth answer 200.- In Search Console, URL Inspection on a sitemap address shows a Google-selected canonical without
.html, or the Pages report lists "Duplicate, Google chose different canonical than user." - A sitemap full of pages that stay in Discovered, currently not indexed for weeks.
While writing this, I ran the same check against a different site of mine that still has the setting on. It has the same split today: its canonicals and sitemap say .html, and the links on its home page say otherwise.
The fix I used, and what it cost
For the flat sites, the fix had four parts, all in one deploy:
- Pretty URLs off, in the UI. Only that setting. Form detection is separate, and the sites' forms needed it.
- Every internal link in the source written as
.html, including the links JavaScript builds for menus and language switchers. - A forced 301 from every extensionless address to its
.htmlpage. In_redirectsthat's/about /about.html 301!, and innetlify.tomlit'sforce = true. Forcing matters because Netlify won't fire a rule when a file already answers at that path, which it calls shadowing (Netlify). Because rule matching ignores the trailing slash, the same rule also catches/about/. I generated the rules from the build output so no page was missed. - A small edge function for folder pages, because with Pretty URLs off nothing answered
/blogwith a redirect to/blog/, and a redirect rule can't add a slash. It answers the slashless path with a 301 and passes everything else through.
Then I checked every page on a draft deploy before production: one 200 per page, one hop for every other spelling, served HTML identical to the uploaded file. One more thing turned up there. With Pretty URLs off, Netlify's form detection still rewrote each detected form tag and served its action without .html. A 301 on that path would turn a no JavaScript form submission into a GET and lose it, so the form's thank you page kept a 200 rewrite at its extensionless path instead of a redirect.
After the deploy I resubmitted the sitemaps in Search Console and sent IndexNow both the canonical pages and the old addresses that now redirect. As I write this, Google hasn't recrawled most of them. What should happen over a week or two is that the extensionless copies move to "Page with redirect," which Search Console defines as "a non-canonical URL that redirects to another page. As such, this URL will not be indexed." That status is the fix working, not a new problem. Don't click Validate fix on it; those addresses will redirect forever, and in my experience that validation can't pass.
If your site is flat and you'd rather have clean addresses, don't fight Netlify for them. Change what your generator writes so each page is a folder page, leave Pretty URLs on, and make every tag use the slash form. On a folder site I measured, /about.html answered a 301 to /about/ when only the folder page existed, so old flat addresses kept working. The Netlify guide walks through all three paths, including the mixed case.
What Bing, IndexNow and AI crawlers see
Bing. Bing reads the same served HTML and the same sitemap. Its webmaster blog calls lastmod "a key signal, helping Bing prioritize URLs for recrawling and reindexing," and recommends "combining sitemaps for comprehensive site coverage with IndexNow for fast, URL-level submission" (Bing Webmaster Blog, July 31, 2025). If you change URL forms, change the lastmod of every page whose tags or links changed.
IndexNow. One submission reaches every participating engine. As of October 10, 2026, the list at indexnow.org names Bing, Yandex, Seznam, Naver, Yep, the Internet Archive and Amazonbot (IndexNow). Google isn't on it, so for Google the sitemap, links and redirects are still the whole message. The IndexNow FAQ says "You should submit redirected URLs" too, so after a fix like this one, send the old spellings along with the new canonical pages (IndexNow). The IndexNow Submission Audit checks the key file and the submission before you rely on it.
AI crawlers. OpenAI documents OAI-SearchBot as the crawler "used to surface websites in search results in ChatGPT's search features" (OpenAI), and Perplexity documents PerplexityBot as "designed to surface and link websites in search results on Perplexity" (Perplexity). Neither page says anything about canonical tags, redirects or sitemaps, as of October 10, 2026. What they do get is the HTML your host serves, links included, so on a Netlify site with Pretty URLs on, they see the rewritten links. With nothing documented about how they choose between two addresses, I assume they keep whichever one they fetched, which is another reason to make the links, the canonical and the sitemap say the same thing.
Agents and fetch tools. A browser agent or a fetch tool working for a person follows the links on the page. If those links point at an address that redirects, every hop costs it a request, and if they point at a second 200 copy, that copy is the address it reports back. None of this needs special handling. It needs one address per page.
Cloudflare and Vercel answer the same question differently
Of the three, Netlify is the only one I found that edits your HTML. Cloudflare and Vercel change which addresses answer and redirect, but the links on your pages are whatever your build wrote. I found nothing in either company's documentation describing link rewriting, and on a Cloudflare Pages project I checked on October 10, the served HTML matched the deployed file byte for byte.
Cloudflare Pages always strips .html. Its docs: "Pages will also redirect HTML pages to their extension-less counterparts: for instance, /contact.html will be redirected to /contact, and /about/index.html will be redirected to /about/" (Cloudflare). On two Pages projects I checked, those redirects were 308s, which Google treats as permanent, and I found no documented switch to turn the behavior off. So on Pages the trouble is the reverse of Netlify's: if your generator prints .html in the canonical tag and sitemap, every address you declare is a redirect. The fix is to print the form Pages serves. Pages _redirects rules default to a 302, so always write the status, and a file is limited to 2,000 static and 100 dynamic redirects (Cloudflare).
Cloudflare Workers static assets let you choose with assets.html_handling: auto-trailing-slash (the default), force-trailing-slash, drop-trailing-slash or none. Cloudflare's own tables show the built-in redirects as 307s (Cloudflare). Google's redirect docs class 307 as temporary: "the indexing pipeline doesn't use the redirect as a signal that the redirect target should be canonical" (Google Search Central). Your tags and links have to carry the canonical signal, and old addresses that were already indexed deserve real 301s in _redirects.
Vercel leaves both decisions to two vercel.json settings. cleanUrls defaults to false; set to true, Vercel serves about.html at /about and answers /about.html with a 308. trailingSlash defaults to undefined, which means "visiting a path with or without a trailing slash will not redirect," and Vercel's own docs add: "This is not recommended because it could lead to search engines indexing two different pages with duplicate content" (Vercel). Next.js has its own default, redirecting /about/ to /about (Next.js). I haven't measured a Vercel site for this post, so the Vercel guide sticks to what Vercel documents and has you probe your own.
| Netlify | Cloudflare Pages | Workers static assets | Vercel | |
|---|---|---|---|---|
| Edits the links in your HTML | Yes, with Pretty URLs on (the default) | Not documented; none seen on the project I checked | Not documented | Not documented |
/about.html when about.html exists |
200 | 308 to /about |
307 to /about (default mode) |
200, or 308 to /about with cleanUrls |
/about when about.html exists |
200, setting on or off | 200 | 200 (default mode) | 200 with cleanUrls; probe yours without it |
/about for a folder page |
301 to /about/ with the setting on, 200 with it off |
308 to /about/ |
307 to /about/ (default mode) |
200 at both unless trailingSlash is set |
| What to set | Pretty URLs to match your files, plus forced 301s on flat sites | Canonicals and sitemap in the form Pages serves | An html_handling mode, plus 301s for indexed URLs |
cleanUrls and trailingSlash, explicitly |
The Netlify and Cloudflare Pages cells are from my own probes; the rest come from each company's documentation.
Check your own site in ten minutes
- Probe one page. Run
curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n'against/about,/about.html,/about/and/about/index.htmlon your domain. One should answer 200 and the rest should redirect to it in one hop. - Compare view source with the file you deployed. If the links changed between the two, your host rewrote them.
- Read the canonical tag in the served page, then request that exact address. It has to answer 200, not a redirect.
- Open your sitemap and check the same page's
<loc>against the canonical, character for character. - Check hreflang on any page with language versions: absolute URLs, same form, each version listing the others.
- Inspect a sitemap address in Search Console. Compare the user-declared canonical with the Google-selected one in the indexed result. Google notes that "the live test cannot predict whether or not the tested version will be considered canonical" (Search Console Help).
Each guide includes a short read-only Node script that does steps 3 to 5 for every page in your sitemap and flags any link that reaches a second address. On the site of mine that still has the split, it found eleven second addresses linked from the first eight pages in the sitemap. If you'd rather click than type, the Canonical and Redirect Graph, the Redirect Chain Audit, the Sitemap Audit and the hreflang Audit cover the same ground in the browser, free and without a signup.
If you're doing this kind of work on your own business's site rather than paying someone for it, The $20 Dollar Agency is the playbook I wrote for that: getting found in search and in AI answers with cheap tools and your own hands.
The fix guides
- Netlify: clean-urls-fix-netlify.md (save). Pretty URLs on or off by site type, forced 301s, the folder slash edge function, the forms caveat.
- Cloudflare Pages and Workers static assets: clean-urls-fix-cloudflare.md (save). Matching your tags to what Pages serves, choosing an
html_handlingmode, replacing 307s with 301s. - Vercel: clean-urls-fix-vercel.md (save). Setting
cleanUrlsandtrailingSlashon purpose, Next.js, and redirects within the 2,048 limit.
Related reading
- Netlify force = true and the redirect loop you ship by accident
- Which URL is the real one? Off-domain canonicals and sitemaps with two homepages
- Circular canonicals and canonical cluster mapping
- Sitemap delta: the URLs you link to that are not in your sitemap, and the reverse
- Why hreflang has to point both ways
Fact-check notes and sources
Every source below was read on October 10, 2026. The behavior I describe as measured comes from my own requests to live Netlify and Cloudflare Pages sites that I maintain, between October 8 and 10, 2026; hosts change this behavior without notice, so probe your own site. The Search Console figures (eleven extensionless copies indexed, 42 pages discovered and never crawled) are from one of those sites on October 8, 2026. "Not documented" and "says nothing about" statements are absence claims as of October 10, 2026.
- Source: https://docs.netlify.com/build/post-processing/overview/ (last updated September 17, 2026) establishes the post processing settings location, the Pretty URLs description and the redeploy note.
- Source: https://docs.netlify.com/manage/routing/redirects/redirect-options/ establishes that Pretty URLs is enabled by default, that rules match with or without a trailing slash, that a rule cannot add or remove a trailing slash, and the infinite redirect example.
- Source: https://docs.netlify.com/manage/routing/redirects/rewrites-proxies/#shadowing establishes shadowing and the
!override. - Source: https://docs.netlify.com/build/configure-builds/file-based-configuration/ establishes
[build.processing.html] pretty_urlsand thatnetlify.tomlsettings override the UI. - Source: https://www.netlify.com/blog/deprecation-of-post-processing-asset-optimization-feature/ establishes the July 17, 2023 deprecation, the October 17, 2023 end of service and that related
netlify.tomlsettings are ignored. - Source: https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls (last updated July 10, 2026) establishes the signal order, linking to the canonical URL, and not mixing canonicals across methods.
- Source: https://developers.google.com/search/docs/crawling-indexing/canonicalization (last updated August 20, 2026) establishes "a hint, not a rule" and that duplicates are crawled less often than canonicals.
- Source: https://developers.google.com/search/docs/crawling-indexing/301-redirects establishes that 301 and 308 are permanent and that 302, 303 and 307 are temporary and not used as a canonical signal.
- Source: https://developers.google.com/search/docs/specialty/international/localized-versions (last updated September 21, 2026) establishes fully qualified hreflang URLs and that one way hreflang is ignored.
- Source: https://support.google.com/webmasters/answer/7440203 establishes the definitions of Discovered, currently not indexed; Duplicate, Google chose different canonical than user; and Page with redirect.
- Source: https://support.google.com/webmasters/answer/9012289 establishes the user-declared and Google-selected canonical fields and that the live test cannot predict canonical selection.
- Source: https://blogs.bing.com/webmaster/July-2025/Keeping-Content-Discoverable-with-Sitemaps-in-AI-Powered-Search (July 31, 2025) establishes Bing's description of lastmod and its pairing of sitemaps with IndexNow.
- Source: https://www.indexnow.org/searchengines.json establishes the participating engines; https://www.indexnow.org/faq establishes that redirected URLs should be submitted and that each engine decides on its own whether to index.
- Source: https://developers.openai.com/api/docs/bots and https://docs.perplexity.ai/guides/bots establish the purposes of OAI-SearchBot and PerplexityBot and contain nothing about canonical tags, redirects or sitemaps.
- Source: https://developers.cloudflare.com/pages/configuration/serving-pages/ establishes Pages route matching and its
.htmland/index.htmlredirects; https://developers.cloudflare.com/pages/configuration/redirects/ establishes the 302 default, the limits and the Functions caveat; https://developers.cloudflare.com/workers/static-assets/routing/advanced/html-handling/ establishes the fourhtml_handlingmodes and their 307 redirects. - Source: https://vercel.com/docs/project-configuration/vercel-json (last updated August 14, 2026) establishes the
cleanUrlsandtrailingSlashdefaults and behavior, Vercel's duplicate content warning, the 308 default for redirects and the filesystem precedence for rewrites; https://vercel.com/docs/routing/redirects/configuration-redirects establishes the 2,048 redirect limit. - Source: https://www.11ty.dev/docs/permalinks/, https://gohugo.io/configuration/ugly-urls/, https://docs.astro.build/en/reference/configuration-reference/, https://jekyllrb.com/docs/permalinks/, https://jekyllrb.com/docs/pages/, https://jekyllrb.com/docs/configuration/default/, https://nextjs.org/docs/app/guides/static-exports and https://nextjs.org/docs/app/api-reference/config/next-config-js/trailingSlash establish each generator's default output form and the setting that changes it.
This post is informational, not SEO consulting advice. Mentions of Netlify, Cloudflare, Vercel, Google, Microsoft Bing, OpenAI, Perplexity and the static site generators named here are nominative fair use. No affiliation is implied.