How Often Does Google Crawl a Site? 14 Days of Our Logs

Googlebot came to each of the three content sites we run every day, but not to every page. It came back to a typical page every 1.8 days on one site and about every 4 days on the other two, across the 14 days to October 22, 2026. In those two weeks it never asked for 7%, 22% and 36% of the pages in their sitemaps.

This post shows how we measured it from our server log, which pages Google came back to least, what a brand-new site looked like, and how to check the numbers for your own site.

What Google says about crawl frequency

Google doesn't publish a crawl interval. Its crawl budget guide describes two limits. The first is the crawl capacity limit: "Google wants to crawl your site without overwhelming your servers," so it goes up when the site "responds consistently" and down when it slows or returns 5xx errors. The second is demand, which for Googlebot "varies based on a site's size, update frequency, page quality, and relevance, compared to other sites." Pages that are "more popular on the Internet tend to be crawled more often," and Google's systems "want to recrawl documents frequently enough to pick up any changes."

Two numbers from Google set the scale. Its Googlebot page says that "for most sites, Googlebot shouldn't access your site more than once every few seconds on average." And when you ask for a recrawl, "crawling can take anywhere from a few days to a few weeks."

The crawl budget guide also says most sites can stop there: if your pages "seem to be crawled the same day that they are published," keeping the sitemap current and checking the Page Indexing report "is adequate."

How we measured it

Our eight sites share one server and one nginx access log, which has no host name field. We took every request from October 8, 00:00 UTC, to October 22, 04:51 UTC (14.2 days) whose user agent named Googlebot and kept only those from an address in Google's published list for its common crawlers. Then we credited a request to a site only when its path appeared in that site's sitemap and in no other site's (58 shared paths, such as the home page and /about, were left out).

We counted page fetches from Googlebot Smartphone and Googlebot Desktop, not Googlebot-Image. Two fetches of the same page less than a minute apart count as one visit. For each page with two or more visits we took the median gap between them, then the median of those across the site.

We set the log against two things Google gave us: each page's impressions in Search Console over the 28 days to October 19, and the last crawl date URL Inspection reported on October 20.

Our three sites: every 1.8 to 4 days

On the card game site the median page was fetched five times in two weeks, and 207 pages were fetched eight times or more. On the manhwa site the median page was fetched once, and 208 pages weren't fetched at all; 23 of those carry a noindex tag.

URL Inspection agreed. On October 20 the median page on the card game site had last been crawled 2.7 days earlier, against 6.2 days on the game news site (557 of its pages inspected) and 5.2 on the manhwa site. The long tail is where the sites differ most: 10 card game pages had gone more than 60 days without a crawl, against 27 on the game news site and 48 on the manhwa site, where the oldest crawl date was 157 days back.

Most fetches came from the phone crawler, as Google says they should for most sites: 83% to 93% of page fetches on the three sites were Googlebot Smartphone.

Pages without impressions get crawled least

We grouped each site's sitemap pages by their impressions in Search Console and counted how many Googlebot fetched at least once in the 14 days. We left out the pages URL Inspection reported as noindex on October 20, because a noindex page can't show in search. On the manhwa and card game sites those were 41 pages, and Google fetched 11 of them in the 14 days. Both sitemaps still listed them on Oct 22, a mistake of ours: noindex pages don't belong in a sitemap. Kept in, they'd make the first row look worse than it is.

On every site, pages that never showed in search were the ones Google left alone. Above the first hundred impressions the share barely moves: 69% against 73% on the manhwa site, 85% against 84% on the game news site, 97% on both rows of the card game site. That fits Google's description of demand, but it's a correlation from one fortnight, and a page with no impressions may also be one Google crawled less in the first place.

Traffic and publishing didn't set the pace

The card game site was crawled most often, and it had by far the least search traffic. In the 28 days to October 19 it had 316,507 impressions, against 1.6 million on the manhwa site and 1.8 million on the game news site. It wasn't publishing more either: the three sites added 20 to 25 articles each in the week of October 5, and in the week of October 12 the card game site added one, the game news site 22 and the manhwa site none.

So neither a site's search traffic nor its publishing pace explains the difference on our sites. All three run on the same server, so the capacity side should be similar too; the server answered 99% of the verified page fetches with a 200 or a 304. We don't know what drives the card game site's higher demand. Its 245 card pages got a median of six visits each in the two weeks, so that page type may be part of it.

A new site: our first eight days

elmrank.com went live on October 14, 2026, and we've logged every Googlebot fetch since. Googlebot found 22 of the 25 launch posts that night, after we submitted the sitemap and before any of our Request Indexing tries got through (the first day is in how long it takes Google to index a new page). The next morning, between about 10:10 and noon Vietnam time, we sent all 25 through Request Indexing, and Googlebot fetched 21 of them between 10:22 and 11:48. For 18 posts that was the second visit, so it measures how fast Google answered our request, not when it would have come back by itself. From October 20, new URLs also went out through IndexNow.

Google's own return rate starts after that batch. In the seven days left before our log ends on October 22, 12 of those 21 posts got another visit, a median of 1.8 days later and up to 6.7 days, and the other nine hadn't been fetched again when the log ends. Count those nine as still waiting and the middle post of the 21 waited about six and a half days, so 1.8 days is the flattering figure. Left alone, the new posts fell into the same multi-day rhythm as our older sites. Our fixed pages saw even less of Googlebot: 10 of 14, including the three service pages and the six category pages, were fetched once in eight days. Of the 18 posts published from October 15 to 21, four have had a second visit so far. Every one of the 111 page fetches came from the phone crawler.

One request in five that said Googlebot wasn't

Across all eight sites, 36,414 requests in the 14 days came from verified Googlebot addresses, between 2,081 and 3,120 a day. Another 8,416 used Googlebot's name from addresses outside Google's list, about one in five of everything that claimed to be Googlebot. If you count crawl frequency from a log by user agent alone, you'll overcount.

Google's own advice is the check we used: "match the source IP against the Googlebot IP ranges," or run a reverse DNS lookup and confirm the name ends in googlebot.com, google.com or googleusercontent.com. One detail for scripts: the old googlebot.json address now answers with a 301 redirect to common-crawlers.json, so a script that doesn't follow redirects gets a redirect page instead of the list.

How to check how often Google crawls your site

1. 1Crawl Stats reportSearch Console, Settings: requests per day, Refresh vs Discovery, response codes
   →
2. 2URL InspectionLast crawl date for one page; the API returns lastCrawlTime
   →
3. 3Your server logVerified Googlebot only, visits per page, gaps between them
   →
4. 4Set it against impressionsPages with none are the ones Google skips first

The Crawl Stats report splits requests by purpose: "Discovery: The URL requested was never crawled by Google before" and "Refresh: A recrawl of a known page." For a site that has stopped growing, Refresh is the number that answers this question. Google adds that the report's counts can differ slightly from your own request logs.

If a page you care about has gone weeks without a visit, start with what Google's crawl budget guide asks for: keep the page in the sitemap, with a lastmod when its content changes, answer quickly, and support 304 Not Modified so a recrawl of an unchanged page costs little. Take the URLs you don't want crawled out of Googlebot's way. We also link such a page from pages that get crawled often. For one page you've just changed, Request Indexing is the direct route. Pages Google has found but not fetched at all have their own status, covered in Discovered - currently not indexed.

Where crawling sits in our plan

The first month of our technical SEO service includes an audit of crawling and indexing, and we make the fixes ourselves, in your code. The plan is $500 for the first month and $300 a month after, and the technical SEO audit checklist shows the checks we start with. To see what we'd fix first on your site, send it to us: the list is free, and we reply within 12 hours.

Questions

Can I make Google crawl my site more often?

Not with a setting. Google's guides describe how to reduce Googlebot's crawl rate and what raises demand, but no setting that raises it. Google says demand comes from a site's size, update frequency, page quality and relevance, and that you help most by keeping unwanted URLs out of its way, keeping the sitemap current and answering fast. For one page you changed, Request Indexing in URL Inspection asks for a recrawl, and Google says crawling can take a few days to a few weeks.

Is it a problem if Google hasn't crawled a page in weeks?

Not by itself. On October 20, 48 of the 586 pages on our manhwa site that had a crawl date hadn't been crawled for more than 60 days: 19 were indexed anyway, 26 carried a noindex tag and 3 were crawled but not indexed. It matters when you changed the page and want the change in search, or when the page has no impressions at all; on our sites those pages were crawled least.

How do I check when Google last crawled a page?

Open URL Inspection in Search Console and read the last crawl date under Page indexing, or call the URL Inspection API, which returns lastCrawlTime. For the whole site, the Crawl Stats report under Settings shows requests per day and splits them into Refresh (pages Google knew) and Discovery (new ones).

How we know this

Written by Tyler Phan. Every source below was read on the date shown; the page was last reviewed on Oct 25, 2026.

1. Google Crawling Infrastructure, Optimize your crawl budget, accessed Oct 22, 2026
2. Google Search Central, Googlebot, accessed Oct 22, 2026
3. Google Search Central, Ask Google to recrawl your URLs, accessed Oct 22, 2026
4. Google Crawling Infrastructure, Verify requests from Google crawlers and fetchers, accessed Oct 22, 2026
5. Google, common crawlers IP ranges (common-crawlers.json), accessed Oct 22, 2026
6. Search Console Help, Crawl Stats report, accessed Oct 22, 2026
7. Elmrank, our VPS access log (eight sites), October 8 to 22, 2026, with the sites' live sitemaps, accessed Oct 22, 2026
8. Elmrank, Search Console page data and URL Inspection results for the three content sites we run, pulled October 20 and 22, 2026, accessed Oct 22, 2026

Send us your site.

We reply within 12 hours with what we'd fix first. Then we run it for you, technical SEO and content: $500 for the first month, then $300 a month for up to 30 articles. You don't need any technical knowledge, and you can cancel any time.

Get my free first-fix listSee what's included