How Much of My Website Traffic Is Bots? How to Tell and What to Do

VenbitTony CarnevaleOctober 5, 20266 min read

The short answer

There is no fixed share. A host log counts every request, so it runs high. A script tracker misses crawlers that never run its script, so it runs lower. Venbit drops automation before anything is counted and names the crawlers it caught. On the demo site, Meta AI read 90 pages, none of them in the visit count.

Key takeaways

  • Google's own help says GA4 excludes known bots automatically and the exclusion cannot be turned off. A crawler not on that list, or a script posing as a browser, is counted.
  • Venbit detects bots by user agent, by an empty user agent, and by datacenter cities such as Ashburn, then hides them from every number by default.
  • Googlebot and Bingbot are the index, GPTBot and ClaudeBot are how an assistant learns to quote you, and Ahrefs or Semrush bots are a competitor reading your pages.
  • A spike is judged by what the visits did. One page, zero seconds, no scroll, no click, all from one town, is a script.
  • Direct means the visit arrived with no referrer, which is what a script, a pasted link and a bookmark all look like.
  • Your own browser is a visit too. Settings has a switch to stop counting it.

Nobody can give you a percentage for your website without looking at your website, and the tool you look with decides most of the number. Your host's stats page counts every request that hit the server, crawlers included, so it runs high. A tracker that runs as a script only sees browsers that run scripts, and a crawler that reads the text and leaves does not, so it runs lower. Venbit drops the automation it does see before anything is counted, which is why our visit counts run lower than the estimator tools claim.

On the demo site, WebsiteMaintenance.com, the dashboard counted 347 visits in the last 30 days, every one of them a person. The AI search section under it shows Meta AI read 90 pages and other bots read 18 in the same window, and none of those reads is in the 347. Any site's answer has that shape, a human count with the bot count kept beside it.

Three counts for one website

The host log is the biggest number because it is the most literal. Every request for a page, an image, a stylesheet or a feed is a line in that log, whether a person or a script asked for it. Uptime monitors hit the home page around the clock. A host's stats page adds those lines up and calls them visits.

A script tracker is smaller. GPTBot and ClaudeBot read the page's text and leave without running its scripts, which the demo site's AI search section says in a note under the crawler list. Those reads never reach a script-based tool at all. Google's own help says GA4 excludes traffic from known bots and spiders automatically and that the exclusion cannot be turned off, so a GA4 count is a filtered one before you touch it.

The third count is the one I built Venbit analytics to show: visits from people, with the bots recorded but hidden from every number by default. You can open the bots when you want to know who read you. They stay out of the chart you grow.

How bot traffic gets filtered out

The definitions page in the portal states the rule in one sentence. Crawlers, scanners and datacenter traffic, detected by user agent, by empty user agents, and by known cloud-region cities such as Ashburn. Bots are recorded but hidden from every number by default.

The user agent is the string a browser or a script sends to say what it is. Honest crawlers announce themselves in it, Google's as Googlebot and OpenAI's as GPTBot, and the tracker works out the name at the moment the visit is recorded and stores it on the visit, because the user agent itself is not kept. Scripts that send no user agent at all are dropped for that silence.

Datacenter cities are the third test. A person's connection resolves to roughly where they live. A cloud server's connection resolves to the town the datacenter sits in, and Ashburn, Virginia, is the best known of them. City is read from the IP address and is approximate, so a datacenter town is only a suspicion. A real person browsing through a VPN that exits in one starts out hidden with the machines, and the flag is withdrawn the moment they send a form. A crawler that names itself is never let back in.

One more case matters for the inbox. A headless browser filling out your contact form can look clean at the moment it submits, then give itself away on the thank-you page twenty seconds later. The bot flag lands after the submit, and the lead leaves the inbox with it.

The AI search section of the Venbit dashboard: ChatGPT sent 26 visits and 0 leads, Meta AI read 90 pages, other bots read 18 pages, with a note that GPTBot and ClaudeBot read pages without running scripts.
The AI search section on the demo site. The 26 ChatGPT visits are people who clicked out of an answer. The 90 and the 18 are crawler reads, counted on their own and kept out of every visit figure.

AI crawlers and the other bots worth knowing by name

Half of the bots on a small business site matter, and the names tell you which half. Googlebot and Bingbot are the index: if they stop coming, you stop appearing. An OpenAI or Anthropic crawler reading your pages is how your site gets quoted in an assistant's answer, which on the demo site turned into 26 visits from ChatGPT last month and no leads yet. Ahrefs and Semrush bots are a competitor's tool reading your pages. A bare curl request is somebody's script.

The AI search section lists the AI crawlers with the pages each one read, next to the people each assistant sent and whether any of those people became a lead, so the reading and what it turned into sit on one screen.

A sudden spike from one country

The worry that sends most people to this question is a spike, a Tuesday with five times the usual visits, all from a country you do not sell to. Whether that was a script or a crowd is written in the rows under the number.

Open the visitor trails list for that day. Automation lands on one page, stays zero seconds, scrolls nowhere, clicks nothing and leaves, and a hundred rows like that from one town is a script. People read for a while, move between pages, and a few of them reach the form. The engagement bar on each row is the quickest tell.

Check the referrer column too. Those rows will say direct, and direct does not mean somebody typed your address. It means the visit arrived with no referrer, which is what a script, a pasted link and a bookmark all look like.

On the demo site the visitor map says 336 of 347 visits came from the United States, where the business sells. A spike that moved that line toward a country the business has never served deserves a look. Settings has an alert for exactly this, a traffic spike above a number of visits in an hour that you choose, so you hear about it in the hour it happens. If you would rather watch it live, the Live screen shows who is on the site right now, with the page each person is reading.

A spike is one half of the worry. The drop that is the other half has its own list of causes, and most of them have nothing to do with bots.

The Venbit Analytics dashboard for the last 30 days: 347 visits up 173 percent, 12 leads, 295 visitors, 576 page views, engaged time 51 seconds, bounce rate 54 percent, conversion 3.5 percent, each with the previous period beside it, and a traffic sources donut with Ads 221, Direct 70, AI Assistants 26, Social 24 and Search 5.
The demo site's dashboard. Every number here is a person. Crawler reads were dropped before the 347 was added up, so the rise against the previous period is growth in people.

Visitors but zero engagement

The quieter version of the worry is a count that looks fine with nothing behind it. Visits, but no scrolls, no clicks, no forms. Two definitions settle most of these.

Engaged time counts only while the tab is visible and focused, so a tab left open in the background adds nothing. A bounce is a visit that saw one page, stayed under ten seconds and did nothing, no scroll, click or lead. On the demo site, engaged time is 51 seconds and the bounce rate is 54 percent, which is a site people read. A site with visits and no engaged time is a site nobody is reading, and if those visits also passed the bot filter, they are either people who left at once or automation the filter did not catch.

The other inflater is you. Every time you open your own site to check a change, that is a visit, and on a quiet site you and your designer are a visible share of them. Settings has a switch for it: stop counting this browser. Turn it on in every browser you use, and ask whoever works on the site to do the same.

The Settings page of the Venbit portal: the one script line that installs the tracker with its allowed domains, a switch to stop counting your own browser, and the business type picker for what counts as a win on this site.
Settings on the demo site. The middle section stops your own browser from being counted, which on a quiet site matters more than any bot.

Real visits without leads is a different problem with a different fix, and that one starts at the funnel, not at the bot filter.

What to do with the number

Judge growth on the human count and nothing else, at your desk or on the iPhone app, which shows the same figure. A chart that includes bots moves when a crawler changes its schedule, and you will spend a morning on a line that means nothing. The demo site's 347 against the previous period is people compared with people, which is the only comparison that tells you about the business.

If your current tool gives you a bigger number than ours will, the two tools disagree about what a visit is and about nothing else. The estimator tools and the host logs are measuring requests. How much traffic a small business site should get has real figures behind it, and all of them are counts of people.

More on reading your website's numbers

Every question in this series, from Website Analytics for Small Business Owners.

Leads inbox4
Ad performance4
SEO tracking3
Visitor behaviour4
Feed and notifications3
Goals and measurement4
Privacy, AI and teams3
Venbit

Tony Carnevale

Founder, Venbit

Tony started Venbit in Mill Creek, WA in 2011 and still does the work: web design, SEO, and the website analytics product the company builds and runs for its own clients. The numbers and screens in his articles come from that product, not from a content mill.

Common questions

Questions, answered straight.

Straight answers about web analytics for your business. If yours isn't here, ask us directly and we'll give it to you straight.

Ask the team

Google's own help says GA4 excludes traffic from known bots and spiders automatically, using a list maintained by the IAB, and that this exclusion cannot be turned off. Known is the operative word. A crawler not on that list, or a script pretending to be a browser, is counted. Any script-based tool, GA4 and ours included, also never sees crawlers that read the page without running scripts.

Because it is counting requests, not people. Every crawler, uptime monitor and scanner that fetches a page is a line in the server log, and the host's stats page adds them up. A script tracker only records browsers that ran the script, and Venbit then drops the automation it did see. The host figure answers a different question.

They are how an assistant learns what your site says, and that learning is what later sends a person who asked ChatGPT for a recommendation. On the demo site, ChatGPT sent 26 visits in 30 days. The crawlers read pages without running scripts, so they cost you nothing in the visit count. What they cost in server load is a question for your host, not your analytics.

One page, zero engaged time, no scroll, no click, no form, and a referrer of direct because a script sends none. One row like that is a person who left fast. Fifty rows like that from one town in one hour, especially a datacenter town such as Ashburn, is automation, and the tracker will have hidden most of them already.

It can try, and some do. The submit itself can look clean, so the judgement does not end there. A headless browser that gives itself away on the thank-you page seconds later is flagged as a bot after the fact, and the lead goes out of the inbox with the flag. People with a pitch are a different problem, and the spam filter in Settings is for them.

Free 30-minute strategy call

Let's talk about your project.

Tell us what you need and we'll give you an honest read on the project, the timeline, and what it takes, before you spend a dollar. Based in Mill Creek, WA, working across the Puget Sound.

4.8 on Google 5.0 on Yelp