> ## Content Index
> Fetch the complete content index at: https://blog.dgactual.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# Build for People. Don't Lock Out Their Assistants.
- URL: https://blog.dgactual.com/build-for-people-dont-lock-out-their-assistants/
- Published: 2026-11-05T21:14:03.000Z
- Updated: 2026-11-05T21:14:03.000Z
- Description: An AI assistant built a real, priced wine cart at one store and got a locked door at another. We ran the same test on 84 franchise dealer websites: 58 turned away a cloud browser, and none of the blocked sites we could check asked AI to stay out.
- Author: Daniel
- Tags: Deep Dive, Market Intelligence

JUMP TO SECTION

[Overview](#overview)[Two Wine Shops](#two-shops)[The Door](#the-door)[Why Now](#why-now)[The Dealer Test](#dealer-test)[Build for People](#build-for-people)[Dealer Desk](#dealer-desk)[Fair Pushback](#fair-pushback)[How We Tested](#how-we-tested)

On Friday I went wine shopping twice with an AI assistant. At Astor Wines it read the shelf, matched bottles to how I actually drink, and filled a real cart: seven bottles, delivery set to Ohio, shipping and tax calculated, $543.07 to the penny, one click short of checkout. At Wine.com the same assistant hit a bot check at the front door and had to work from cached copies of product pages, some of them two weeks old, with a polite note to recheck every price at checkout. Same shopper, same assistant, same day. One store got a ready order. The other got a shopper who might buy somewhere else.

## **Overview**

I've said for years that if a person can't easily find it, read it and do it, AI can't either. Friday proved that rule is half right. Astor's site is plain and a little dated, and it worked perfectly because it explains its products well and lets visitors in. Wine.com is easy for people and still failed, because its firewall stopped the assistant before it saw a single bottle.

So the rule needs a second line: build for people, and don't lock out their assistants. Those are two separate checks, and a store can pass the first while failing the second without knowing it.

We then ran the same test on 84 randomly chosen franchise dealer websites. About 69% of them, 58 stores, turned away a standard Chrome browser running from a cloud server, which is how a growing share of AI shopping agents travel. Only 11 had used-car prices sitting in the page code where any simple reader can find them. Details, caveats and a three-step check you can run on your own store are below.

## **Two Wine Shops**

Start with what made Astor easy. Every item has its own simple web address, the price is written right into the page, and the cart works through ordinary buttons. None of that is built for AI. It's just a website that hands over what it knows to anyone who asks.

Then there's the content, which did the real work. Astor's staff notes read like a clerk talking to you in the aisle. The amontillado note says it's traditionally served before a meal with salty snacks and keeps two to three weeks in the fridge once opened, according to [Astor's product page](https://www.astorwines.com/item/56984?ref=blog.dgactual.com). That's the kind of detail that let the assistant match a bottle to a Negroni drinker who leaves bottles open for weeks. Written for people, useful to machines.

Wine.com is the odd one. Its [robots.txt file](https://www.wine.com/robots.txt?ref=blog.dgactual.com), the public note a website leaves telling automated visitors where they may go, names Perplexity, ChatGPT, Claude and other assistants and tells them they're welcome. Then its bot-protection service, DataDome, answers automated visitors from the cloud with a 403, the web's version of a locked door, and a puzzle page. The welcome mat says come in. The bouncer says not you.

To be fair to Wine.com, the bouncer is doing its job. Bot services exist to stop price copying, fake accounts and card-testing fraud, and they work by spotting traffic that doesn't look like a person at a home computer. An AI agent shopping for you from a cloud server looks a lot like the bad guys.

## **The Door**

This is the part most store owners never see, because their own visits always get through. Who gets in now depends on who's knocking, and the rules sit with whoever runs the website's security, often a vendor setting the store never touched.

Our own checks showed how fine-grained this is. Six dealer homepages that refused our cloud browser opened right up for a different automated reader, the kind AI search tools use to fetch a page and summarize it. So many dealer sites will let an assistant read about a car while stopping an assistant that tries to browse, click and build a deal. Wine.com refused both.

The tools to sort this out already exist. Cloudflare now sorts AI traffic into three buckets: search crawlers that index pages, agents acting in real time for a person, and training crawlers feeding models. Customers can block or allow each one, per [Cloudflare's documentation](https://developers.cloudflare.com/bots/additional-configurations/block-ai-bots/?ref=blog.dgactual.com). DataDome's own 2026 report warns that "businesses that block all automated traffic lose the real value it can create," according to [DataDome](https://datadome.co/threat-research/state-of-bot-and-agent-security-report-2026-key-takeaways/?ref=blog.dgactual.com). The vendor selling the lock is telling you not to lock everything.

Fun fact from the making of this post: when our cloud link checker went to confirm the sources, DataDome's own report pages turned it away, and so did OpenAI's and Cars.com's. A recognized reader got through to all of them earlier the same day. The door really does depend on who's knocking.

## **Why Now**

![Adobe Analytics: AI-referred retail visits convert 90% better and add to cart 53% more often; product pages are 66% readable by AI](https://storage.ghost.io/c/99/c9/99c9791b-bee0-4e16-950a-32c6472afab3/content/images/2026/10/card_adobe.png)

Why care in October? Because the assistant shopper is already the better shopper. In September, visits that reached U.S. retail sites from AI tools converted 90% better than other visits and added items to carts at a 53% higher rate, according to [Adobe Analytics](https://business.adobe.com/blog/ai-traffic-higher-revenue-per-visit-retail-holiday?ref=blog.dgactual.com). A year earlier those visits converted worse. Adobe expects AI traffic on Thanksgiving to run 159% above last year.

The buying is starting to happen inside the assistant, too. The share of U.S. AI users who completed a purchase with AI rose from 10% in December 2025 to 17% in June, per [McKinsey](https://www.mckinsey.com/capabilities/growth-marketing-and-sales/our-insights/who-wins-when-ai-does-the-shopping-new-evidence-on-agentic-commerce?ref=blog.dgactual.com). [Morgan Stanley](https://www.morganstanley.com/insights/articles/agentic-commerce-market-impact-outlook?ref=blog.dgactual.com) puts agent-driven shopping at $190 billion to $385 billion of U.S. online sales by 2030\. OpenAI and Stripe built an open standard so agents can check out on a shopper's behalf, described in [OpenAI's launch post](https://openai.com/index/buy-it-in-chatgpt/?ref=blog.dgactual.com).

The agents are moving to the cloud, which matters for who gets stopped. In September, ChatGPT's cloud browser made up 47% of the agent-browser traffic DataDome tracked, and total agent-browser traffic grew 4.7 times from August to September, per [DataDome's Q3 report](https://datadome.co/threat-research/ai-traffic-report-q3-2026/?ref=blog.dgactual.com). Cloud agents don't come from a home internet connection, so they trip the alarms built to catch bots. Running the assistant at home isn't a guaranteed pass, though. Our Wine.com attempt ran in Perplexity's Comet browser, which browses from the shopper's own device and connection, and DataDome stopped it anyway. DataDome tracks Comet by name in that same report, so a site that wants to stop agents can spot one running on a shopper's own machine.

And the courts are leaning toward the shopper. In August the Ninth Circuit lifted an order that had blocked Perplexity's Comet browser from shopping Amazon, saying it's the user, not the AI company, who accesses the site, according to [Reuters](https://www.reuters.com/business/retail-consumer/amazon-loses-us-court-ban-perplexitys-ai-shopping-tools-2026-08-04/?ref=blog.dgactual.com). The case continues, but walling out agents just got harder to defend as a legal matter.

Car shoppers are on board. In a November Cars.com survey of 936 in-market and recent car shoppers, 44% had used AI car-search tools, per [Cars.com](https://www.cars.com/articles/cars-com-survey-reveals-ais-growing-influence-on-car-shopping-97-of-ai-users-say-it-will-impact-purchase-decisions-and-almost-half-have-already-leveraged-the-tech-for-car-shopping-518967/?ref=blog.dgactual.com). And in February CarMax put more than 45,000 vehicles inside ChatGPT, letting shoppers describe a car in plain English and start a trade offer without leaving the chat, per [CarMax](https://investors.carmax.com/news-and-events/news/news-details/2026/CarMax-Launches-First-of-Its-Kind-Car-Shopping-and-Selling-Experience-in-ChatGPT-App-Store/default.aspx?ref=blog.dgactual.com).

## **The Dealer Test**

So we tested dealers the same way. We drew 90 stores at random from the 18,106 franchise rooftops on our [Dealer Reputation Scoreboard](https://reviews.dgactual.com/?ref=blog.dgactual.com) and found 88 websites. We dropped four: three roster entries that turned out not to be franchise stores, and one site that was down for everyone. That left 84.

![DGActual test of 84 franchise dealer websites: 58 turned away a cloud browser, 11 had used prices in the page code, 0 of 25 blocked sites ask AI to stay out in robots.txt](https://storage.ghost.io/c/99/c9/99c9791b-bee0-4e16-950a-32c6472afab3/content/images/2026/10/card_dealer.png)

Here's what a real Chrome browser running from a cloud server saw on Friday. Of the 84 homepages, 58 answered with a block page instead of the store. That's about 69%. Of those, 31 were plain "Access Denied" pages and 26 were Cloudflare "just a moment" puzzles that didn't clear during our test. Nearly identical block pages showed up across unrelated stores, which points to website-platform settings rather than 58 separate decisions.

Now the twist. We could read the robots.txt file for 25 of those 58 blocked stores, and none of them asked AI tools to stay out. Across all 84 sites, only two robots files shut out an AI crawler, and both of those stores let our browser in anyway. Nearly every dealer we could check is saying come on in with the welcome mat while the bouncer turns people away.

Of the 26 stores that opened, 11 had used-car prices written right into the page code. Another 13 showed prices only after the page ran its scripts, which a full browser handles fine but a quick reader skips. Two we couldn't classify.

## **Build for People**

The other half of the rule is the one dealers already know: a page has to be good for a person first. Adobe scored retail product pages at 66% readable by machines, meaning about a third of what's on a typical product page is invisible to AI, worse than homepages at 75%, per [Adobe's April report](https://business.adobe.com/blog/ai-traffic-surge-retail-sites-not-machine-readable?ref=blog.dgactual.com). The page that matters most for a sale scored lowest.

Think about what Astor's note did for the cognac: age, proof, cru, how much was made, how it drinks. Now picture a used VDP, the vehicle detail page, that says "call for price," lists no reconditioning, and has photos with no words. A person can't decide from that page, so an assistant can't either. It'll go find a page it can read, and that's often a marketplace listing, not yours.

## **Dealer Desk**

Three checks, about ten minutes, no AI budget required.

First, the person test. Open one used VDP on your phone and ask whether a stranger could decide from it: real price, mileage, stock number, what you fixed, what's still wrong. If not, fix that before anything else.

Second, the door test. Ask an assistant with browsing turned on for the price of a specific stock number on your site. If it quotes your live page, you're open. If it quotes a marketplace, a cached copy or nothing, something is stopping it. Then ask your website provider one question: what does our firewall do with AI search, AI agents and AI training traffic, separately?

Third, the match test. Type your domain followed by /robots.txt into a browser. If it welcomes AI tools, make sure your firewall agrees. If you want to keep training crawlers out, that's a fair call. Just make it a decision, not a default.

## **Fair Pushback**

Blocking isn't dumb. AI agents hitting login pages grew more than eightfold in the first half of 2026, per [DataDome](https://datadome.co/resources/bot-and-agent-security-report/ai-traffic-trends/?ref=blog.dgactual.com). Dealers have real reasons to guard customer portals, credit apps and lead forms. Guard those. The ask is narrower: let a shopper's assistant read a car and its price.

Our test is also harsher than real life in one way. A shopper using an AI browser on their own laptop at home probably gets through more sites than our cloud browser did. That's why we count it as an exposure marker, not a lost sale. Probably isn't always, though: Wine.com stopped Comet on the shopper's own device, on a single attempt on a single evening, so treat that as one shopper's experience, not a measured rate. And the fast-growing agents, the ones that shop while you're doing something else, run from the cloud, and that's who these walls stop.

## **How We Tested**

Sample: 90 stores drawn at random from our 18,106-store franchise roster, websites found through public business listings, 84 kept after four exclusions. With 84 stores, the 69% blocked figure carries a margin of error of about 10 points either way. Tests ran October 30, 2026 from one cloud network: a headless Chrome browser loaded each homepage and the store's used-inventory page, counted prices in the raw page and after scripts ran, and read robots.txt. Wine.com and Astor were checked the same day with the same tools.

Disclosure: both wine shopping trips, the Astor cart and the Wine.com attempt, used Perplexity's Comet browser, and our test ran on Perplexity's tools. Perplexity is the company in the Amazon case above.

Sources: Adobe Analytics, McKinsey, Morgan Stanley, DataDome, Cloudflare, OpenAI, Reuters, Cars.com, CarMax, Astor Wines and Wine.com, linked where cited. Dealer test data: DGActual. More from us at [data.dgactual.com](https://data.dgactual.com/?ref=blog.dgactual.com) and [reviews.dgactual.com](https://reviews.dgactual.com/?ref=blog.dgactual.com).