A test voyage through your store
TL;DR: User testing can pinpoint exactly where customers abandon their carts – but classic testing with real people is expensive and slow. So we built an open-source skill that lets AI walk through an online store as several different customers on mobile and desktop, get all the way to checkout, and write up a prioritized report. In this article we use a real (anonymized) test of one Shoptet store to show what it uncovers.
AI and online stores are our daily work. This tool came out of a simple frustration: a store owner "feels" that something in the buying journey is off, but has neither the time nor the budget for a classic usability test. So we handed that work to AI.
How AI testing of an online store works
The tool works in three phases. It's built on Playwright (it drives a real browser, so it handles even the JavaScript-heavy Shoptet, where ordinary scrapers fail) and on Claude models.
- Research and persona design. The AI crawls the store – homepage, categories, products, info pages – and based on what you sell, it proposes four specific personas tailored to your store. Not templates, but realistic customers. You approve them or adjust them.
- Parallel testing. Four personas test at the same time, each in its own browser and on a different device – iPhone, Android, desktop, and a laptop at 125% zoom for older users. Each one goes through the whole journey: from the homepage through the product and the cart to checkout, collecting observations and screenshots. One persona also places a test order.
- Synthesis. The AI finds patterns across the personas and generates a report ranked by priority – from critical problems that several people ran into, down to minor details.
The result is a folder with screenshots, feedback from every persona, and a summary SUMMARY.md you can read in five minutes and know what to fix first.
A real test: a strong core, holes at checkout
Let's take one specific run on a family-run online store selling herbal syrups on Shoptet (the name and details are anonymized). Four personas ran the test:
- Tereza (31, mom on maternity leave, iPhone) – buying a product for a two-year-old.
- Petr (47, IT project manager, desktop) – looking for a gift for Sunday, a decided buyer.
- Hana (39, teacher, Android) – needs a gift in time for Saturday.
- Jindřich (67, retiree, laptop with zoom) – cautious, pays cash on delivery.
The report confirmed that the store has a strong core: detailed product pages with ingredients and dosage, authentic photos, a wide range of payment methods (card, Apple/Google Pay, and cash on delivery, all with no surcharge), complete company details, and the owner's human tone. But it was losing conversions exactly where they matter most – in the last steps of checkout. Here are the five most important findings:
1. No delivery date estimate anywhere
Three of the four personas were shopping against a deadline (a gift for the weekend), but nowhere – not on the product, in the cart, or in the shipping methods – did the store tell them when the package would arrive. For buying a gift, that's a dealbreaker. Suggestion: show a specific date under every shipping method ("Delivery on Friday, May 8") and a countdown to the cutoff on the product ("Order by 2 PM and it arrives Saturday").
2. Separate "Street" and "House number" fields
Three of the four personas typed the street together with the number into a single field and had to fix a red error. Friction at exactly the moment when they just want the purchase over with. Suggestion: one "Street and number" field with normalization in the background, or at least a smart split once it's filled in.
3. A static "ORDER AND PAY" button for cash on delivery
Jindřich picked cash on delivery – he pays the courier – but the final button shouted "PAY" at him. He stopped: "I'm not clicking until I know what happens." For less confident customers, that's a brake right before the finish line. Suggestion: change the button text based on the payment method – "SUBMIT ORDER" for cash on delivery and bank transfer, "PAY" only for card.
4. Free in-person pickup with no visible address
The pickup address was hidden behind a question-mark tooltip that is hard to hit with a thumb on mobile. And Hana lived 10 km from the production facility – if she had seen the address, she would have saved on shipping and finished the order. Suggestion: put the address and opening hours right under the in-person pickup option, without a tooltip.
5. Free shipping threshold set too high
The free shipping threshold was set well above the usual order value, so none of the four personas reached it and shipping felt like an "extra tax". Suggestion: A/B test a lower threshold, or offer free shipping based on the number of items instead of order value.
On top of that, the report added dozens of other findings – from a rejected e-mail address with a + sign (legal, but validation refused it), through inconsistent cart behavior on different phones, down to small stuff like a cramped footer at higher zoom.
What helps the most
The strength of this approach isn't that AI "clicked through the site". It's in the different perspectives at once. The same buying journey was walked by a mom under time pressure, a decided buyer, a teacher on Android, and a cautious senior – and each one ran into something different. What's a minor detail for one is a reason to leave for another. And because they test in parallel and in your own language, you get the report in a single afternoon, not after two weeks of recruiting testers.
Important: this doesn't replace real user tests, it complements them. An AI persona has neither the wallet nor the emotions of a real customer. But as a fast, cheap, and repeatable first pass – before every bigger release, for example – it's invaluable. And because the skill is open source, you can run it yourself, or leave it to us.
Want your online store tested?
If you want to know where your online store loses orders, get in touch. We'll have AI walk your buying journey through the eyes of several types of customers, and you'll get the same report with fixes ranked by impact. And if you want to work with AI in your store yourself, we'll teach you at our Shoptet AI training – from testing through the MCP server for Shoptet to automation.