A product answer is one step in a purchase, not the purchase itself. To evaluate an AI agent, follow the customer from the first question to the cart and then into checkout. This walkthrough uses the Everyday Oxford Shirt from Obsedia’s website demonstration: an illustrative ivory shirt, listed at €59, with sample sizes S, M and L. It shows what the agent can contribute and what still belongs to the storefront and the customer.
1. The customer describes a use, not a product ID
Customer: “I need a shirt for work, not too formal.” A search-style response might return every shirt. A more useful response asks for a detail that changes the choice: “Which size do you wear, and do you prefer a regular fit?”
At this point the task is product discovery. The agent should keep the conversation focused on the purchase and avoid asking for unrelated personal information. A short clarification is enough if it gives the agent what it needs to identify a relevant catalog item.
2. The next answer uses the customer’s constraint
Customer: “M, regular fit. A neutral colour.” The sample response can now introduce the Everyday Oxford: ivory, regular fit and 100% cotton, with the demo price of €59. Each detail should come from the example catalog rather than a general assumption about Oxford shirts.
Review whether the agent remembered size M and the neutral-color request. If the conversation shifts to a different garment, those requirements still matter. Losing them forces the customer to repeat the same information and can lead to the wrong variant.
3. The product card makes the recommendation concrete
The chat displays the shirt image, product name, price and size selector. The customer can inspect the actual recommendation and change the option. A card should remove ambiguity about which item the agent means.
For a live store, verify that the card reflects the current eligible product and variant. Check whether a selection change affects the price or availability. The public website demonstration uses sample products and prices; it is not evidence of live inventory in a merchant’s store.
4. The customer chooses Add to cart
The customer selects M and presses Add to cart. In the demo, the item appears in the demo cart. In a configured storefront, the supported action should add the eligible Shopify variant to that store’s cart.
Check the cart contents, not only the chat response. Confirm the product, size and quantity, and test what happens if the item is unavailable. The customer should receive a clear outcome. A recommendation or a conversational “yes” should not be described as automatic checkout authorization.
5. Checkout remains a separate customer step
After the cart, the customer continues through the store’s checkout, where shipping, taxes, payment and other purchase conditions are handled. Shopify’s cart model distinguishes cart operations from the checkout process. Adding an item is not a completed order.
If the customer stops here, investigate the relevant step. A good product conversation cannot repair an unavailable payment method. Likewise, do not attribute a checkout failure to the agent without checking whether it selected the wrong item or whether the problem occurred later in the journey.
Review the entire path before launch
Repeat this walkthrough with your own product and one alternative variant. Start from the storefront, ask a realistic question and check each transition. Then run a failure case: missing information, unavailable variant or a request about an existing order. Ecommerce V1 should not be assumed to access protected order or customer data.
Write down the expected outcome at each step. The answer should use approved facts, the card should identify the right item, the cart should reflect the chosen variant and checkout should remain under the customer’s control. That is a more useful acceptance test than judging the assistant by a polished first reply.
Measure the transitions that matter
A practical review separates answer quality, product selection, cart success and purchase completion. This makes it easier to find the cause when a journey fails. It also prevents a cart addition from being reported as a sale.
To try the sequence, open the Ecommerce Agent page and use its sample shopping conversation. Then adapt the same steps to a configured Shopify test environment. Use the demonstration to understand the interaction; use store testing and appropriate analytics to evaluate real business results.
Verify the question, answer, product card, variant, cart and checkout as separate steps in one journey. A correct first answer is only the beginning.
