The designer's version was better. Customers still wanted the controls.

In 2014 I was working on ecommerce for a small-business marketing product. Many of our customers wanted their online store to look like the website they already had, and getting there meant working in HTML and CSS. That's a lot to ask of someone who runs a business and doesn't build websites.

I wanted to explore skipping that work entirely. A customer's website already had the colors, images, and overall look they'd chosen. We could read it, pull those pieces out, and style their store automatically. No configuration step at all.

I loved the idea. I was also worried about asking engineering to build something that expensive before we knew it solved the problem. So before anyone wrote code, I asked one of our designers to do by hand what the feature would have done. For a few local customers, the designer took images and colors from their existing websites and used them to style their storefronts.

The designer's result was better than anything our product could have generated. Customers were looking at something close to the best possible version of the idea. If automatic styling was going to win anywhere, it should have won there.

It didn't. Within a few days, it was clear that these customers were cautious about how their brand was represented. They wanted to be able to change every part of what we'd done for them.

Because the styling was good, their reaction wasn't about quality, and smarter automation wouldn't have answered it. "Configuration" had been covering two different things. One was the work: knowing CSS, finding the right colors, getting images to sit properly on the page. The other was a set of choices about how their business would look to people who had never met them. Our proposal removed both. Customers were glad to lose the work and wanted to keep the choices.

My best guess about why: the store was public, and it had their name on it. A few customers can't settle that.

We changed the priority to making customization easy, so customers could adjust the look of their store without writing code. That was far less work to build than automatic styling, which meant useful improvements could reach customers much sooner.

When the evidence points toward the cheaper option, I get a little suspicious, because that's the answer everyone wants to hear. Two things made this one easier to trust. The test had been tilted in favor of the expensive idea. And the editing tools were going to be needed in every version of the product. If we built automatic styling later, customers would still want to change what it produced. Building the controls first kept automation possible and built something it would need anyway.

The test also left a question open. Customers saw styling that had been done for them, and they wanted control over it. We never tested the same automation offered as a first draft they expected to edit. That's a different offer, and it might have gone over well.

I think about this more now than I did then, because producing a first draft of almost anything has become cheap. A common way to think about AI products is that people hold on to control until the system earns their trust, then hand it over a piece at a time. That's often right. These customers seemed to hold on for a different reason. The work was better than we could have automated. They kept control because the decisions were theirs.

When someone keeps editing what software produced for them, it's worth finding out which situation you're in. If they don't trust the result, the answer is a better result. If they consider the choice theirs, a better result won't change their mind, and the product should make their choices easier.