A global automaker — testing like an outsider.

The company is famous for the volume of its consumer research: market testing, quality assurance, constant customer feedback. By any automotive standard, they knew their customer cold.

But shipping brake pads is not the same as shipping software. The company recognized that its real competition was no longer other carmakers; it was the tech industry, where the pace of innovation and the quality of the experience set a bar the automotive playbook wasn't built to clear. The world-class company was used to shipping software it designed, built, and tested five years prior, and that was exactly the problem.

The brief: bring independent testing, free of internal confirmation bias, plus design and UX alternatives for the experience inside and outside the vehicle. Working across three internal divisions and with Aperio Insights, I helped build and run the company's first-ever usability tests for its in-car systems.

Cars are now judged against phones. A head unit, a voice assistant, a companion app. Customers measure them against the best software they use every day, not against last year's dashboard. Automotive QA was built to catch defects, not to judge an experience against the tech industry's bar.

And the gap was invisible from the inside. The teams building the systems were the same teams evaluating them, so the research kept confirming the company's own assumptions instead of surfacing where real users actually struggled.

Their competition was no longer other carmakers. It was the tech industry's pace and polish.

The problem: a company famous for research could no longer trust its own research to keep pace with tech-industry UX, and needed independent, tech-speed design and testing to close the gap before launch, not after.

The obvious move was to lean harder on the existing research apparatus: more studies, same machine. I didn't. Its scale was real, but so was its blind spot: it was structured to validate, not to challenge. The only way to get honest answers was to put the testing in independent hands.

So we stood up a dedicated, outside testing practice. Over the program, that meant more than 500 moderated, in-person usability tests and 3,500-plus hours of live-streamed, recorded, logged, and reported research. Rotating teams of three to five researchers tested ten to twelve hours a day, up to three days a week, across 17 separate studies: head units, dashboards, at-speed voice UI, mobile apps, and more.

New thinking needed new tools, so we built them: our own logging, study-tracking, and rapid-analysis tooling, and even our own way to structure simulated driving. We were, by the company's own account, the first team to run usability studies on their software. And the point was never to run tests. It was to wire results straight back to the design teams fast enough to iterate inside the same engagement, a closed loop instead of a report.

The work here is the machine that produced the answers, not a screen. Three parts of it carried the program.

The program caused major ripples across the company globally. Everyone wanted to see how the North American team was designing, testing, and building the new HMI so quickly and with such high quality. The company's chairman himself stopped by to watch a session, and teams from around the world asked to adopt the practice. Once the model proved out, the same approach went on to test power wheelchairs, at-speed voice UI, mobile apps, head units, and dashboards.

The decision I'd reverse: I built our relationship with the automaker and let the relationship with the design vendor whose work we were testing fend for itself. With the automaker sitting between us as the middleman, the structure quietly pitted two vendors against each other. We were there to validate and elevate their work; too often it must have felt like we were there to break it. I should have fought for a direct, close partnership with them from day one, because research is only as useful as the design team's willingness to act on it, and I left that to chance.

The surprise: how readily drivers pull their attention off the road. It reads as obvious, but it was visceral in testing: people lifting their hands off the simulated wheel to lean in and squint at a font, or getting lost three menus deep just to do something simple. A big touchscreen that makes you look away to operate it is a genuinely bad idea. Simple, high-level, glanceable interaction you can use without looking went from "nice" to mission-critical.

What I carried forward: never be afraid to step into the dark and build the thing that doesn't exist yet. We had to invent most of our apparatus: the logging, the study-tracking, the rapid analysis, even how to structure simulated driving, to run studies no one at the company had run before. If the tool you need doesn't exist, you build it. It's the same instinct behind Speakeazy.