A global automaker — testing like an outsider.

The company is famous for the volume of its consumer research: market testing, quality assurance, constant customer feedback. By any automotive standard, they knew their customer cold.

But shipping brake pads is not the same as shipping software. The company recognized that its real competition was no longer other carmakers; it was the tech industry, where the pace of innovation and the quality of the experience set a bar the automotive playbook wasn't built to clear. The world-class company was used to shipping software it designed, built, and tested five years prior, and that was exactly the problem.

The brief: bring independent testing, free of internal confirmation bias, plus design and UX alternatives for the experience inside and outside the vehicle. Working across three internal divisions and with Aperio Insights, I helped build and run the company's first-ever usability tests for its in-car systems.

Cars get judged against phones now. When someone uses a head unit, a voice assistant, or a companion app, they're comparing it to the best software on their phone, not to last year's dashboard. Automotive QA was built to catch defects, not to judge an experience against the tech industry's bar.

And the gap was invisible from the inside. The teams building the systems were the same teams evaluating them, so the research kept confirming the company's own assumptions instead of surfacing where real users actually struggled.

They weren't competing with other carmakers anymore. They were competing with the phone in your pocket.

A company famous for research could no longer trust its own research on software. It needed somebody independent, working at tech speed, to find the problems before launch instead of after.

The obvious move was more of the same: more studies through the same machine. I didn't do that. The machine was huge, but it was built to confirm what the company already believed, and that was the blind spot. The only way to get honest answers was to put the testing in independent hands.

So we stood up a dedicated, outside testing practice. Over the program, that meant more than 500 moderated, in-person usability tests and 3,500-plus hours of live-streamed, recorded, logged, and reported research. Rotating teams of three to five researchers tested ten to twelve hours a day, up to three days a week, across 17 separate studies: head units, dashboards, at-speed voice UI, mobile apps, and more.

New thinking needed new tools, so we built them: our own logging, study-tracking, and rapid-analysis tooling, and even our own way to structure simulated driving. We were, by the company's own account, the first team to run usability studies on their software. And running tests was never really the point. The point was getting results back to the design teams fast enough that they could fix things inside the same engagement, instead of reading about them in a report months later.

There's no screen to show you here. The work was the operation itself. Three parts of it mattered most.

The program caused major ripples across the company globally. Everyone wanted to see how the North American team was designing, testing, and building the new HMI so quickly and with such high quality. The company's chairman himself stopped by to watch a session, and teams from around the world asked to adopt the practice. Once the model proved out, the same approach went on to test power wheelchairs, at-speed voice UI, mobile apps, head units, and dashboards.

What I'd do differently: I built our relationship with the automaker and let the relationship with the design vendor whose work we were testing fend for itself. With the automaker sitting between us as the middleman, the structure quietly pitted two vendors against each other. We were there to validate and elevate their work; too often it must have felt like we were there to break it. I should have fought for a direct, close partnership with them from day one, because research is only as useful as the design team's willingness to act on it, and I left that to chance.

What surprised me: how readily drivers pull their attention off the road. It reads as obvious, but it was visceral in testing: people lifting their hands off the simulated wheel to lean in and squint at a font, or getting lost three menus deep just to do something simple. A big touchscreen that makes you look away to operate it is a genuinely bad idea. Simple, high-level, glanceable interaction you can use without looking went from "nice" to mission-critical.

What I took with me: never be afraid to step into the dark and build the thing that doesn't exist yet. We had to invent most of our apparatus: the logging, the study-tracking, the rapid analysis, even how to structure simulated driving, to run studies no one at the company had run before. If the tool you need doesn't exist, you build it. It's the same instinct behind Speakeazy.