The redesign they loved, and the problem it hid

Systems map · Systems thinking

How a thumbs-up turned into a roadmap

The study was framed as a simple go or no-go: did the rebuilt explorer beat the old one? It did, and stopping there would have produced a clean, satisfying, and slightly hollow readout.

The redesign itself was a real improvement. It moved the chart from a cramped side panel to the center of the screen, brought controls that used to be buried in menus into plain sight, and let people stack and compare several metrics on one chart.

The applause was hiding the real problem

Reading the interviews and the survey together, one thing repeated regardless of company or skill level: people liked the new layout but were not confident they were querying their data the right way. The satisfaction scores were really measuring the layout. The thing that would make or break the product sat one layer down, in building the query itself.

• A clear preference. A strong majority preferred the rebuilt explorer to the old version, which felt cleaner, faster, and easier to learn.
• The real struggle. The hardest task was not the layout but choosing the right metrics and shaping them into the right query with confidence; comparing several queries at once ranked close behind.
• People learn by doing. Most people learn the tool through trial and error rather than documentation or videos, so the interface itself has to teach, with worked examples and in-product guidance.
• A loved feature with gaps. Comparing several metrics on one chart was the most valued new ability, yet many found those busier charts hard to read and some never noticed the feature at all.

Built-in quality check

One survey item was written as a deliberately extreme statement, that the tool was already perfect and could not be improved, to confirm people were reading carefully rather than agreeing by default. They disagreed, which both validated the responses and showed they saw room to grow.

That reframe pointed the next phase at the query itself: guide people toward the right one with examples, in-product help, and eventually plain-language querying, so getting it right stops depending on trial and error.

What I almost missed · Critical thinking

What I almost missed

The study was set up to answer one question: do users like the rebuilt explorer? They did, clearly, and the easy version of this readout was a single slide that said success, ship it. That would have been true and nearly useless. What stopped me was reading the interviews and the survey side by side rather than separately. The same struggle surfaced in both, at every skill level: people were not confident they were building the right query to answer their question, especially once it involved several steps or several metrics. The satisfaction scores were measuring the layout. The real risk to the product sat one layer underneath, in the querying itself.