It’s impossible not to think a lot about data these days. We’re generating it all the time, constantly. On our phones, on our televisions, on our laptops, in public spaces. And increasingly the best startups and Internet giants are using data to make better and better product decisions and designs.
Today at Greylock we announced that DJ Patil is joining us as Data Scientist in Residence, as far as I know the first time any VC has had a position quite like that. It’s a huge addition for us, and the expression of a bunch of deeply held beliefs about the state of the art in designing great products.
But as I talk about using data for design, I find that there’s a lot of misunderstanding about it — some people have the sense that it somehow makes designers less powerful, that you’re basing decisions based purely on mechanical measures rather than designer intuition and genius.
In my view, however, data is what makes designers not only strong, but primary. It’s what turns designers from artists into the most important decision makers in a company, because it’s understanding the data that lets you understand what your users are doing, how they’re using (or not using) your products, and what you can be doing better.
It made me think back a bit to my own training as a UX designer (we called it HCI then) at Stanford in the mid-nineties, when the field was just starting to develop. We would spent a lot of time on ethnography, need finding, doing paper prototypes and then doing basic mockups and user testing. And we’d get 80% of the way there then go and build it.
Nowadays, the state of the art is to still do need finding and some mockups early, but to get to a working prototype as quickly as you can, that’s instrumented so that you can tell what’s happening and figure out whether you’re on the right track or not.
I think that’s generally the right approach, but it’s worth noting: instrumented prototypes can really only get you to local maxima — they can help you find ways to tweak and optimize the basic design you’ve got, but they can never help you find a radically different and better solution.
So when I talk about using data — and I talk about it a LOT — what I’m talking about is a mixture of the artisan/designer-led designs along with using data to figure out what’s best.
Thinking about it the other day, I was reminded of one of my favorite sayings that I learned from Bob Sutton: “Fight like you’re right, listen like you’re wrong.” Bob’s an organizational theorist, and what he means is really a paraphrase of something that I think Andy Grove said, which is that he wanted all his people to have strong beliefs, loosely held. In other words, he always wanted people to come in with a point of view — a design, as it were — but to be willing to moved off of that point of view in the face of data.
So the modern, design oriented framing is this:
“Design like you’re right. Read the data like you’re wrong.”
In other words, you should always design the product you think/believe/know is what people want — there’s a genius in that activity that no instrumentation, no data report, no analysis will ever replace. But at the same time you should be relentless in looking at the data on how people actually use what you’ve built, and you should be looking for things that show which assumptions you’ve made are wrong, because those are the clues to what can be made better. We all like to see all the up-and-to-the-right happy MBA charts, and those are important. But they don’t help you get any better than you already are.
I wish we taught more of this blend, because all of the products we use would get better.
So: design like you’re right; listen like you’re wrong.
I can’t help but think that the user experience guys need to understand the second part; ‘listen like you are wrong’.
Sometimes, trying to give them constructive criticism seems about as productive as bashing your head against a brick wall.