John’s Blog

Update 5/22/08: First, I do want to thank everyone for commenting & discussing — this is a serious and important subject that people should care deeply about.

To make a few things clear:

1. There is no secret data project.
2. There is no secret plan to collect user data.
3. We are not already secretly collecting data.
4. Yes, we are trying to figure out how to accumulate and open better data about how people use the web and their browsers; like everything Mozilla does, that starts with discussion like this, and we expect people to have many, many opinions.

As I told Mike, there’s no staff, no project plan, no nothing, really, except a desire to level the playing field in ways that open source itself has. Like everything else at Mozilla, anything we do will be rooted in the fundamentals of user control, data privacy, and transparency with our community and users.

——-

Had a great conversation with Mike Arrington from TechCrunch yesterday — resulted in a nice writeup of one of the projects we’ve been thinking about here at Mozilla over the last few months. He highlights the opportunity quite well, I think, and I’d like to add some context here so everyone knows where we’re coming from. One correction that I need to make up front: “stealth project” should read “very early stage project that Mozilla has been open about,” but that’s probably not quite as catchy a headline. 🙂

Meeting Context
For most of our hour we spent was talking about the upcoming release of Firefox 3 — which is on the way very soon now, and is a release that everyone involved is very proud of.

Towards the end of our time, Mike asked: “What’s next?”

We answered similarly to our conversation with Matt Asay a while back, in that we’ve got about 4 things we’re spending a fair bit of time thinking about: (1) the future of Firefox and the technology that powers it (the new version we’re calling moz2), (2) mobile Firefox (code-named Fennec), (3) online services like Weave, and (4) data. Mike was particularly interested in the 4th item, data, and saw a lot of possibility in it.

Key Insight
The key insight is not so much that rich clients or web sites are able to collect information about what people do, but rather that this data is one of the most important pieces to faciliate understanding (and innovation), and is also one of the most under-explored areas of the modern web.

I’ll say it again another way: while technology has gotten cheaper & cheaper to deploy, and the connected nature of the global web means that you can start up a new worldwide service practically overnight for very little capital, there remain worlds of information about how people use the web that are locked up and not currently shared.

So we asked ourselves what we can do to help unlock some of this latent potential — and started thinking about whether there’s a project we can do at Mozilla that does a few things:

1. Collects & shares data in a way that embodies the user control & privacy options which are at Mozilla’s core.
2. Enables everyone — from individual researchers and entrepreneurs (both the social and capitalist types) to the largest organizations in the world — to take usage data, mix it up, mash it up, derive insight, and hopefully share some of that insight with others.
3. Helps move the conversation around data collection and web usage forward, to help consumers make more informed decisions.

It seems obvious to us that there’s lots to be done here, and lots that we can do, if we can work with our broad community to figure it out.

The Current Situation at Mozilla
First things first: we track very very little data today. I’ve posted before that we use our Application Update Service (AUS) pings to get a sense of where our main usage around the world is and to try to spot problems when they happen (it’s notable that this is a secondary usage of that system — the primary function of AUS is to enable timely updates to the softeware we release — in Firefox 3 and future versions of Firefox 2, we’ll watch instead an analogous ping that checks whether updates to add-ons are available.) Beyond that, we don’t collect much data in the product at all.

We’ve got a couple of projects started at a small level in this area — one is called Spectator, an add-on mostly used to improve the user interface of Firefox, and another is a project in Mozilla Labs called Test Pilot. They’re both early and very limited in scope.

Beyond that, we’re thinking about it and talking about it, but haven’t staffed it very much — we don’t even have a name for the project yet. What we do know is that data is important, and that there’s a ton of potential for everyone.

We’ve had most of the substance of these conversations in the open, like most everything we do, and we want to have more. Key to us doing anything is having even more conversations like this in public, and figuring out a set of core principles that go beyond just the level of opting-in.

So I’m glad that Mike wrote about it & sees some of the promise here. It’s early days, but it seems to me at least that opening up all sorts of data — from web usage to the social graph & beyond — is going to be the topic of conversation for a long time to come.