The value of subjective data, and how Google used it to build successful teams: NYT story and comment

I've argued before that subjective information is among the most important measurements of the success of emerging technology research, and that you can add subjective data into a metrics stream by assigning number values to subjective ideas.  It's a great concept, one that is not often given much weight in enterprise metrics.

I recently hurt my back at the gym, and got a referral to see an orthopedist.  Among the very first things in our discussion was the question, "how much does it hurt on a scale from one to ten?"  "Well, it's an 8 when I have a back spasm, but just sitting here right now talking with you, I'd say it's a 2 or 3," was my answer. That discussion is a practical demonstration of applying hard numbers to subjective feelings as a way to benchmark them.  It's not touchy-feely at all, and it's surprisingly practical.

image from the NYT article

And in that conversation, we have an important lesson about the way we measure performance within our enterprises, and how Google applied numeric values to subjective data to find out why some teams succeed and others don't.

After exhaustive number crunching about teams that worked really well, one of the main findings was that all team members need to have a sense of what they call "psychological safety" within the team.  In successful, high-performing teams, all members feel safe to offer any contribution without fear of ridicule or reprisal.

Here is the article:

NYT: Google research on subjective metrics and successful teams

Here is a brief excerpt:  ‘‘ By adopting the data-driven approach of Silicon Valley, Project Aristotle has encouraged emotional conversations and discussions of norms among people who might otherwise be uncomfortable talking about how they feel.  "Googlers love data,’’  (Google manager, Matt) Sakaguchi told me. But it’s not only Google that loves numbers, or Silicon Valley that shies away from emotional conversations. Most work­places do. ‘‘By putting things like empathy and sensitivity into charts and data reports, it makes them easier to talk about,’’ Sakaguchi told me. ‘‘It’s easier to talk about our feelings when we can point to a number.’

"The data helped me feel safe enough to do what I thought was right,’’  is the concluding quote in the NYT story.

Let's take this away from the idea of just talking about feelings.  As Google's research bears out, feelings turn out to have measurable value for enterprise success.  But the goal to get value-add from subjective data goes much farther.  When it comes to perceiving nuance and big picture--as opposed to big data--there is still no more powerful computer on earth than the human brain, and you have them in your organization in abundance.  Unfortunately, because it's tougher to report nuance as a hard number, it's relegated to the "subjective, hence unimportant" bin.

We already have historical evidence of subjective input outperforming choices based upon pure datasets: crowdsourced stock predictions.

Here is a practical example of measuring subjective data in business.  At each stagegate of your projects, ask participants to report their sense of the project: "on a scale of one to ten, how do you feel about this project's likelihood of success?"  A caveat, though: unless you have made giving an opinion openly a safe thing to do*, the data you receive is likely to be highly biased.  Anonymity through a third-party survey site might be in order.  Average the results, chart it over time against your other KPIs for the project, and I don't think it will take very long to get a very useful Project Health Index that you can benchmark against variables within the project management, and against other projects.  That is an early warning system, a business metric you can take to the bank.

By the way, my back is doing better.  Occasional incidents of a 5 or so, but the trend is good.



*The whole topic of "safety in reporting" is profoundly important to business metrics. I'm at work on a subsequent article on the idea, and will report steps former Ford CEO Alan Mulally took to change the fear culture that used to permeate Ford.