Having prided myself on being well read on industry information, I thought I knew how to have an intelligent conversation on analytics. I’ve pitched A/B testing in previous jobs, explained value-stream mapping, and collected more than my fair share of metrics on team success. With all of the sites devoted to gathering and analyzing feedback, you can fool yourself to thinking you know enough.

Then I met a real analytics expert.

Our agency hired one shortly after I started, and I made it my mission to understand how she thinks. Why something should be measured and why it shouldn’t. What should the real value in a particular question or the numbers associated with it be. LB was a life saver, because she taught me the value in asking the right question.

Over lunch, one day, we discussed gathering data on Agile teams. Having sat in on a few project teams, she saw a retrospective up close and personal. My frustration was that we gather all this useful information on how to improve our work, and because projects are short lived at Bottle Rocket the data from retrospectives sort of disappear into the ether with the close of a project. I sought to institutionalize the lessons learned, but I didn’t know how to get my fellow PMs on board.

I took her through ideas like Jeff Sutherland’s happiness metric, Deming’s performance appraiser formula, and the various ways we currently gather feedback in Scrum retrospectives. It was a sea of isolated information, without a way to contextualize it for senior leadership. Her face then lit up as she explained how she runs usability testing for our apps.

“By itself, a number is simply that…a number,” she told me. “Value-stream mapping holds value only if you point the survey to the right follow up question.” 

She guided me to the conclusion that asking open ended questions is great for mature, self-aware team members who can easily pinpoint what’s happening with each sprint. Paring it with a scale of 1-10, however, can prime Agile teams to understand how exactly they feel in the moment. Once they establish how they feel about something, it’s that much easier to put an explanation with it.

For example, if you ask me to rate my personal performance in the last sprint from 1 to 10, I could say that I was a 7. When I’m asked to follow up with why I would rate myself thusly, I could point to me meeting my sprint goals and all the associated tasks. What I wasn’t great at was communicating with my strategist because I was really heads down on other work. If I can’t communicate better, then the team will silo off.

That, my friends, is valuable information that leads to immediate action items to be gathered.

To boot, if I can get questions like this answered in advance of the retrospective, we can be more solution-based in our discussion and focus on what the right action items are to address in the next sprint. Imagine how productive your time together as a team could be if the agenda is that clear. Trust is established. Performance gaps are identified, griping is minimized and improvements are maximized.

Thanks, LB, for helping me realize that the time I devote trying to understand the team is not something to keep to myself. If your work is like mine, and resources tend to move around a lot, wouldn’t it be an amazing thing to standardize retrospectives and collect the improvements for all to share? It establishes that you aren’t the only one to experience a particular struggle, and helps the company keep from making the same mistake 20 times. 

Now, I can sell to leadership that we can settle for making a mistake only once.