At the open source accountability workshop earlier this week, Stephen Walli used a phrase that really resonated with me. He called his role (and mine) open source diplomats. We do much the same thing that diplomats do – lots of conversations, negotiations, wheedling and persuading – which is all incredibly hard to measure.
He mentioned that he investigated how actual diplomats are measured, and, turns out, that’s really hard too.
A large part of my work is what we snarkily refer to as preventing dumpster fires. We identify potential errors in open source engagement, and we work with the teams in question to ensure better outcomes. This might be something as simple as helping an engineer persuade a project to welcome their patch, in ways that are appropriate to that community. Or it might be something as potentially catastrophic as avoiding a fork of a major open source project, which might harm reputation and careers, and burn trust in critical communities.
That’s hard to measure. I mean, I could put in my monthly report that I avoided 10 catastrophes, but there’s only my word that those catastrophes would have happened otherwise. And, even then, quantifying the cost or impact of those catastrophes, or of avoiding them, is also impossible. Would customers care? Would it affect revenue? Would it be a blip in the tech news? So many of these things are impossible to predict. I have only my decades of experience and a gut feeling to guide me in most of these scenarios.
It’s also easy to abuse this kind of situation. I could, for example, simply state that, by my wisdom alone, we avoided a billion dollar loss. Yay me. Who’s to say otherwise? And, indeed, I have, in the past, worked with people who did exactly that, and got away with it, for years. Which is in large part why I care about trying to find ways to quantify what I do, and why it matters — often to people who don’t understand the nuances of open source community engagement.
Unfortunately, this particular part of the workshop produced more questions, and anecdotes, than actual solutions, but one thing was raised by several people. Many engineering roles at large corporations are measured by what you released/launched/shipped. As such, engineers working in open source are at a disadvantage for promotions or recognition because that’s not often an actual thing in the projects that they work with. Moreover, you can realistically work for a year on a feature and then have the community say, no thanks. (Granted, there are ways to mitigate that, but that’s another article.) This makes it very important that people who understand open source are involved in discussions around promotion and recognition, and find ways to reward open source participation in that process. Notably, everyone in the room said that their employer is bad at that, so a lot more discussion is warranted here.