Open source stats – but what do the numbers *mean*?

I recently sent a report to project management containing some numbers that purport to describe the status of the RDO project.

I got a long and thoughtful response from one of the managers – we’ll call him Mark – and it seems worthwhile sharing some of his insights. To summarize, what he said was, don’t bother collecting stats if they don’t tell a story.

1. Focus on the goals

Listing a bunch of numbers without context – even with pretty graphs – doesn’t tell us anything unless you relate them to goals that we’re trying to achieve.

Several weeks ago I presented a “stakeholder review” to this same audience. Any statistics that I present in the future should be directly related to a goal in that review, or they are just meaningless numbers, and possibly a distraction, and, worse still, might cause people to work towards growing the wrong metric. (Google for “be careful what you measure” and read any of those articles for more commentary on this point.)

2. Focus on the people

One of the stats that I provided was about how certain words and phrases feature in the questions on ask.openstack.org. Mark looked beyond the numbers and saw three people who are very active on that website, two of whom are not obviously engaged in the RDO community itself. Why not? How can we help them? How can they help us? What’s their story? Why are we ignoring them?

3. Focus on the blips

In February, our Twitter mentions, retweets, visits, and so on, went through the roof. Why? And why didn’t we do that same thing again in March?

As it turns out, in February there were two conferences that contributed to this. But, specifically, we captured a lot of video at those events, and the Twitter traffic was all around those videos. So clearly we should be doing more of that kind of content, right?

4. Ignore the stuff that doesn’t seem to mean anything

We track “downloads” of RDO, which roughly speaking means every time someone runs the quickstart and it grabs the RPM. Except RDO is on a mirror network, so that number is false – or, at best, it reflects what the trends might be across the rest of the mirror network. So we have no idea what this metric means. So why are we bothering to track it? Just stop.

5. Ask not-the-usual-suspects

This last one wasn’t one of Mark’s observations, but is what I’m taking from this interaction. We tend to ask the same people the same questions year after year, and then are surprised that we get the same answers.

By taking this data to a new audience, I got new answers. Seems obvious, right? But it’s the kind of obvious thing we overlook all the time. Mark provided insight that I’ve been overlooking because I’m staring so hard at the same things every day.

By the way, I’ve presented Mark’s insight very bluntly here, because it’s important to be clear and honest about the places where we’re not doing our job as well as we can be. Mark’s actual response was much kinder and less judgmental, because Mark is always kind and supportive.

3 ways you find the right type of contributor and where to find them

Another one of Stormy’s questions caught my eye:

“What are 3 ways you find the right type of contributor, and where do you find them?”

Thinking back to the last few years of work on RDO, several answers come to mind:

The people asking the questions

Watch the traffic on your mailing list(s) and on the various support forums. The people that are asking the most questions are often great potential contributors. They are using the project, so they are invested in its success. They are experiencing problems with it, so they know where there are problems that need to be addressed. And they are outspoken enough, or brave enough, to talk about their difficulties publicly, so they are likely to be just as willing to talk about their solutions.

These are usually good people to approach ask ask to write about their user experience. This can often be done collaboratively, combining their questions with the eventual answers that they encountered.

If they appear to be the type of person who is implementing solutions to their problems, ask them to bring those solutions back to your community for inclusion in the code.

In RDO, people that participate in the rdo-list mailing list will sometimes end up contributing their solutions to the project, and eventually becoming regular contributors. We probably need to do a better job of encouraging them to do this, rather than just hoping it’s going to happen on its own.

The people answering the questions

In watching the question-and-answer on ask.openstack.org I often see names I don’t recognize, answering questions related to RDO. Sometimes these are Red Hat engineers who have recently joined the project, and I haven’t met yet. But sometimes, it’s people from outside of Red Hat who have developed expertise on RDO and are now starting to pay that back.

These are the people that I then try to approach via the private messaging feature of that website, and ask them what their story is. This occasionally evolves into a conversation that brings them to more active involvement in the project.

Most people like to be asked, and so asking someone specifically if they’d be willing to come hang out in the IRC channel, or answer questions on the mailing list, tends to be fairly effective, and is a good step towards getting them more involved in the project.

The people complaining

This a tricky one. People complain because something is broken, or doesn’t work as they expect it to. The traditional response to this in the free software world is “Patches Welcome.” This is shorthand for “Fix it yourself and stop bugging me.”

The trick here is to recognize that people usually take the trouble to complain because they want it to work and they’re looking for help. This passion to just make things work can often be harnessed into contributions to the project, if these people are treated with patience and respect.

Patience, because they’re already frustrated, and responding with frustration or defensiveness is just going to make them angry.

Respect, because they are trying to solve an actual real world problem, and they’re not just there to hassle you.

It’s important that you fix their problem. Or at least, try to move them towards that solution.

Once the problem is addressed, ask them to stay. Ask them to write about their situation, and the fix to it. Ask them to stick around and answer other questions, since they have demonstrated that they care about the project, at least to the point of getting it working.

When people complain about your project, you also have a great opportunity to brush them off and persuade them that you are uncaring and unwelcoming, which they will then go tell all of their friends and Twitter followers. This can be a very expensive thing to do, for your community. Don’t do that.

When people come to #rdo on Freenode IRC to ask their RDO and OpenStack questions, I frequently (usually) don’t know the answer myself, but I try to make an effort to connect them with the people that do know the answer. Fortunately, we have an awesome community, and once you bring a question to the attention of the right person, they will usually see it through to the right solution.

Retaining community contributors

Stormy, at OpenSource.com, recently asked the question “What are 10 steps to keeping new contributors once you have their attention”

I’m not sure I have 10, but here’s what I’d say:

Celebrate their contribution, loudly and publicly

People like to be noticed, and they like to be thanked. Tell them that you noticed their contribution by telling the whole world about what they did.

There’s a number of ways to do this, from tweeting every first contribution, to including a Thanks section in your monthly newsletter, to doing a weekly blog post that lists new contributors for that week. Whichever approach you take, make sure that you personally send them a message telling them about it, rather than just hoping that they notice it.

If you have very few new contributors every week/month, then take the time to describe what the new contributor did. If you have dozens every time, this can be more of a challenge, but sending them a personal note is still a very critical part of the process if you want them to stick around.

Give them the keys

A lot of people in open source think that everyone should have to earn their place on the contributor list, by consistent, high-quality contributions over a long period of time.

I’m at the other end of the spectrum. I think you should hand out committer rights like candy. If someone shows initiative and contributes once, give them the opportunity to contribute again, immediately, with no barrier to entry.

What’s the worst that can happen? You have to revert a patch. This is why you have revision control.

So, give them the keys on day one. But there’s a caveat here, which is the next item on my list:

Give constructive criticism

Nobody likes a patronizing pat on the head. Give constructive feedback. After thanking them, tell them patiently and kindly what they can improve. This shows that you consider them to be a contributor to a serious project, and not merely a statistic to report to your boss.

If you have a style guide, and they didn’t live up to it, point them to it, gently and kindly, and give them specific pointers on what they need to improve.

Don’t be one of those jerks that throws it back in their face with a nasty remark. That will guarantee that they won’t come back. Remember that your goal is to retain new contributors, and train them up in the way that your project operates, not merely to act as a filter to keep out unfavorable contributions.

Ask them to do it again, with a specific request

People like to be asked. Asking someone, personally, to do a particular task, is much more effective than a shotgun “Hey, can someone …” sent out to an anonymous faceless mob on a mailing list.

Once someone has contributed something, that’s your opportunity to say, “Hey, I notice that you have some interest in the WhizBang feature, and Ticket 31245 is about that, too. Maybe you could take a look?”

Ask them to write about it

The new contributor experience is a rare and valuable thing. You don’t know what it’s like any more. You remember bits of it, but mostly you’ve forgotten, and become jaded with your second, third, and 700th contribution.

Encourage this new contributor to write about their experience. What was too hard. What was fun. What was unexpected, frightening, exciting. Who helped them. Who got in their way.

These are hugely valuable data points for your project. And they are also a great way to tell this new contributor that their input is valuable, and that you’re trying to make your project more welcoming.

And, of course, once they have written about it, share it with the developer community, along with the name of the contributor, and say, here are some ways maybe we can make this better for the next person.

Point them to the documentation

You have docs, and then you have hidden docs, and then you have the stuff that “everybody knows”. Steer this new contributor to the docs, and take the opportunity to turn the other two categories into actual docs.

Some of this will come out of the blog posts that these new contributors write. Some you will need to write yourself, based on their feedback of the new contributor experience.

This is tedious, but, remember, you’re making an investment in the next time. The better you make this process, the easier it will be for the next person, and the sooner you can pass this part of the job on to someone else.

Ask them to mentor the next new contributor

Ask them to turn that first-timer experience into mentoring another first-timer, as soon as possible. They’ll have the recent experience. They’ll know the potholes that you have long since learned to steer around. They’ll know which people in the community are helpful, and which ones are jerks, who you’ve long ago dismissed as “Oh, that’s just how Larry is.”

What’s new in OpenStack Ocata

OpenStack Ocata has now been out for a little over a month – https://releases.openstack.org/ – and we’re about to see the first milestone of the Pike release. Past cycles show that now’s about the time when people start looking at the new release to see if they should consider moving to it. So here’s a quick overview of what’s new in this release.

First, it’s important to remember that the Ocata cycle was very short. We usually do a release every six months, but with the rescheduling of the OpenStack Summit and OpenStack PTG (Project Team Gathering) events, Ocata was squeezed into 4 months to realign the releases with these events. So, while some projects squeezed a surprising amount of work into that time, most projects spent the time on smaller features, and finishing up tasks leftover from the previous release.

At a high level, the Ocata release was all about upgrades and containers – themes that I heard from almost every team I interviewed. (For the full interview series, see https://goo.gl/3aCQ2d ) How can we make upgrades smoother, and how can we deploy bits of the infrastructure in containers. These two things are closely related, and there seems to be more cross-project collaboration this time around than I’ve noticed in the past.

And the themes of upgrades and containers will continue to be prominent in the Pike cycle.

Highlights in the Ocata cycle include:

Auto-healing: Work was done in Heat to make it easier to recover from service failure. When an outage is detected, you can have Heat automatically spin up a replacement service, and swap it out without any intervention on the part of the operator.

Composability: Composable roles are a feature whereby you can specify details of how things are deployed, rather than allowing OpenStack to choose. You can, for example, specify that a particular hardware configuration be used for particular services. This is termed Composable Roles. Work was done in Ocata to expand this to composable upgrades, so that these roles are respected across upgrades as well.

Multi-factor auth in Keystone: Work was done in Keystone to improve support of MFA, including OTP (One Time Password) support, and per-user token expiration rules.

NFV: Network Function Virtualization continues to be an area where we’re seeing a lot of activity, and so a lot of the work in Nova, Neutron, and various other projects focuses on these developments. NFV has become more stable in this release, and is more fully integrated into TripleO for ease of deployment. This effort is happening under the Apex project – https://wiki.opnfv.org/display/apex/Apex

Upgrades: Upgrades were a common theme across all projects, with the emphasis being the ability to upgrade from one release to the next with as close to zero downtime as possible. Much of this work centers around TripleO, Heat, and Mistral, for orchestration and automation of the process.

Containers: While centered around the Kolla project, containerization was a theme in many of the projects this cycle. The eventual goal, at least according to some, is that OpenStack services will be deployed in containers by default by the Pike release. This of course poses a real challenge for the Ocata -> Pike upgrade path (migrating from non-container to container in the course of the upgrade), and that’s something that the TripleO people are working hard on.

Security: TLS-everywhere made strides forward in Ocata, with connections between services moving to TLS. This involves changes to Barbican as well, for key management for the shared keys between services, to ensure that your traffic is secure between components of your cloud, which may be located in different data centers around the world.

Collaboration: Something I heard more this year than in previous years was talk of collaboration between projects. This has, of course, always been happening. However at the PTG in Atlanta, it was a major focus, with time set aside for cross-project meetings focusing on the interface between one service and another. I also heard from several people that the PTG allowed a focus, and a camaraderie, that was not possible when the design summit was part of OpenStack Summit. This resulted in fewer interpersonal tensions, and a lot more work getting done.

Everything else: The difficulty with OpenStack is that it’s just so big. While these are the things that stood out to me, someone else is likely to pull out different highlights, depending on their interests. So I encourage you to look at some of the other “What’s new in Ocata” articles out there, including especially “53 new things to look for in OpenStack Ocata” – https://www.mirantis.com/blog/53-new-things-to-look-for-in-openstack-ocata/ and, if you have a lot of time, or have interest in a particular project, check out the official release notes – https://releases.openstack.org/ocata/  And take a moment to watch my interviews with various Red Hat OpenStack engineers, from the Atlanta PTG, here: https://goo.gl/3aCQ2d.

Poetry month, day 2

Day 2 prompt: Write about a poem about a superhero coming to your house and confronting you about something. Somewhere in the poem, you have to state what your superpower is.

The Documentor!

His shoulders fill the doorway
and his cape flutters slightly in the wind
only he feels.

You must use words like
Super, Ultra, Mega
when you refer to me.

I must be best, perfect, finest, outstanding, leading,
Optimum

But I hold him entirely in thrall
Sure, he can fly,
lift trains from their tracks,
trace a segfault to its root cause and patch it

But I write about it
and without me
he doesn’t exist.

 

Poetry month: Day 1 prompt – Unbroken

Working from http://www.agodon.com/uploads/2/9/4/3/2943768/writing_prompts_by_kelli_russell_agodon.pdf  for daily writing prompts, here’s day 1.

The prompt: 1. Grab the closest book. Go to page 29. Write down 10 words that catch your eye. Use 7 of words in a poem. For extra credit, have 4 of them appear at the end of a line.

Unbroken

High Bridge Road, Some time in September, 1991

I sit quietly
in the middle of the deserted field
the tick of the metal cooling
the moment frozen,
silent

Silence amplified
to the point of being
almost deafening.

The engine block steams
as the toxic fluids
spill on the grass,
the anger drains away

The telephone pole, upright,
shorn off
inches from the ground
sways gently, suspended from the wires
conversations uninterrupted

Yesterday was simpler
It usually is

But you cannot go back
to the unbroken,
the unwounded,
put the oil back in the pan

So, I sit
waiting
waiting for the past to be
unbroken
waiting for the future
to come slamming back,
as the engine cools,
the coolant seeps into the soil

 

Moving to CentOS

TL;DR: Leaving OpenStack; Moving to CentOS; Still at Red Hat.

4 years ago, I came to Red Hat, and started as the OpenStack Community Liaison, working primarily with the RDO project, but more generally with all of Red Hat’s involvement in the upstream OpenStack project.

I took over from Dave Neary, but it took a while to actually replace him. His depth of knowledge and experience with the community were not easy to step into.

Over those 4 years, I’ve become much more knowledgeable about OpenStack – the community as well as the technology. It’s a wonderful community, with a passion for open source, for doing things transparently and collaboratively, and for doing things well. The individuals in the community have been great to get to know – both people here at Red Hat, as well as people in other organizations, and at the OpenStack Foundation. I could certainly call out dozens of individuals who have made my time with OpenStack smoother. The names that come to mind are Haikel Guemar, Stefano Maffuli, Perry Meyers, Eliska Malikova, Alan Pevec, Jakub Ruzicka, Rain Leander … see, I knew that as soon as I got started I would find that there’s no end in sight.

Rain, in particular, has really stood out as someone that was hugely passionate about the community around OpenStack, and was just such a delight to work with, particularly when we were able to attend the same events and work together in person.

This is why I’m so excited to announce that Rain will be taking my place as the RDO/OpenStack community manager for Red Hat, effective immediately. I cannot think of anybody more qualified, in skills and temperament, for this position, and I am completely confident that I’m leaving the community in good hands. One develops a lot of ownership of a project over four years, but I have no doubt she’ll take care of the project.

I’m not leaving Red Hat, though. Instead, I’m moving over to be more active in the CentOS community. CentOS is an exciting community that is absolutely critical to Red Hat. It’s the place where community projects, like RDO, as well as many others, do their development and testing, before being deployed and supported on Red Hat Enterprise Linux. I’ll be focusing on a variety of things, including CentOS in HPC (High Performance Computing) and IoT (Internet of Things).

CentOS presents a number of challenges from a community perspective, and I’m very pleased to be more active there. It will be an interesting and challenging place to be, and I’ll be, once again, working with an awesome group of people. I’m sure I’ll be telling lots of CentOS stories on this website in the years to come, so stay tuned.

Follow RDO on Twitter at @rdocommunity and follow CentOS on Twitter at @centos to keep up to date with the two communities, as well as learn how we work together.

World traveler maple syrup

This is very well traveled maple syrup. 

In November, Shane brought this syrup to Seville for me, from Boston. He forgot to give it to me until I had already left for the airport. So be gave it to Ruth to give me the next time we traveled together. 

Unfortunately, Ruth forgot to take it to LA when we were there last month. So she brought it to Boston to give it to me. And I just had brunch with Shane. 

So the syrup has come back home to Boston, and is now in my suitcase heading home.

Apache Board

This time last year, I was nominated for the Apache Board of Directors, and I wasn’t very keen on it. I was on the down-swing of getting burned out with ASF stuff, which comes every few years of any volunteer effort, no matter how much you believe in the mission of the organization. It does one good to step back from it every few years.

I made a rather half-hearted position statement that basically said: “all of the people on the ballot are great, and any board chosen randomly from this slate will be great.” I wasn’t elected to the board.

However, midway through the term, there were some resignations, and I found myself on the board again.

This time around, I am much more passionate about it and have actually made an attempt to state what I think the board needs. In short, it’s more delegation.

We have arrived at the size where we must delegate, trust those to whom we delegate, and resist the urge to tinker in the minutiae. This is very, very hard, and is especially hard for our current slate of directors, because we are all so deeply passionate about the mission of the Foundation.

Non-profit and volunteer-based organizations have the tendency to rely on heroes – people who take on 90% of the work, rather than delegating. The thing with heroes is that they are awesome as long as they are around. But eventually, they burn out, or retire, or move on to other things, and then suddenly you have a crisis. We need to find a way to identify areas where one person is doing a disproportionate amount of the work, and find a way to delegate that work to several other people, and then be willing to let them do the work, even when we think they’re not doing it exactly the way we’d do it ourselves.

The election of the new board will happen at the ASF members meeting on March 28th. And whether or not I’m elected, I have my work cut out for me in the coming year at Apache. There’s just so much to do.

Green

For St. Patrick’s Day, and for Poetry Friday, here’s Green, perhaps the best poem I ever wrote:

Oh, and here’s me reading it, too, over here: https://drbacchus.com/green/

Green
Dec 5, 2007

My favorite color?
Well, the question lacks context
and therefore meaning.

My favorite sky is
sharp aching blue,
the kind of blue you can
cut your fingers on
until they bleed into
and African Sunset
and plunge into a
deep purple African night,
with the diamonds of
faraway worlds scattered
like the dust thrown up
by the passing herds.

My favorite sea is
gray-green-blue stormy
swirling seaweed churned
from depths beyond imagining,
the sky a reflection of a
reflection of infinity,
winking back at you from
a million million miles down.

My favorite earth is the
dark maroon red brown
of the Maasai clay
brittle and cracked
under the Tsavo sun,
gummy sticky under the
monsoon rains
sweeping up from Victoria,
thundering past on their way
to green the highlands,
glisten on the tea leaves,
pound the coffee flowers
from their branches,
and drench the faces
of the beautiful children
running around in what
God dressed them in,
laughing and shouting
in a language I will never know
but that sings fluently
in the memory of my heart.

So, you see,
the question is unfair.
As well ask a man
to choose one food
to live on forever,
one wine to drink,
one song to sing,
one painting to gaze at
for all my days.

But if I must choose,
I choose the green
of the Kericho fields,
stretching to the horizon,
the beginning and ending
of my world.
The green of a
1952 Ferrari Barchetta,
with its greedy grinning grill
sucking in the wind
of a winding Italian mountain road
tires squealing around the corners,
flashing past a
sleeping countryside
content to be stuck in a simpler time.
The green of
the foothills of Ngong,
acacias and baobabs
clawing at the
dark, angry sky,
promising threatening delivering rain,
the hills singing to my heart,
come and walk our paths,
come and feel the
wind tugging at your hair
come and
lay on your back and
watch the clouds dance
across the sky,
dance from one end of the sky
to the edge of the world,
where the
blue falls back once again
into the red dust
of the far Mara horizon.
The green of my voice,
singing across the years,
telling my stories to
the attentive ears,
the deep green eyes
of the friend of my heart.

The Margin Is Too Narrow