Tag Archives: openstack

3 ways you find the right type of contributor and where to find them

Another one of Stormy’s questions caught my eye:

“What are 3 ways you find the right type of contributor, and where do you find them?”

Thinking back to the last few years of work on RDO, several answers come to mind:

The people asking the questions

Watch the traffic on your mailing list(s) and on the various support forums. The people that are asking the most questions are often great potential contributors. They are using the project, so they are invested in its success. They are experiencing problems with it, so they know where there are problems that need to be addressed. And they are outspoken enough, or brave enough, to talk about their difficulties publicly, so they are likely to be just as willing to talk about their solutions.

These are usually good people to approach ask ask to write about their user experience. This can often be done collaboratively, combining their questions with the eventual answers that they encountered.

If they appear to be the type of person who is implementing solutions to their problems, ask them to bring those solutions back to your community for inclusion in the code.

In RDO, people that participate in the rdo-list mailing list will sometimes end up contributing their solutions to the project, and eventually becoming regular contributors. We probably need to do a better job of encouraging them to do this, rather than just hoping it’s going to happen on its own.

The people answering the questions

In watching the question-and-answer on ask.openstack.org I often see names I don’t recognize, answering questions related to RDO. Sometimes these are Red Hat engineers who have recently joined the project, and I haven’t met yet. But sometimes, it’s people from outside of Red Hat who have developed expertise on RDO and are now starting to pay that back.

These are the people that I then try to approach via the private messaging feature of that website, and ask them what their story is. This occasionally evolves into a conversation that brings them to more active involvement in the project.

Most people like to be asked, and so asking someone specifically if they’d be willing to come hang out in the IRC channel, or answer questions on the mailing list, tends to be fairly effective, and is a good step towards getting them more involved in the project.

The people complaining

This a tricky one. People complain because something is broken, or doesn’t work as they expect it to. The traditional response to this in the free software world is “Patches Welcome.” This is shorthand for “Fix it yourself and stop bugging me.”

The trick here is to recognize that people usually take the trouble to complain because they want it to work and they’re looking for help. This passion to just make things work can often be harnessed into contributions to the project, if these people are treated with patience and respect.

Patience, because they’re already frustrated, and responding with frustration or defensiveness is just going to make them angry.

Respect, because they are trying to solve an actual real world problem, and they’re not just there to hassle you.

It’s important that you fix their problem. Or at least, try to move them towards that solution.

Once the problem is addressed, ask them to stay. Ask them to write about their situation, and the fix to it. Ask them to stick around and answer other questions, since they have demonstrated that they care about the project, at least to the point of getting it working.

When people complain about your project, you also have a great opportunity to brush them off and persuade them that you are uncaring and unwelcoming, which they will then go tell all of their friends and Twitter followers. This can be a very expensive thing to do, for your community. Don’t do that.

When people come to #rdo on Freenode IRC to ask their RDO and OpenStack questions, I frequently (usually) don’t know the answer myself, but I try to make an effort to connect them with the people that do know the answer. Fortunately, we have an awesome community, and once you bring a question to the attention of the right person, they will usually see it through to the right solution.

What’s new in OpenStack Ocata

OpenStack Ocata has now been out for a little over a month – https://releases.openstack.org/ – and we’re about to see the first milestone of the Pike release. Past cycles show that now’s about the time when people start looking at the new release to see if they should consider moving to it. So here’s a quick overview of what’s new in this release.

First, it’s important to remember that the Ocata cycle was very short. We usually do a release every six months, but with the rescheduling of the OpenStack Summit and OpenStack PTG (Project Team Gathering) events, Ocata was squeezed into 4 months to realign the releases with these events. So, while some projects squeezed a surprising amount of work into that time, most projects spent the time on smaller features, and finishing up tasks leftover from the previous release.

At a high level, the Ocata release was all about upgrades and containers – themes that I heard from almost every team I interviewed. (For the full interview series, see https://goo.gl/3aCQ2d ) How can we make upgrades smoother, and how can we deploy bits of the infrastructure in containers. These two things are closely related, and there seems to be more cross-project collaboration this time around than I’ve noticed in the past.

And the themes of upgrades and containers will continue to be prominent in the Pike cycle.

Highlights in the Ocata cycle include:

Auto-healing: Work was done in Heat to make it easier to recover from service failure. When an outage is detected, you can have Heat automatically spin up a replacement service, and swap it out without any intervention on the part of the operator.

Composability: Composable roles are a feature whereby you can specify details of how things are deployed, rather than allowing OpenStack to choose. You can, for example, specify that a particular hardware configuration be used for particular services. This is termed Composable Roles. Work was done in Ocata to expand this to composable upgrades, so that these roles are respected across upgrades as well.

Multi-factor auth in Keystone: Work was done in Keystone to improve support of MFA, including OTP (One Time Password) support, and per-user token expiration rules.

NFV: Network Function Virtualization continues to be an area where we’re seeing a lot of activity, and so a lot of the work in Nova, Neutron, and various other projects focuses on these developments. NFV has become more stable in this release, and is more fully integrated into TripleO for ease of deployment. This effort is happening under the Apex project – https://wiki.opnfv.org/display/apex/Apex

Upgrades: Upgrades were a common theme across all projects, with the emphasis being the ability to upgrade from one release to the next with as close to zero downtime as possible. Much of this work centers around TripleO, Heat, and Mistral, for orchestration and automation of the process.

Containers: While centered around the Kolla project, containerization was a theme in many of the projects this cycle. The eventual goal, at least according to some, is that OpenStack services will be deployed in containers by default by the Pike release. This of course poses a real challenge for the Ocata -> Pike upgrade path (migrating from non-container to container in the course of the upgrade), and that’s something that the TripleO people are working hard on.

Security: TLS-everywhere made strides forward in Ocata, with connections between services moving to TLS. This involves changes to Barbican as well, for key management for the shared keys between services, to ensure that your traffic is secure between components of your cloud, which may be located in different data centers around the world.

Collaboration: Something I heard more this year than in previous years was talk of collaboration between projects. This has, of course, always been happening. However at the PTG in Atlanta, it was a major focus, with time set aside for cross-project meetings focusing on the interface between one service and another. I also heard from several people that the PTG allowed a focus, and a camaraderie, that was not possible when the design summit was part of OpenStack Summit. This resulted in fewer interpersonal tensions, and a lot more work getting done.

Everything else: The difficulty with OpenStack is that it’s just so big. While these are the things that stood out to me, someone else is likely to pull out different highlights, depending on their interests. So I encourage you to look at some of the other “What’s new in Ocata” articles out there, including especially “53 new things to look for in OpenStack Ocata” – https://www.mirantis.com/blog/53-new-things-to-look-for-in-openstack-ocata/ and, if you have a lot of time, or have interest in a particular project, check out the official release notes – https://releases.openstack.org/ocata/  And take a moment to watch my interviews with various Red Hat OpenStack engineers, from the Atlanta PTG, here: https://goo.gl/3aCQ2d.

OpenStack PTG, trip report

last week, I attended the OpenStack PTG (Project Teams Gathering) in Atlanta.

Even more in depth: PTG info at https://www.openstack.org/blog/2016/05/faq-evolving-the-openstack-design-summit/

TL;DR:

1) This is a hugely productive event, with project teams getting an enormous amount of work done without the distractions that are usually present at a conference.

2) I remain very concerned about how this event will effect the
character of OpenStack Summit – removing the bulk of the engineers from that event, and making it more product/marketing/sales focused. Time will tell

At the gathering, I did 23 interviews with Red Hat engineers about what they did in the Ocata release. You can see some of those interview on the RDO YouTube Channel. I’m not done editing them all yet, but they will appear over the coming weeks as part of various blog posts, as well as all of them appearing in that YouTube playlist.

I am constantly blown away by the passion, expertise, and
professionalism of the folks I get to work with. Wow.

Anyways, more about the PTG.

I was (and, really, still am) very skeptical about this new format.
Splitting OpenStack Summit into four events rather than two has already had significant impact on travel budgets, not just at Red Hat, but also at other companies involved in OpenStack. A lot of companies, for example, didn’t send anyone to FOSDEM, and we had a hard time staffing the OpenStack table there. Usually people work one shift at the table, but this year several of us worked 4 and 5 shifts to cover all the slots.

I am concerned that splitting the engineers off into their own event
will significantly change the character of OpenStack Summit from being a community-centric, tech-centric event, to more of a sales and marketing event, light on technical depth.

But this event, for what was intended, has already been amazing.
Everyone is commenting on how much is getting done, how much less distracted the team meetings are, how much better the teams are gelling than they have at any previous event. This is a working event, and people are here to get work done. They are meeting all day, every day, working on plans and blueprints, and fighting out agreements on things that would take weeks in email, and everyone seems VERY pleased with outcomes.

So, perhaps the trade off will be worth it. Time will tell. Regardless, Erin Disney and her team put on an amazing event that fulfilled, and exceeded, its goals.

On Wednesday  night, everyone that has ever contributed a patch to RDO was invited for drinks and hors d’oeuvres at the SideBar, and while there the RDO Ocata release announcement was sent out.

We had about 50 people in attendance, who ate and drank up all of my budget in about 2 hours.

Here’s some pictures.

 

Project Leader

I was recently asked to write something about the project that I work on – RDO – and one of the questions that was asked was:

A healthy project has a visible lead(s). Who is the project lead(s) for this project?

This struck me as a strange question because, for the most part, the open source projects that I choose to work on don’t have a project lead, but are, rather, led by community consensus, as well as a healthy dose of “Just Do It”. This is also the case with RDO, where decisions are discussed in public on the mailing list, and on IRC meetings, and those that step up to do the work have more practical influence than those that just talk about it.

Now, this isn’t to say that nobody takes leadership or ownership of the projects. In many senses, everyone does. But, of course, certain people do rise to prominence from time to time, just based on the volume of work that they do, and these people are the de facto leaders for that moment.

There’s a lot of different leadership styles in open source, and a lot of projects do in fact choose to have one technical leader who has the final say on all contributions. That model can work well, and does in many cases. But I think it’s important for a project to ask itself a few questions:

  • What do we do when a significant number of the community disagrees with the direction that this leader is taking things?
  • What happens when the leader leaves? This can happen for many different reasons, from vacation time, to losing interest in the project, to death.
  • What do we do when the project grows in scope to the point that a single leader can no longer be an expert on everything?

A strong leader who cares about their project and community will be able to delegate, and designate replacements, to address these concerns. A leader who is more concerned with power or ego than with the needs of their community is likely to fail on one or more of these tests.

But, I find that I greatly prefer projects where project governance is of the people, by the people, and for the people.

OpenStack Summit Barcelona, 3 of N

Continuing the saga of OpenStack Summit Barcelona …

Wednesday was a very long day – about 10 hours working the Red Hat booth, with hundreds and hundreds of people dropping by to chat, get tshirts, ask questions about our various products, and just hang out.

While Tuesday was “day two”, Wednesday is the second full day of the expo hall, and so this is the time when you get even less frantic swag-seeking, and more people looking for information. Which is good, because we were starting to run out of the swag.

What stood out on Wednesday was the sheer number of people who stopped by just to say how awesome RDO is, and how much they appreciate the work that the RDO community does. Several people also asked how they could get involved in making things happen that are not happening yet. So, this was very encouraging.

Wednesday evening was a quiet dinner-then-bed evening, as is usually the case by this point in a conference, when the true exhaustion of long days and jet lag really catches up. I had dinner with a small group of friends and coworkers, which was really delicious, but I have no idea how to tell you what I had. 🙂

And Thursday was the final day. We got the expo hall open time wrong, so I was actually there in the booth for 2 hours before it opened. Gah! And then, another really long day of answering questions, and giving away the last of the ducks.

img_20161030_154636The duck tradition that was started in Paris continues, and we plan to have more ducks in Boston. We only had 800 ducks in Barcelona, because I waited kind of late to order, and the vendor was low on inventory. Any suggestions of how the duck should dress for Boston?

We closed up just after 4, packed everything up, and I headed to the beach. Unfortunately, right about the time I got there, it started to get overcast and windy, so I only got about a half hour on the beach before I gave up and left, shivering. Oh, well, better than nothing.

Thursday night was another dinner-and-fall-in-bed-early evening, as I was completely exhausted. Unfortunately, as usual when I have a flight in the morning, I didn’t get much sleep. I have this bad habit of waking up every few hours to check the clock, even though I have several alarms set. Some day, I hope to get over that.

On Friday, I shared a cab to the airport with Jen, and we boarded the flight for JFK.

It’s good to be home.

OpenStack Summit, Barcelona, 2 of n

Tuesday, the first day of the main event, was, as always, very busy. I spent most of the day working the Red Hat booth. We started at 10 setting up, and the mob came in around 10:45.

Day two of booth duty is always interesting, because it’s after the swag feeding frenzy has died down a bit that you start hearing from the people that actually care about what you’re “selling”. You get the questions. And what’s been fascinating in the 6 summits I’ve attended is that the bar has been raised a LOT on the questions. In Hong Kong, my first Summit, there were still a lot of people asking what OpenStack was, and nobody had any idea what RDO was. Now, the questions are about specific deployment scenarios, projects that aren’t yet being packaged, the future of TripleO, and so on, with only a handful of people asking what RDO is.

OpenStack has clearly made the transition from “something to consider some day” to “of course we are, and what are you going to do to make it better?”

Another awesome improvement this Summit was how the RDO community stepped up to help in the booth. Every single hour of the day, I had at least one, and usually two, members of the RDO community in the booth with me, either doing an “Ask Me Anything About RDO”, or doing some kind of a demo. It was *awesome*. Maybe next year, I’ll just stay home. 😉

The highlight of the day was the RDO/Ceph community meetup. We had 4 hours at the Gym Bar in the Princess Hotel.

Members of the Ceph and RDO community presented, lightning talk style (5 minute presentations) on a variety of topic. Speakers were threatened with being thrown in the pool if they went over 5 minutes, but we managed to restrain ourselves.

By the end, we had checked in 215 people overall, and we had 12 speakers. The food was good. The speakers were awesome. The only complaint was that the people not actually listening to the talks would NOT shut up, so it was hard to hear. Eventually, one of the speakers shouted at them to shut up or get out, and most of them moved to the other side of the room.

I have a recording of the event, but I don’t expect it’s going to be usable, due to the noise level. I haven’t had a chance to review it yet. Next week, for sure. I also hope to have (some of?) the presentation slides from the various speakers posted somewhere. Watch rdo-list and/or @rdocommunity for details.

After the talks were over, we had an hour or so left, and I cowardly skipped out. There comes a time when I have just had too much social interaction, and I need quiet time.

So, that was Tuesday. Another success, and another day to be glad that I work with such an awesome community.

 

OpenStack Summit, Barcelona, 1 of n

I have the best intentions of blogging every day of an event. But every day is always so full, from morning until the time I drop into bed exhausted.

I used to imagine wandering around the world like Hemingway, seeing exotic places, and writing witty stories about the interesting people I met. I have the great good fortune to have the traveling as part of my job. If only I could find the time for the stories.

Anyways … I’m in Barcelona for OpenStack Summit. This is always an impressive, and somewhat overwhelming, event, with 7000+ people attending, dozens of companies presenting their various products, hundreds of technical presentations, and after-hour events every evening.

The photos are on flickr, at https://www.flickr.com/photos/rbowen/albums/72157675620637236

On Sunday, I met up with Jen and various other members of the events team to scope out the venue. We walked over to the Princess Hotel where a number of our meetings and social events are to be held. And we walked down to El Boo, where the employee party was to be held. Both venues were just great.

Later in the day I visited Sagrada Familia, a cathedral which has been under construction since 1884, on and off, and isn’t done yet. It was weird and improbable, and fascinating, and beautiful. I think the architect’s driving passion was to be different from anything you’ve ever seen before.

I also spent a little time on the lovely beach, Mar Bella, which is right in front of my hotel.

On Monday, we had the RDO Infrastructure gathering in one of the rooms at the Princess. we had about 20 people in attendance, and made good progress on a number of issues. The Ocata cycle is going to see more improvements to how RDO works.  More details on this meeting coming to the RDO mailing list soon.

Although the conference didn’t officially start until Tuesday morning, the “booth crawl” was Monday evening – the odd ritual where swag-hungry attendees rush from booth to booth, grabbing all the free stuff they can carry, and having the occasional conversation with booth staff. Sometimes, the booth crawl can be a depressing experience, with people refusing to make eye contact, and just grabbing the stuff. But this was actually really great, with people excited about RDO, and wanting to learn more about we had to show.

Monday night was the Red Hat employee party at El Boo. It’s a boat-shaped restaurant sitting on one of the stone piers along the beach, and we had the whole place to ourselves for the entire night. It was a lot of fun. I stayed until midnight, when several friends toasted my birthday.

All together, it was a lovely start to the week. And there’s so much more to come.

OpenStack 6th Birthday, Lexington, KY

IMG_1580

Yesterday I spent the day at the University of Kentucky at the OpenStack 6th Birthday Meetup. The day was arranged by Cody Bumgardner and Kathryn Wong from the UK College of Engineering.

UK has an OpenStack cloud that they use for instruction, as well as for research, and they’ve got a 6PB Ceph cluster hanging off of it. There were presentations about the various aspects of this cloud, and how it’s being used.

I gave an introduction to OpenStack – the Foundation, the software, and the community – for the attendees that were just getting started. Patrick McGarry gave a talk about how Ceph works.

Nassir Hussamddin closed the day with a really cool presentation about CloudLab, which is a tool shared by a number of universities that allows users to spin up an OpenStack cloud (not just a VM, but an entire cloud) on demand for testing purposes. Definitely worth looking into further.

Big thanks to Dell, the University of Kentucky, and the OpenStack Foundation, who, along with RDO, sponsored this event.

Red Hat Summit in Review

Despite my best intentions of blogging every day at Red Hat Summit, time got away from me, as it often does at these events. There’s always 3 things going on, and it’s hard to find a moment between that first cup of coffee, and stumbling into bed at the end of the night.

I spent almost the entire event working the RDO booth in the Community Central section of the expo hall. While traffic wasn’t as heavy as at OpenStack Summit, it was still pretty constant.

In the swag department, I had our “what does RDO stand for” tshirts, and TripleO QuickStart USBkeys.

IMG_20160629_122305

Several things stood out to me from this audience.

First, I was delighted to hear story after story of companies that use RDO in the test/dev/lab environment, and use Red Hat OpenStack Platform in their public/production environments. This is what I really want to see happening, so it’s very gratifying when I get anecdotal evidence that it is happening. Now, if I can only convince those folks to follow up with case study writeups for the user stories page.

Second, from people who were not quite as familiar with either RDO or OpenStack, if there was a consistent thread in the questions, it was confusion as to the overlap between oVirt (or Red Hat Enterprise Virtualization), OpenStack, and OpenShift, and when one might use one vs. the others. This looks like a good opportunity for some blog posts around what the overlap is, what the distinctions are, and what recommendations are for using one or another.

Brian and I did an OpenStack vs oVirt comparison talk at last year’s Red Hat Summit, but I don’t believe we ever wrote it up anywhere. And OpenShift has the added confusion of having a similar name, which gets people kind of mixed up before they even consider the feature set.

IMG_20160702_114227

And, finally, the week was yet another reminder that I work for the best company in the world, with the best coworkers. I feel sorry for the rest of you.

 

The OpenStack Big Tent

I’ll be giving a presentation at LinuxCon next week about the ‘Big Tent’ at OpenStack. It’ll go something like this …

The OpenStack Big Tent

OpenStack is big and complicated. It’s composed of many moving parts, and it can be somewhat intimidating to figure out what all the bits do, what’s required, what’s optional, and how to put all the bits together.

The Problem

In the attempt to tame this confusion, the OpenStack Technical Committee defined what’s part of the Integrated Release and what’s not, so that you, the consumer, know what’s in and what’s out. One of the unintended side effects of this was that new projects were treated as second class citizens, and had trouble getting resources, developers, and a seat at the table at the developer summit.

As OpenStack continues to grow, this became more and more of a problem.

With the Liberty cycle, the Technical Committee has taken another look at what makes a project part of OpenStack, to make things better for the projects, as well as for the consumers.

Are You OpenStack?

The question that has been asked all along about any project wanting to be part of OpenStack was, is this thing OpenStack? To answer this question, a number of criteria were applied, including interoperability with existing bits, maturity, diversity (i.e., is this thing entirely developed by one company, or does it have broader participation?), and other things. This process was called Incubation, and once a project graduated from Incubation, it could be part of the integrated release.

As the stack grew, these questions became harder to answer, and more projects were getting left out of the tent, to everyone’s detriment, and to the growing confusion of the folks trying to use the software.

So, in recent months, the Technical Committee (TC) has decided to turn the question around. Rather than asking “Is thing thing OpenStack?” the new question is “Are You OpenStack?”

This changes how we look at making the determination on a few fronts.

OpenStack is People!

As Thierry Carrez Sean Dague said in their Summit presentation, OpenStack is composed of teams of people, working towards the betterment of the overall project. To that end, we’ll now welcome everyone to the table, if they are OpenStack.

So … how’s this defined?

Something is OpenStack if it:

1) Aligns with the OpenStack Mission: to produce the ubiquitous Open Source Cloud Computing platform that will meet the needs of public and private clouds regardless of size, by being simple to implement and massively scalable.

2) Follows the OpenStack Way – Open Source, Open Community, Open Development, and Open Design. (More here)

3) Strives for interoperability with other things that are OpenStack.

4) Subjects itself to the governance of the Technical Committee

Tags

But while this solves one problem, it creates another. As a user of the OpenStack software, I really still need to know what’s in and what’s out.

There is no longer going to be a single release that is defined to be OpenStack, how do I know which bits I need, and which bits I can live without?

To help sort this out, a system of community-defined tags will be applied to the various pieces of OpenStack, starting with “tc-approved-release” which will, initially, just reflect what was already the integrated release. These tags will indicate project maturity, as well as other considerations. Packagers, like the CentOS Cloud Sig, can then use those tags to determine what they want to include in distributions.

Who’s In

As a result of this change, we immediately have several new projects that are part of OpenStack, that were previously held at arm’s length:

What’s Next?

People are still going to expect a release, and exactly what that means going forward is a little unclear. Every six months there will be a release which will include stuff tagged ‘tc-approved-release’. It will be opt-in – that is, projects can participate, or not, as they like. Or they can release on their own cadence, as was discussed about a year ago.

There are still some details to be worked out, but the overall benefit to the community seems like it’s going to be huge, as we include more great ideas, and more passionate people, inside the Big Tent.