Tag Archives: openstack

SnowpenStack

I’m heading home from SnowpenStack and it was quite a ride. As Theirry said in our interview at the end of Friday (coming soon to a YouTube channel near you), rather than spoiling things, the freak storm and subsequent closure of the event venue served to create a shared experience and camaraderie that made it even better.

In the end I believe I got 29 interviews, and I’ll hopefully be supplementing this with a dozen online interviews in the coming weeks.

If you missed your interview, or weren’t at the PTG, please contact me and we’ll set something up. And I’ll be in touch with all of the PTLs who were not already represented in one of my interviews.

A huge thank you to everyone that made time to do an interview, and to Erin and Kendall for making everything onsite go so smoothly.

OpenStack PTG and the Beast From The East

I’m at the OpenStack PTG in Dublin. I’ve started posting some of my videos on my personal YouTube channel – http://youtube.com/RichBowen – as well as on my work channel – http://youtube.com/RDOCommunity.

It turns out we’ve planned an event in the middle of the storm of the century, which the press is calling the Beast From The East.

So far it hasn’t amounted to a lot, but there’s a LOT more snow promised for this afternoon, and the government has warned people to stay off the roads after 4 unless they have a really good reason. Which is disappointing because I have a party planned to start at 6. I’m still trying to get hold of the venue to decide what happens next.

Yesterday I suddenly realized that I had bought my plane ticket for Sunday instead of Saturday by mistake. I quickly booked another hotel room for Saturday night, closer to the airport. Well, it turns out this may have been the most fortunate travel error I’ve made in a long time, as pretty much everything is cancelled for the next few days, and getting out of here on Saturday might have been impossible.

For now, we’re just watching the weather reports, and hoping for the best.

CERN CentOS Dojo, part 3 of 4: Friday Dojo

1 Comment

On Friday, I attended the CentOS Dojo at CERN, in Meyrin Switzerland.

CentOS dojos are small(ish) gatherings of CentOS enthusiasts that happen all over the world. Each one has a different focus depending on where it is held and the people that plan and attend it.

You can read more about dojos HERE.

On Friday, we had roughly 60-70 people in attendance, in a great auditorium provided by CERN. We had 97 people registered, and 75% is pretty standard turnout for free-to-register events, so we were very pleased.

You can get a general idea of the size of the crowd in this video:

The full schedule of talks can be seen here: https://indico.cern.ch/event/649159/timetable/#20171020

There was an emphasis on large-scale computing, since that’s what CERN does. And the day started with an overview of the CERN cloud computing cluster. Every time I attend this talk (and I’ve seen it perhaps 6 times now) the numbers are bigger and more impressive.

CERN and Geneva

This time, they reported 279 thousands cores in their cluster. That’s a lot. And it’s all running RDO. This makes me insanely proud to be a small part of that endeavor.

Other presentations included reports from various SIGs. SIGs are Special Interest Groups within CentOS. This is where the work is done to develop projects on top of CentOS, including packaging, testing, and promotion of those projects. You can read more about the SIGs here: https://wiki.centos.org/SpecialInterestGroup

If you want to see your project distributed in the CentOS distro, a SIG is the way to make this happen. Drop by the centos-devel mailing list to propose a SIG or join an existing one.

The entire day was recorded, so watch this space for the videos and slides from the various presentations.

The CERN folks appeared very pleased with the day, and stated their intention to do the event again on an annual basis, if all works out. These things aren’t free to produce, of course (even though we strive to make them always free to attend) so if your organization is interested in sponsoring future dojos, please contact me. I’ll also be publishing a blog post over on seven.centos.org in the coming days about what’s involved in doing one of these events, in case you’d like to host one at your own facility..

CERN Centos Dojo, event report: 2 of 4 – CERN tours

3 Comments

(This post is the second in a series of four. They are gathered here.)

The second half of Thursday was where we got to geek out and tour various parts of CERN.

I was a physics minor in college, many years ago, and had studied not just CERN, but many of the actual pieces of equipment we got to tour, so this was a great privilege.

We started by touring the data center where the data from all of the various physics experiments is crunched into useful information and discoveries. This was amazing for a number of reasons.

From the professional side, CERN is the largest installation of RDO – the project I work with at work – that we know of. 279 thousand cores running RDO OpenStack.

For those not part of my geek world, that translates into hundreds of thousands of physical computers, arranged in racks, crunching data to unlock the secrets of the universe.

For those that are part of my geek world, you can understand why this was an exciting thing to see in person and walk through.

The full photo album is here, but I want to particularly show a couple of shots:

Visiting CERN

Here we have several members of the RDO and CentOS team standing in front of some of the systems that run RDO.

Visiting CERN

And here we have a photo that only a geek can love – this is the actual computer on which the very first website ran. Yes, boys and girls, that’s Tim Berners-Lee’s desktop computer from the very first days of the World Wide Web. It’s ok to be jealous.

There will also be some video over on my YouTube channel, but I haven’t yet had an opportunity to edit and post that stuff.

Next, we visited the exhibit about the Superconducting Super Collider, also known as the Large Hadron Collider. This was stuff that I studied in college, and have geeked out about for the years since then.

There are pictures from this in the larger album, but I want to point out one particular picture of something that absolutely blew my mind.

Most of the experiments in the LHC involve accelerating sub-atomic particles (mostly protons) to very high speeds – very close to the speed of light – and then crashing them into something. When this happens, bits of it fly off in random directions, and the equipment has to detect those bits and learn things about them – their mass, speed, momentum, and so on.

In the early days, one of the the ways that they did this was to build a large chamber and string very fine wires across it, so that when the particles hit those wires it would cause electrical impulses.

Those electrical impulses were captured by these:

CERN visit

Those are individual circuit boards. THOUSANDS of them, each individually hand-soldered. Those are individual resistors, capacitors, and ICs, individually soldered to boards. The amount of work involved – the dedication, time, and attention to detail – is simply staggering. This photo is perhaps 1/1000th of the total number of boards. If you’ve done any hand-soldering or electronic projects, you’ll have a small sense of the scale of this thing. I was absolutely staggered by this device.

Outside on the lawn were several pieces of gigantic equipment that were used in the very early days of particle physics, and this was like having the pages of my college text book there in front of me. I think my colleagues thought I’d lost my mind a little.

College was a long time ago, and most of the stuff I learned has gone away, but I still have the sense of awe of it all. That an idea (let’s smash protons together!) resulted in this stuff – and more than 10,000 people working in one place to make it happen, is really a testament to the power of the human mind. I know some of my colleagues were bored by it all, but I am still reeling a little from being there, and seeing and touching these things. I am so grateful to Tim Bell and Thomas Oulevey for making this astonishing opportunity available to me.

Finally, we visited the ATLAS experiment, where they have turned the control room into a fish tank where you can watch the scientists at work.

CERN visit

What struck me particularly here was that most of the people in the room were so young. I hope they have a sense of the amazing opportunity that they have here. I expect that a lot of these kids will go on to change the world in ways that we haven’t even thought of yet. I am immensely jealous of them.

So, that was the geek chapter of our visit. Please read the rest of the series for the whole story.

CERN Centos Dojo 2017, Event report (0 of 4)

3 Comments

For the last few days I’ve been in Geneva for the CentOS dojo at CERN.

What’s CERN? – http://cern.ch/

What’s a dojo? – https://wiki.centos.org/Events/Dojo/

What’s CentOS? – http://centos.org/

A lot has happened that I want to write about, so I’ll be breaking this into several posts:

(As usual, if you’re attempting to follow along on Facebook, you’ll be missing all of the photos and videos, so you’ll really want to go directly to my blog, at https://drbacchus.com/)

 

Event report: OpenStack PTG

Last week I attended the second OpenStack PTG, in Denver. The first one was held in Atlanta back in February.

This is not an event for everyone, and isn’t your standard conference. It’s a working meeting – a developers’ summit at which the next release of the OpenStack software is planned. The website is pretty blunt about who should, and should not, attend. So don’t sign up without knowing what your purpose is there, or you’ll spend a lot of time wondering why you’re there.

I went to do the second installment of my video series, interviewing the various project teams about what they did in the just-released version, and what they anticipate coming in the next release.

The first of these, at the PTG in Atlanta, featured only Red Hat engineers. (Those videos are HERE.) However, after reflection, I decided that it would be best to not limit it, but to expand it to the entire community, focusing on cross-company collaboration, and trying to get as many projects represented as I could.

So, in Denver I asked the various project PTL (project technical leads) to do an interview, or to assemble a group of people from the project to do interviews. I did 22 interviews, and I’m publishing those on the RDO YouTube channel – http://youtube.com/RDOCommunity – just as fast as I can get them edited.

I also hosted an RDO get-together to celebrate the Pike release, and we had just over 60 people show up for that. Thank you all so much for attending! (Photos and video from that coming soon to a blog near you!)

So, watch my YouTube channel, and hopefully by the end of next week I’ll have all of those posted.

I love working with the OpenStack community because they remind me of Open Source in the old days, when developers cared about the project and the community at least as much, and often more, than about the company that happens to pay their paycheck. It’s very inspiring to listen to these brilliant men and women talking about their projects.

Event report: Red Hat Summit, OpenStack Summit

Event report: Red Hat Summit, OpenStack Summit

May 1-5, 2017 and May 8-11, 2017

During the first two weeks of May, I attended Red Hat Summit, followed by OpenStack Summit. Since both events were in Boston (although not at the same venue), many aspects of them have run together.

Mini-cluster

On the first day of Red Hat Summit, I received the mini-cluster, which had been built in Brno for the April Brno open house. There were one or two steps missing from the setup instructions, so with a great deal of help from Hugh Brock, it too most of the first day to get the cluster running. We’ll be publishing more details about the mini-cluster on the RDO blog in the next week or two. However, most of the problems were 1) it was physically connected incorrectly (ie, my fault) and 2) there were some routing table changes that were apparently not saved after initial setup.

Once the cluster was up, we connected to the ManageIQ cluster on the other side of our booth, and they were able to manage our OpenStack deployment. Thus, we were able to demonstrate the two projects working together.

In future events, we’d like to bring more projects into this arrangement – say, use Ceph for storage, or have ManageIQ managing OpenStack and oVirt, for example.

After we got the cluster working, in subsequent days, we just had to power it on, follow the startup instructions, and be patient. Again, more details of this will be in the RDO blog post in the coming  weeks.

Upcoming CentOS Dojos

I had conversations with two groups about planning upcoming CentOS Dojos.

The first of these will be at Oak Ridge National Labs (ORNL), and is
now tentatively scheduled for the first Tuesday in September.  (If you saw my internal event report, I mentioned July/August. This has since changed.) They’re interested in doing a gathering that would be about both CentOS and OpenStack, and draw together some of the local developer community. This will be held in conjunction with the local LOPSA group.

The second Dojo that we’re planning will be at CERN, where we have a great relationship with the cloud computing group, who run what we believe to be the largest RDO installation in the world. We have a tentative date of October 20th, immediately before Open Source Summit in Prague to make it easier to combine two trips for those traveling internationally. This event, too, would cover CentOS topics as well as OpenStack/RDO topics.

If you’re interested in participating in either one of these events, you need to be on the centos-promo mailing list. Send mail to centos-promo-subscribe@centos.org to subscribe, or visit https://lists.centos.org/mailman/listinfo/centos-promo for the
clicky-clicky version.

General Impressions

The Community Central area at Red Hat Summit was awesome. Sharing center stage with the product booths was a big win for our upstream first message, and we had a ton of great conversations with people who grasped the “X is the upstream for Red Hat X” concept, seemingly, for the first time. The “The Roots Are In The Community” posters resonated with a lot of people, so huge thanks to Tigert for pulling those together at the last minute.

The collaboration between RDO and ManageIQ was very rewarding, and helped promote the CloudForms message even more, because people could see it in action, and see how the communities work together for the greater good of humanity. I look forward to expanding this collaboration to all of the projects in the Community Central area by next year.

The space for Red Hat Summit was huge, making the crowd seem a lot smaller than it actually was. The opposite was true for  OpenStack Summit, where it was always crowded and seemed very busy, even though the crowd was smaller than last year.

 

Where next?

In three weeks I’ll be heading to the High Performance Computing event in Frankfurt. My mission there is to talk with people that are using CentOS and RDO in HPC, and collect user stories.

3 ways you find the right type of contributor and where to find them

Another one of Stormy’s questions caught my eye:

“What are 3 ways you find the right type of contributor, and where do you find them?”

Thinking back to the last few years of work on RDO, several answers come to mind:

The people asking the questions

Watch the traffic on your mailing list(s) and on the various support forums. The people that are asking the most questions are often great potential contributors. They are using the project, so they are invested in its success. They are experiencing problems with it, so they know where there are problems that need to be addressed. And they are outspoken enough, or brave enough, to talk about their difficulties publicly, so they are likely to be just as willing to talk about their solutions.

These are usually good people to approach ask ask to write about their user experience. This can often be done collaboratively, combining their questions with the eventual answers that they encountered.

If they appear to be the type of person who is implementing solutions to their problems, ask them to bring those solutions back to your community for inclusion in the code.

In RDO, people that participate in the rdo-list mailing list will sometimes end up contributing their solutions to the project, and eventually becoming regular contributors. We probably need to do a better job of encouraging them to do this, rather than just hoping it’s going to happen on its own.

The people answering the questions

In watching the question-and-answer on ask.openstack.org I often see names I don’t recognize, answering questions related to RDO. Sometimes these are Red Hat engineers who have recently joined the project, and I haven’t met yet. But sometimes, it’s people from outside of Red Hat who have developed expertise on RDO and are now starting to pay that back.

These are the people that I then try to approach via the private messaging feature of that website, and ask them what their story is. This occasionally evolves into a conversation that brings them to more active involvement in the project.

Most people like to be asked, and so asking someone specifically if they’d be willing to come hang out in the IRC channel, or answer questions on the mailing list, tends to be fairly effective, and is a good step towards getting them more involved in the project.

The people complaining

This a tricky one. People complain because something is broken, or doesn’t work as they expect it to. The traditional response to this in the free software world is “Patches Welcome.” This is shorthand for “Fix it yourself and stop bugging me.”

The trick here is to recognize that people usually take the trouble to complain because they want it to work and they’re looking for help. This passion to just make things work can often be harnessed into contributions to the project, if these people are treated with patience and respect.

Patience, because they’re already frustrated, and responding with frustration or defensiveness is just going to make them angry.

Respect, because they are trying to solve an actual real world problem, and they’re not just there to hassle you.

It’s important that you fix their problem. Or at least, try to move them towards that solution.

Once the problem is addressed, ask them to stay. Ask them to write about their situation, and the fix to it. Ask them to stick around and answer other questions, since they have demonstrated that they care about the project, at least to the point of getting it working.

When people complain about your project, you also have a great opportunity to brush them off and persuade them that you are uncaring and unwelcoming, which they will then go tell all of their friends and Twitter followers. This can be a very expensive thing to do, for your community. Don’t do that.

When people come to #rdo on Freenode IRC to ask their RDO and OpenStack questions, I frequently (usually) don’t know the answer myself, but I try to make an effort to connect them with the people that do know the answer. Fortunately, we have an awesome community, and once you bring a question to the attention of the right person, they will usually see it through to the right solution.

What’s new in OpenStack Ocata

OpenStack Ocata has now been out for a little over a month – https://releases.openstack.org/ – and we’re about to see the first milestone of the Pike release. Past cycles show that now’s about the time when people start looking at the new release to see if they should consider moving to it. So here’s a quick overview of what’s new in this release.

First, it’s important to remember that the Ocata cycle was very short. We usually do a release every six months, but with the rescheduling of the OpenStack Summit and OpenStack PTG (Project Team Gathering) events, Ocata was squeezed into 4 months to realign the releases with these events. So, while some projects squeezed a surprising amount of work into that time, most projects spent the time on smaller features, and finishing up tasks leftover from the previous release.

At a high level, the Ocata release was all about upgrades and containers – themes that I heard from almost every team I interviewed. (For the full interview series, see https://goo.gl/3aCQ2d ) How can we make upgrades smoother, and how can we deploy bits of the infrastructure in containers. These two things are closely related, and there seems to be more cross-project collaboration this time around than I’ve noticed in the past.

And the themes of upgrades and containers will continue to be prominent in the Pike cycle.

Highlights in the Ocata cycle include:

Auto-healing: Work was done in Heat to make it easier to recover from service failure. When an outage is detected, you can have Heat automatically spin up a replacement service, and swap it out without any intervention on the part of the operator.

Composability: Composable roles are a feature whereby you can specify details of how things are deployed, rather than allowing OpenStack to choose. You can, for example, specify that a particular hardware configuration be used for particular services. This is termed Composable Roles. Work was done in Ocata to expand this to composable upgrades, so that these roles are respected across upgrades as well.

Multi-factor auth in Keystone: Work was done in Keystone to improve support of MFA, including OTP (One Time Password) support, and per-user token expiration rules.

NFV: Network Function Virtualization continues to be an area where we’re seeing a lot of activity, and so a lot of the work in Nova, Neutron, and various other projects focuses on these developments. NFV has become more stable in this release, and is more fully integrated into TripleO for ease of deployment. This effort is happening under the Apex project – https://wiki.opnfv.org/display/apex/Apex

Upgrades: Upgrades were a common theme across all projects, with the emphasis being the ability to upgrade from one release to the next with as close to zero downtime as possible. Much of this work centers around TripleO, Heat, and Mistral, for orchestration and automation of the process.

Containers: While centered around the Kolla project, containerization was a theme in many of the projects this cycle. The eventual goal, at least according to some, is that OpenStack services will be deployed in containers by default by the Pike release. This of course poses a real challenge for the Ocata -> Pike upgrade path (migrating from non-container to container in the course of the upgrade), and that’s something that the TripleO people are working hard on.

Security: TLS-everywhere made strides forward in Ocata, with connections between services moving to TLS. This involves changes to Barbican as well, for key management for the shared keys between services, to ensure that your traffic is secure between components of your cloud, which may be located in different data centers around the world.

Collaboration: Something I heard more this year than in previous years was talk of collaboration between projects. This has, of course, always been happening. However at the PTG in Atlanta, it was a major focus, with time set aside for cross-project meetings focusing on the interface between one service and another. I also heard from several people that the PTG allowed a focus, and a camaraderie, that was not possible when the design summit was part of OpenStack Summit. This resulted in fewer interpersonal tensions, and a lot more work getting done.

Everything else: The difficulty with OpenStack is that it’s just so big. While these are the things that stood out to me, someone else is likely to pull out different highlights, depending on their interests. So I encourage you to look at some of the other “What’s new in Ocata” articles out there, including especially “53 new things to look for in OpenStack Ocata” – https://www.mirantis.com/blog/53-new-things-to-look-for-in-openstack-ocata/ and, if you have a lot of time, or have interest in a particular project, check out the official release notes – https://releases.openstack.org/ocata/  And take a moment to watch my interviews with various Red Hat OpenStack engineers, from the Atlanta PTG, here: https://goo.gl/3aCQ2d.

Moving to CentOS

TL;DR: Leaving OpenStack; Moving to CentOS; Still at Red Hat.

4 years ago, I came to Red Hat, and started as the OpenStack Community Liaison, working primarily with the RDO project, but more generally with all of Red Hat’s involvement in the upstream OpenStack project.

I took over from Dave Neary, but it took a while to actually replace him. His depth of knowledge and experience with the community were not easy to step into.

Over those 4 years, I’ve become much more knowledgeable about OpenStack – the community as well as the technology. It’s a wonderful community, with a passion for open source, for doing things transparently and collaboratively, and for doing things well. The individuals in the community have been great to get to know – both people here at Red Hat, as well as people in other organizations, and at the OpenStack Foundation. I could certainly call out dozens of individuals who have made my time with OpenStack smoother. The names that come to mind are Haikel Guemar, Stefano Maffuli, Perry Meyers, Eliska Malikova, Alan Pevec, Jakub Ruzicka, Rain Leander … see, I knew that as soon as I got started I would find that there’s no end in sight.

Rain, in particular, has really stood out as someone that was hugely passionate about the community around OpenStack, and was just such a delight to work with, particularly when we were able to attend the same events and work together in person.

This is why I’m so excited to announce that Rain will be taking my place as the RDO/OpenStack community manager for Red Hat, effective immediately. I cannot think of anybody more qualified, in skills and temperament, for this position, and I am completely confident that I’m leaving the community in good hands. One develops a lot of ownership of a project over four years, but I have no doubt she’ll take care of the project.

I’m not leaving Red Hat, though. Instead, I’m moving over to be more active in the CentOS community. CentOS is an exciting community that is absolutely critical to Red Hat. It’s the place where community projects, like RDO, as well as many others, do their development and testing, before being deployed and supported on Red Hat Enterprise Linux. I’ll be focusing on a variety of things, including CentOS in HPC (High Performance Computing) and IoT (Internet of Things).

CentOS presents a number of challenges from a community perspective, and I’m very pleased to be more active there. It will be an interesting and challenging place to be, and I’ll be, once again, working with an awesome group of people. I’m sure I’ll be telling lots of CentOS stories on this website in the years to come, so stay tuned.

Follow RDO on Twitter at @rdocommunity and follow CentOS on Twitter at @centos to keep up to date with the two communities, as well as learn how we work together.