1) This is a hugely productive event, with project teams getting an enormous amount of work done without the distractions that are usually present at a conference.
2) I remain very concerned about how this event will effect the
character of OpenStack Summit – removing the bulk of the engineers from that event, and making it more product/marketing/sales focused. Time will tell
At the gathering, I did 23 interviews with Red Hat engineers about what they did in the Ocata release. You can see some of those interview on the RDO YouTube Channel. I’m not done editing them all yet, but they will appear over the coming weeks as part of various blog posts, as well as all of them appearing in that YouTube playlist.
I am constantly blown away by the passion, expertise, and
professionalism of the folks I get to work with. Wow.
Anyways, more about the PTG.
I was (and, really, still am) very skeptical about this new format.
Splitting OpenStack Summit into four events rather than two has already had significant impact on travel budgets, not just at Red Hat, but also at other companies involved in OpenStack. A lot of companies, for example, didn’t send anyone to FOSDEM, and we had a hard time staffing the OpenStack table there. Usually people work one shift at the table, but this year several of us worked 4 and 5 shifts to cover all the slots.
I am concerned that splitting the engineers off into their own event
will significantly change the character of OpenStack Summit from being a community-centric, tech-centric event, to more of a sales and marketing event, light on technical depth.
But this event, for what was intended, has already been amazing.
Everyone is commenting on how much is getting done, how much less distracted the team meetings are, how much better the teams are gelling than they have at any previous event. This is a working event, and people are here to get work done. They are meeting all day, every day, working on plans and blueprints, and fighting out agreements on things that would take weeks in email, and everyone seems VERY pleased with outcomes.
So, perhaps the trade off will be worth it. Time will tell. Regardless, Erin Disney and her team put on an amazing event that fulfilled, and exceeded, its goals.
On Wednesday night, everyone that has ever contributed a patch to RDO was invited for drinks and hors d’oeuvres at the SideBar, and while there the RDO Ocata release announcement was sent out.
We had about 50 people in attendance, who ate and drank up all of my budget in about 2 hours.
I was recently asked to write something about the project that I work on – RDO – and one of the questions that was asked was:
A healthy project has a visible lead(s). Who is the project lead(s) for this project?
This struck me as a strange question because, for the most part, the open source projects that I choose to work on don’t have a project lead, but are, rather, led by community consensus, as well as a healthy dose of “Just Do It”. This is also the case with RDO, where decisions are discussed in public on the mailing list, and on IRC meetings, and those that step up to do the work have more practical influence than those that just talk about it.
Now, this isn’t to say that nobody takes leadership or ownership of the projects. In many senses, everyone does. But, of course, certain people do rise to prominence from time to time, just based on the volume of work that they do, and these people are the de facto leaders for that moment.
There’s a lot of different leadership styles in open source, and a lot of projects do in fact choose to have one technical leader who has the final say on all contributions. That model can work well, and does in many cases. But I think it’s important for a project to ask itself a few questions:
What do we do when a significant number of the community disagrees with the direction that this leader is taking things?
What happens when the leader leaves? This can happen for many different reasons, from vacation time, to losing interest in the project, to death.
What do we do when the project grows in scope to the point that a single leader can no longer be an expert on everything?
A strong leader who cares about their project and community will be able to delegate, and designate replacements, to address these concerns. A leader who is more concerned with power or ego than with the needs of their community is likely to fail on one or more of these tests.
But, I find that I greatly prefer projects where project governance is of the people, by the people, and for the people.
Continuing the saga of OpenStack Summit Barcelona …
Wednesday was a very long day – about 10 hours working the Red Hat booth, with hundreds and hundreds of people dropping by to chat, get tshirts, ask questions about our various products, and just hang out.
While Tuesday was “day two”, Wednesday is the second full day of the expo hall, and so this is the time when you get even less frantic swag-seeking, and more people looking for information. Which is good, because we were starting to run out of the swag.
What stood out on Wednesday was the sheer number of people who stopped by just to say how awesome RDO is, and how much they appreciate the work that the RDO community does. Several people also asked how they could get involved in making things happen that are not happening yet. So, this was very encouraging.
Wednesday evening was a quiet dinner-then-bed evening, as is usually the case by this point in a conference, when the true exhaustion of long days and jet lag really catches up. I had dinner with a small group of friends and coworkers, which was really delicious, but I have no idea how to tell you what I had. 🙂
And Thursday was the final day. We got the expo hall open time wrong, so I was actually there in the booth for 2 hours before it opened. Gah! And then, another really long day of answering questions, and giving away the last of the ducks.
The duck tradition that was started in Paris continues, and we plan to have more ducks in Boston. We only had 800 ducks in Barcelona, because I waited kind of late to order, and the vendor was low on inventory. Any suggestions of how the duck should dress for Boston?
We closed up just after 4, packed everything up, and I headed to the beach. Unfortunately, right about the time I got there, it started to get overcast and windy, so I only got about a half hour on the beach before I gave up and left, shivering. Oh, well, better than nothing.
Thursday night was another dinner-and-fall-in-bed-early evening, as I was completely exhausted. Unfortunately, as usual when I have a flight in the morning, I didn’t get much sleep. I have this bad habit of waking up every few hours to check the clock, even though I have several alarms set. Some day, I hope to get over that.
On Friday, I shared a cab to the airport with Jen, and we boarded the flight for JFK.
Tuesday, the first day of the main event, was, as always, very busy. I spent most of the day working the Red Hat booth. We started at 10 setting up, and the mob came in around 10:45.
Day two of booth duty is always interesting, because it’s after the swag feeding frenzy has died down a bit that you start hearing from the people that actually care about what you’re “selling”. You get the questions. And what’s been fascinating in the 6 summits I’ve attended is that the bar has been raised a LOT on the questions. In Hong Kong, my first Summit, there were still a lot of people asking what OpenStack was, and nobody had any idea what RDO was. Now, the questions are about specific deployment scenarios, projects that aren’t yet being packaged, the future of TripleO, and so on, with only a handful of people asking what RDO is.
OpenStack has clearly made the transition from “something to consider some day” to “of course we are, and what are you going to do to make it better?”
Another awesome improvement this Summit was how the RDO community stepped up to help in the booth. Every single hour of the day, I had at least one, and usually two, members of the RDO community in the booth with me, either doing an “Ask Me Anything About RDO”, or doing some kind of a demo. It was *awesome*. Maybe next year, I’ll just stay home. 😉
The highlight of the day was the RDO/Ceph community meetup. We had 4 hours at the Gym Bar in the Princess Hotel.
Members of the Ceph and RDO community presented, lightning talk style (5 minute presentations) on a variety of topic. Speakers were threatened with being thrown in the pool if they went over 5 minutes, but we managed to restrain ourselves.
By the end, we had checked in 215 people overall, and we had 12 speakers. The food was good. The speakers were awesome. The only complaint was that the people not actually listening to the talks would NOT shut up, so it was hard to hear. Eventually, one of the speakers shouted at them to shut up or get out, and most of them moved to the other side of the room.
I have a recording of the event, but I don’t expect it’s going to be usable, due to the noise level. I haven’t had a chance to review it yet. Next week, for sure. I also hope to have (some of?) the presentation slides from the various speakers posted somewhere. Watch rdo-list and/or @rdocommunity for details.
After the talks were over, we had an hour or so left, and I cowardly skipped out. There comes a time when I have just had too much social interaction, and I need quiet time.
So, that was Tuesday. Another success, and another day to be glad that I work with such an awesome community.
I have the best intentions of blogging every day of an event. But every day is always so full, from morning until the time I drop into bed exhausted.
I used to imagine wandering around the world like Hemingway, seeing exotic places, and writing witty stories about the interesting people I met. I have the great good fortune to have the traveling as part of my job. If only I could find the time for the stories.
Anyways … I’m in Barcelona for OpenStack Summit. This is always an impressive, and somewhat overwhelming, event, with 7000+ people attending, dozens of companies presenting their various products, hundreds of technical presentations, and after-hour events every evening.
On Sunday, I met up with Jen and various other members of the events team to scope out the venue. We walked over to the Princess Hotel where a number of our meetings and social events are to be held. And we walked down to El Boo, where the employee party was to be held. Both venues were just great.
Later in the day I visited Sagrada Familia, a cathedral which has been under construction since 1884, on and off, and isn’t done yet. It was weird and improbable, and fascinating, and beautiful. I think the architect’s driving passion was to be different from anything you’ve ever seen before.
I also spent a little time on the lovely beach, Mar Bella, which is right in front of my hotel.
On Monday, we had the RDO Infrastructure gathering in one of the rooms at the Princess. we had about 20 people in attendance, and made good progress on a number of issues. The Ocata cycle is going to see more improvements to how RDO works. Â More details on this meeting coming to the RDO mailing list soon.
Although the conference didn’t officially start until Tuesday morning, the “booth crawl” was Monday evening – the odd ritual where swag-hungry attendees rush from booth to booth, grabbing all the free stuff they can carry, and having the occasional conversation with booth staff. Sometimes, the booth crawl can be a depressing experience, with people refusing to make eye contact, and just grabbing the stuff. But this was actually really great, with people excited about RDO, and wanting to learn more about we had to show.
Monday night was the Red Hat employee party at El Boo. It’s a boat-shaped restaurant sitting on one of the stone piers along the beach, and we had the whole place to ourselves for the entire night. It was a lot of fun. I stayed until midnight, when several friends toasted my birthday.
All together, it was a lovely start to the week. And there’s so much more to come.
Yesterday I spent the day at the University of Kentucky at the OpenStack 6th Birthday Meetup. The day was arranged by Cody Bumgardner and Kathryn Wong from the UKÂ College of Engineering.
UK has an OpenStack cloud that they use for instruction, as well as for research, and they’ve got a 6PB Ceph cluster hanging off of it. There were presentations about the various aspects of this cloud, and how it’s being used.
I gave an introduction to OpenStack – the Foundation, the software, and the community – for the attendees that were just getting started. Patrick McGarry gave a talk about how Ceph works.
Nassir Hussamddin closed the day with a really cool presentation about CloudLab, which is a tool shared by a number of universities that allows users to spin up an OpenStack cloud (not just a VM, but an entire cloud) on demand for testing purposes. Definitely worth looking into further.
Big thanks to Dell, the University of Kentucky, and the OpenStack Foundation, who, along with RDO, sponsored this event.
Despite my best intentions of blogging every day at Red Hat Summit, time got away from me, as it often does at these events. There’s always 3 things going on, and it’s hard to find a moment between that first cup of coffee, and stumbling into bed at the end of the night.
I spent almost the entire event working the RDO booth in the Community Central section of the expo hall. While traffic wasn’t as heavy as at OpenStack Summit, it was still pretty constant.
In the swag department, I had our “what does RDO stand for” tshirts, and TripleO QuickStart USBkeys.
Several things stood out to me from this audience.
First, I was delighted to hear story after story of companies that use RDO in the test/dev/lab environment, and use Red Hat OpenStack Platform in their public/production environments. This is what I really want to see happening, so it’s very gratifying when I get anecdotal evidence that it is happening. Now, if I can only convince those folks to follow up with case study writeups for the user stories page.
Second, from people who were not quite as familiar with either RDO or OpenStack, if there was a consistent thread in the questions, it was confusion as to the overlap between oVirt (or Red Hat Enterprise Virtualization), OpenStack, and OpenShift, and when one might use one vs. the others. This looks like a good opportunity for some blog posts around what the overlap is, what the distinctions are, and what recommendations are for using one or another.
Brian and I did an OpenStack vs oVirt comparison talk at last year’s Red Hat Summit, but I don’t believe we ever wrote it up anywhere. And OpenShift has the added confusion of having a similar name, which gets people kind of mixed up before they even consider the feature set.
And, finally, the week was yet another reminder that I work for the best company in the world, with the best coworkers. I feel sorry for the rest of you.
I’ll be giving a presentation at LinuxCon next week about the ‘Big Tent’ at OpenStack. It’ll go something like this …
The OpenStack Big Tent
OpenStack is big and complicated. It’s composed of many moving parts, and it can be somewhat intimidating to figure out what all the bits do, what’s required, what’s optional, and how to put all the bits together.
The Problem
In the attempt to tame this confusion, the OpenStack Technical Committee defined what’s part of the Integrated Release and what’s not, so that you, the consumer, know what’s in and what’s out. One of the unintended side effects of this was that new projects were treated as second class citizens, and had trouble getting resources, developers, and a seat at the table at the developer summit.
As OpenStack continues to grow, this became more and more of a problem.
With the Liberty cycle, the Technical Committee has taken another look at what makes a project part of OpenStack, to make things better for the projects, as well as for the consumers.
Are You OpenStack?
The question that has been asked all along about any project wanting to be part of OpenStack was, is this thing OpenStack? To answer this question, a number of criteria were applied, including interoperability with existing bits, maturity, diversity (i.e., is this thing entirely developed by one company, or does it have broader participation?), and other things. This process was called Incubation, and once a project graduated from Incubation, it could be part of the integrated release.
As the stack grew, these questions became harder to answer, and more projects were getting left out of the tent, to everyone’s detriment, and to the growing confusion of the folks trying to use the software.
So, in recent months, the Technical Committee (TC) has decided to turn the question around. Rather than asking “Is thing thing OpenStack?” the new question is “Are You OpenStack?”
This changes how we look at making the determination on a few fronts.
OpenStack is People!
As Thierry Carrez Sean Dague said in their Summit presentation, OpenStack is composed of teams of people, working towards the betterment of the overall project. To that end, we’ll now welcome everyone to the table, if they are OpenStack.
So … how’s this defined?
Something is OpenStack if it:
1) Aligns with the OpenStack Mission: to produce the ubiquitous Open Source Cloud Computing platform that will meet the needs of public and private clouds regardless of size, by being simple to implement and massively scalable.
2) Follows the OpenStack Way – Open Source, Open Community, Open Development, and Open Design. (More here)
3) Strives for interoperability with other things that are OpenStack.
4) Subjects itself to the governance of the Technical Committee
Tags
But while this solves one problem, it creates another. As a user of the OpenStack software, I really still need to know what’s in and what’s out.
There is no longer going to be a single release that is defined to be OpenStack, how do I know which bits I need, and which bits I can live without?
To help sort this out, a system of community-defined tags will be applied to the various pieces of OpenStack, starting with “tc-approved-release” which will, initially, just reflect what was already the integrated release. These tags will indicate project maturity, as well as other considerations. Packagers, like the CentOS Cloud Sig, can then use those tags to determine what they want to include in distributions.
Who’s In
As a result of this change, we immediately have several new projects that are part of OpenStack, that were previously held at arm’s length:
People are still going to expect a release, and exactly what that means going forward is a little unclear. Every six months there will be a release which will include stuff tagged ‘tc-approved-release’. It will be opt-in – that is, projects can participate, or not, as they like. Or they can release on their own cadence, as was discussed about a year ago.
There are still some details to be worked out, but the overall benefit to the community seems like it’s going to be huge, as we include more great ideas, and more passionate people, inside the Big Tent.
Zaqar (formerly called Marconi) is the messaging service in OpenStack. I recently had an opportunity to interview Flavio Percoco, who is the PTL (Project Technical Lead) of that project, about what’s new in Kilo, and what’s coming in Liberty.
The recording is here, and the transcript follows below.
R: This is Rich Bowen. I am the RDO community liaison at Red Hat, and
I’m speaking with Flavio Percoco, who is the PTL of the Zaqar project.
We spoke two years ago about the project, and at that time it had a
different name. I was hoping you could tell us what has been happening
in the Kilo cycle, and what we can expect to see in Liberty.
F: Thanks, Rich, for having me here. Yes, we spoke two years ago, back
in Hong Kong, while the project was called Marconi. Many things have
happened in these last few years. We developed new APIs, we’ve added
new features to the project.
At that time, we had version 1 of the API, and we were still figuring
out what the project was supposed to be like, and what features we
wanted to support, and after that we released a version 1.1 of the
API, which was pretty much the same thing, but with a few changes, and
a few things that would make consuming Zaqar easier for the final
user.
Some other things changed. The community provided a lot of feedback to
the project team. We’ve attempted to graduate two times, and then the
Big Tent discussion happened, and we just fell into the category of
projects that would be a good part of the community – of the Big Tent
discussion. So we are now officially part of OpenStack. We’re part of
this Big Tent group.
We changed the API a little bit. The impression that the old API gave
was that it was a queueing service, whereas what we really wanted to
do was a messaging service. There is a fundamental difference between
the two. Our focus is to provide a messaging API for OpenStack that
would not just allow users to send messages from one point to another,
but it would also allow users to have notifications right away from
that API. So we’ll take advantage of the common storage that we’ll use
for both features, for different services living within the same
service. That’s a big thing, and something we probably didn’t talk
about back then.
The other thing is that in Kilo we dedicated a lot of time to work on
these versions of the API and making sure that all of the feedback
that we got from the community was taken care of and that we were
improving the API based on that feedback, and those long discussions
that we had on the mailing list.
In Liberty, we’ve dedicated time to integrating with other project, as
in, having other projects consume the API. So we’re very excited to
say that in Liberty a few patches have landed in Heat that rely on
Zaqar for having notifications, or to send messages, and communicate
with other parts of the Heat service. This is very exciting for us,
because we have some stories of production environments, but we didn’t
have stories of other projects consuming Zaqar, and this definitely
puts us in a better position to improve the service, and get more
feedback from the community.
In terms of features for the Liberty cycle, we’ve dedicated time to
improve the websocket transport which we started in Kilo, but didn’t
have enough time to complete there. This websocket transport will
allow for persistent connections to be made against the Zaqar service,
so you’ll just connect to the service once, and you’ll keep that
connection alive. This is ideal for several scenarios, and one of
those is connecting to Zaqar from a browser and having Javascript
communication directory to Zaqar, which is something we really want to
have.
Another interesting feature that we implemented in Liberty is called
pre-signed URLs, and what it does is something very similar – if folks
are familiar with Swift temp URLs –
http://docs.openstack.org/kilo/config-reference/content/object-storage-tempurl.html
– this is something very similar to that. It generates a URL that
can expire. You will share that URL with people or services that don’t
have an username in Zaqar, so that they can connect to the service and
still send messages. This URL is limited to a single tenant and a
single queue, and it has privileges and policies attached to it so
that we can protect all the data that is going through the service.
I believe those are the two features that excite me the most from the
Liberty cycle. But what excites me the most about this cycle is that
we have other services using Zaqar, and that will allow us to improve
our service a lot.
R: Looking forward to the future, is there anything that you would
like to see in the M cycle? What is the next big thing for Zaqar?
F: In the M cycle, I still see us working on having more projects
consuming Zaqar. There’s several use cases that we’ve talked about
that are not being taken care of in the community. For instance,
talking to guest agents. We have several services that need to have an
agent running in the instances. We can talk about Trove, we can talk
about Sahara, and Murano. We are looking forward to address that use
case, which is what we built presigned URLs for. I’m not sure we’re
going to make it in Liberty, because we’re already on the last
milestone of the cycle, but we’ll still try to make it in Liberty. If
we can’t make it in Liberty, that’s definitely one of the topics we’ll
need to dedicate time to in the M cycle.
But as a higher level view, I
would really like to see a better story for Zaqar in terms of operations
support and deployment – make it very simple for people to go there
and say they want Zaqar, this is all I need, I have my Puppet
manifest, or Anisible playbooks, or whatever people are using now – we
want to address that area that we haven’t paid much attention to.
There is already some effort in the Puppet community to create
manifests for Zaqar, which is amazing. We want to complete that work,
we want to tell operations, hey, you don’t have to struggle to make that
happen, you don’t have to struggle to run Zaqar, this is all you need.
And the second thing that I would like to see Zaqar doing in the
future is to have a better opinion of what the storage it wants to
rely on is. So far we have support for two storages that are unicode
based and there’s a proposal to support a third storage, but in
reality what we would really like to do is have a more opinionated
Zaqar instance of storage, so that we can build a better API, make it
consistent, and make sure it is dependable, and provide specific
features that are supported and that it doesn’t matter what storage
you are using, it doesn’t matter how you deploy Zaqar, you’ll always
get the same API, which is something that right now it’s not true. If
you deploy Redis, for instance, you will not have support for FIFO
queues, which are optional right now in the service. You won’t be able
to have them because that’s something that’s related to the storage
itself. You don’t get the same guarantees that you’d get with other
storage. We want to have a single story that we can tell to users,
regardless of what storage they are using. This doesn’t mean that ops
cannot use their own storage. If you deploy Zaqar and you really want
to use a different storage, that’s fine, we’re not going to remove
plugability from the service. But in terms of support, I would like
Zaqar to be more opinionated.
