For the second time this year, I’m sitting in a place that I have read about for decades. I’m in a conference room at CERN, attending a CentOS meeting. CERN is the home of the Superconducting Super Collider, as well as being the birth place of the World Wide Web.
This afternoon I’ll tour the data center where a lot of the computing for the SSC happens, as well as seeing some of the actual experiments. Photos to come later this evening.
The earlier visit referenced above was Oak Ridge National Labs. I actually visited ORNL when I was in college, but on my most recent visit I got to visit Titan – one of the world’s largest supercomputers – and actually walk through it. It was very loud.
I woke up this morning, to make a long story short, with pain in my chest and left arm. We called 911 and the ambulance came. The EMTs checked me out and recommended that I go to the hospital. 5 hours later, I was discharged. They couldn’t find anything wrong. We figure that I just have a pulled muscle in my neck and shoulder.
Anyways, we had planned to go hiking today, so we decided to go ahead and do it anyway. We climbed chimney tops. Isaiah and I made it to the top, you can see my pictures on Flickr.
I’m not certain, but I think the last time I climbed chimney tops was at least 20 years ago. Perhaps a little bit more. It was a lot harder hike this time than it was then. I remember jogging up to the peak with Tim and Tony and Eddie.
But we made it to the top and back with no ill effects. My neck and chest and shoulder feel the same now as they did this morning. So I guess there’s nothing to worry about.
Last week I attended the second OpenStack PTG, in Denver. The first one was held in Atlanta back in February.
This is not an event for everyone, and isn’t your standard conference. It’s a working meeting – a developers’ summit at which the next release of the OpenStack software is planned. The website is pretty blunt about who should, and should not, attend. So don’t sign up without knowing what your purpose is there, or you’ll spend a lot of time wondering why you’re there.
I went to do the second installment of my video series, interviewing the various project teams about what they did in the just-released version, and what they anticipate coming in the next release.
The first of these, at the PTG in Atlanta, featured only Red Hat engineers. (Those videos are HERE.) However, after reflection, I decided that it would be best to not limit it, but to expand it to the entire community, focusing on cross-company collaboration, and trying to get as many projects represented as I could.
So, in Denver I asked the various project PTL (project technical leads) to do an interview, or to assemble a group of people from the project to do interviews. I did 22 interviews, and I’m publishing those on the RDO YouTube channel – http://youtube.com/RDOCommunity – just as fast as I can get them edited.
I also hosted an RDO get-together to celebrate the Pike release, and we had just over 60 people show up for that. Thank you all so much for attending! (Photos and video from that coming soon to a blog near you!)
So, watch my YouTube channel, and hopefully by the end of next week I’ll have all of those posted.
I love working with the OpenStack community because they remind me of Open Source in the old days, when developers cared about the project and the community at least as much, and often more, than about the company that happens to pay their paycheck. It’s very inspiring to listen to these brilliant men and women talking about their projects.
I’ve been doing interviews at the OpenStack PTG this week, and the space I started in was very noisy. I’ve been trying to figure out how to reduce background noise, and I think I finally figured it out.
(I’m sure that the same process works with other tools, but these are the ones that I use.)
In short, the process is this:
Extract audio from original recording
Noise reduction in Audacity
Add video and noise-reduced track to kdenlive.
Mute video track, and “group” the audio and video tracks
Edit as usual
In more detail, here’s how you do that:
Extract audio from original recording:
At the command line, convert the video directly to an mp3. ffmpeg takes care of the details:
ffmpeg -i original_video.mp4 original_audio.mp3
2. Open the audio track in Audacity, and noise-reduce
Select a section of the audio where nobody’s talking. This gives you the typical background noise that you want to remove. The longer this section is, the better your overall noise reduction will be.
Click “Effect” -> “Noise Reduction”, and then press the “Get Noise Profile” button.
Now, unselect the section (ie, by clicking anywhere in the track) then remove that noise from the entire track by clicking “Effect” -> “Noise Reduction” and clicking “OK”.
DO NOT make any edits that change the length of the track, as it must be exactly the same length as the video clip.
Now export the track again as an mp3.
3. In kdenlive, add the original video, and the new audio.
Add the two clips – the original video and the new audio to the timeline and make sure that they line up exactly.
4. Mute the video track by clicking the speaker icon next to the track.
Click the video track, then control-click the audio track. Press control-G (or right-click and select the “Group clips” option) to group the tracks into a single unit.
5. Edit as usual
In particular, when you cut or drag one clip, it will also happen to the other, so cutting out sections will affect both tracks, and they’ll stay in sync.
Every time I go to a conference, I pack pretty much the same things. Each item has its place in my bag.
Here they all are. Click on the picture for the full size. Image annotations were made in Inkscape, with assistance by Ruth.
Missing from this photo is my laptop, which of course I also always take. Except for that one time I forgot it, and Maria had to rush it to the airport for me. Oops. The laptop is a Lenovo T450s ThinkPad running Fedora 26. It has roughly 78,000 stickers on it.
The numbered items in the picture are as follows:
This is a lovely leather satchel which Maria ordered for me, from China. People ask about it all the time. No, I don’t know what site it came from. It is awesome. It has all the pockets. So many pockets.
This is a “travel charm” that Daniel got for me in a shinto shrine downtown Tokyo. The next time I was in Tokyo I got one for each of the coworkers I travel with the most. No, I’m not superstitious. It’s just a nice reminder of going to Tokyo, and a lovely gift from a friend.
Tablet. This one is a Samsung Galaxy Tab A. It has numerous downloaded movies from Netflix on it, as well as a dozen books. It keeps me entertained on the plane.
Phone. Google Nexus 6P
Plastic bag of micro SD cards, which I use in the various audio and video recording devices I carry with me.
Power supply for my laptop.
This little guy is named Zipper Pull Man, and was made for me by my daughter, Rhiannon, probably 15 years ago. Possibly longer. It is made of plastic beads strung on plastic cords, and has traveled with me to 5 continents. In October it’ll go to the 6th. It has been repaired and restrung many many times.
Leather cable wrap. Maria made this out of scrap leather. It holds various USB cables and chargers.
Another leather cable wrap. This one usually carries AV stuff, like video adapters and audio cables. It’s got USB cables in it right now. You can never have too many cables.
22,000mAh phone charger battery. Can charge my phone about 5 times, or my tablet twice, from dead.
Phone charger battery. Can usually get one charge out of this one.
Micro SD card reader. Plugs into a USB port.
Mini power strip. 3 outlets.
Yet another cable wrap. I might slim it down to just two. I might have too many USB cables.
Reading glasses. Yes, I’m that old.
Spare batteries for #15
Gorillapod flexible tripod for #13
Hearing aid batteries.
Flashlight and bottle opener
Another USB SD card reader. Handles micro SD cards and regular SD cards.
Power supply for #15
Remote slide clicker thingy
Handle for man-on-the-street interviews using #13
Remote bluetooth microphone for my hearing aid
Monocular. That’s like binoculars, but just one.
Various business cards.
Fancy USB-C charger for #4
These items usually end up in my suitcase rather than my carry-on, since they are pointy. The black case holds my darts. The weird monkey thing is a PocketMonkey tool. And the knife is my current favorite pocket knife.
In this mornings community managers meeting at work, I presented on using video as part of our community promotion. As I said at the beginning, this is a hobby which I enjoy – although I’m far from being an expert. And I’m trying to figure out if it’s actually useful as part of promoting our projects.
Here’s a summary of what I talked about, and some of the questions that were asked.
This camera is available in a number of different models. I got the one that had the largest on-board storage, so that I didn’t have to mess with SD cards. However, you can get it a lot more cheaply with less on-board storage, and get as large an SD card as you think you’re going to need. For reference, a minute of raw video at default settings on my camera takes roughly 150MB. Plan accordingly.
However, the on-board mic on the camera is pretty good.
Remember that you have a better video camera in your pocket than most movie makers have had in the history of film. Full feature films are being made with iPhones these days.
Meanwhile, I’ve been trying to justify the expense of a GoPro for some time. I really want one, but I realize this is mostly geek appeal, and I probably wouldn’t get enough use out of it to justify the cost. The main reason I didn’t buy one as my original camera is that it doesn’t have an audio input jack, making it less than ideal for the kind of interview situations that I started with.
I edit video with kdenlive – https://kdenlive.org/ – which is free and available for most modern operating systems. It took me perhaps 5 hours to get comfortable with it, and another 10 to feel like I’m really good at it.
Now, if you’re going to be a professional, you’ll probably end up using some expensive software from Adobe, or from Apple. And that’s great. I’m sure they are objectively better, in ways that professional videographers can appreciate. But for us amateurs, kdenlive has everything that we’re likely to need. And it’s free.
Several years ago I tried to teach myself to use iMovie – back when I was a Mac user – and found it incredibly intimidating. I’m curious how I would feel about it now that I know more.
There is a huge selection of free music available at Free Music Archive – http://freemusicarchive.org/ – in a wide variety of genres. I have always found something that seemed to fit the video.
Here’s one of my favorite videos featuring music from there:
(Note: If you’re reading this on Facebook, you won’t see the video. Follow the link at the top to my website.)
What’s unclear to me about FMA is what their business model is, and how all of these talented artists are making a living. Sometimes I feel bad about that.
Reasons for making videos:
There’s lots of reasons for making videos. What your reasons are will greatly affect how much time and effort you put into it, what kind of video you make, how you promote it, and so on.
Some reasons include:
Because making videos is fun
To show off a project/hobby/interest/yourself/something cool
To draw people in, so that they go to another site
To advertise a product/project
To put a human face on some project/product/concept
For my personal channel, it’s primarily the first two reasons. This morning I posted a video of a caterpillar, because I wanted to. There’s really no other reason.
For work, though, I have to justify the time that I spend, in terms of what the measurable benefits are. Mostyly, in that case, it’s the other four reasons.
Which leads to …
Measuring whether it’s effective:
Measuring whether what we do at work advances the goals of the company/project/whatever is tricky. If I enjoy doing something (like making videos) then I’m likely to think that it’s a useful thing to do, and look for reasons that support that.
But it’s important that it is actually effective, for some definition of effective. Does it bring traffic to the site? Does it educate people in performing some particular task. In fact, is anybody at all watching the videos, or are they just sitting there sad and lonely?
What to make videos of:
In short, everything. However, this can be extremely time consuming if you want to do postproduction, so you might want to be selective.
For the purposes of work, I recommend:
Events/Presentations. If you’re at an event, capture video of the sessions you attend. The people that weren’t there sometimes appreciate this. Make sure you ask the speaker if it’s ok. Make sure you have a mic, and that the speaker users it.
Meetings. Maybe. If you use some kind of video conferencing meeting service, and if the content of the meeting might be useful to someone else that couldn’t make it, press record. Why not? I wish I’d recorded the meeting this morning, for example, since I’m sure I’m forgetting something that was discussed.
People. People like to talk. Most of them, anyways. If you ask them to write a blog post or article, they’ll all say yes, but almost none of them will actually do it. But get them in front of a camera and start asking questions, and most people will have something useful to say. Everyone is an expert on something, and they like to talk about it.
YouTube has been kind enough to provide us with infinite storage for anything we want to create. Granted, they have their own reasons for this. But you might as well take advantage of it.
Other topics and questions that came up:
Q: Do you ask people to sign any kind of waver when you take video, saying that it’s ok to put it online for the whole world to see?
A: I never thought of that. I probably should. Seems that it’s also company policy for me to do so, so I should start right away. However, I always clearly set the expectation – “This will be on YouTube, ok?” – and get verbal assent, and this hasn’t, so far, resulted in any problems.
Q: How much time does it take?
A: It depends. If you’re getting a video of a presentation or a meetup, just post the raw video and be done with it. Total time: Length of event plus upload time. If you’re doing a more formal interview, it can take around 5-10 minutes per minute of final product. And if you’re doing something more elaborate like piecing together several clips and trying to tell a coherent story, it can take hours to make a 5 minute product.
And I’m sure that people that do this for a living are both better than I am at it (so they can do it faster) and more careful (so they do more work in that time, for a better finished product).
One thing I didn’t mention is that rendering the video (the process which produces the final uploadable file) takes a really long time. Around 5-10 minutes per minute of video, depending on your computer.
So, really, you need to experiment, and figure this out.
Q: How do you have people prepare for an interview?
A: I always provide a list of questions ahead of time. Usually a day or two before, so that they can think about what they’re going to say. Giving them a sample video of someone else answering similar questions is a great way to get them prepared. Then at the end, I always ask “was there something that I didn’t ask?” so that they can put in what they actually wanted to say. I then edit that bit into a relevant place in the conversation.
Q: Who are some YouTube’ers that you like to watch.
A: I like Casey Neistat – https://www.youtube.com/user/caseyneistat – because he’s really good at this, and his videos are, mostly, a lot of fun. And some of them aren’t. But he just post stuff because it interests him, and this seems to work out for him. (Warning: He’s not careful about his salty language. Sometimes.) My kids watch hours and hours of Good Mythical Morning. https://www.youtube.com/user/rhettandlink2 These guys are insane.
Related, I hate technical videos that take information that could be adequately stated in 2 lines of text, and make a 15 minute rambling video about it. There’s thousands of “how to install Whatever on CentOS” videos out there that, tell you to type “sudo yum install kdenlive” but take 15 minutes of boring voiceover to do it. Thanks. I’ll do without.
Q: What’s the hardest thing about starting making videos?
A: Feeling that you don’t have anything that anybody will want to watch, and it’ll just be stupid. Solve this by just looking at Youtube. 300 hours of videoare uploaded to YouTube every minute. That’s almost 50 YEARS of video every day. And, sure, most of it, nobody ever watches. But I guarantee you have something more valuable to say than about 45 of those 50 years of content.
(Note: If you’re reading this on Facebook, you’re missing half of the story. Please follow the link to my website to see the rest of the post.)
I’ve been tinkering with making videos for a while, and have been learning a lot over the last year or so. Last week I decided to make a video about something not work related, and picked home automation. As it turns out, what I have to say about home automation fits naturally into 4 (or possibly more) episodes.
Here’s the first one.
What I learned from doing this video:
Lighting is important. This is way too dark
Need to actually show more of what you’re talking about, than just talking about it.
And here’s my second one:
I’m very pleased with this second video, but I still learned a few things from it. I learned a lot about the tool I’m using (kdenlive) and what you can do to paste various tracks together. Also, watching it again, I’m sure I can do better on sound levels. There’s just too much difference between the tracks where I’m sitting at my desk, and when I’m doing to “on site” clips. The former, I’m using my desk mic, and the latter is using the onboard mic on my camera. My desk mic is a better mic, but I just have it turned down lower. I can probably also adjust this when I’m editing the video.
In my next episode, I’ll be talking about the Phillips Hue products, and in the fourth, the Osram/Sylvania Lightify product.
I came into the ISC event pretty ignorant. Here’s some of the things I’ve learned.
Supercomputers run Linux. All of them. This isn’t even a topic of discussion. Yes, I’m sure there are some that don’t, but everyone here just assumes that you are running Linux. And probably two or three Apache products.
Supercomputing isn’t about software. This is a hardware conference.
Supercomputing is primarily about how fast you can get rid of heat. And these people are serious about cooling. I’ve seen some amazingly cool cooling rigs. Perhaps the coolest of them was this one: https://youtu.be/hs9WG0ZA79Q That unit is called the AIC24, and is manufactured by Asperitas, and is a full submersion rack. You lower your blades into oil, which is in turn cooled by a water cooling pump. This is much quieter than fans, and much more efficient. The oil was cool enough to touch. Enormous supercomputing centers are locating on the edge of lakes specifically so that they can pump cool water from the lake into cooling units like this.
I also saw this cool demo: https://youtu.be/aaEQN8DH0kM You can actually see the oil boiling on the processor. The vapor is then condensed on a cooling unit in the back and trickles back down into the tank.
I have also been blown away by the Student Cluster Competition. These kids have access to hardware that would have blown my mind when I was in school. There’s 11 teams competing on a variety of metrics, and they have these astonishing supercomputers at their disposal. I was also amazed to discover thatLINPACK is still one of the standard benchmarks. I used that when I was in college!
The student hardware is all sponsored by the vendors that are here at this event – presumably so that they can benefit from the publicity when they win the contest. Check out some of these rigs:
I was pleasantly pleased to discover that of the 11 teams competing, 8 are running CentOS. One other was running Fedora – they wanted to run CentOS, but needed a newer kernel for something (I wasn’t very clear on what that was. I’ll try to go find out more information today.) The other two were running Ubuntu. CentOS also appears to be the preferred platform for the various research institutes I’ve talked to. However, these are the groups that chose to come over to the Red Hat booth and talk to me, so I do acknowledge that this is a rather self-selected sample. The sign on the SuSE booth claims that SuSE is the Linux “most used by the top 100 supercomputers.” More research is warranted here. But it appears clear, at least from this small sample, and from conversations with the students, that CentOS is just What You Run when you’re doing supercomputing.
And finally, I’ve learned (not that it’s a big surprise) that one year of high school German, 30 years ago, is not a great deal of help. And that people are amazingly patient and kind with my ignorance – something that I’ve discovered almost everywhere in the world
In George R R Martin’s books “A Song of Fire and Ice” (which you may know by the name “A Game of Thrones”), the people of Braavos,
have a saying – “Valar Morghulis” – which means “All men must die.” As you follow the story, you quickly realize that this statement is not made in a morbid, or defeatist sense, but reflects on what we must do while alive so that the death, while inevitable, isn’t meaningless. Thus, the traditional response is “Valar Dohaeris” – all men must serve – to give meaning to their life.
So it is with software. All software must die. And this should be viewed as a natural part of the life cycle of software development, not as a blight, or something to be embarrassed about.
Software is about solving problems – whether that problem is calculating launch trajectories, optimizing your financial investments, or entertaining your kids. And problems evolve over time. In the short term, this leads to the evolution of the software solving them. Eventually, however, it may lead to the death of the software. It’s important what you choose to do next.
You win, or you die
One of the often-cited advantages of open source is that anybody can pick up a project and carry it forward, even if the original developers have given up on it. While this is, of course, true, the reality is more complicated.
As we say at the Apache Software Foundation, “Community > Code”. Which is to say, software is more than just lines of source code in a text file. It’s a community of users, and a community of developers. It’s documentation, tutorial videos, and local meetups. It’s conferences, business deals and interpersonal relationships. And it’s real people solving real-world problems, while trying to beat deadlines and get home to their families.
So, yes, you can pick up the source code, and you can make your changes and solve your own problems – scratch your itch, as the saying goes. But a software project, as a whole, cannot necessarily be kept on life support just because someone publishes the code publicly. One must also plan for the support of the ecosystem that grows up around any successful software project.
Eric Raymond just recently released the source code for the 1970s
computer game Colossal Cave Adventure on Github. This is cool, for us greybeard geeks, and also for computer historians. It remains to be seen whether the software actually becomes an active open source project, or if it has merely moved to its final resting place.
The problem that the software solved – people want to be entertained – still exists, but that problem has greatly evolved over the years, as new and different games have emerged, and our expectations of computer games have radically changed. The software itself is still an enjoyable game, and has a huge nostalgia factor for those of us who played it on greenscreens all those years ago. But it doesn’t measure up to the alternatives that are now available.
Software Morghulis. Not because it’s awful, but because its time has
Winter is coming
The words of the house of Stark in “A Song of Fire and Ice”, are “Winter is coming.” As with “Valar Morghulis,” this is about planning ahead for the inevitable, and not being caught surprised and unprepared.
How we plan for our own death, with insurance, wills, and data backups, isn’t morbid or defeatist. Rather, it is looking out for those that will survive us. We try to ensure continuity of those things which are possible, and closure for those things which are not.
Similarly, Planning ahead for the inevitable death of a project isn’t defeatist. Rather, it shows concern for the community. When a software project winds down, there will often be a number of people who will continue to use it. This may be because they have built a business around it. It may be because it perfectly solves their particular problem. And it may be that they simply can’t afford the time, or cost, of migrating to something else.
How we plan for the death of the project prioritizes the needs of this community, rather than focusing merely on the fact that we, the developers, are no longer interested in working on it, and have moved on to something else.
At Apache, we have established the Attic as a place for software projects to come to rest once the developer community has dwindled. While the project itself may reach a point where they can no longer adequately shepherd the project, the Foundation as a whole still has a responsibility to the users, companies, and customers, who rely on the software itself.
The Apache Attic provides a place for the code, downloadable releases, documentation, and archived mailing lists, for projects that are no longer actively developed.
In some cases, these projects are picked up and rejuvenated by a new community of developers and users. However, this is uncommon, since there’s usually a very good reason that a project has ceased operation. In many cases, it’s because a newer, better solution has been developed for the problem that the project solved. And in many cases, it’s because, with the evolution of technology, the problem is no longer important to a large enough audience.
However, if you do rely on a particular piece of software, you can rely on it always being available there.
The Attic does not provide ongoing bug fixes or make additional releases. Nor does it make any attempt to restart communities. It is
merely there, like your grandmother’s attic, to provide long-term storage. And, occasionally, you’ll find something useful and reusable as you’re looking through what’s in there.
The Apache Software Foundation exists to provide software for the public good. That’s our stated mission. And so we must always be looking out for that public good. One critical aspect of that is ensuring that software projects are able to provide adequate oversight, and continuing support.
One measure of this is that there are always (at least) three members of the Project Management Committee (PMC) who can review commits, approve releases, and ensure timely security fixes. And when that’s no longer the case, we must take action, so that the community depending on the code has clear and correct expectations of what they’re downloading.
In the end, software is a tool to accomplish a task. All software must serve. When it no longer serves, it must die.