An Evening with OpenStack and the DevOps Community

I had the chance to attend the Triangle DevOps meetup, along with my podcast co-host Aaron Delp (@aarondelp), which was being held at the Teradata offices and lead by their DevOps person (Felix Maldonado) responsible for their OpenStack environment. It was an interesting event for a few reasons:

1. We've had a few guests (here, here, here) on the podcast to talk about OpenStack, but our hands-on exposure had been limited in comparison to years of VMware environments.
2. Most of the people we had spoken with were deeply involved with the OpenStack development, or worked directly at Rackspace, so their viewpoints were slightly skewed to the positive.
3. Aaron had just written a blog about his experience with the new Rackspace Private Cloud edition. We had compared it to another write-up from Cody Bunch (@cody_bunch), noting that Aaron's experience seemed much simpler. Another experience here.

The presenter was very transparent about his experience with OpenStack. They had work that needed to be done, but they were short on budget, so they decided to spend their money on new server hardware instead of software licenses. They were a development group, but their output needed to go into a production environment, so the system ultimately needed to work. He was tasked with getting a small "cloud" up and running (3 servers, hosting about 80 VMs of various sizes).

[I'll stop here and clarify that I don't intend this to be a commentary on OpenStack, but rather a set of observations from both the audience ("DevOps crowd") and myself, somewhat in the context of experience with other systems.]


While he made it clear that he had the environment working, and was happy with it's performance, he did highlight a number of challenges. He highlighted these for the audience, who were there to learn about OpenStack (most had never tested it in any way).

• Getting things to work took quite a bit of trial and error. He had to rebuild the system several times before it worked as expected. [Note: All done using community distro, not any of the new pre-built distros from Rackspace, Redhat, Piston Cloud, Nebula)
• The documentation is terrible. It makes many assumptions that people know about OpenStack architecture or concepts, and then mixes many scripted and manual configurations together.
• No built in management or monitoring tools (capacity, performance, etc.)
• No way to move VMs dynamically from controller to controller (manually copy config files)
• No native load-balancing function
• No way to have HA-like functionality of VMs
• Not clear where KVM or Xen tools go once OpenStack is managing via Nova-Controller
• Making changes to the system has all sorts of unexpected side-effects, often times causing a crash.
• Updates only come out every six months, which he believed to be too short, especially when bugs were reported through various channels.

[NOTE: Please let us know where there is incorrect information above]

He pointed out that he uses customized Nagios as his primary means of monitoring performance, and was exploring other tools like Zenoss. He also explained that having to go through the setup trial and error allowed him to learn quite a bit about the underlying OpenStack architecture. Having used AWS EC2 before, he made many comparisons between the functionality of the two systems (EBS <-> Snapshots; Instance Sizes <-> Flavors, etc.)

I walked away from the event with several thoughts:

1. It's been two years, but OpenStack still feels like it's very early days. One person in attendance compared it to ESX 2.0, back before vCenter and all the tools became commonplace.
2. The installation process should get considerably easier, with all the pre-buit distros coming into the market.
3. There are still plenty of unanswered questions about how to architect the system; when it makes sense to deploy OpenStack vs. using native hypervisor tools; and which types of applications make the most sense in this architecture.
4. At one point, I thought it was impressive that OpenStack releases had shipped consistently every six months. But the audience seem quite surprised that bug updates weren't made available more often. Either way, it's a much more rapid development cycle than some commercial implementations.
5. OpenStack already has support for many configuration tools, such as Puppet, Chef, enStratus, RightScale, so some of the "it's not included" comments are covered through external tools. 

For me, it's always good to see technology outside of the high-profile events, away from the headlines and with people that are only focused on making things work. This group walked away moderately interested in OpenStack, but not overwhelmed. OpenStack has a ways to go, but it has come a long way in just a few years. Time will tell...