Mr OpenStack, Tear Down This Tent!

I’ve been an active member of the OpenStack community since 2012. In that time I’ve seen plenty of changes, but none as big as in 2015 when the OpenStack Technical Committee (TC) introduced The Big Tent. This new governance model for OpenStack was intended to solve the following problems:

There was no agreed definition of what should constitute the ‘Integrated Release’
There were too many projects vying to become part of the ‘Integrated Release’

The solution implemented by the TC was to abolish the ‘Integrated Release’ entirely, and effectively open the gates to any project that wanted to officially become a part of OpenStack. This, in my opinion, was a big, big mistake.

The rationale behind this change was noble. OpenStack always strives to be as open and inclusive as possible. Our project and our community are built upon the Four Opens. Allowing all projects to become first class citizens was a huge step forward in that quest for community inclusivity, but it has come at a cost to our primary mission – powering the datacenter. As it’s put on our website:

OpenStack is a cloud operating system that controls large pools of compute, storage, and networking resources throughout a datacenter, all managed through a dashboard that gives administrators control while empowering their users to provision resources through a web interface.

This definition is how I have always thought about OpenStack, and I think that this is how much of the industry views us. Before The Big Tent, anything that didn’t fit within this definition would sit outside of the ‘Integrated Release’. The answer to the question, “What is OpenStack?”, was the ‘Integrated Release’. After The Big Tent, the answer to the question, “What is OpenStack?”, is everything developed within the OpenStack ecosystem. As a list, this looks like:

Sixty individual projects. All OpenStack. Many with very little or no coupling at all. While bringing all of these projects together under The Big Tent has made it easier for developers to start new initiatives, it has made it significantly harder for operators to adopt and maintain OpenStack as the fabric of their datacenter. It has made it harder for customers to assess whether OpenStack is right for them, and it’s made it impossible to simply answer the question, “What is OpenStack?” (just take a look at the patch that’s currently trying to do exactly that). If we are not constantly striving to make all of these things easier, then we are not doing our jobs properly.

OpenStack is always changing, always evolving. Part of this evolution process is being able to take an honest look at the way we work and decide what’s working, and what isn’t. I believe that now is the time to be open and honest in our assessment that The Big Tent has failed to bring the improvements many hoped for. It’s time to move on, and make OpenStack a cohesive project once again.

If I were on the Technical Committee

I’m a believer that there’s little value in pointing out problems without offering solutions, so if it were up to me this is what I would change:

No More Big Tent

I think the reasons for this are well described above.

Provide a clear, narrower definition of OpenStack

We desperately need a consistent, concise answer to the “what is OpenStack?” question. Something that operators can be pointed towards, to download in a single package. To me, this means agreeing on the core set of services that we as a community consider to be the “standard” deployment. As a starting point, let’s take the existing list of core services:

Cinder
Glance
Horizon
Keystone
Neutron
Nova
Swift

I’ve added Horizon to this list due to it’s inclusion in the mission statement above. Anything outside of that list is not “OpenStack”. However, there are many excellent projects that compliment this list and are found in many deployments at a variety of scales. Therefore, we need to:

Introduce a new place for complimentary projects to live – The OpenStack Family

The name is just a placeholder, but I think the concept is solid. We absolutely want these projects to continue to be a part of the OpenStack ecosystem. We should continue to provide them with the level of support they’re used to today, and make them an integral part of our summits and promotional material. The list of projects in this category will look something like:

Barbican
Ceilometer
Congress
Designate
Heat
Ironic
Magnum
Manila
Murano
Sahara
Trove
Zaqar

I’m sure that some of these could be argued to fit into the core OpenStack definition by now, so this distinction will need to be made carefully. These two categories will capture all of the projects mature enough to get the OpenStack seal of approval, but new projects still need somewhere to incubate. This is why we should:

~~Bring back Stackforge!~~

UPDATE: It’s been pointed out that my understanding of the Stackforge retirement was wrong. Though the Stackforge namespace was retired, the incubation framework remains to this day. I’m going to leave the original section here so others can learn from my mistake, and add a new section about the graduation workflow. Thanks to those who brought this to my attention.

Stackforge was an excellent program for incubating new projects without diluting OpenStack as a whole. It was retired in October 2015, at which point all projects had to move into the OpenStack Big Tent or leave entirely. I think we should bring it back and continue to foster innovation within the community and beyond. Developing a project in Stackforge will be the beginning of the path towards becoming part of OpenStack or the OpenStack Family. Everything that is currently considered part of OpenStack but doesn’t appear in the two lists above, will move into Stackforge.

While I sympathize with the TC’s position that there were too many Stackforge projects to review for graduation into OpenStack, I disagree that the right solution was to have no graduation process at all. I believe that the introduction of the OpenStack Family, together with a clear set of requirements for graduation from Stackforge will help to alleviate the review workload on the TC.

Update the graduation path

With the addition of the OpenStack Family, we’ll need a new path for projects that graduate from StackForge. The way I see it, there will be two types of graduation:

StackForge -> OpenStack Family
OpenStack Family -> OpenStack

The first will be based upon maturity and conforming with community rules (the Four Opens), the second will generally be based on adoption rate. Graduation into the OpenStack Family will indicate that we as a community are vouching for a given project’s compliance with our rules, and that it shows a degree of maturity/usability. Graduation into the OpenStack core will probably be a much rarer event, as it indicates that a given project has become sufficiently widespread in its use that we consider it to be an integral part of a reference OpenStack deployment. There will likely be extra rules applied to OpenStack core projects that are not applied to the OpenStack Family, such as needing to follow the same release cadence.

This added layer of granularity will help us to much better tell the story about which projects are OpenStack, and which projects enhance OpenStack.

Also note that this does not necessitate separate git namepsaces for the three different project categories (neither did the proposal in the retracted section above, but that may not have been clear). The retirement of the StackForge namespace was a big step forward in managing the lifecycle of a project and it’s not something I’d advocate for reverting.

In Summary

I believe that this proposal, or one derived from it, would go a long way towards providing a tighter focus for OpenStack that has been missing since the introduction of The Big Tent. It’s not final, but it’s a good starting point, and I enthusiastically welcome debate. Please feel free to reach out to me here, on Twitter, or on OpenStack irc (john-davidge), whether you think this is the best idea or the worst.

A blog by John

Updated sometimes, maybe