CoreOS Stackanetes puts OpenStack in containers for easy management

CoreOS Stackanetes puts OpenStack in containers for easy management

Stackanetes uses Kubernetes to deploy OpenStack as a set of containerized apps, simplifying management of OpenStack components

The ongoing effort to make OpenStack easier to deploy and maintain has received an assist from an unexpected source: CoreOS and its new Stackanetes project, announced today at the OpenStack Summit in Austin.

Containers are generally seen as a lighter-weight solution to many of the problems addressed by OpenStack. But CoreOS sees Stackanetes as a vehicle to deliver OpenStack’s benefits — an open source IaaS with access to raw VMs — via Kubernetes and its management methodology.

A rich dev and test toolchain, collaborative end-to-end workflow, and improved Windows support put Chef

OpenStack in Kubernetes

Kubernetes, originally created by Google, manages containerized applications across a cluster. Its emphasis is on keeping apps healthy and responsive with a minimal amount of management. Stackanetes uses Kubernetes to deploy OpenStack as a set of containerized applications, one container for each service.

The single biggest benefit, according to CoreOS, is “radically simplified management of OpenStack components,” a common goal of most recent  OpenStack initiatives.

But Stackanetes is also a “single platform for consistently managing both IaaS and container workloads.” OpenStack has its own container management service, Magnum, used mainly as an interface to run Docker and, yes, Kubernetes instances within OpenStack. Stackanetes stands this on its head, and OpenStack becomes another containerized application running alongside all the rest in a cluster.

Easy management = admin appeal

Other projects that deploy OpenStack as a containerized service have popped up but have taken a different approach. Kolla, an OpenStack “big tent” project, uses Docker containers and Ansible playbooks to deploy an OpenStack configuration. The basic deployment is “highly opinionated,” meaning it comes heavily preconfigured but can be customized after deployment.

Stackanetes is mostly concerned with making sure individual services within OpenStack remain running — what CoreOS describes as the self-healing capacity. It’s less concerned with under-the-hood configurations of individual OpenStack components — which OpenStack has been trying to make less painful.

One long-standing OpenStack issue that Stackanetes does try to address is upgrades to individual OpenStack components. In a demo video, CoreOS CEO Alex Polvi showed how Stackanetes could shift workloads from nodes running an older version of the Horizon service to nodes running a newer version. The whole process involved only a couple of clicks.

With Stackanetes, CoreOS is betting more people would rather use Kubernetes as a deployment and management mechanism for containers than OpenStack. At the least, Kubernetes gives admins a straightforward way to stand up and manage the pieces of an OpenStack cluster — and that, by itself, has admin appeal

Getting started with basics of building your own cloud

Openstack Cloud tutorial

My daily routine involves too much of AWS Cloud infrastructure. And let me tell you AWS now has grown to an extent that it has now become the synonym of Cloud. I mean they have grown without leap and bounds in the past few years and believe me many other major players are not even near them in the cloud arena (Yeah of course Google and Microsoft does have their own cloud solutions which are pretty brilliant for all use cases, but nobody has the user/customer base that aws has in their public cloud architecture).

Nothing can match the flexibility, elasticity, and ease of use that cloud provides.  Because I remember when I use to work with physical hardware machines (I had to literally wait for hours to get one ready up and running for an emergency requirement. Then if I need additional storage for that machine again wait some more time.) . And if you are using the cloud, then you can spin up a few cloud servers in seconds (believe me in seconds) and test whatever you want.

What is OpenStack Cloud?

An year ago I happen to read an article from netcraft regarding their findings on AWS. According to them in 2013 itself AWS has crossed the mark of 158K in the total number of public facing computers.

Now imagine if you get the same features that AWS cloud provides with something open source that you can build in your own data centre. Isn’t that amazing? Well that’s the reason why tech giants like IBM, HP, Intel, Red Hat, CISCO, Juniper, Yahoo, Dell, Netapp, Vmware, Godaddy, Paypal, Canonical(Ubuntu) support and fund such a project.

This open source project is called as Open Stack, and is currently supported by more than 150 tech companies worldwide. It all started as a combined project by NASA and Rackspace in 2009 (well both were independently developing their own individual projects, which at a later point got together and later called as OpenStack). Well NASA was behind a project called as NOVA(which is very analogous to amazon ec2 and provided computing feature), and Rackspace built another tool called as Swift(a highly scalable object storage solution, very similar to AWS S3).

Apart from these, there are other components that help make openstack very much same as aws cloud(we will be discussing each of them shortly, and in upcoming tutorials, we will configure each of them to build our own cloud).

Openstack can be used by anybody who wants their own cloud infrastructure, similar to AWS. Although its origin will trace back to NASA, its not actively developed/supported by NASA any more.

And they are currently leveraging aws public cloud infrastructure J

If you want to simply use openstack public cloud, then you can use Rackspace Cloud, ENovance, HP cloud etc(these are very much similar to aws cloud.) with their cost associated. Apart from these public openstack cloud offerings, there are plug and play cloud services, where you have dedicated hardware appliance for openstack. Just purchasing it and plugging it would turn it into an openstack cloud service without any further configurations.

Let’s now discuss some of the crucial components of OpenStack, which when combined together will make a robust cloud like any other commercial cloud (Like AWS), that too in your datacenter, completely managed and controlled by your team.

When you talk about cloud, the first thing that comes to your mind will be virtualization. Because virtualization is the technology that caused this cloud revolution possible. Virtualization basically is nothing but the method of slicing resources of a physical machine to smaller/required parts, and those slices will act as independent hosts sharing resources with other slices on the machine.  This enables optimal use of computing resources.

  • OpenStack Compute:  So one of the main component of cloud is virtual machines, that can scale without bounds. This need of the cloud in openstack is fulfilled by something called as Nova. Nova is the name of the software component in OpenStack cloud, that offers and manages virtual machines.

Apart from the compute requirements, the second major requirement is storage. There are two different types of storage in the cloud, one is block storage(very similar to the way how you use RAID partition on any of your servers and format it and use it for all kind of local storage needs), or  normal disk storage, where your operating system files are installed etc.

  • OpenStack block storage (Cynder): will work similar to attaching and detaching an external hard drive to your operating system, for its local use. Block storage is useful for database storage, or raw storage for the server(like format it, mount it and use it), or else you can combine several for distributed file system needs (like you can make a large gluster volume, out of several block storage devices attached to a virtual machine launched by Nova).

The second type of storage full fills the scaling needs, without bounds. You need a storage that can scale without worry. Where your storage need is of static objects. This can be used for storing static large data like backups, archives etc. It can be accessed with its own API, and is replicated cross datacenter, to withstand large disasters.

  • OpenStack Object storage(Swift): is suitable for storing multimedia content like videos, images, virtual machine images, backups, email storage, archives etc. This type of data needs to grow without any limitation, and needs to be replicated. This is exactly what OpenStack swift is designed to do.

Last but not the least, comes Networking. Networking in the cloud has become so matured that you can create your own private networks, access control lists, create routes between them, interconnect different networks, connect to remote network using VPN etc. Almost all of these needs of an enterprise cloud is taken care by openstack networking.

  • Openstack Networking(Nova-networking, or Neutron): When I say openstack networking, think of it as something that manages networking for all our virtual hosts(instances), and provide IP address both private and public. You might be thinking that networking in virtualization is quite easy by setting up a bridge adapter and routing all traffic through it, similar to many virtual adapters. But here we are talking about an entire cloud, that should have public ip’s, that can be attached, detached from the instances that we launch inside, there must be one fixed ip for each instance, and then there must never be a single point of failure etc.

According to me openstack networking is the most complex thing that needs to be designed by taking extreme care. We will be discussing openstack networking in very detail, in a dedicated post, because of its complexity, and importance. Also it can be done with two different tools. One is called as nova-networking, and the other is called as neutron. Please note the fact that each and every component of openstack cloud needs special attention on its own, as they are each very distinct and work combined together to form a cloud. Hence i will be doing dedicated post for each of its major components.

Openstack is very highly configurable, due to this very reason, its quite difficult to mention all of its possible configurations in a tutorial. You will come to know about this, at a later point, when we start configuring things in the upcoming series of posts.

Higher Level Overview of Openstack Architecture

Component Name Used for Similar to
Horizon A dashboard for end users or administrators to access other backend services AWS Management Web Console
Nova Compute Manages virtualization and takes requests from end user through dashboard or API to form virtual Instances AWS Elastic Compute
Cynder For Block storage, directly attachable to any virtual instance, similar to an external hard drive EBS(Elastic Block Store)
Glance This is used for maintaining a catalog for images and is kind of a repository for images. AMI (Amazon Machine Images)
Swift This is used for Object storage that can be used by your applications or instances to store static objects like multimedia files, backups, store images, archives etc. AWS S3
Keystone This component is responsible for managing authentication services for all components. Like a credentials and authorization, and authentication for users AWS Identity And Access Management(IAM)

You might have got an idea of what OpenStack Cloud actually is till now. Let’s now answer some questions, that can really prove helpful in getting a little bit more idea of what openstack really is, or say how these individual components fit together to form a cloud.

What is Horizon Dashboard?

Its nothing but a web interface for users and administrators to interact with your OpenStack cloud. Its basically a Django Web Application implemented in mod_wsgi and Apache. Its primary objective is to interact with the backend API’s of other components and execute requests initiated by users. It interacts with keystone authentication service, to authorize requests before doing anything

Does nova-compute perform virtualization?

Well, nova-compute basically is a daemon that does the job of creating and terminating virtual machines. It does this job through virtual machine API calls. There is something called as a libvirt library. Libvirt is nothing but an API for interacting with Linux virtualization technologies(its a free and open source software that needs to be installed with nova as a dependency).

Basically libvirt gives nova-compute, the functionality to send API requests to KVM, Xen, LXC, OpenVZ, Virtualbox, Vmware, Parallels hypervisors.

So when a user in openstack requests to launch a cloud instance, what actually happens is nova-compute sending requests to hypervisors using libvirt. Well other than libvirt, nova-compute can send requests directly to Xen-Api, vSphere API etc. This wide support of different virtualization technologies is the main strength of nova.

How does Swift Work?

Well swift is a highly scalable object storage. Object Storage in itself, is a big topic, so i recommend reading the below post.

Unlike block storage, files are not organized in hierarchical name space. But they are organized in a flat name space. Although it can give you an illusion of a folder with contents inside, all files inside all folders are in a single name space, due to which scaling becomes much easier compared to block storage.

Swift uses multiple commodity servers and backend storage devices to combine together and form a large pool of storage as per the requirement of the end user. This can be scaled without bounds, by simply adding more nodes in the future.

swift object storage

What is keystone?

Its a single point of contact for policy, authentication, and identity management in openstack cloud. It can work with different authentication backends like Ldap, SQL or a simple key value store.

Keystone has two primary functions

  • Manage Users. Like tracking of all users, and their permissions.
  • Service list/catalog. This is nothing but providing information regarding what services are available and their respective API endpoint details.

What is Openstack Cinder?

As discussed before and shown in the diagram, cinder is nothing but a block storage service. It provides a software block storage on top of basic traditional block storage devices to instances that nova-compute launches.

In simple terms we can say that cinder does the job of virtualizing pools of block storage(any traditional storage device) and makes it available to end users via API. Users use those virtual block storage volume inside their virtual machines, without knowing where the volume is actually deployed in the architecture, or knowing details about the underlying device of the storage.

Google is said to endorse ARM server chips, but don’t get excited yet

Google is said to be working with Qualcomm to design servers based on ARM processors, which would be a significant endorsement for ARM as it tries to challenge Intel’s dominance in data centers.

Google will give its public backing for Qualcomm’s chips at an investor meeting next week, according to a Bloomberg report Wednesday that cities unnamed sources. If the chips meets certain performance goals, the report says, Google will commit to using them.

It would be a big vote of confidence for both ARM and Qualcomm, but if history is a guide then it’s too early to say how significant the news really is. ARM won’t be the first x86 alternative that Google has rallied behind, and it’s unclear if the last effort has come very far.

Google's IBM Power server board

Two years ago, Google made a big show of support for IBM’s Power processor. It was a founding member of IBM’s OpenPower initiative, which allows companies to design and build Power servers for use in, among other things, cloud data centers like those run by Google.

Google even showed a Power server board it had designed itself. “We’re always looking to deliver the highest quality of service for our users, and so we built this server to port our software stack to Power,” a Google engineer said at the time.

But there’s been little news about the partnership since. Google hasn’t revealed whether it’s using Power servers in production, and last year it made only vague statements that it’s keeping its options open.

Google is secretive about the technologies it uses, and it might well have plans to use both ARM and Power, but public endorsements don’t tell us much, and in the case of ARM it’s likely even Google doesn’t know for sure.

The search giant could have several reasons for showing support for non-x86 architectures. Google probably does want to test Qualcomm’s server chips, just as it tested IBM’s, to see if a different architecture can shave costs off of running its vast infrastructure. A show of support from Google encourages development of the ecosystem as a whole, including tools and software, which will be important if Google decides to put a new architecture in production.

Such statements also serve to pressure Intel, giving Google some price leverage and pushing Intel to develop new, more power-efficient parts — something Intel has done since the ARM threat emerged a few years ago.

There’s been a lot of debate about whether “brawny” cores, like Power, or “wimpy” cores, like ARM, are more efficient for cloud workloads. It depends partly what workloads you’re talking about, and there are also costs to consider like porting software.

Urs Holzle, who’s in charge of Google’s data centers, once published a paper on the topic titled “Brawny cores still beat wimpy cores, most of the time.” But that was in 2010, and the ARM architecture has evolved a lot since then.

Qualcomm's ARM server chip

 Qualcomm disclosed its plan to sell ARM server chips in October, joining rivals like AppliedMicro. It showed a test chip with 24 cores running a Linux software stack, but it still hasn’t said when a finished product will go on sale.

Derek Aberle, Qualcomm’s president, told investors last week that shipments would begin “probably within the next year or so.” But he suggested significant sales are still “out a few years.”

A vote from Google could do a lot to boost its chances. But it’s also hard to know where all of this will end up. The only sure thing is that the processor business is a lot more interesting than it was a few years ago

Microsoft launches IoT Hub to ingest data from the physical world

Microsoft’s Azure IoT Hub for managing fleets of devices that collect data about the the physical world will be made generally available on Thursday.
The product, which was announced in September, allows companies to provision, manage and ingest data from millions of devices that are part of the internet of things. It’s designed to help companies ignore the messy business of figuring out how to handle a fleet of IoT devices and focus on the hardware and software.

At its core, Azure IoT Hub serves as the backend for telling embedded devices what to do, and then getting data back from them in order to power applications. One of the advantages to using it over building a similar system from scratch is that Azure IoT Hub is designed to integrate with Microsoft’s other cloud services.

That means developers who want to build an intelligent software backend for their devices would have an easier time getting the information into something like Azure Machine Learning for processing and action.

Users pay for Azure IoT hub on a per-message basis, whether it’s a message from a device to Azure, or one from the cloud to a physical device. Microsoft offers a free tier that lets users try out the service with up to 500 devices and sending up to 8,000 messages per day, and users can pay for more expensive plans that eliminate the device limits and increase the cap on the number of messages sent through IoT hub.

Azure IoT Hub is a part of Microsoft’s cloud IoT Suite, which (as the name implies) includes a set of tools designed to help companies manage their integrated devices.

In addition to the software news, Microsoft also announced partnerships with Advantech, Dell, HPE and Libelium on the Azure Certified for IoT program, which lets manufacturers show that their IoT devices are able to easily integrate with Azure products like IoT hub.

A New Front in OpenStack Networking

OpenStack Networking Roll Cloud

Yesterday Aptira, Akanda and Cumulus Networks announced a partnership to deliver innovative network virtualisation and orchestration solutions on OpenStack using Astara and Cumulus networking infrastructure. You can read the press releases if that’s your thing.

At Aptira we’re really excited about this. Dave talked recently about needing new types of technology to allow organisations to innovate, and the OpenStack Astara platform on Cumulus Networks is exactly what we’ve been talking about.

These technologies give service providers the ability to not just streamline their existing cloud environments, but to also create entirely new offerings. A converged control plane with built-in overlay/physical interworking reduces duplication and enables high value hybrid cloud and NFV products. Entirely novel service chains can be created.

But don’t take our word for it. Contact Aptira today and explore the what this new partnership can do for your business.

Here is what happens inside Nova when you provision a VM

At the Essex conference summit this past month, we presented a session on  OpenStack Essex architecture. As a part of that workshop we visually demonstrated the request flow for provisioning a VM and went over Essex arthicture. There was a lot of interest in this material; it’s now posted in Slideshare:

  • Recap of the request flow
  • Essex architecture survey/overview

In fact, we’ve packaged up the architecture survey/overview as part of our 2-day Bootcamp for OpenStack. The next session is scheduled 14-15 June. This time around will carry out the training at the Santa Clara CA offices of our friends at Nexenta. Last course was delivered at our Mountain View office right before the OpenStack summit in April to a sold out crowd.

OpenStack Glance, Murano, Community app catalog: links in a chain


On Wednesday I did a Glance API V3 talk with Brian Rosmaita, hoping to send a proper message to those who are going to use our APIs. We also had some fruitful conversations on Artifacts before and after the sessions; it seems like the topic still attracts attention.

After the session, we tried to do some internal (Mirantis-only) planning, but we were suddenly “attacked” by a very friendly (and very talkative) person who wanted to talk about Murano. It’s funny, we worked so hard to get some traction, and when we finally did, it caused problems for my schedule. Not that I’m complaining, of course. That’s what summits are for, and we appreciate the outside perspectives!
Cross-project property protection

The idea here is that Glance has ability to “protect” (i.e. to make read only or to hide) some of the image metadata properties based on policies. However, there are projects which create, update or read the images on behalf of the user but still should have other privileges to access those properties. For example, if Nova creates a Snapshot of the VM, it should be able to set some special properties, while a regular user – that is, someone accessing the images API – should not be able to do that. Right now, Glance has no way to differentiate between these two types of requests; both of them are authenticated by the user’s token.

It’s been suggested that we utilize so-called “Service Tokens” – additional authentication tokens being passed with the request, and identifying that these requests are made by some service (e.g. Nova) on behalf of the user. In this case, the request being made by the service will have both tokens, and the property protection will take into account the roles of both the user and the server. This streamlines the process.

Another related topic is property protection for read operations. Projects such as Searchlight are indexing image properties based on notifications emitted by Glance, and they (probably) should not be allow to search by the protected properties if the searching user does not have privileges to access them. However, this requires Searchlight to duplicate the property-protection settings of glance, and that’s a bad idea, as duplication is always error-prone. But we don’t know what to do with it. One of the ideas may be not to include the protected properties into the notification at all, but this will prevent Searchlight from indexing such data at all, even for admins.
Glance: Image Signing

I attended a small session about needed improvements to the image signing feature, which was recently added and landed in Liberty. The list of improvements includes adding new signing algorithms (MD5 is definitely outdated, so more should be added, but we need to maintain backwards compatibility) and increasing the key size. The latter requires us to increase the size of the signature, and this may lead to some issues; the size of the signature property will have to be more than 255 chars, and since it is passed in the headers, that may cause some deployment issues.

The team has agreed on the first topic (more algorithms) and identified some open issues with the second (signature size). It was agreed that we should look more deeply into it, but that it should be solvable.
Glance: DefCore updates

The DefCore guys joined us to explain DefCore’s rules and guidance about APIs, their test coverage, the differences between public and private (or admin) APIs and their view of them. It was a useful and informative session, mostly aimed at syncing up and clarifying these things. No particular decisions were made, but the picture became a lot clearer.

One important thing that did come out of this session is that DefCore will provide a set of compatibility requirements within 3 months of the summit, at which point the community can provide feedback.

For, artifacts my understanding is that the plugin-specific APIs are fine, as we just do not include them into DefCore, and that’s fine. The “public” part should be DefCore-compatible, however. So, this actually matches what I thought before, so we don’t have to make any changes there.
Glance: Artifacts

Ok, this one was my session, and here is an important decision we’ve made: we are separating the API endpoints of Glance Images (v1/v2) and Glance Artifacts (Glare) into two different processes (and thus two different endpoints in Keystone). This actually means that the Glance v3 API is removed in favor of Artifacts v1. The project itself remains in the Glance codebase, and under Glance project guidance.

It was also decided to merge the v3 feature-branch of python-glanceclient into the master, so we have a single client library. In the meantime, the Glare CLI should be done in the OpenStack CLI itself, not in python-glanceclient.

So, this is quite a radical change for us. It is ok, and it makes sense, however it actually puts the things back to where it was before the Vancouver. It’s frustrating because these changes are confusing people, so I feel like we better pick something and stay at it, instead of changing the decisions every cycle with every new PTL.
Glance Priority List

We decided to have a special tag in the commit message that indicates the work done by that commit is considered a priority for the Glance team during the Mitaka cycle. This tage then enables reviewers filter the reviews in the dashboard based on the approved team priorities.

As for as the actual priorities, security fixes have top priority, of course. Also, the conversion from Nova v1 to v2 is considered a primary priority. Most of the patches in this process will target Nova, but some things may need to land in Glance. These patches should indicate appropriate needs in the commit messages.

We agreed, that for the Mitaka cycle, most other features are of lower priority. (They are listed in the spec to be landed this week.) We also decided that Glare API stabilization is a priority on its own, but it’s actual priority level was undecided. We’ll file a separate spec on that ASAP.

For AppCatalog, there was a priority and roadmap presentation, followed by a design session.

The presentation clearly identified the potential conflict of AppCat, Murano and GlanceArtifacts, also it was expressed a very serious security concern on the fact that AppCat does not provide any way to verify the resources. AppCat should have https endpoint support, as well as the ability to sign resources. We decided to document the current issues, start fixing the https issue (as it’s easier) and then think about how to do the consistency guarantees.

During the design session we discussed the workflow needed to update the assets. We agreed that the existing objects should never be modified. Instead, a new version of the object should be published.

The Murano Contributors’ Meetup actually turned into a cross-project work session for Murano, Glance and AppCatalog to define the borders between projects, and agree on shared responsibilities and reuse of common functionality.

We’ve started with what had been called Glance V3 (and now is called Artifacts V1, aka Glare) and its applications for Community App Catalog, i.e. with what was defined in this article. It turned out that we’ve finally reached an agreement with the AppCat guys on the fact that Glare should be used for the backend. We’ve identified the immediate action items for this to happen (pluggable DB models, CORS API support and more) and agreed to collaborate in this direction. Also, I’ve made a demo of the proof-of-concept we made for them.

Then we covered the overlap in functionality between the Murano Dashboard and the AppCatalog UI. And this one looks important; we’ve decided to join the UI efforts here.

Instead of having a separate “Murano” dashboard with its “Application Catalog” -> “Applications” panel which looks really similar in the name and the concept to the “Applications” panel of App Catalog UI, we’ve decided to have a single “Applications” Dashboard (note, that there will be no “Murano” or other code names in the UI: users usually don’t care about the project names). It will provide capabilities to browse both remote applications (i.e. search in and the local one (Glare – if it is installed, of just Glance V2 if not) for all types of supported assets. It will users allow to install assets such as apps locally (if supported by local config) and launch the apps from the local copies (i.e. spawn a VM out of an Image, run a Heat stack out of a template or deploy a Murano application out of package).

The same dashboard will contain a “Running Apps” page to list what has already been deployed, with available actions, such as Stop or calling Murano Actions. Different versions of that dashboard will be available depending on what is currently installed in the cloud.

So this is a really big thing, which may turn into a new project with its own git repo and reviewers team. We need more discussion on this internally (we need to be sure that we don’t shoot ourselves in the foot with this: we don’t want a simple “launch a VM” or “run a Heat stack” to prevent people from using Murano), but overall that was a big success for the Murano team.