Joyent: The Docker-friendly cloud you’ve never heard of
Joyent started the container party, later validated by Docker. Despite superior technology, does an independent public cloud like Joyent have a chance?
One of the privileges of writing for InfoWorld is that I get to meet some of the brightest minds of the industry — and they actually talk to me. In October I met Bryan Cantrill, the CTO of Joyent, while attending Couchbase Live NY.
Cantrill is one of those people who could give a truly entertaining technical or technomarketing talk no matter what the content happens to be. He weaves in computer history and pops off the stage at you whenever he speaks. The last person I met who could present on anything and make it as interesting was Marc Fleury back in my JBoss days.
Microsoft is adopting Apple’s approach to PC management, while also keeping the familiar Configuration
The PaaS that wasn’t
Before we get to my conversation with Cantrill, a little background: In 2012 I wrote “Which freaking PaaS should I use?” under the theory that, before long, everything that wasn’t SaaS would quickly move to PaaS.
After that, crickets — I mean, it took Cloud Foundry a rewrite, a spin-off, and a long time to gain any traction whatsoever. Red Hat’s OpenShift has improved a lot, but it isn’t really a public cloud play and has yet to take over the world. I barely remember that Heroku exists, and for a little while, it looked like there was Amazon, that upstart Microsoft, and everyone else.
But something changed: the emergence of Docker. Docker let us have most of the benefits of PaaS while still letting us lovingly tune the software layout and use specific … everything (because who among us doesn’t like to coil the toaster in the process of making toast — or deploying apps).
While Cantrill’s company Joyent is most known for being “the Node.js people,” that isn’t how Joyent keeps the lights on. Joyent is a cloud provider with both compute and storage options, as well as its own data centers. It also sells a supported open source environment similar to Cloud Foundry or OpenShift but based entirely on container technology.
In Cantrill’s words, Joyent’s is better or at least ahead of the market:
Joyent has been in the right place at the wrong time for an extended period of time. We’re a company really based around containers and the belief in container-based virtualization, which is great. But we were way ahead of the market. And it’s only been the last year or so that the market has caught up and realized, hey, wait a minute, that container idea is actually a pretty great idea.
In Cantrill’s presentation he did what I’d been dying to see: connect Docker to Solaris Zones. Cantrill, an ex-Sun guy, totally understood the history and made that connection. But Joyent can be defined as much by what it is not as well as by what it is. Cantrill acknowledges that, as CTO of a relatively little-known public cloud provider, he faces an uphill battle in mind share: “We are not Amazon and we are not OpenStack.”
On the other hand, who wants to be OpenStack? Cantrill likens OpenStack to the old Solaris CDE; having several companies attempt to develop something together rarely (possibly never) works out well. There are a lot more players in OpenStack than there were in CDE. Cantrill thinks OpenStack’s time has passed:
I think people are now beginning to realize that, actually. OpenStack is an also-ran for yesterday’s revolution. We’re really much more focused on what we think is tomorrow’s revolution. Today’s, hopefully, but tomorrow’s in terms of the container revolution and being an all-container-based stack. So we’ve got it all — and the software we develop is open source.
Like other vendors, Joyent has both a public and a private cloud offering. Joyent’s container technology, Triton, offers kernel-native containers but supports Docker packaging. Joyent also claims bare-metal performance. After all, Triton is based on decades-mature Solaris Zones.
Consider the people trying to run Docker images on Amazon. That is container-style virtualization running on actual virtualization. Get out your wallet because Bezos needs extra jet fuel money! Not only is Joyent’s pricing cheaper for CPU and utilization compared to Amazon, but you’re less likely to use as much because you’re not running your container on top of a virtual machine.
This un-Hadoop/un-S3 storage is not EMC approved
SANs are kind of dumb. The idea of putting your storage way across the network from compute and shoving all the disks in a box was based on a rather specific model of client-server computing. Newer, more resilient software designs don’t depend on emulating that resiliency with appliances full of disks (sorry, Dell).
This is where Joyent runs ahead of the curve — and a bit ahead of HDFS, EMR, and S3 — with Manta, an object storage solution with integrated compute:
I’ll give you an example of something we’ve already built [Manta], we use every day, that the market is still not ready for, so you don’t hear me mention it, because people are just not ready for it. And that is the ability to break container’s storage. So we’ve got an object store that’s like S3, but if you want to actually compute on your objects, instead of having to drag them out of the object store, you can spin up a container of where those objects actually live.
I observed, snarkily, that Cantrill didn’t seem to believe that stuffing a bunch of disks in a box, connecting it with a network cable, and sticking it “way over there” is a fundamentally good thing:
No, I don’t think that centralized storage is a great idea… Apps can compute; the divide between storage and compute doesn’t make sense. It feels like it makes sense, and it has this kind of nice property that your compute becomes totally transient. This machine dies, you can just spin up another one, because your storage is over here, which is great, but it’s kinda like the end of the good news. Because, by the way, your data’s over here, and your compute is over here. And by the way, the component that you’re trying to optimize for failure — compute — is, like, the most reliable component we’ve got. It’s actually the spindle that is the most unreliable component we have. The spindle is still the component whose death is assured before depreciation.
With Manta you can spin up storage via an API and run your massively parallel processes written in R, Python, Node.js, Perl, Ruby, Java, C/C++, and more. Manta also supports streaming. However, Manta is its own thing. It isn’t Spark, and it isn’t Hadoop.
It’s hard not to think of history repeating itself. Without a compatible API and an ecosystem, Manta may be yet another technology that’s “ahead of the market” — it’s better, but can’t reach mass adoption because either people don’t understand it or they care more about the ecosystem than the superiority of the underlying technology.
Is better good enough?
When Joyent first developed its container technology, it had no industry standard or Linux-based API. Docker ended up creating the de facto standard that made people want to jump on board the container bandwagon — which has in turn created a market for Joyent.
Maybe when Spark becomes a first-class API, the same will happen with Manta. It’s clearly a good design and open source, so Manta could capture the imagination of people looking for something better than HDFS or SANs or the current storage mess around virtualization.
But ecosystems are as much social as they are technical or economic. Is having better technology enough to enable Joyent to break through? Joyent is picking a lot of fights at once: Amazon, Hadoop/Spark, and friends, everyone dreaming of the “hybrid cloud.” Is better good enough to win customers while you make enemies? Time will tell.