OpenStack has become something of a lightning rod for media attention – attention which, sadly, focuses reliably on the fear, uncertainty and doubt that any disruptive ecosystem will naturally produce. Much like the eddies that form on the edges of fast-moving water, this open-source project throws off a fair amount of confusion, conjecture and rumor.

I’m hoping to get in front of the upcoming round of media frenzy (which is certain to spin out as soon as someone from the Register, CIO Magazine or Network World bothers to skim recent posts to the OpenStack-PPB list), and try to answer a simple question – what does it mean for a project to be “part of” OpenStack?

Ignoring, for just a moment, the eloquent-but-high-level mission statement of the OpenStack project, I will make three simple observations:

1. OpenStack means: a cloud operating system that meets Rackspace’s needs.
2. OpenStack means: a cloud operating system that meets NASA’s needs.
3. OpenStack means: a community effort that meets the needs of its’ members.

While the relative *order* of these observations is likely to provoke outrage from many members of the OpenStack community, I think it’s still reasonably fair to describe it in this fashion – if OpenStack ceases to meet the needs of Rackspace or of NASA, the founding partners, then it will cease to be OpenStack.

What constraints does this put on new projects?

Many.

What has been talked about, at length and by many of the most prominent members of our community, is scale. Rackspace (and indeed many others in our ecosystem) is a service provider, with (at least) tens of thousands of customer accounts. OpenStack has got to scale.

Also discussed, at least in the earliest days, is development philosophy and methods. No, I’m not talking about “development in the open” (although I think it’s generally a good idea), I’m talking about agile, fail-fast, and test-driven development. I’m talking about continuous integration and good unit test coverage. I’m talking about a working-code-trumps-designs-or-promises attitude.

One thing that *hasn’t* been discussed much, is language. Or more precisely, language consistency. Yes, a foolish consistency is the hobgoblin of little minds – and yet, I think it’s no accident that OpenStack is, thus far, 98% Python (plus or minus a smattering of bash).

I sit on the OpenStack Project Policy Board, and I’m getting ready to make myself fairly unpopular. There are a *large* number of projects in the “affiliated with OpenStack” phase, and a few of them are gearing up to apply for official OpenStack status.

Which brings me back to what it *means* to be OpenStack, and particularly NASA’s needs.
(DISCLAIMER: I no longer work for, at, or in any particular association with NASA. I’m speaking merely from my historical position as the architect of the NASA Nebula project, the precursor to OpenStack.)

NASA, like any US Federal agency, is under near-constant IT attack. Far more than any service provider, we have had to focus on the security of our platform. No, I’m not saying that python is more secure than any other language (and NASA has had its share of security issues with EVERY language and platform) – simply that limiting OpenStack to *as few languages as necessary* keeps the strike surface smaller. It keeps system complexity down. And it makes OpenStack monitoring and maintenance a more straightforward process.

Which brings me back to the process of becoming unpopular.

To the best of my knowledge, there are “related” projects in:

• PHP
• JAVA
• Erlang
• Ruby
• C++(?)

Every one of these projects will be considered on the basis of its own merits, by the Project Policy Board and in good time. But for my part, I’ll be looking at security. I’ll be looking at test coverage. I’ll be looking at scale and, yes, I’ll be looking at language.

This applies equally to new project submissions, as well as to legacy projects from both Rackspace and NASA – even a few that I had always thought *were* part of OpenStack.

Oh, and a final thought – to me, OpenStack has always been about being *pythonic*, even in the rare case when we weren’t (yet) writing Python. Which means, as opposed to Perl, there really is “one way to do it”. I love the fact that OpenStack supports every possible hypervisor – but I hate the fact that the test coverage is not equal for each. I love that we have several (many?) different network models available – but I find the current proposal to write, big-design-up-front, a new-from-scratch OpenStack network component misguided at best, and counter-productive at worst. Ditto for the efforts to write a new-from-scratch replacement for the nova-volumes component. Again back to the principles of agile, test-driven development – unless there’s something fundamentally *wrong* with the architecture of the existing components, this strikes me as a bit of NIH syndrome. Iterative improvement, a long set of carefully reviewed changes to an existing codebase, will keep code quality high and interface pain down.

So if I vote against your project tomorrow – it’s not personal. Really. I’ll be voting against some of my own as well – and yes, that really *is* as weird as it sounds.

Too bad.

Have I lost my passion for open source, open data, open science or open government?

Certainly not.

I’m still on the OpenStack Project Oversight Committee, and I plan on kicking ass, now more than ever. At the same time, I have never seen a conflict between a capitalist agenda, and a social agenda – and I’ve now got a great occasion to prove that.

While I was tripping around Italy over new years, I had a chance to tour Pompeii with my children. It’s a great place to pause for a moment and appreciate permanence, and the idea of building something to last.

What are *you* working on? If it was dug out of the ashes 1,000 years from now, would you feel proud of it?

To deal with this, I whipped up a quick OutputStream and matching Proxy interface:

package org.gem;
import java.io.IOException;
import java.io.OutputStream;
public class PythonOutputStream extends OutputStream {
    private IPythonPipe thispipe;
    public void setPythonStdout(IPythonPipe mypipe) {
        thispipe = mypipe;
    }
    @Override
    public void write(int arg0) throws IOException {
        thispipe.write((char) arg0);
    }

package org.gem;
public interface IPythonPipe {
    public void write(char output);
}

mystream = jpype.JProxy("org.gem.IPythonPipe", inst=sys.stdout)
errstream = jpype.JProxy("org.gem.IPythonPipe", inst=sys.stderr)
outputstream = jpype.JClass("org.gem.PythonOutputStream")()
err_stream = jpype.JClass("org.gem.PythonOutputStream")()
outputstream.setPythonStdout(mystream)
err_stream.setPythonStdout(errstream)
ps = jpype.JClass("java.io.PrintStream")
jpype.java.lang.System.setOut(ps(outputstream))
jpype.java.lang.System.setErr(ps(err_stream))

The view from the office

EDIT: Feel free to contact me by twitter DM as well: http://twitter.com/jmckenty

As most of you have by now figured out from my FourSquare stream, I’ve been spending a fair bit of time in Europe. In fact, I’ve taken a sabbatical from NASA to work on the Global Earthquake Model. This international project is headquartered in Pavia, Italy, with the bulk of the development team in Zurich, Switzerland, at the ETH university.

I’m helping to reboot the project. In fact, we’re changing everything – language (from exclusively Java to mostly Python), methodology (from a traditional waterfall approach to agile using Scrum), and development approach (continuous integration and test-driven development FTW).

Everyone, by now, has heard about the Haiti earthquake. You know that more than 250,000 people (roughly 40% of the affected population) died. What you might not have realized, is that similar sized earthquakse in California routinely kill only dozens. The difference?

Without earthquake models, you can’t assess hazard.

Without hazard assessment, you can’t make building codes that are appropriate for the seismic risk.

Without building codes (and the right kind of building reinforcement, even for traditional construction techniques), most of a large city turns into rubble.

Without software, people die.

Here’s a quote from our job posting (full link at http://www.globalquakemodel.org/system/files/SED_GEM-Software_Engineer.pdf):

GEM, the Global Earthquake Model, is a non-profit, international effort to tackle the understanding of earthquake risk worldwide. In parallel with our scientific and diplomatic efforts, we are developing a unique, open-source software system that computes all aspects of earthquake hazard, risk, and impact. This is high-performance, scientific computing at its finest. And you could be a part of it.

We’re looking for only the best and brightest, regardless of where you currently are in the world, to come to Zurich and join a small, tight-knit team of engineers, scientists, and software developers. If you’ve got the aptitude and attitude, we’ll train you in the rest.

That’s right – we’re writing open-source software to save lives. Who’s with me?

Desperate to get sleep, but I’m terrified to wake up to 1,000 new bug reports. It’s like streaking the quad, but with code. #openstack @jmckenty

There was a video up from an interview I had given the week before.

There were over 100 developers camped out in the #openstack channel, asking probing questions. Most of them were politely worded versions of: “What the hell were you thinking?”

Patches started coming in. Then a New York Times blog post. More bug reports. More patches. More press.

Original Nebula Logo

I’d love to say that we shared a moment of solidarity as a team, toasted ourselves, and wrote more code. But truthfully, half of the NASA Nebula team were on a red-eye flight to DC, to run another set of training workshops for new Nebula users. Some were holed up with the legal teams, trying to figure out how to take the Nova example (brute-force software release) and use it to reshape NASA’s open source software release policy. Vish was bravely manning IRC, answering the flood of questions.

And in a classic case of nostalgia, I started thinking back over the crazy history of this “thing” that has had more names, than developers.

The code name of the “compute” component in Open Stack is called “Nova”. While, if you had the appropriate access to NASA internal systems, you could find early references to the “Novae” task in Nebula development (which was originally the PXE and Puppet-based configuration management system we used for deploying Nebula to physical hardware), the earliest public record of nova is this:

“NASA Nebula Bonding Lunch – We Are the Nova”

Which is possibly one of the worst photos of me on the internet. C’est la via.

While in Austin, Texas last week, at the first Open Stack Developer’s Summit, I had an opportunity to give a brief presentation about NASA Nebula – why we were in the Cloud business, some of the history that led us to this point, etc. While I never prepare slide decks to be standalone (I prefer to use them as audio-visual aids to the substance of the event, which is obviously paying attention to me), you can have a look at the whole thing:

It was based, loosely, on an internal memo I sent a while back to some members of the Nebula team (excerpts below):

There are, literally, a ton of people on the Nebula team now. Which is awesome. But it also mean there are dozens of you that I’ve never had a chance to interact with personally, who may have missed a lot of our early (ancient) discussions about WHY we’re all working so damn hard.

It’s not about the money.

Which is to say, Nebula is not, primarily, about cost recovery. It’s not (mostly) about saving money, for missions, projects, or the Agency at large.

And on the flip side, we’re not DOING this just because it’s our job. Nebula is not the place for “work to rule” – if you don’t wake up every morning with a burning desire to get to your keyboard and roll the Nebula ball forward, then there are honestly better places and projects for you to be working on.

It’s not about being on the “cutting edge“.

We get a ton of press on Nebula, most of it excited over how innovative NASAhas become, how we’re embracing new technologies, how we’re leading the government forward. And all of that is great – but that’s not why we’re doing it. This is not innovation, for the sake of innovation.

It’s about the Science.

I haven’t really been at NASA long enough to have a good grasp of its long and illustrious history. So in the same fashion that I get away with practicing Computer Science without having formally studied it, I have to work from first principles. In this case, the first principles are a happy little document titled, the “Space Act” (Specifically the Declaration of Policy and Purpose, section 102):

Sec. 102.  (a) The Congress hereby declares that it is the policy of the United States that activities in space should be devoted to peaceful purposes for the benefit of all mankind.

(7) Cooperation by the United States with other nations and groups of nations in work done pursuant to this Act and in the peaceful application of the results thereof;

(8) The most effective utilization of the scientific and engineering resources of the United States, with close cooperation among all interested agencies of the United States in order to avoid unnecessary duplication of effort, facilities, and equipment;

So I’d just like to point out that the mission statement of the Agency, includes specifically section 102(d)(8) – is to cooperate with other agencies in order to make effective utilization of scientific and engineering resources. AKA… Cloud Computing.

Open Source, by which I mean participation in an open source process, as well as the release of source code under an open license, is an activity NASA can partake of under section 102(a). It is way more meaningful than most of the rest of what we’re doing.

http://novacc.org

From the website:

Nova is a cloud computing fabric controller (the main part of an IaaS system) built to match the popular AWS EC2 and S3 APIs. It is written in Python, using the Tornadoand Twisted frameworks, and relies on the standard AMQP messaging protocol, and the Redis distributed KVS.

Nova is intended to be easy to extend, and adapt.

Jeff as a webhook

PROBLEM: Help people understand how cool webhooks are.
PROBLEM: Find out when there’s new content on a site.

SOLUTION: Put a webhook into the site’s search indexing tool – SOLR.
SOLUTION: Custom SOLR Token Filter.

The code, while JAVA and therefore evil, is trivial.

WHY?

Every cool web app (sooner or later) has some cool content in it. So you add a search engine. When your users want to build mashups with that content, you add webhooks to that engine. And you’re done.

Starting point: http://e-mats.org/2008/06/writing-a-solr-analysis-filter-plugin/

GOTCHAS:
JAVA – Evil and Stupid
Java classpath (on Mac) – also Evil and Stupid
Java permissions (to connect to remote servers) – http://forums.sun.com/thread.jspa?threadID=5309619

WHERE IS THIS GOING TO LIVE?

Webhooks coming to NASA soon...

FINAL TAKEAWAY: Fitz was right – everything should be HTTP.

As is often the case with the large, complex issues facing humanity, most of our effort is (dare I say it?) wasted on trying to fix the wrong things. Like changing our lightbulbs. Even changing our cars is, most of the time, the wrong thing. Why?

Making a thing, almost ANY thing, takes an enormous amount of energy input. These days, most of that energy input is petroleum, generating huge amounts of waste carbon. So regardless of how much BETTER the end product is, it needs to fulfill two criteria to justify the switch:

1. It needs to last forever (or some reasonable facsimile).
2. It needs to use so much less carbon, that it offsets the output of buying something NEW in the first place.

Our problem, as a culture, is less an OIL addiction, as it is an addiction to CONSUMING. Endlessly.

Let’s take the principles of the “Tucker” automobile, and apply them to all consumer goods. Do away with “designed obsolescence“, and try building things to LAST, for a change.

My favorite example of this? A Stanley #2 plane.

(‘But wait!’ you cry. ‘What about the Rabbits?’)

Ah yes. The Rabbits.

Even in “Peak Everything”, when the author discusses how America may need to shift back to a nation with 20-40% of our workforce involved directly in agriculture (in a move called re-ruralization), which mimics much of what Cuba did after the trade embargo was put into place – there’s still too much emphasis on simple vegetables and grain. As a passionate carnivore, I have to ask – Where’s the beef?!?

What’s wrong with cows, anyway? A lot, as it turns out.

1. They produce tons of methane, a greenhouse gas that’s hundreds of times worse than simple carbon dioxide.
2. They’re horribly inefficient at converting feed or pasture to meat.
3. They’re grazed at great distances from where they’re processed, and then transported great distances AGAIN to where they’re consumed.

4. They require relatively high-quality rangeland, and human-edible feed – which contributes directly to the worldwide shortage of arable land.

What’s better? Locally-grown, locally-processed rabbits. Why?

1. Rabbits are incredibly efficient at converting vegetable scraps, and simple non-human-edible vegetation (dandelions, for instance) into meat.
2. Their meat is lean, perhaps cutting down on the obesity endemic.
3. They’re fast and simple to process, with very little toxic byproduct from slaughtering.
4. They breed like, well, like rabbits.
5. They’re simple to breed on a highly-localized scale, reducing transport and attendant fuel costs, pollution, etc.

So here’s my pitch, Mr. Cuban – strike some direct-to-restaurant business deals with major fast-food chains (I’m thinking Wendy’s or McD-scale) for the “Bunny Burger”. Work with these guys on some compelling branding to take the stigma away from breaking fast on the bunny scramble, and put together massively distributed production of bunnies (and their attendant feed). Compost and sell the manure. Feed them kitchen scraps from the same restaurants that are buying your bunny burgers.

Hire the homeless as Neo-shepherds, to keep stray dogs and eagles away from your flock. And where do you put them? How bout the parking garages that will be abandoned as our global economy (and our ability to make ridiculous monthly payments) hits the toilet?

Rabbits. FTW.

How to use VPN to solve VoIP problems

As some of you may be aware, when I’m not busy with my day job at NASA, I run a small VoIP company. The why and when of this company is a story for another day, with intrigue, betrayal, ex-CIA agents, con artists, lawsuits and utter, utter sleep deprivation. But, as I said, that will have to wait.

It’s a simple enough business – multi-line phone service for homes and offices, complete with distinctive ring, cool automated phone trees, time-of-day dialing, etc. And it has always had one nasty technology problem in it – NAT.

Anyone who has ever dealt with VoIP has run into all the various ways that NAT can go wrong. And I thought I had worked out ways to deal with most of them – at least, until I started working with some new hardware.

Let’s skip the gory details, and summarize by saying that I’ve been forced to deal with this problem in a decisive, and final, fashion.

No more NAT. Instead – OpenVPN.

There are usually two different ways to handle getting a VoIP phone connected to a remote VPN server – either connect the phone directly (if it supports that), or put in some sort of VPN concentrator. There are even routers that support VPN tunneling – if you’re willing to pay for it.

There’s also another way – a happy story called DD-WRT. This is an alternative, open-source firmware for many of today’s most common hardware routers – including the WRT54G2 that I’ve been dealing with. Along with being just really cool, it offers both a VoIP SIP Proxy (which I opted against), and an OpenVPN implementation.

So this is what I’ve ended up with – an office LAN full of happy telephones, connected through a VPN tunnel that originates in their gateway device, and terminates in my Asterisk Server. No muss, no fuss.

And a few happy side effects, to boot:

• Because the VPN traffic is encrypted (and prioritized by the ISP), I’m no longer vulnerable to any nasty stateful packet inspection and downgrading of third-party VoIP (see the lawsuit between Vonage and Shaw for details).
• This VPN link works both ways – I can remotely configure all those phones without having to expose them to the public internet. Goodbye, site visits.

(Some credit for this solution goes to Chiral Software – although I didn’t find this post until after I had decided on an OpenVPN solution, it still had some helpful hints. But I believe using DD-WRT makes the whole thing more elegant.)

Turning 404 Errors into “I’m Feeling Lucky” Searches

Jesse Andrews and I often take turns copying each other. (Of course, since he’s ridiculously famous, and I’m only mildly annoying, people usually notice me copying him. Alas). Anyway, Jesse was the first one to point out Humanized Messages to me. After which, I used (and modified) a jQuery plugin to put Humanized Messages into a Firefox Extension.

Jesse had to go one better, and built almost the same effect into Searchy, one of HIS Firefox Extensions. But he wrote it from scratch, and made it actually much nicer.

Anyway, I had been hacking on various ways to monetize browser extensions at the time, and harking back to the days when we wrote the “HP Browser Booster” at Mercurial. One of the many things it did, was to redirect 404 errors to the AOL Search Page. I figured I could do better than that – and I wrote “Four-Oh”.

This happy extension detects a 404 error (well before the 404 page is displayed, which was a happy little piece of coding), strips the requested path off the base domain name, converts slashes into spaces, and then uses that as an “I’m feeling lucky” query for the site:<domainname.com> of the original request.

Lost?

It means that trying to go to http://overstimulate.com/taboo – will automatically land you on http://overstimulate.com/projects/taboo.

Not satisfied with simply a useful little utility, I also parse an RSS feed of random jokes, and pop one of them up on top of the resulting page – in a Humanized Message window that looks remarkably like Searchy.

Jesse, your turn.

Making money on the internet, at least at a small scale, is exactly the opposite of making money in the real world. Trying something is so cheap, you should just keep trying things until something works. Unfortunately, just because it works a little bit – doesn’t mean it’ll work much more than that.

I started writing software when I was seven years old. Most of the years I clung to the idea that, someday, I would write a piece of software that people would pay money to OWN. When I began working, first as a coder and then later as a software architect, I still held out dreams of building the next Google – a gigantic, brilliant piece of engineering that would make me the richest man in the world.

And then I discovered Toyota.

This was also just about the time I was going broke, chasing one of these big, complicated visions.

To take my mind off things, I decided to start copying Jesse Andrews (the godfather of UserScripts.org). Build simple, clean, simple, tiny, simple… really SIMPLE pieces of software that do only one thing.

But I took a little twist on Jesse’s way of doing it – I would only work on things that I could code, test, and ship – in one sitting.

And we called it “Tiny Apps“. And it was good.

But not that good. A great little app, thrown together in a few hours and tossed out into the surf of the Intarwebs, is a bit of a sitting duck. People started shooting them – by which, I mean ripping our ideas off EXACTLY – and then getting tons of press.

US: Whoissocial.com. Four days later: usernamecheck.com

US: Gastracker.cognition.ca. Several months later: A whole bunch of them.

My original tiny app, a grid-computer-powered price-watching service, remains the only Tiny App to make much money. Okay, any money. Really. Freddy Mangum and I worked on developing this concept further, and prototyped a few variants – GoogAzon, which adds Amazon and eBay search results to google pages; and “The Kraken“, which was a tongue-in-cheek response to the “Pirates of the Amazon” extension that came out late last year.

The Kraken was notable enough to get us some substantial press (and a fair number of downloads, too) – but it’s never made a cent.

Here’s how I’ve made money on teh internet:

• Google Adsense. But only a little. ($46 in three years, roughly).
• Amazon Affiliate revenue – both embedded in software, and in blog posts. (Some many bunches of dollars).
• eBay Affiliate revenue – About $3.

Now, if you’ve bothered to read all the way down here, this is where I’m going to give away a little secret. And by secret, I don’t mean something really clever that I dreamed up on my own – I mean a strange and lucky coincidence that happened to lead to something cool.

As of right now, I make more money from my blog, than from anything else.

Seriously.

Actually, to be fair – I make more money off of a single post on my blog.

This post.

Which brings me back to Seth Godin, and Jesse Andrews. Just build cool stuff, solve your own problems – and share.

Oh, and get an Amazon Affiliate account.