A picture of me standing at a lectern, working on a laptop computer, on the stage of the FWD50 digital government conference

Hi! I’m Alistair. I write surprisingly useful books, run unexpectedly interesting events, & build things humans need for the future.

Using hackathons to drive internal innovation at Intel

293px-Intel-logo.svgCathy Spence is the Enterprise Architect and PaaS Lead for Intel IT’s Cloud Computing program. She’s been at the company for nearly a decade, and has a background in IT and software engineering. Recently, she championed the use of Platform-as-a-Service computing within the software and hardware giant, and used hackathons to build support. You can read more about her work here.

In this interview she talks about innovation, Intel’s involvement in the Open Data Center Alliance, the metrics the company uses to measure progress for internal initiatives, and what worked best to get developers on board. The full video and audio are included at the bottom of the post.

Some background on moving to cloud computing

We’re going to look at how Cathy tested ideas and grew supporters. These lessons apply to many businesses, but Cathy works on cloud computing, so it might help to explain what that is a bit first.

Cloud computing has significantly changed how companies use IT resources. Rather than order a server, spend days preparing it, and managing it, they can get access to virtual hardware on demand. This is known as Infrastructure-as-a-Service (IaaS).

Unfortunately, even IaaS takes some work. Developers still need to manage those machines, which can undermine the benefits the cloud promised. One way to fix this is to offer Platform-as-a-Service (PaaS) to developers, where they can simply paste their code into an interface and have it work. Public cloud providers like Force.com, Google App Engine, and Engineyard offer this today.

If you want to deploy a PaaS within an organization, you have some options (like VMWare’s Cloud Foundry.) But none of this makes any difference if your developers won’t use it—and developers like what they know. So making a bet on IT deployments is risky at best. Cathy used an iterative approach and a series of hackathons to test demand, quickly identify shortcomings, and build management buy-in for what amounted to a serious overhaul of how Intel develops software.

Intel and innovation

Cathy’s employer is no stranger to innovation. Intel’s Clay Christensen quite literally wrote the book on disruptive innovation, and the company’s culture reflects this. “That article about software eating the world is my favorite example to point to in terms of just trying to think about how the world is changing,” she says.

Cathy’s not sold on centralizing and isolating innovation. “There has been a lot of work done to study how innovation happens in large companies. And a lot of it suggests that you create a center for it—a group of people dedicated to innovation. [But] what’s happening now is that we need that innovation to be more pervasive within an organization. Not just a particular team that is off innovating, and then they have to cross the chasm of taking that seed innovation and putting it into individual products.”

Intel has an innovation group—Intel Labs. “They are like the really advanced special forces teams that go out and explore new ideas. They have very strong relationships with universities, and they have a formal university program in place looking for targeted research projects around particular areas where we want to innovate.” But to make these ideas find a home within Intel, they still need incubation. “They do the innovation in some type of an incubator. And then they bring it back and they try to introduce it into the more mainstream process that are going on.”

I challenged Cathy on the idea of knowing the areas in which you want to innovate—because after all, knowing these ideas up front suggests you’ve already framed the problem. “The special forces teams are trying to do the targeted thing, but we are trying to make the actual innovation part of your regular job,” she says. “Take user experience. I was just talking to one of our human factors engineers yesterday, and I asked her if she was going to own user experience. And she said, ‘I can’t really own that.’ Because it has to be really part of everything.”

“I think that innovation is in the same category as that. It has to really be part of your daily job, and you have to be looking for ways to innovate … so you have got the advance team working on specialized topics, but then you have your individuals that are actually in the heat of working on your projects and trying to embed that innovative thinking into your regular working model.”

David McRaney said that much of what looks like luck is opportunism. But it is very hard to say, “go and be opportunistic,” and still comply with the traditional incentive models of a large company—quarterly earnings, KPI’s, clear goals. So I wanted Cathy to explain how Intel reconciles that predictability with the opportunism needed for innovation.

“I have been working on platform as a service for about two years. It is really tough, because you go into your Management Review Committee meeting to pitch your idea: ‘We want to offer a Platform as a Service,’ which is very, very new and nobody even really understands what that means. It is very nascent from a technology perspective, but it is an area that we want to take some leadership. You’re asked questions all the time like ‘Where is your demand?’ and ‘What developers are asking you to provide platform as a service?’”

“The problem is, the developers have never even heard of Platform as a Service. I can tell you, when I started working on this, no one came up to me and said, ‘I want platform as a service.’ It was up to me to paint the picture of what the future could look like.”

Ultimately, Cathy got approval and used gradual, iterative testing to grow the project. “We took the  path of ‘I try a little proof of concept type of thing, and then I did a wider implementation.’ I did some surveying of people. So I got some data because you have to be able to do data driven decision making. But at the same time, it was still very speculative as to whether or not this would work.”

Despite this approach, many of Cathy’s metrics were technical, not user-centric. The management team wanted to measure things like the time it took to procure a server. “We looked at some actual data of how long that takes,” said Cathy. “And it was on the order of 70 days. This proposal was we want to make it possible to land applications in less than a day.”

Cathy is trying to measure things like application throughput time, resource efficiency, and business agility—technical criteria that are familiar to the IT team. So while Intel has created a climate of risk tolerance where experimentation is acceptable, this is sustaining innovation—Cathy knows what she needs to do (make IT more efficient) and how to do it (use clouds and PaaS.)

I asked her if they also look at user-centric data such as adoption. “I keep track of that … I have to track how many apps we have and how many users we have. What does that uptake look like? Does it look like people are experimenting with it? Or are we really starting to land some real applications there. And even now, it’s still pretty nascent. Also, we think—regardless of whether you are using PaaS or IaaS—that there is a big value around having cloud-aware applications.”

Since Cathy’s users are internal IT, I wondered if they really had a choice—or if they would simply do what she told them. “They do have a choice,” she explained. “Through the service catalog. But I have also been doing other outreach-type things to engage with the development community.” Cathy mentions that there are many different kinds of developer at Intel—internal and external, hardware and software, and so on. “Trying to figure out where my market is and who the people I am trying to communicate with are is a little bit of a challenge. I try to over-communicate; one of the things I did this past year is I hosted a series of hack-a-thons internally as an alternate to doing instructor-led training.”

Cathy ran eight of these hack-a-thons last year, which had roughly a hundred participants worldwide, aimed at a total of about 400 developers who would be PaaS IT developers, but only once she’d done a broad-based survey of their needs, sent to roughly a thousand internal emails, which yielded 2-300 responses.

She asked them what kind of application would belong in a data center, and what tools they use today. Then she explained the concept and asked whether they would use it as described. “My last question in the survey was, hey, you know, we’re going to be piloting this, do you want to be part of the pilot? And I actually ended up with 25 volunteers as a result of that, which I thought was actually pretty good. So it was worth it for me to do the survey to see kind of the tools and technologies and direction that people are going. Introduce them to this concept—which was very new at the time. And then invite them to participate.”

The hack-a-thons needed minimal budget for food, and some prizes. Ultimately, Cathy used Intel’s internal recognition system, which can handle reward compensation and pay the necessary taxes, since global rewards are complex. “We had some really interesting hack-a-thons—especially in some of the other developing geos. In Asia Pacific, they wanted to use it as an opportunity to bring everybody together and actually do some innovation around a theme.” At these events, some people brought their own applications, while others started some at the event itself. “You would get bonus points if you actually got it deployed and working and you could demonstrate stuff,” explained Cathy. “I have a rubric, basically, and you get different points for different things, like if you actually got it all working. But in general, you can’t finish anything in a day.”

Judging was tricky as well. “We had one hack-a-thon—I think there were 20 different applications. If you just spend 5 minutes on each application trying to judge, it could take a while. It is like doing science fair.” Cathy had three judges, including herself, a local site champion, and a third person.

While it was a lot of work, Cathy jokes that it actually stemmed from laziness. “The funny thing is that I did this because I didn’t want to create instructor-led training and make everybody do the same exercises. I was avoiding work myself by creating all of that. It is a ton of work. I didn’t want to do that, right? I have a lot of other things I wanted to do. But then people loved it and all of a sudden people are contacting me within the company asking, “hey, how are you running this hack-a-thon, and can I reuse some of these concepts?” Intel has since done a events around hardware and Maker Faire style events.

I wanted to know whether Intel had seen the cloud adoption Cathy was hoping for, and how they tracked actual developer engagement versus one-time dabbling. “I haven’t done that actually. I probably could have done a lot better job on that—but the hack-a-thon did create demand, so my evil plan worked, right? And I got a lot of questions from different groups asking, ‘when is it going to be available for production?’”

The hack-a-thons were also a good way to reveal limitations and gaps in what Cathy was trying to launch. “The first hack-a-thon that we had, I had my whole team there—and we are not a huge team; we’re only 7 or 8 people. The concept around the hack-a-thon I call ‘roadside assistance.’ We actually have people wandering around the room helping you with, getting your cloud application written, answering questions, and so forth. We have a QA team that goes through different scenarios, and we have a test harness and all that kind of stuff. But when you get real people using the platform you learn a lot. That was one of the huge benefits from doing the hack-a-thon. In one example, if you deployed your application and it wasn’t starting properly, it would all of a sudden start to consume a bunch of resources. And if they didn’t shut it down properly, then it would start to cause a problem in terms of our capacity.”

There’s plenty more insight into hack-a-thons and internal adoption in the interview, which lasts around 30 minutes.

Here’s the audio:

And here’s the video of our discussion: