Using pilots within an innovation strategy – CIOs weigh in

Current and former CIOs explain how they're using pilots as part of their innovation strategy.

A hand moves a pawn through a conceptual maze of circuits. [navigating/negotiating/strategy/tactics]
Olivier Le Moal / Getty Images

Numerous books have been written about the value of pilots and experiments. According to Michael Schrage in The Innovator’s Hypothesis, "simple, fast, and frugal experimentation is the smartest investment that serious innovators can make." At the same time, it has become increasingly important to build flatter, more diverse organizations and to charter innovation teams to deliver upon innovation goals. Doing this right is critical to establishing business competitive advantage in the digital age.

To better understand how they're being considered as part of business and IT strategy, we asked CIOs how they're using pilots as part of their innovation strategy. We asked five questions of CIOs Martin Davis, Page Francis, Deb Gildersleeve, and Jay Ferro, and former CIOs Joanna Young, Isaac Sacolick and Mike Kail.

Do you use pilots before you invest?

Martin Davis says, "definitely, I try to go for a smaller blueprint project up front that identifies the best solution and high-level design. Taking this step enables you to make better decisions when starting the full project." Page Francis agrees, explaining she "pushes for pilots when we already have a defined need and are confident of funding but want to try a variety of solutions or when we want to test/develop a cheap idea for improvement."

Numerous books have been written about the value of pilots and experiments. According to Michael Schrage in The Innovator’s Hypothesis, "simple, fast, and frugal experimentation is the smartest investment that serious innovators can make." At the same time, it has become increasingly important to build flatter, more diverse organizations and to charter innovation teams to deliver upon innovation goals. Doing this right is critical to establishing business competitive advantage in the digital age.

To better understand how they're being considered as part of business and IT strategy, we asked CIOs how they're using pilots as part of their innovation strategy. We asked five questions of CIOs Martin Davis, Page Francis, Deb Gildersleeve, and Jay Ferro, and former CIOs Joanna Young, Isaac Sacolick and Mike Kail.

Do you use pilots before you invest?

Martin Davis says, "definitely, I try to go for a smaller blueprint project up front that identifies the best solution and high-level design. Taking this step enables you to make better decisions when starting the full project." Page Francis agrees, explaining she "pushes for pilots when we already have a defined need and are confident of funding but want to try a variety of solutions or when we want to test/develop a cheap idea for improvement."

Similarly, Deb Gildersleeve claims, "for big investments definitely, I need to test the design and assumptions to determine the scope of investment. Cost is always a factor at some level", and former CIO Joanna Young says she "pushes for pilots before material investment is made. We can learn whether the concept will delight or disturb quickly."

Isaac Sacolick also find pilots to be a useful pre-investment tool: "proofs on concept flush out risks and unknowns especially for new technologies, upgrades, and integrations." While Mike Kail believes that "pilots can serve as a binding agent", explaining he "pushes for a true partnership on pilots with tight feedback loops between teams/companies. Done right this results in the proverbial win-win."

What kinds of things do you pilot?

For Page Francis, pilots provide an opportunity to prove and validate concepts: "this includes testing multiple vendors product/service and having an open mind to the new and different including reimagining things. I like to test the service with people. The best technology in the world means zip if the provided service model is unsavory to deal with." Ferro agrees, saying he "wraps a little structure around his quick wins program and celebrates them regularly without sucking the life out of things with needless red tape and permissions". Likewise, Young recommends you "prioritize pilots for those items that will create the greatest impact/change for the intended customer/user."

Who are the best leaders for pilots?

Francis "prefers it to be the functional service owners guaranteeing things. If things go south, IT will own the ugly parts. To be clear, we have to consciously yet carefully push for others to embrace the new. I pilot nothing without pre-educating myself on all the options. No, vendor has served us that well. Evaluation is done by round table discussions with participants who will manage and own the product or process with access to results from others' experiences."

Taking a contrasting position, Young claims she "typically want the innovators/disruptors running pilots/POCs, as long as they don't run amok". Likewise, Gildersleeve claims pilot leaders "need to be nimble and be able to apply learnings from any result. That can be a different skillset from large project leaders."

Should pilots be alleviated of traditional disciplines?

The CIOs we spoke to had conflicting opinions on this. Davis doesn’t disagree and says, "as with any project you need to scale the management needs to the size of the project, no exceptions for a pilot, but it should be very small scale, therefore, require less of most things." While Young suggests, "There should be different guidelines for pilots and POCs; but these still be mindful of security and careful to not impact existing production systems."

What can CIOs do to encourage teams to pilot new ideas?

CIOs had many ideas for encouraging pilots:

  • Push for pilots before material investment
  • Use pilot/POC environments well away from existing products
  • Only pilot where and when necessary
  • Provide finite budget and quick turnarounds for most client solutions
  • Remove fear, but ensure appropriate structure
  • Try to learn whether the concept will delight or disturb quickly
  • Put the onus on vendors to show their product works for our world
  • Celebrate every step
  • Recognize team leaders for the wins
  • Make sure the organization learns whatever the result

Parting words

Based on our conversations, CIOs consider pilots to be valuable to innovation. They believe pilots reduce risk and help drive out more valuable products in the end. However, it is clear that pilots need to be well managed to achieve their business objectives.