Technology demonstration and hardware challenges seek prototypes or fully developed solutions to catalyze and demonstrate breakthrough technical innovations. The Orteig Prize that inspired Charles Lindbergh’s transatlantic flight and the Ansari XPRIZE for commercial spaceflight are two well-known technology demonstration and hardware development challenges. In practice, many of these challenges are not limited to hardware innovation, since large-scale technology development challenges can require both software and hardware components to accomplish the task, especially in topic areas like robotics.
Large competitions often take place over many years, demonstrate sophisticated technology and take advantage of multiple steps where entrants have benchmark evaluations before advancing. Precursors to large challenges, or even standalone prototyping hackathons, have been very successful in making use of relatively inexpensive electronics components and 3-D printers to develop a diverse set of proof-of-concepts. These types of challenges vary in scope, and not every technology development challenge may be as large as these examples. You may find that a smaller demonstration challenge fits your needs or is needed to more completely define your requirement set for a long-term challenge. It’s important to recognize that technology demonstration and hardware challenges can be designed to develop technology along the entire technology readiness level (TRL) spectrum, from paper concepts to final products ready for infusion.
It’s important to survey the current state of innovation early during the problem-definition phase of technology and hardware challenges. This analysis serves several purposes:
- It objectively defines state-of-the-art metrics for the technology’s performance.
- It establishes the level of sophistication and performance of technologies and hardware currently available to consumers.
- It determines which product has market share and why.
- It uncovers barriers to innovation in the sector.
- It determines what technologies are in development by querying industry.
- It reveals whether innovation in this space would build upon existing technologies and standards in the broader ecosystem, and defines implications for the intended user community.
- It reveals who is already working in this space and who might be competitors or collaborators in a challenge.
This method of research typically consists of both primary (interviews with stakeholders) and secondary (literature review) research. State-of-innovation analysis establishes an evidence-based framework for determining the prize amount and the desired outcomes of the competition. The analysis can help you understand if the technology you seek will require research and development or if a solution can be developed by combining existing technologies.
2. Consider from the outset if your goal is limited to technology development and demonstration or if you also seek to simulate markets through your challenge.
Technology development and demonstration challenges seek to reward the successful demonstration of a specific set of performance criteria. These performance criteria are typically many times more ambitious than the current state of innovation. For many challenges, achieving this increase in capability is a worthy enough goal. However, for some technology demonstration challenges, stimulating new markets for innovative technologies both on the supply and demand side is a critical complementary goal. A key outcome of some challenges is that a new product becomes available to consumers in the marketplace. If this is your goal, strategize from the outset how your agency can help with commercialization objectives. Methods used to stimulate markets through a challenge may have additional legal considerations. Consult your counsel, especially if your challenge involves:
- judging a business plan or the ability to “go to market” as part of the challenge’s evaluation criteria
- providing seed money for a startup company
- matching winning ideas with potential funders or customers, or at least exposing them to one another
- engaging in joint media outreach and interviews
- allowing the winner to include a logo from the challenge in its advertising material
- working together with the winner on future high-profile projects
Supporting technology commercialization in your challenge will require a completely different challenge structure. Be clear with your leadership from the beginning if both technology demonstration and technology commercialization are desired outcomes of your challenge. If they are, it may require larger prize purse and additional operational costs.
A technology or hardware competition requires the use of an appropriate testing facility for the demonstration portion. For many innovators, the opportunity to test their technology in an appropriate facility is at least as valuable as the prize purse. Several government agencies have testing facilities and trained technicians who can assist in the demonstration portion of the competition. Collaborating with these agencies can result in meaningful partnerships and avoid duplicative funding for programs with cross-agency goals. In the Desal Prize, the U.S. Agency for International Development (USAID) partnered with the U.S. Bureau of Reclamation to use their Brackish Groundwater National Desalination Research Facility in New Mexico. Similarly, the XPrize Wendy Schmidt Oil Clean-up Challenge used The National Oil Spill Response Research & Renewable Energy Test Facility.
Access to resources also can be more valuable than the prize purse, especially if that access helps overcome significant barriers to entry. For example, NASA’s Cubequest Challenge is offering three teams a launch into space through manifested slots on NASA’s Space Launch System Rocket. This required an internal NASA partnership between two different divisions. The challenge asks teams to perform cubesat communications at a lunar distance from earth. NASA’s offering of launch slots increased the number of competitors up front by removing an access barrier—transportation to the moon.
Beyond providing access to test facilities and other resources, partners can play meaningful roles in the administration of a competition. For example, the NASA Centennial Challenges Program often partners with third party, nonprofit “allied organizations” to administer their challenges. These partners support judging, communication, registration and many other activities in the conduct of a challenge. Choosing the right allied organization partner is critical. Partnering comes with risks. A community may not be as far-reaching or compatible as desired for a given challenge, or conflicting legal conditions may exist. You need to ensure each alliance is appropriate before committing for the long-term challenge.
Technology demonstration and hardware challenges run the gamut of technology readiness levels, and there are many shapes and sizes. Consider breaking a large challenge into smaller, incrementally more complicated challenges, especially if your ultimate goal is a large-scale demonstration. These might include design competitions where solvers submit written concepts for their dream technologies or a weekend “hardware-hackathon” where you bring solvers together to build prototypes in real time. These types of smaller-scale events can help you refine your problem statements and inform your design of future challenges. They may even be the means to achieving your ultimate goal. They can also increase audience awareness of other technology sectors. For example, NASA’s 3D-Printed Habitat Challenge taps into the additive manufacturing technology sector to print a habitat on a non-Earth planetary surface. The challenge kicked off with an architectural design competition to spark interest in the challenge. NASA partnered with the publishers of MAKE: Magazine to conduct the challenge judging and presented awards at a Maker Faire. This allowed for increased promotion of the challenge and competitors.
Many successful technology demonstration and hardware challenges have multiple evaluation stages for several reasons:
- Multiple steps allow for the ability to award prize money for incremental advancements in the technology. For complex technologies, this allows teams to receive seed funding in the form of prize money giving them greater clout for fundraising.
- By building in multiple steps, you have the ability to determine if teams are converging on a solution or not. A failed approach caught early can be turned into a success. Proceeding blindly without checkpoints on a large-scale hardware challenge can have major financial and public image costs.
- You may need a proof-of-concept of the challenge process itself to gain internal support for a large-scale hardware challenge. The enthusiasm of potential competitors can be a useful measure of a hardware challenge’s potential success.
Just as some software challenges aim to highlight the usefulness of a certain dataset, some hardware challenges highlight a specific hardware platform or promote innovation in a single industry or field. These smaller-scale events attempt to produce proof-of-concepts in a very short period of time. The next step would be to increase a project’s technology readiness level by moving on to a larger event or even being sponsored by an additional funding source.
In larger-scale competitions, individuals, teams or companies who participate are required to invest significant time and expense. In kind, the challenges themselves have audacious goals that are intended to spark imagination and drive significant awareness to the problem and potential solutions. Both the sponsors and participants in these events can benefit from a small-scale approach which lessens barriers to entry yet still has potential for long-term impact.
Often a kickoff workshop intended to get everyone on a level playing field before initial registration can help inform potential competitors and allow them to collaborate early. This is especially true if the challenge involves multiple technology sectors or if solutions are likely to encounter regulatory issues. For example, a pre-registration workshop for a cubesat challenge could engage industry leaders and representatives from regulatory agencies to discuss space communications best practices. This gives confidence to the teams, demonstrates that you are planning for them to be successful and encourages potential participants to network with each other prior to any formal commitments.
6. Provide potential participants and partner organizations enough time to review challenge requirements, rules and judging criteria prior to kickoff, and be iterative in development of those products.
A challenge’s rules and requirements will significantly steer the productivity, efficiency and attitude of your participants and partners. Leveling expectations is critical to avoid issues during the challenge itself. Since individual perspectives and culture differences can sometimes alter the reception of rules, requirements and judging criteria, it is imperative that the community as a whole have a chance to provide meaningful feedback to the baseline set of ground rules. You can do this in a number of ways:
- Conduct a series of phone calls that allow for unstructured, early input from a variety of stakeholders and potential competitors. Make sure to review the structure of your input sessions with your attorneys so you don’t violate the Paperwork Reduction Act or the Federal Advisory Committee Act.
- Release a draft set of your prize concept and rules for a public comment period through the Federal Register, making sure to alert potential competitors and stakeholders of the deadlines and location of the notice. This is a common practice of the Centennial Challenges
- Conduct an online dialogue to collect comments on your draft prize structure and rules. USAID’s Desal Prize used an online platform to collect comments on a number of questions it was seeking feedback on.