This challenge is no longer accepting new submissions.
Fusion of Face Recognition Algorithms
Fuse the outputs of multiple face recognition algorithms to improve accuracy
Office of Director of National Intelligence - Intelligence Advanced Research Project Activity
Type of Challenge: Software and apps
Submission Start: 05/23/2018 12:00 AM ET
Submission End: 08/06/2018 11:59 PM ET
DescriptionCan you leverage the data outputs of multiple face recognition algorithms to improve overall accuracy? There is a large literature on biometric fusion intended to improve accuracy via fusion of multiple modalities (e.g., face + fingerprint), multiple algorithms, or multiple samples. However, most of the research has only addressed 1:1 verification at the score-level. This prize challenge is aimed at stimulating research into methods to improve one-to-many (i.e., 1:N) identification accuracy via template-level fusion. Can further accuracy gains be realized by fusing feature-level templates or through more innovative score-level fusion methods informed by modern data science?
Who We Are: The Intelligence Advanced Research Projects Activity (IARPA), within the Office of the Director of National Intelligence (ODNI), focuses on high-risk, high-payoff research programs to tackle difficult challenges of the agencies and disciplines in the intelligence community. IARPA’s challenges invite experts from the broader research community to participate in IARPA research in a convenient, efficient, and non-contractual way. What We’re Doing: The 2018 FOFRA challenge tasks developers with fusing the outputs of multiple face recognition algorithms applied to the same input images. Fusion can be conducted on templates, before recognition, or on scores produced by recognition, with prizes awarded for best improvements to verification and identification accuracy. Why We’re Doing This: Face recognition error rates, particularly on uncontrolled face imagery, are well above zero. While algorithm development has seen considerable investment, other mechanisms for improving accuracy are known. Among them, there is a large academic literature on biometric fusion, covering multimodal and multi-algorithmic fusion. It shows that substantial accuracy gains can be made over using a single mode, or a single algorithm alone, and this can be achieved, in large part, using quite simple methods. The gains reduce when the fused inputs are correlated. The vast majority of the literature addresses biometric verification, rather than identification. Moreover, the literature covers score-level fusion rather than feature (i.e. template) level fusion. The latter, on information theoretic grounds, offers greater accuracy gains at the expense of some complexity. Where and When We’re Doing This: Registration to join the challenge will take place through Challenge.gov. You can find the challenge details and rules at that site. From the Challenge.gov webpage, participants will be directed to register with the National Institute of Standards and Technology (NIST). Registration closes on August 6, 2018.
- When does FOFRA registration begin? May 23, 2018
- Where to learn more about the challenge, including rules, criteria and eligibility requirements?
- Where do participants register? NIST FOFRA Support Website
- When is the deadline for comments on the API and other documents? June 6, 2018
- When is the registration deadline? August 6, 2018
- When will winners be announced? October 2018
PrizesVerification Score-level Fusion
Cash Prize Amount: $8000
1:1 Verification accuracy gains from multi-algorithm score-level fusion. Prize value includes $2000 base award, $2000 open-source award, and $4000 workshop attendance award.
Verification Template-level Fusion
Cash Prize Amount: $16000
1:1 Verification accuracy gains from multi-algorithm template-level fusion. Prize value includes $8000 base award, $4000 open-source award, and $4000 workshop attendance award.
Identification Score-level Fusion
Cash Prize Amount: $11000
1:1 Identification accuracy gains from multi-algorithm score-level fusion. Prize value includes $5000 base award, $2000 open-source award, and $4000 workshop attendance award.
Identification Template-level Fusion (Two-way)
Cash Prize Amount: $18000
1:1 Identification accuracy gains from two-way multi-algorithm template-level fusion. Prize value includes $10,000 base award, $4000 open-source award, and $4000 workshop attendance award.
Identification Template-level Fusion (Three-way)
Cash Prize Amount: $17000
1:1 Identification accuracy gains from three-way multi-algorithm template-level fusion. Prize value includes $9000 base award, $4000 open-source award, and $4000 workshop attendance award.
RulesThe NIST FOFRA Support Website provides detailed instructions on participation, registration, and submissions. An overview of the FOFRA Prize Challenge, including the challenge concept and planned evaluations can be found here. The full NIST Concept of Operations and API Specifications can be found here. Participation Eligibility To be eligible to win a prize under this competition, an individual or entity:
- Must have visited the FOFRA website at Challenge.gov;
- Must have read the FOFRA overview slides;
- Must have completed and submitted a FOFRA participation agreement form (see NIST FOFRA Support Website);
- Must have complied with all the requirements under these rules; and
- Must be (1) an individual or team member each of whose members are 18 years of age and over, or (2) an entity incorporated.
- May not be a federal entity or federal employee acting within the scope of their employment. An individual or entity shall not be deemed ineligible because the individual or entity used federal facilities or consulted with federal employees during a competition if the facilities and employees are made available to all individuals and entities participating in the competition on an equitable basis.
- Employees of IARPA, NIST, their affiliates, and/or any other individual or entity associated with the development, evaluation, or administration of the competition as well as members of such persons’ immediate families (spouses, children, siblings, parents), and persons living in the same household as such persons, whether or not related, are not eligible to participate in the competition.
- Foreign Nationals & International Developers: All Developers can participate with this exception: residents of, Iran, Cuba, North Korea, Crimea Region of Ukraine, Sudan or Syria or other countries prohibited on the U.S. State Department’s State Sponsors of Terrorism list. In addition, Developers are not eligible to participate if they are on the Specially Designated National list promulgated and amended, from time to time, by the United States Department of the Treasury. It is the responsibility of the Developer to ensure that they are allowed to export their technology solution to the United States for the Live Test. Additionally, it is the responsibility of participants to ensure that no US law export control restrictions would prevent them from participating when foreign nationals are involved. If there are US export control concerns, please contact IARPA and we will attempt to make reasonable accommodations if possible. IARPA will not be held responsible for devices shipped or transported to the U.S. that are confiscated by local authorities or that violate local export laws.
- Federal grantees may not use federal funds to develop challenge applications unless consistent with the purpose of their grant award. Federal contractors may not use federal funds from a contract to develop challenge applications or to fund efforts in support of a challenge submission without written approval of both IARPA and their Federal government sponsor. Entities affiliated with the IARPA Janus program may participate, but are ineligible to win prize monies.
- Entrants must agree to assume any and all risks and waive claims against the federal government and its related entities, except in the case of willful misconduct, for any injury, death, damage, or loss of property, revenue, or profits, whether direct, indirect, or consequential, arising from their participation in a competition, whether the injury, death, damage, or loss arises through negligence or otherwise.
- Entrants must also agree to indemnify the federal government against third-party claims for damages arising from or related to competition activities. Entrants are not required to obtain liability insurance or demonstrate financial responsibility in order to participate in the competition.
- By participating in the competition, each entrant agrees to comply with and abide by these rules and the decisions of IARPA, NIST and/or the individual judges, which shall be final and binding in all respects.
FOFRA Prize Challenge Evaluations
See the NIST FOFRA Support Website for details on the challenge evaluation methodologies and criteria.