follow us on facebook follow us on twitter email us

Passenger Screening Algorithm Challenge

About the Challenge
Can you locate hidden objects?

Posted By: Department of Homeland Security
Category: Scientific/Engineering
Skill: Algorithms Interest: Science & Research Partnership With: Science and Technology
Transportation Security Administration
Submission Dates: 12 p.m. ET, Jun 22, 2017 - 7:59 p.m. ET, Dec 15, 2017

About the Challenge: The Department of Homeland Security (DHS) gives notice of the availability of the “Department of Homeland Security’s Person Screening Algorithm Challenge Prize Competition and rules.” The DHS Science and Technology Directorate ( S&T) Homeland Security Advanced Research Projects Agency (HSARPA) Explosives Division (EXD) and the Transportation Security Administration (TSA) (Competition Sponsor) are seeking new automated detection algorithms from individuals and entities that improve the speed, accuracy, and detection of small threat objects and other prohibited items during the airport passenger screening process.

Algorithms developed from this Competition, or through further research and development under a limited intellectual property use agreement, have the potential to improve the speed, the detection of prohibited items, and the accuracy of Advanced Imaging Technology (AIT) scanners. A comprehensive set of new automated detection algorithms have the potential to be integrated into the latest screening equipment.

In 2012, Congress required all person screening technology to have automated target recognition. DHS S&T and TSA are developing better screening technologies and want to promote the development of the best automated detection algorithms for TSA screening equipment that have the potential to advance the DHS mission.

Traditional algorithms used in automated screening equipment are company proprietary and must be adapted to unique detection hardware geometrics. When a single change in screening requirements or threat detection is required, each Original Equipment Manufacturer (OEM) must adapt their proprietary algorithm. Instead of contracting with specific OEM’s on a one-on-one basis, DHS intends to use this Prize Competition to explore the possibility of discovering a comprehensive or set of automated detection algorithms that can improve the speed, detection of prohibited items, and accuracy in both OEM and government developed screening equipment. The goal of the Prize Competition is to reach beyond the traditional screening equipment manufacturers to entities who are exclusively skilled at algorithm development. This provides an opportunity to discover new talent that may otherwise be overlooked by DHS or current technology providers.

This Prize Competition consists of two stages. The first stage will give competitors six months to train and build automated threat detection algorithms using a dataset of hundreds of laboratory study volunteer mock passenger research images in a variety of formats. The second stage will evaluate those models against a holdout set of images for the purpose of validating algorithm performance.

The total cash prize payout for this Competition is up to $1,500,000 (USD).

Overview: The Competition will be administered and run on Kaggle.com. Submissions, in the form of predictions on the dataset, can be made to the Kaggle platform during the advertised competition submission period. The accuracy of those submissions will be evaluated using the multi-class logarithmic loss metric. The top eight ranked teams at the end of the second stage will be eligible to receive prize money, distributed as specified under “Prizes.”

 

Privacy: Personal information provided by entrants (Solvers) on the nomination form through the Prize Competition website will be used to contact selected finalists. Information is not collected for commercial marketing. Winners are permitted to cite that they won this Competition after it has been officially announced by the Department of Homeland Security. The names, cities, and states of selected winner or entity will be made available in promotional materials and at recognition events.

Prizes: The top eight ranked teams at the end of the second stage Competition will be eligible to receive a cash prize, distributed as follows:

1st Place: $500,000

2nd Place: $300,000

3rd Place: $200,000

4th – 8th Place: $100,000 each

Payment of the Prize: Prizes awarded under this Competition will be paid by the Department of Homeland Security, Science and Technology Directorate and must be received by the Solver(s) via electronic funds transfer. All Federal, state, and local taxes are the sole responsibility of the winner(s). DHS Science and Technology and the Transportation Security Administration will comply with the Internal Review Service withholding and reporting requirements, where applicable.

List of Judges and their Organization

William Garrett, DHS Transportation Security Administration

John Fortune, DHS Science and Technology, Explosives Division

Barry Masters, DHS Science and Technology, Transportation Security Laboratory

Judging Criteria

Overview

This Competition is a challenge of skill and the final results are determined solely by leaderboard ranking of the second stage Competition (subject to compliance with the Competition Rules). Participants' scores and ranks on the Competition Website at any given stage of the Competition will be based on the evaluation metric described on the Competition Website, as determined by applying the predictions in the Submission to the ground truth of a validation dataset whose instances were a fixed set sampled from the Data.

The Logarithmic Loss evaluation metric used for scoring and ranking Submissions will be displayed on the Competition Website. (https://www.kaggle.com/wiki/LogarithmicLoss/) Logarithmic Loss takes a confidence score and assigns points based on the confidence level and correctness. This scoring method is purely objective based on algorithm performance against the Data.

Federal judges will use the evaluation metric scores to determine the ranking of winning solutions to this Competition. The submission of an entry to this challenge does not guarantee acceptance, a cash or non-cash award, or further use or consideration of the algorithm by the Department of Homeland Security.

How to Enter

To register for this Competition and obtain additional challenge data material go to the challenge information site:

https://www.kaggle.com/c/passenger-screening-algorithm-challenge

Prizes
Prize Purse $1,500,000.00 Prizes: The top eight ranked teams at the end of the second stage Competition will be eligible to receive a cash prize, distributed as follows: 1st Place: $500,000; 2nd Place: $300,000; 3rd Place: $200,000; and 4th – 8th Place: $100,000 each
Solutions
No solutions have been posted for this challenge yet.
Rules

Submission Requirements: To be eligible to win a prize under this Competition, the Solver shall have registered to participate in the contest under the process and Solver User Agreement identified on the contest’s official entry site which may be accessed at https://www.kaggle.com/c/passenger-screening-algorithm-challenge (hereafter referenced simply as “Competition Website”).

All Solvers will receive an automatically calculated score for their submission on a subset of public leaderboard data. However, final evaluation of a solver’s algorithm performance will be subject to evaluation against a private holdout dataset in the second stage of the Competition.

Registration: To register for this Competition and obtain additional challenge data material go to the challenge information site:

https://www.kaggle.com/c/passenger-screening-algorithm-challenge

Eligibility Requirements: A complete set of rules and eligibility requirements may be found on the contest’s official entry site located at https://www.kaggle.com/c/passenger-screening-algorithm-challenge (hereafter referenced simply as “Competition Rules”).

The Competition is open to all individuals (non-felons) over the age of 18, United States Citizens or legal permanent residents at the time of entry and to all validly formed legal U.S. entities whose primary place of business is in the U.S. that have not declared or been declared in bankruptcy. Eligibility is subject to verification by the Competition Sponsor before Cash Prizes are awarded.

Officers, directors, advisory board members, employees, and support contractors of the Department of Homeland Security, Science and Technology Directorate, the Transportation Security Administration, Kaggle and their respective affiliate, subsidiaries, Kaggle contractors (with the express exception of Kaggle’s authorized Kaggle Community Evangelists), and judges are ineligible to compete in this Competition. Likewise, members of their immediate family (spouses, children, step-children, siblings, step-siblings, parents, step-parents), and persons living in the same household, whether or not related, are not eligible to participate in any portion of this Competition. Note: The members of an individual’s household include any other person who shares the same residence as such individual for at least three months out of the year. See the Challenge.gov (https://www.challenge.gov) public notice for full set of exclusions and eligibility requirements.

You are not eligible to receive any Cash Prize in the Competition if you are a resident designated by the United States Treasury’s Office of Foreign Assets Control (see http://www.treasury.gov/resource-center/sanctions/SDN-List/Pages/default.aspx for additional information).

To be eligible to win a cash prize under this Competition, an individual or entity:

  1. Shall have registered to participate in the Competition under the rules promulgated by the Department of Homeland Security, Science and Technology Directorate, and the Transportation Security Administration (Competition Sponsors) in accordance with the description provided on the Competition Website.
  2. Shall have complied with all of the requirements on the official Competition Website.
  3. Pursuant to the America COMPETES Act of 2010, monetary and non-monetary awards for this Prize Competition may only be given to an individual that is a citizen or legal permanent resident of the United States or an entity that is incorporated in and whose primary place of business is in the United States, subject to verification by the Competition Sponsor before prizes are awarded; an individual or private entity must be the registered entrant to be eligible to win a prize
  4. Entities selected as a Competition Cash Prize Winner must register or be previously registered in the System for Awards Management (http://www.sam.gov) in order to receive a cash prize. Registration in the System for Awards Management is not a prerequisite for submitting an entry to this Prize Competition. Failure to register in the System for Awards Management within 30 days of notification by Kaggle, Inc. will result in a disqualification of the winning entry. An individual or entity that is determined to be on the Excluded Parties List (www.sam.gov) is ineligible to receive a cash prize award and will not be selected as a Competition Cash Prize Winner.
  5. Individuals and entities, otherwise eligible to win a prize, may form and submit a team entry; however, each eligible individual or entity must first register as an individual or entity participant for the contest before they form a team. Failure to follow this procedure as outlined on the contest website will disqualify the contest submission. Team winnings, as determined by the Contest Sponsor, will be equally distributed among team individuals or entities.
  6. Contestants to this Prize Competition must: agree to be bound by the rules of the Prize Competition, agree that the decision of the judges for this Prize Competition are final and binding, and acknowledge that their submission may be the subject of a Freedom of Information Act (FOIA) request and that they are responsible for identifying and marking all business confidential and proprietary information in their submission.
  7. Individuals and Individual Team Members selected as a Competition Cash Prize Winner must submit all required taxpayer identification and bank account information required to complete an electronic payment of the cash prize. Failure to provide the government required documents for electronic payment within 30 days of notification by Kaggle, Inc. will result in a disqualification of the winning entry.
  8. Participants in this Prize Competition agree, as a condition for winning a cash prize, to complete and submit all required winner verification documents to Kaggle, Inc. within 30 days of notification. Failure to return all required verification documents by the date specified in the notification may be a basis for disqualification of the winning entry.
  9. Contestants to this Prize Competition must agree and consent, as a condition for receiving a cash prize, to the use of their name, entity, city and state, likeness or image, comments, and a short synopsis of their winning solution as a part of the Department of Homeland Security’s promotion of this Prize Competition.
  10. By participating in this Competition, contestants must agree to indemnify the Federal Government against third party claims for damages arising from or related to Competition activities and for damage or loss to government property resulting from such activity.
  11. Contestants must own or have access at their own expense to a computer, an Internet connection, and any other electronic devices, documentation, software, or other items that contestants may deem necessary to create and enter a submission.
  12. The following Participants (including any individuals participating as part of a team or entity) are not eligible regardless of whether they meet the criteria set forth above:  Any Federal entity or Federal employee acting within the scope of his or her employment or otherwise be prohibited by Federal law (employees should consult their agency ethics officials). Note: Federal ethical conduct rules may restrict or prohibit federal employees from engaging in certain outside activities; any federal employee not excluded under the prior paragraph seeking to participate in this Competition outside the scope of employment should consult his/her agency’s ethics official prior to developing a submission;; Any individual or entity that used DHS Federal facilities or relied upon significant consultation with DHS Federal employees to develop a submission, unless the facilities and employees were made available to all Participants in this Competition on an equal basis; Any individual or entity that used Federal funds to develop a submission, unless such use is consistent with the grant award, or other applicable Federal funds awarding document. If a grantee using Federal funds enters and wins this Prize Competition, the prize monies must be treated as program income for purposes of the original grant in accordance with applicable Office of Management and Budget Circulars. Federal contractors may not use Federal funds from a contract to develop a submission for this Competition;
  13. Per 15 U.S.C. 3719(h), an individual or entity shall not be deemed ineligible under these eligibility rules because the individual or entity used Federal facilities or consulted with Federal employees during a competition if the facilities and employees are made available to all individuals and entities participating in the Competition on an equitable basis.
  14. Use of Marks: Except as expressly set forth in the Participant Agreement or the Competition rules, participants shall not use the names, trademarks, service marks, logos, insignias, trade dress, or any other designation of source or origin subject to legal protection, copyrighted material or similar intellectual property (“Marks”) of the organizers or other Prize Competition partners, sponsors, or collaborators in any way without such party’s prior written permission in each instance, which such party may grant or withhold at its sole and absolute discretion.
  15. An individual or entity that is determined to be on the GSA Excluded Parties List (www.sam.gov) is ineligible to receive a cash prize award and will not be selected as a Prize Competition winner.

Prize awards are subject to verification of eligibility and compliance with the Competition Rules. All decisions of the Competition Sponsor and judges will be final and binding on all matters relating to this Competition. The Competition Sponsor reserves the right to examine the Submission and any associated code or documentation for compliance with these Competition Rules. In the event that the Submission demonstrates a breach of these Competition Rules, Competition Sponsor may, at its discretion, take either of the following actions:

  • disqualify a Submission(s)
  • require that remediation within one week all issues identified in your Submission(s) (including, without limitation, the resolution of license conflicts, the fulfillment of all obligations required by software licenses, and the removal of any software that violates the software restrictions)

Individuals and entities, identified above, who are ineligible to receive a prize or participate as in this Challenge may register and submit an entry but will be considered ineligible to receive cash or non-cash prize award.  The Contest Sponsor, may at their dissection, provide honorable mention of promising algorithm solutions and contact ineligible participants after the close of the Challenge.

Intellectual Property

Each winner grants to the Competition Sponsor and its designees a worldwide, non-exclusive, sub-licensable, transferable, fully paid-up, royalty-free, perpetual, irrevocable right to use, not use, reproduce, distribute, create derivative works of, publicly perform, publicly display, digitally perform, make, have made, sell, offer for sale and import their winning Submission and the source code used to generate the Submission, in any media now known or hereafter developed, for any purpose whatsoever, commercial or otherwise, without further approval by or payment to Participant and represents that he/she/it has the unrestricted right to grant that license.

Submit Solution
Submissions for this competition are being accepted on a third-party site. Please visit the external site for instructions on submitting: https://www.kaggle.com/c/passenger-screening-algorithm-challenge
Challenge Followers
Public Profile: 1
Private Profile: 4