This challenge is no longer accepting new submissions.
Army Signal Classification Challenge
The Army is seeking innovative approaches, leveraging machine learning/deep learning, to conduct blind radio frequency signal analysis.
Department of Defense
Type of Challenge: Scientific
Partner Agencies | Federal: 1
Submission Start: 04/30/2018 12:00 AM ET
Submission End: 08/13/2018 12:00 AM ET
This challenge is externally hosted.
You can view the challenge details here: https://sites.mitre.org/armychallenge
DescriptionRegistration will be conducted on a third-party site: https://sites.mitre.org/armychallenge/ Did you know the Army Rapid Capabilities Office (RCO) is sponsoring a Blind Signal Classification Challenge where the winners will split $150,000? Set to launch on April 30, the challenge is seeking new technologies that apply artificial intelligence and machine learning to signal identification and classification. This effort invites anyone who can bring machine learning algorithms and the supporting processes, methods and tools needed to improve the speed and agility of blind signal identification and classification within the electromagnetic spectrum. The RCO hopes the technology from the challenge will lead to advancements in how Electronic Warfare Officers identify and react to these signals on the battlefield. The Government intent is for solvers to present advanced algorithms and AI implementations with a high degree of classification accuracy and performance (e.g., speed of classification, low CPU resource requirement) that would allow for ease of integration within existing systems. This research goal is to promote innovation and advancement in the area of signal processing. In exchange for cash prizes, the government is seeking a brief technical paper describing the methodology of the specific implementation, to include both the model architecture and training process, including the hyper-parameter optimization process. In order to fully understand the approach, essential source code elements must be provided that show the implementation of the model and training process. Participants’ solutions for each test dataset will be scored independently based on a cross-entropy loss function. Participants will have an aggregate score based on their individual CSV file submissions for each test dataset. Upon registration and approval participants will be given access to a third-party site to obtain access to the training dataset.
Judging CriteriaMulti-Class Logarithmic Loss Metric Scoring Metric Submissions are evaluated using a multi-class logarithmic loss metric, with scores ranging from 0 to 100. Each test instance has been labeled with one true class of 24 possible classes. For each test instance, i, you must submit a set of 24 predicted probabilities, pi,j., where j=1, 2, …, 24. The formula used to compute your score is then,
PrizesSignal Classification Solver Awards
Cash Prize Amount: $150000
Solver with winning algorithms who meet government terms and conditions will receive monetary awards from the government.