follow us on facebook follow us on twitter email us

PerfLoc Prize Competition

About the Challenge
Develop an indoor localization and tracking app using NIST test data!

Posted By: Public Safety Communications Innovation Accelerator
Category: Software/Apps
Skill: Software/Apps Interest: Public Safety Submission Dates: 12 p.m. ET, Mar 22, 2017 - 5 p.m. ET, Jan 17, 2018 Winners Announced: Feb 16, 2018

— NOTE: competition timeline has been extended until January 17, 2018 and up to three winners will now be sent to an international conference in April 2018 (updated August 9, 2017) — 

This competition is about developing the best possible indoor localization and tracking app for Android smartphones. Indoor localization is the capability to determine/estimate the location of an entity to be localized or tracked, such as a person, a robot, or some other object equipped with an appropriate electronic device in buildings and subterranean structures such as tunnels, caves, and underground mines. Localization and tracking, whether indoors or outdoors, has a wide range of applications, including emergency response and law enforcement. Other applications include fleet management, asset tracking in factories/warehouses/hospitals, tracking the elderly/children, personal navigation in museums and shopping malls, social networking, and the Internet of Things (IoT).

PerfLoc Prize Competition participants are required to use the extensive Android smartphone data repository developed by NIST to develop their apps and have their apps’ performance evaluated through the PerfLoc Performance Evaluation Portal.  The Prize Purse for the PerfLoc Prize Competition is a total of $35,000. In addition, up to five finalists will be provided invitational travel funds to travel to the NIST campus in Gaithersburg, MD for live tests of their apps at App Demo Days February 15-16, 2018. Up to three finalists will be selected as winners at the App Demo Days at NIST. Up to three winners will be provided invitational travel funds by NIST to showcase their apps at the 2018 IEEE IPSN Conference in Porto, Portugal, April 11-1, 2018.

To Enter:

  1. Visit the website for the PerfLoc Prize Competition
  2. Create an account at the PerfLoc website (see lower left bar)
  3. Following the Rules and other information on the PerfLoc website, develop and test your App at the PerfLoc Performance Evaluation Portal up to 150 times

Submissions are ONLY accepted at the PerfLoc website.

 

Judging Criteria

Overall Performance on the PerfLoc Performance Evaluation Portal - 80%

Scores will be posted in real-time throughout the competition period. The total maximum score possible is 80 points (see Rules for details).

Performance During App Demo Days at NIST - 20%

Up to five finalists eligible to win a cash prize will be invited to participate in App Demo Days at NIST Gaithersburg campus, February 15-16, 2018. There, the finalists may receive up to 20 additional points based on the real-time localization accuracy and latency of their app in the Demo Days.

Prizes
1st Place $20,000.00
2nd Place $10,000.00
3rd Place $5,000.00

One Discussion for "PerfLoc Prize Competition"

  • hevans
    Want to know more about the PerfLoc Prize Competition? Attend the webinar on Thursday May 18th at 1:00 PM ET to learn about the rules and the technical details! Register here: https://www.nist.gov/news-events/events/2017/05/perfloc-prize-competition-webinar

Add to the Discussion

Solutions
No solutions have been posted for this challenge yet.
Rules

Rules for being eligible to participate in the Competition:

— NOTE: competition timeline has been extended until January 17, 2018 and up to three winners will now be sent to an international conference in April 2018 (updated August 9, 2017) — 

Participation in the PerfLoc Prize Competition is open to ALL; however, not all participants are eligible to win cash prizes as explained in the next section.

Each Competition Participant (individual, team, or legal entity) is required to register on the PerfLoc website and create ONE password-protected user account. There shall be one Official Representative for each Competition Participant. To create a user account, the Official Representative must provide a name, username, password, email address, and geographical location, and affirm that he/she has read and consents to be governed by the Competition Rules. Multiple accounts per Participant are NOT allowed. At NIST’s discretion, any violation of this rule will be grounds for disqualification from the Competition. Multiple individuals and/or legal entities may collaborate as a team to submit a single entry, in which case the designated Official Representative will be responsible for meeting all entry and evaluation requirements. Participation is subject to all U.S. federal, state and local laws and regulations. Individuals entering on behalf of or representing a company, institution or other legal entity are responsible for confirming that their entry does not violate any policies of that company, institution or legal entity.

During the competition testing period, Participants are allowed a total of 150 uses of the PerfLoc Performance Evaluation Portal and a maximum of three uses per day. At NIST’s discretion, any violation of this requirement will be grounds for disqualification from the Competition. Modifications and improvements to a Participant’s App during the competition testing period are to be expected, but the Participant should have one final App ready at the end of the competition testing period.

Rules for being eligible to win a cash prize:

To be eligible for a cash prize:

  • A Participant (whether an individual, team, or legal entity) must have registered to participate and complied with all of the requirements under section 3719 of title 15, United States Code as contained herein.
  • At the time of Entry, the Official Representative (individual or team lead, in the case of a group project) must be age 18 or older and a U.S. citizen or permanent resident of the United States or its territories.
  • In the case of a private entity, the business shall be incorporated in and maintain a primary place of business in the United States or its territories.
  • Participants may not be a Federal entity or Federal employee acting within the scope of their employment. NIST employees and NIST Research Associates as well as direct recipients of NIST funding awards for NIST projects in the development of an Android app in the area of the challenge are not eligible to participate. Non-NIST Federal employees acting in their personal capacities should consult with their respective agency ethics officials to determine whether their participation in this Competition is permissible.
  • Any other individuals or legal entities involved with the design, production, execution, distribution or evaluation of the PerfLoc Prize Competition are also not eligible to participate.
  • A Participant shall not be deemed ineligible because the Participant consulted with Federal employees or used Federal facilities in preparing its submission to the PerfLoc Prize Competition if the Federal employees and facilities are made available to all Participants on an equitable basis.

In addition, interested Participants who do not meet the eligibility requirements to win a prize (i.e., individuals who are neither a US citizen nor a permanent resident of the United States or non-US-based entities) are encouraged to participate in the Competition. They are invited to register on the PerfLoc website, download the data sets, and use the PerfLoc Performance Evaluation Portal according to the PerfLoc Prize Competition Rules. The performance obtained by these Participants will be displayed on the PerfLoc website in the same manner as the performance obtained by Participants who are eligible to win cash prizes.

How to participate in the Competition:

An interested party (individual, team, or legal entity) must initiate the process of participating in the Competition by registering at the PerfLoc website and creating a user account. The party is then allowed to download a number of smartphone data sets collected by NIST from the PerfLoc website. In the event NIST determines modifications to the data sets are needed, all website registrants will be notified by email.

The party becomes a Participant once he/she has built a smartphone indoor localization and tracking Android App (i.e., working software application that operates on a Wi-Fi and GPS enabled Android smartphone with possibly cellular service) AND has had the App evaluated at least once through the PerfLoc Performance Evaluation Portal. An entry consists of the file(s) uploaded to the PerfLoc Performance Evaluation Portal and, in the case of the Finalists, the App demonstrated at NIST (hereinafter referred to as an “Entry”).

Types of PerfLoc Data

The PerfLoc website contains two types of data sets: training data and test data, collected by NIST researchers in four buildings using four different Android smartphones. Details for these two types of data sets are provided below:

  1. Training data: These are similar to solved problems in a textbook. There are a total of four training datasets, one for each of the buildings in which NIST collected data. For each training data set, NIST is providing the ground-truth locations of the test subject that collected the data at a number of time instances during the course of data collection. Participants can compare the location estimates generated by their Apps at the abovementioned time instances with the ground-truth locations to decide how well their Apps are doing. Based on such comparisons, the Participants can modify their Apps and hopefully develop better versions. Participants should not upload location estimates for training data sets, because the ground truth locations for designated time instances in those data sets have already been provided to them.
  2. Test data: There are multiple test data sets, corresponding to various Test & Evaluation Scenarios, for each of the four buildings in which NIST collected data. There are also a number of time instances associated with each test data set that are available for download from the PerfLoc website. Participants are asked to upload location estimates generated by their App for ALL the designated time instances for each test data set. The Participants must provide such location estimates for ALL test data sets.

App Requirements

Horizontal error
Horizontal Error: One of the performance metric specified in ISO/IEC 18305

 

Participants should create an original indoor localization and tracking Android-based App. Note that Finalists invited to the PerfLoc App Demo Days will be required to run their Apps on a NIST-provided Google Pixel XL smartphone running on Android 7.1.2 Nougat OS. The Apps must be able to compute location estimates in real-time based on the data collected by the smartphone in the course of the tests to be administered at that event. Each time requested to generate a location estimate, the App should produce the estimate within five seconds.

PerfLoc Performance Evaluation Portal Scoring

Each App is evaluated over the web instantly when the Participant uploads on the PerfLoc Performance Evaluation Portal the location estimates generated by the App at the designated time instances. The Participant has to upload such location estimates for all test data sets, but not for training data sets. Each upload of location estimates for test data sets constitutes one use of the PerfLoc Performance Evaluation Portal.

The PerfLoc Performance Evaluation Portal will instantly return five values of a single performance metric, namely spherical error 95% (SE95), whose definition is included in the Appendix to this document. One SE95 value is returned for each of the four buildings and an overall SE95 value is returned for all time instances and all test data sets. This is another way for the Participant to know how well his/her App is performing and what steps to take, if any, to make it better.

The following formula will be used for assigning up to 80 points based on the overall SE95 performance (in meters):

score={80(e90.1×SE951)e91;if SE95900;otherwisescore={80(e9−0.1×SE95−1)e9−1;if SE95≤900;otherwise

This function is plotted below and some of its sample values are given in the table to the right.

SE95 Score
0 80
1 72.39
2 65.50
3 59.26
4 53.62
5 48.52
6 43.90
7 39.72
8 35.94
9 32.52
10 29.42
11 26.62
12 24.09
13 21.80
14 19.72
15 17.84
16 16.14
17 14.61
18 13.22
19 11.96
20 10.82

 

All Participant data uploads must be completed by 5:00 PM EDT (UTC-05:00) on August 17, 2017. Uploads to the PerfLoc Performance Evaluation Portal before or after the competition testing period will not be evaluated or considered for award. Entries sent to NIST in any manner other than through the Event Website will not be evaluated or considered for award.

The best performance of an App (i.e., the best score of data generated by the App and uploaded to the portal) will be listed in the publically available rank-ordered table of performances, along with the Participant’s username, name, geographic location and the date and time of the data upload. If a Participant modifies his/her App, but the modification results in a lower score than the highest score obtained by the Participant previously, this will count as one use of the PerfLoc Performance Evaluation Portal, but the lower score will not replace the highest one obtained by the Participant previously.

At the end of the competition testing period, in addition to SE95, many other performance metrics specified in the international standard ISO/IEC 18305 will be computed for the best version of the App developed by each Participant and will be published on the PerfLoc Website.

Judging

The NIST Director will appoint a highly qualified judge. The judge will be a NIST employee with expertise in public safety and advanced communications. The Judge will determine winners according to the Judging Criteria described below. The Judge may not have personal or financial interests in, or be an employee, officer, director, or agent of, any entity that is a registered Participant in the Competition and may not have a familial or financial relationship with an individual who is a registered Participant. In the event of such a conflict, the Judge must recuse himself or herself and a new Judge may be appointed.

Basis on which finalists and winners will be selected:

Finalists: The top 3-5 Participants, as determined by their standing on the PerfLoc Performance Evaluation Portal at the end of the competition testing period, who are also eligible to receive a cash prize and have complied with the Rules of the Competition, will be selected as the Finalists. A minimum of three Finalists will be selected assuming that are at least three Participants who meet the eligibility and compliance requirements stated above. NIST may select up to two more Participants satisfying the abovementioned requirements as Finalists if the overall SE95 performance of their Apps are within 20% of the overall SE95 performance of the top Finalist. This is an opportunity for the fourth and fifth Finalists to break into the top three slots and declared as winners of cash prizes based on the points awarded at the PerfLoc App Demo Days at NIST. Therefore, there is an incentive for the Participants to develop not just an algorithm that yields the best overall SE95 performance, but also an App that runs in real-time and, each time requested, generates a location estimate as fast as possible (e.g.,in less than five seconds).

Upon receipt of an email from NIST informing you of your selection as a Finalist, you must respond within 48 hours to confirm your eligibility and willingness to attend the PerfLoc App Demo Days at NIST in person. If any of the selected Finalists is unable to attend the PerfLoc App Demo Days, NIST will make an attempt to select other Participants as Finalists, time permitting and if warranted. In the unlikely event of tie scores on the Leaderboard, Participants will be rank ordered by the time stamp of their best overall SE95 performance, with the earliest time ranking highest. These Participants (individuals, teams, or legal entities) constitute the Finalists and they (the Official Representative in the case of a team or legal entity) will be invited to attend the PerfLoc App Demo Days on the NIST campus in Gaithersburg, MD. NIST will arrange and pay for the Official Representatives for the Finalists to travel to NIST to take part in the PerfLoc App Demo Days.

At the PerfLoc App Demo Days Event, the finalists will install their Apps on a NIST-provided Google Pixel XL smartphone running on Android 7.1.2 Nougat OS and undergo testing in a NIST building. Each Finalist will also make a 10-minute presentation followed by audience question and answer. NIST reserves the right to run security scans on the installed App to ensure there is no risk to NIST IT systems during the Perfloc App Demo Days Event. At the event’s conclusion, NIST will remove the Participants’ app software from the NIST smartphones.

Winners: The Winners of cash prizes will be determined by a Judge appointed by the NIST Director, using the Judging Criteria outlined here.

Judging Criteria:

  1. Overall Performance on the PerfLoc Performance Evaluation Portal (up to 80 points)
    1. Using the final ranking at the close of the competition testing period, a score of up to 80 points will be calculated using the Scoring Methodology described above.
  2. Performance during App Demo Days at NIST (20 points)
    1. Localization accuracy and latency of the App

If a Finalist at the PerfLoc App Demo Days does not have a functioning App, he/she will be disqualified from winning a prize. Specifically, an App is required to generate a location estimate, each time requested, in real-time (e.g., less than five seconds).

NIST will announce the Finalists via the Event Website as well as those Finalists that have been awarded a cash award (each, an “Award”). The anticipated number and amount of the Awards that will be awarded for this Competition is set forth in these rules; however, NIST is not obligated to make all or any Awards, and reserves the right to award fewer than the anticipated number of Awards in the event an insufficient number of eligible Apps meet any one or more of the Judging Criteria for this Competition, based on NIST’s sole discretion. Using the Judging Criteria, a highly-qualified Judge (NIST employee appointed by the NIST Director) will determine the winners of the Perfloc Prize Competition.

The winner verification process for being eligible to receive an Award includes providing the full legal name, tax identification number or social security number, routing number and banking account to which the prize money can be deposited directly. Return of any notification as “undeliverable” will result in disqualification. After verification of eligibility, Awards will be distributed in the form of electronic funds transfer addressed to the Official Representatives specified in the winning Entries. That Official Representative will have sole responsibility for further distribution of any Award among Participants in a group Entry or within a company or institution that has submitted an Entry through that representative. The list of Entries receiving Awards for the Competition, including the names of all members of a team, will be made public according to the timeline outlined on the Event Website.

Winners are responsible for all taxes and reporting related to any Award received as part of the Competition.

All costs incurred in the preparation of Competition Entries are to be borne by Participants.

Demonstration of winning Apps at 2018 Microsoft Indoor Localization Competition:

 

Up to three Winners selected at the PerfLoc App Demo Days at NIST may be invited by NIST to attend 2018 Microsoft Indoor Localization Competition, tentatively to be held at 2018 IEEE IPSN, Porto, Portugal, April 11-13, 2018. The Winners will be required to go through another round of demos and evaluations, even though these evaluations will not affect their status as Winners. The purpose of these demos and evaluations is for the conference participants, who do not have the opportunity to attend PerfLoc App Demo Days at NIST, to see how well the Winning Apps perform. The Winners may also make a 10-minute presentation on their Apps followed by audience question and answer.

NIST will arrange and pay for the travel of the Official Representatives of the invited Winners to the conference to cover eligible travel costs such as airfare, conference registration fee, and lodging.

The test and presentation event is also an opportunity for Participants who are not eligible to win cash prizes to showcase their Apps. NIST may invite up to three such Apps to be showcased (demo and presentation)  in place of one or more Winning Entries (cash prize recipients), if their overall SE95 performance is better than one or more of the Winning Entries.

Point of Contact for the Competition:

Questions about the PerfLoc Prize Competition can be directed to Nader Moayeri, moayeri@nist.gov.

— TERMS AND CONDITIONS —

Intellectual property rights

Other than as set forth herein, NIST does not make any claim to ownership of your Entry or any of your intellectual property or third party intellectual property that it may contain. By participating in the PerfLoc Prize Competition, you are not granting any rights in any patents or pending patent applications related to your Entry; provided that by submitting an entry (i.e., participating in the Competition and uploading data to the PerfLoc Performance Evaluation Portal), you are granting NIST certain limited rights as set forth herein.

By submitting an Entry, you grant to NIST the right to review and score your Entry as described in the section “PerfLoc Performance Evaluation Portal Scoring”, to describe your Entry in connection with any materials created in connection with the Competition including making public your username, name, highest score, and the date/time that score was obtained at the Event Website, and to have the Judge, Competition administrators, and the designees of any of them, review your Entry.

By submitting an Entry in the PerfLoc Prize Competition, you grant a non-exclusive, irrevocable, paid up right and license to NIST to use your name, likeness, biographical information, image, any other personal data submitted with your Entry and the contents in your Entry, in connection with the Perfloc Prize Competition for any purpose, including promotion and advertisement of the competition and future challenges.

You agree that nothing in these Rules grants you a right or license to use any names or logos of NIST or the Department of Commerce, or any other intellectual property or proprietary rights of NIST or the Department of Commerce or their employees or contractors. You grant to NIST the right to include your name and your company or institution name and logo (if your Entry is from a company or institution) as a Participant on the Event Website and in materials from NIST announcing Winners, Finalists, or Participants in the Competition. Other than these uses or as otherwise set forth herein, you are not granting NIST any rights to your trademarks.

Entries containing any matter which, in the sole discretion of NIST, is indecent, defamatory, in obvious bad taste, which demonstrates a lack of respect for public morals or conduct, which promotes discrimination in any form, which shows unlawful acts being performed, which is slanderous or libelous, or which adversely affects the reputation of NIST, will not be accepted. If NIST, in its sole discretion, finds any Entry to be unacceptable, then such Entry shall be deemed disqualified and will not be evaluated or considered for award. NIST shall have the right to remove any content from the Event Website in its sole discretion at any time and for any reason, including, but not limited to, any online comment or posting related to the Competition.

To advance the goals of NIST’s public safety communications research and other relevant technical programs, NIST may negotiate a license for the use of intellectual property developed by a registered participant in the PerfLoc Prize Competition.

Confidential Information

By making a submission to the PerfLoc Prize Competition, you agree that no part of your submission includes any confidential or proprietary information, ideas or products, including but not limited to information, ideas or products within the scope of the Trade Secrets Act, 18 USC §1905. Because NIST will not receive or hold any submitted materials “in confidence,” it is agreed that, with respect to your Entry, no confidential or fiduciary relationship or obligation of secrecy is established between NIST and you, your Entry team, the company or institution you represent when submitting an Entry, or any other person or entity associated with any part of your Entry.

Warranties

By participating in the PerfLoc Prize Competition, you represent and warrant that all information you submit is true and complete to the best of your knowledge, that you have the right and authority to submit the data generated by your App on your own behalf or on behalf of the persons and entities that you specify, and that your data uploaded to the PerfLoc Performance Evaluation Portal (both the information submitted in the upload and the underlying technologies or concepts in the App):

(a)   is your own original work, or is submitted by permission with full and proper credit given within your Entry;

(b)   does not contain confidential information or trade secrets (yours or anyone else’s);

(c)   does not knowingly violate or infringe upon the patent rights, industrial design rights, copyrights, trademarks, rights in technical data, rights of privacy, publicity or other intellectual property or other rights of any person or entity;

(d)   does not contain malicious code, such as viruses, malware, timebombs, cancelbots, worms, Trojan horses or other potentially harmful programs or other material or information;

(e)   does not and will not violate any applicable law, statute, ordinance, rule or regulation, including, without limitation, United States export laws and regulations, including, but not limited to, the International Traffic in Arms Regulations and the Department of Commerce Export Regulations; and

(f)   does not trigger any reporting or royalty or other obligation to any third party.

Limitation of Liability

By participating in the PerfLoc Prize Competition, you agree to assume any and all risks and to release, indemnify and hold harmless NIST from and against any injuries, losses, damages, claims, actions and any liability of any kind (including attorneys’ fees) resulting from or arising out of your participation in, association with or submission to the PerfLoc Prize Competition (including any claims alleging that your Entry infringes, misappropriates or violates any third party’s intellectual property rights). In addition, you agree to waive claims against the Federal Government and its related entities, except in the case of willful misconduct, for any injury, death, damage, or loss of property, revenue, or profits, whether direct, indirect, or consequential, arising from your participation in the PerfLoc Prize Competition, whether the injury, death, damage, or loss arises through negligence or otherwise.

NIST is not responsible for any miscommunications such as technical failures related to computer, telephone, cable, and unavailable network or server connections, related technical failures, or other failures related to hardware, software or virus, or incomplete or late Entries. Any compromise to the fair and proper conduct of the PerfLoc Prize Competition may result in the disqualification of an Entry or Participant, termination of the PerfLoc Prize Competition, or other remedial action, at the sole discretion of NIST. NIST reserves the right in its sole discretion to extend or modify the dates of the PerfLoc Prize Competition, to modify the test and training data provided at the event website, and to change the terms set forth herein governing any phases taking place after the effective date of any such change. By entering, you agree to the terms set forth herein and to all decisions of NIST and/or all of their respective agents, which are final and binding in all respects.

NIST is not responsible for: (1) Any incorrect or inaccurate information, whether caused by a Participant, printing errors, or by any of the equipment or programming associated with or used in the PerfLoc Prize Competition; (2) unauthorized human intervention in any part of the Entry Process for the PerfLoc Prize Competition; (3) technical or human error that may occur in the administration of the PerfLoc Prize Competition or the processing of Entries; or (4) any injury or damage to persons or property that may be caused, directly or indirectly, in whole or in part, from a Participant’s participation in the PerfLoc Prize Competition or receipt or use or misuse of an Award. If for any reason an Entry is confirmed to have been deleted erroneously, lost, or otherwise destroyed or corrupted, the Participant’s sole remedy is to submit another Entry in the PerfLoc Prize Competition.

Termination and Disqualification

NIST reserves the authority to cancel, suspend, and/or modify the PerfLoc Prize Competition, or any part of it, if any fraud, technical failures, or any other factor beyond NIST’s reasonable control impairs the integrity or proper functioning of the PerfLoc Prize Competition, as determined by NIST in its sole discretion.

NIST reserves the right to disqualify any Participant or Participant team it believes to be tampering with the Entry process or the operation of the PerfLoc Prize Competition or to be acting in violation of any applicable rule or condition.

Any attempt by any person to undermine the legitimate operation of the PerfLoc Prize Competition may be a violation of criminal and civil law, and, should such an attempt be made, NIST reserves the authority to seek damages from any such person to the fullest extent permitted by law.

Verification of Potential Winner(s)

All potential winners are subject to verification by NIST, whose decisions are final and binding in all matters related to the PerfLoc Prize Competition.

Potential winner(s) must continue to comply with all terms and conditions of the PerfLoc Prize Competition Rules described herein, and winning is contingent upon fulfilling all requirements. In the event that a potential winner, or an announced winner, is found to be ineligible or is disqualified for any reason, NIST may make an award, instead, to another Participant.

Privacy and Disclosure under FOIA

Except as provided herein, information submitted throughout the PerfLoc Prize Competition will be used only to communicate with Participants regarding Entries and/or the PerfLoc Prize Competition. Participant Entries and submissions to this competition may be subject to disclosure under the Freedom of Information Act (“FOIA”).

Authority

 

15 U.S.C. 3719.

Appendix: SE95 Definition

Introduction

The performance analysis boils down to comparing the location estimates generated by an App under test at a finite number N of test points with the corresponding ground truth coordinates of those points. For i=1,2,…,N, the following terminology is introduced for “test point” i :

ground truth coordinates: (xi,yi,zi)(xi,yi,zi)

location estimate generated by the App: (^xi,^yi,^zi)(x^i,y^i,z^i)

3D error vector: εi=(εx,i,εy,i,εz,i)=(^xixi,^yiyi,^zizi)εi_=(εx,i,εy,i,εz,i)=(x^i−xi,y^i−yi,z^i−zi)

l2-norm or magnitude of εi: εi=ε2x,i+ε2y,i+ε2z,i‖εi_‖=εx,i2+εy,i2+εz,i2

It is assumed that the z-axis corresponds to the vertical direction.

Spherical Error 95% (SE95)

SE95 is defined as the radius of the smallest sphere centred at the origin of the 3D Cartesian coordinate system that encloses 95% of εi‘s. Specifically, SE95=min{R:R0,{εi:i=1,2,...,N,εiR}0.95N}SE95=min{R:R≥0,∣{εi_:i=1,2,…,N,‖εi_‖≤R}∣≥0.95N}

Once again, it is a good idea to plot the empirical CDF of the magnitude of 3D error. Once this empirical CDF has been plotted, then SE95 is simply the inverse values of this function at 0.95.

Submit Solution
Submissions for this competition are being accepted on a third-party site. Please visit the external site for instructions on submitting: https://perfloc.nist.gov/perfloc-competition-rule.php
Challenge Followers
Public Profile: 0
Private Profile: 6