Skip to main content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

AI for Critical Mineral Assessment Competition graphic
AI for Critical Mineral Assessment Competition

AI for Critical Mineral Assessment Competition

Defense Advanced Research Projects Agency and United States Geological Survey

Help accelerate assessments of U.S. critical mineral resources through automation

Prepared by: The MITRE Corporation and the Jet Propulsion Laboratory – Approved for Public Release; Distribution Unlimited. Public Release Case Number 23-1117. A portion of this document was written at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration (80NM0018D0004). This research was developed with funding from the Defense Advanced Research Projects Agency (DARPA). The views, opinions and/or findings expressed are those of the author and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S. Government.

Summary

Critical minerals are essential to the U.S. economy and national security; however, their supply is vulnerable to disruption. U.S. production and refining of critical minerals has been declining for decades, while production has become more concentrated in fewer countries. The Energy Act of 2020 defines a “critical mineral” as a non-fuel mineral or mineral material essential to the economic or national security of the U.S. that can be vulnerable to a supply chain disruption. USGS also characterizes critical minerals as serving an essential function in the manufacturing of a product, the absence of which would have significant consequences for the economy or national security. Given the urgency to increase and better secure critical mineral supply, DARPA partnered with the U.S. Geological Survey (USGS) to launch the Artificial Intelligence (AI) for Critical Mineral Assessments (CMAs) Competition in August 2022. To perform CMAs, the USGS draws from over a century of accumulated data, contained mostly within geologic maps and reports, which provide the fundamental basis for these resource assessments. Extracting useful and accurate information from these maps is a time-consuming and laborious manual process. A typical assessment for one critical mineral takes approximately two years to prepare. That is because the USGS map catalog consists of over 100,000 geologic maps; only roughly 10% of those maps are available as georeferenced images and approximately half of those are fully digitized vector files needed for analysis. Everything else – 90% of the data – is comprised of scanned images of paper maps.

The aim of the competition was to crowdsource ideas that could drastically reduce the time required to complete parts of the assessment using AI and machine learning (ML) to automate key processes. The team identified the following goals:

  • Increased Throughput of Quantitative Mineral Assessments: USGS will conduct high-quality, quantitative assessments in less time to increase throughput and help the agency meet increasing demand of CMAs.
  • Application of Best Science for Optimal Customer Outcomes: USGS will integrate the best data, methodologies, and technologies to deliver the most accurate assessments.
  • Framework for Future AI/ML Integration: USGS will deploy a custom framework for integrating AI/ML to bring benefits such as enhanced quality and continuous improvements. A general version of the framework can guide other agencies with similar challenges.
  • Timely and Accurate Insights for DOD Supply Chain Decisions: DOD and other national security partners will have timely and accurate quantitative mineral resource assessments to inform supply chain decisions.

Results

After analyzing the mineral assessment workflow, DARPA and its partners, MITRE and NASA Jet Propulsion Laboratory (JPL), recognized the greatest potential for near-term, high impact in solving the data needs associated with georeferencing and extraction of individual geologic features found on USGS maps. As such, DARPA divided the competition into two distinct challenges: 1) Map Georeferencing and 2) Map Feature Extraction.

  • A total of eighteen teams from industry, academia, and even a high school junior, competed for cash prizes of $10,000 for first place, $3,000 for second, and $1,000 for third.
  • For the Map Georeferencing challenge, DARPA tasked participants to find a map within a given scanned image and georeference it by aligning reference points to base maps, such as grid lines, topography, administrative boundaries, roads, or towns. A Canadian company, Uncharted, received top prize for their simple, clean, and organized solution.
  • For the Map Feature Extraction challenge, participants were asked to extract features identified in an image’s map legend. Students and faculty from the University of Southern California Information Sciences Institute and University of Minnesota joined forces, earning first place for their exceptional solution to extract line features as well as polygons and points.
  • USGS experts plan to integrate the best elements of the submissions into a workable solution for mineral assessment workflows, and potentially for other mission area assessments within the agency.
  • The competition was successful and received praises from USGS and DARPA leadership: “The competition has been a valuable opportunity for the USGS to work with leading minds in AI to improve our approach to CMAs,” said David Applegate, USGS Director. “Critical minerals are essential to the national security supply chain, and as such, the agency is approaching the need from multiple angles,” said DARPA Director Stefanie Tompkins

Areas of Excellence

The areas of excellence provide insight into the shaping of the competition through the collaboration of four agencies: DARPA, USGS, JPL and MITRE.

#1. Define the Problem to Be Solved

The DARPA-USGS-MITRE-JPL team quickly explored the current state-of-the-art of CMAs and was able to identify automation opportunities in a timely manner through a methodical approach to understanding CMAs and USGS culture, work, and hurdles. Among the candidates for automation, the team was quick to establish a priority list with low-hanging fruit that could be tackled early, while also being impactful to the USGS partner. There were several key elements to the team’s success in establishing a solvable problem:

  • Presence of the right USGS experts, such as the geoscientists who perform and teach the assessment process, who helped select some of these opportunities over others;
  • The use of collaboration tools such as Zoom or Mural that allowed each team member to provide valuable input while looking at the full workflow for CMA
  • The utilization of AI/ML to aid the process. The team used AI/ML techniques to prepare, improve and/or make machine-readable data to distribute to the candidates. Participants subsequently dove into automation on two separate topics, map georeferencing and feature extraction.
  • Contemplation of what a to-be CMA workflow could look like

The team was then able to adapt these complex problems to a challenge-compatible format through documentation, effective in-person red teaming, thorough datasets preparation, and quick reaction in addressing participants’ hurdles discovered during the duration of the competition.

#2. Build a Team

The strength of this DARPA competition was the strong, unified team behind the organization of the competition. The different partner organizations were able to bring in the right experts that helped accelerate understanding the hurdles of CMAs and identifying the right automation opportunities. This smooth coordination was enabled by a combination of in-person meetings, retrospective exercises, virtual synchronization meetings, and use of collaborative tools.

  • In-person meetings: in-person working meetings allowed the team to make significant progress in a short period of time. The team was also brought closer together by the organization of external-to-work activities, such as a field trips and an outing to a baseball game, which fostered casual conversations, helped the team get to know one another on a deeper lever, and allowed each member to understand individual motivations. These meetings helped to better define everyone’s specific expertise and accelerated assigning the various tasks to the appropriate people.
  • Retrospective exercises: the team conducted retrospective exercises, one mid-competition, and another after the award ceremony. These exercises led to genuine self-reflection by the team as a whole and what they thought made the work successful, how it could improve, and its vision for the future.
  • Collaborative tools: the use of collaborative tools such as Mural and Slack were essential in allowing the team to gather ideas and concerns from every team member and share decisions and news in a coordinated and goal-focused way. This enabled continuous updates, and quick reactions to urgent, unexpected needs (e.g., the need to label more training data).
  • Synchronization meetings: these meetings, whose cadence was adapted to the needs of the competition, were key in enabling collaborative decision making and situational awareness. From weekly meetings to launch the competition, the team moved to daily meetings to quickly tackle participants’ questions and hurdles. At the conclusion of the competition, meetings were scaled back to bi-weekly, which was enough to meet the demands of the transition effort.

The team achieved an efficient and repeatable collaboration process and willingly shared responsibility outside their expertise. For example, when the team recognized an urgent need to quickly label more training data for the participants, the majority of the team volunteered to get the labeling accomplished in a rapid and efficient manner.

Challenge Type: Ideas

This open-ended ideation challenge supported exploring innovative implementations as solutions to the USGS CMA problems of map georeferencing and map feature extraction. By using the competition model, the team was able to source broadly for input and ideas which benefit from a collective effort of multiple disciplines.

In the Map Georeferencing challenge, many of the submitted solutions used Optical Character Recognition (OCR)-based approaches and the OpenCV library to detect features on the maps that could help to geolocate them. Several of the teams focused on corner detection as a primary clue, and the winning solution also attempted to detect which coordinate system the map was in – a step that may have given them a slight edge in the scoring. Several of the teams also trained machine learning models on various features of the maps, although models trained on topographical contour lines did not perform as well as one might have expected.

In the Map Feature Extraction challenge, most teams developed separate approaches for points, lines and polygons; some teams only focused on one or two of these. Nearly all teams developed a pipeline for pre-processing the map images, including resizing them for easier processing and cropping them to reduce false positive pixels. The first and second place teams developed and trained machine learning models, while the third-place team relied on more traditional computer vision techniques such as pixel color matching. Interestingly, each team performed best on one of the three feature types, and the final rankings came down to how many of each feature type each team attempted.

Legal Authority

DARPA launched this challenge using America COMPETES Act Prize Authority.

Challenge Website & Link

Home|AI for Critical Mineral Assessment Competition (darpa.mil)

DARPA Forward Event