Nemotron 3 Super (120B A12B): benchmarks, price and rank

Its headline results

The benchmarks that count most in the rating, with its place among the models measured the same way, and where each number came from.

• BrowseComp31.3%

  Questions whose answers are hidden deep on the web and take many searches and pages to track down.

  53rd of 56. The maker's own number, from NVIDIA, via LLM Stats.

• Humanity's Last Exam22.8%

  About 2,500 questions written by experts in more than a hundred fields to be as hard as they could make them, many needing a diagram or image.

  49th of 74. The maker's own number, from NVIDIA, via LLM Stats.

• AA-LCR58.3%

  Questions that need reasoning across several long documents at once, about 100,000 tokens of text in all.

  15th of 19. The maker's own number, from NVIDIA, via LLM Stats.

• SciCode42.0%

  Research coding problems written by scientists across 16 fields, from physics and chemistry to biology and materials science.

  14th of 24. The maker's own number, from NVIDIA, via LLM Stats.

• SWE-bench Verified53.7%

  Real bug reports and feature requests from popular open-source Python projects. The model has to change the code so the project's hidden tests pass.

  51st of 56. The maker's own number, from NVIDIA, via LLM Stats.

• GPQA Diamond82.7%

  Graduate-level questions in biology, physics and chemistry, written by PhDs so that searching the web does not give the answer away.

  The maker's own number, from NVIDIA, via LLM Stats.

• MMLU-Pro83.7%

  Twelve thousand multiple-choice questions across fourteen subjects, from law and medicine to engineering, with ten options each.

  26th of 56. The maker's own number, from NVIDIA, via LLM Stats.