Nemotron 3.5 Super VL (120B A12B): benchmarks, price and rank

Its headline results

The benchmarks that count most in the rating, with its place among the models measured the same way, and where each number came from.

• BrowseComp78%

  Questions whose answers are hidden deep on the web and take many searches and pages to track down.

  23rd of 56. The maker's own number, from NVIDIA, via LLM Stats.

• SWE-Bench Pro51.8%

  Long changes to real codebases in several languages, often across many files, including code from private and copyleft projects that models are unlikely to have seen.

  45th of 50. The maker's own number, from NVIDIA, via LLM Stats.

• Humanity's Last Exam31.8%

  About 2,500 questions written by experts in more than a hundred fields to be as hard as they could make them, many needing a diagram or image.

  38th of 74. The maker's own number, from NVIDIA, via LLM Stats.

• OSWorld-Verified51.1%

  Ordinary tasks on a real desktop computer using normal apps: a browser, a spreadsheet, an email client, a photo editor, the file manager.

  24th of 25. The maker's own number, from NVIDIA, via LLM Stats.

• LiveCodeBench v688.2%

  Programming contest problems collected after models were trained, from LeetCode, AtCoder and Codeforces.

  8th of 43. The maker's own number, from NVIDIA, via LLM Stats.

• CharXiv Reasoning80.8%

  Questions about real charts from scientific papers that need reasoning across the chart, not just reading off a value.

  26th of 45. The maker's own number, from NVIDIA, via LLM Stats.

• AA-LCR68.6%

  Questions that need reasoning across several long documents at once, about 100,000 tokens of text in all.

  6th of 19. The maker's own number, from NVIDIA, via LLM Stats.

• SciCode47.1%

  Research coding problems written by scientists across 16 fields, from physics and chemistry to biology and materials science.

  10th of 24. The maker's own number, from NVIDIA, via LLM Stats.

• SWE-bench Verified71.1%

  Real bug reports and feature requests from popular open-source Python projects. The model has to change the code so the project's hidden tests pass.

  31st of 56. The maker's own number, from NVIDIA, via LLM Stats.

• GPQA Diamond89.5%

  Graduate-level questions in biology, physics and chemistry, written by PhDs so that searching the web does not give the answer away.

  The maker's own number, from NVIDIA, via LLM Stats.