Seed 2.1 Pro: benchmarks, price and rank

Its headline results

The benchmarks that count most in the rating, with its place among the models measured the same way, and where each number came from.

• APEX-Agents33.8%

  Long tasks written by investment banking analysts, management consultants and corporate lawyers from their own work, done across files and apps in a realistic work setup.

  The maker's own number, from ByteDance, via LLM Stats.

• BrowseComp86.2%

  Questions whose answers are hidden deep on the web and take many searches and pages to track down.

  9th of 56. The maker's own number, from ByteDance, via LLM Stats.

• MCP Atlas83.8%

  Multi-step tasks where the model has to pick and combine many real tools, connected through MCP, the standard way agents plug into apps and data.

  4th of 33. The maker's own number, from ByteDance, via LLM Stats.

• SWE-Bench Pro57.5%

  Long changes to real codebases in several languages, often across many files, including code from private and copyleft projects that models are unlikely to have seen.

  27th of 50. The maker's own number, from ByteDance, via LLM Stats.

• Humanity's Last Exam55.7%

  About 2,500 questions written by experts in more than a hundred fields to be as hard as they could make them, many needing a diagram or image.

  11th of 74. The maker's own number, from ByteDance, via LLM Stats.

• Toolathlon50.6%

  Long tasks across many real apps and their tools, such as email, calendars, spreadsheets, code repositories and online stores.

  18th of 37. The maker's own number, from ByteDance, via LLM Stats.

• MMMU-Pro82.7%

  College exam questions that need a picture to answer, such as charts, diagrams, maps and chemical structures, with more answer options than the original MMMU and some questions shown only as a screenshot.

  5th of 51. The maker's own number, from ByteDance, via LLM Stats.

• CharXiv Reasoning86.4%

  Questions about real charts from scientific papers that need reasoning across the chart, not just reading off a value.

  14th of 45. The maker's own number, from ByteDance, via LLM Stats.

• SciCode59.8%

  Research coding problems written by scientists across 16 fields, from physics and chemistry to biology and materials science.

  2nd of 24. The maker's own number, from ByteDance, via LLM Stats.

• CyberGym70.2%

  Security tasks on real software: finding vulnerabilities and showing they can be triggered.

  12th of 15. The maker's own number, from ByteDance, via LLM Stats.