Muse Glimmer-30B: benchmarks, price and rank

Its headline results

The benchmarks that count most in the rating, with its place among the models measured the same way, and where each number came from.

• MCP Atlas75.5%

  Multi-step tasks where the model has to pick and combine many real tools, connected through MCP, the standard way agents plug into apps and data.

  12th of 33. The maker's own number, from Meta, via LLM Stats.

• SWE-Bench Pro51.2%

  Long changes to real codebases in several languages, often across many files, including code from private and copyleft projects that models are unlikely to have seen.

  46th of 50. The maker's own number, from Meta, via LLM Stats.

• Humanity's Last Exam22%

  About 2,500 questions written by experts in more than a hundred fields to be as hard as they could make them, many needing a diagram or image.

  50th of 74. The maker's own number, from Meta, via LLM Stats.

• OSWorld-Verified65.9%

  Ordinary tasks on a real desktop computer using normal apps: a browser, a spreadsheet, an email client, a photo editor, the file manager.

  18th of 25. The maker's own number, from Meta, via LLM Stats.

• MMMU-Pro74%

  College exam questions that need a picture to answer, such as charts, diagrams, maps and chemical structures, with more answer options than the original MMMU and some questions shown only as a screenshot.

  34th of 51. The maker's own number, from Meta, via LLM Stats.

• CharXiv Reasoning78.8%

  Questions about real charts from scientific papers that need reasoning across the chart, not just reading off a value.

  29th of 45. The maker's own number, from Meta, via LLM Stats.

• ScreenSpot-Pro75.4%

  Screenshots of professional software at full resolution, with an instruction to find the exact button or field to use.

  7th of 18. The maker's own number, from Meta, via LLM Stats.

• AA-LCR80%

  Questions that need reasoning across several long documents at once, about 100,000 tokens of text in all.

  1st of 19. The maker's own number, from Meta, via LLM Stats.

• SciCode43.6%

  Research coding problems written by scientists across 16 fields, from physics and chemistry to biology and materials science.

  13th of 24. The maker's own number, from Meta, via LLM Stats.

• DeepSearchQA74.6%

  Questions that need many searches and several steps of reasoning across sources to answer.

  8th of 8. The maker's own number, from Meta, via LLM Stats.