source: simon willison: kimi k3, and what we can still learn from the pelican benchmark
level: technical
chinese ai lab moonshot ai released kimi k3, a 2.8 trillion parameter model they call the first open 3t-class model. it beats claude opus 4.8 max and gpt-5.5 high on self-reported benchmarks but trails claude fable 5 and gpt-5.6 sol. on artificial analysis's private long-horizon knowledge work evaluation, k3 scored an elo of 1547, a 732-point jump from k2.6, behind only claude fable 5. the model also leads arena.ai's frontend code arena.
pricing is $3 per million input tokens and $15 per million output tokens, matching anthropic's claude sonnet series and making it the most expensive chinese ai lab model yet. this is a steep rise from k2.6's $0.95/$4. token usage dropped 21% in output tokens compared to k2.6. the model uses heavy reasoning: generating an svg of a pelican on a bicycle took 13,241 reasoning tokens and cost 25 cents. a simple 'hi' prompt consumed 86 tokens, hinting at a hidden system prompt.
the pelican test, a simple svg generation task, remains a quick way to try new models. it confirms basic spatial and geometry skills, reveals reasoning effort and cost, and checks vision capabilities. k3's pelican improved over k2.5, and its vision-generated alt text was accurate. the test is not a serious benchmark but forces hands-on use and highlights model traits like token consumption and pricing.
why it matters: kimi k3 shows chinese labs are competing at the top tier with high-cost, high-parameter models, affecting choices for cost-sensitive ai tasks.
source: simon willison: kimi k3, and what we can still learn from the pelican benchmark