GPT-5.6 Sol vs Kimi K3 vs Claude Opus 5: an early coding comparison

Three serious coding models landed in fifteen days.

OpenAI released GPT-5.6 Sol on July 9. Moonshot followed with Kimi K3 on July 16. Anthropic shipped Claude Opus 5 on July 24. The leaderboard changed after every launch, and the simple question — which model is best for coding? — already has three defensible answers.

One caveat matters before going any further: Opus 5 has been publicly available for two days at the time of writing. Its benchmark results are strong, but much of the readily available qualitative evidence comes from Anthropic’s early-access partners. We don’t yet have weeks of independent use, failure reports, or production post-mortems. Treat this as an early comparison, not a settled verdict.

On the evidence available today, Sol has the strongest case for terminal-heavy coding, Kimi recorded the lowest API cost per task in the current Artificial Analysis coding suite, and Opus 5 has made the best first impression on difficult codebase work. Kimi is also the only one with an announced open-weight path, if Moonshot publishes the weights and licence as promised.

The three models at a glance

The specifications come from the OpenAI model documentation, Kimi K3 launch and API guide, and Anthropic’s model overview. Kimi’s output limit shares the 1,048,576-token context with the input; 131,072 is the default completion budget, not a separate window.

A million-token context window is now a checkbox. All three models advertise roughly the same capacity and support image input. The number says nothing about whether a model can find the relevant code, preserve its reasoning through a long tool loop, and verify the result without being told every step.

The benchmark answer depends on which benchmark you open

Artificial Analysis currently scores Opus 5 at 61 on its broad Intelligence Index, narrowly ahead of Sol at 59 and Kimi K3 at 57. Artificial Analysis ran that Opus evaluation with Opus 4.8 fallback enabled. Its newer Coding Agent Index makes the practical result even closer:

A 0.17-point coding lead isn’t a meaningful Opus victory. Sol produced almost the same score in less than half the active agent time, while Kimi gave up roughly five points and cut the observed API cost by more than half.

The treatment still isn’t just the model. Artificial Analysis ran Opus in Claude Code, Sol in Codex, and K3 in Kimi Code. Its index combines 321 software-engineering tasks across DeepSWE, Terminal-Bench, and repository-understanding questions, with three attempts per task. Changing the harness can change the result even when the model stays fixed. These are evaluations of model-plus-agent systems, which is useful for buyers but not a clean measure of isolated model intelligence.

The vendor benchmarks are messier. Frontier-Bench puts Opus at 43.5% and Sol at 34.4%, but Anthropic’s Opus result was an internal run using mini-SWE-agent, averaged five attempts per task, and allowed Opus 4.8 to handle safety-classifier fallbacks. Sol ran with Codex. Kimi isn’t on that public leaderboard yet. Moonshot’s own launch table mixes Kimi Code, Claude Code, Codex, different hardware, and third-party results. Moonshot documents those differences unusually well, but they still prevent a clean row-by-row comparison.

Even the shared context specification needs testing. On Artificial Analysis’s long-context reasoning benchmark, Kimi scored 74.7%, Sol 73.7%, and Opus 70.0%. Its documents range from 10K to 100K tokens, so the benchmark provides no evidence about reasoning quality at 500K or one million.

The evidence supports narrower claims:

• All three are frontier coding models.
• Opus 5 has the best opening result on Frontier-Bench.
• Sol and Opus are effectively tied on current coding-agent and terminal evaluations.
• Kimi is behind that pair on agentic coding, but materially cheaper.
• No public evaluation yet tells us which one creates the fewest bad pull requests over a month.

That last measure is the one teams need.

GPT-5.6 Sol: the best-supported all-round agent today

Sol’s advantage isn’t a single benchmark. It’s the combination of a capable model, Codex, and two weeks of public evidence that the pair can sustain real terminal work.

OpenAI’s launch evaluation reports 88.8% on Terminal-Bench 2.1 and 72.7% on DeepSWE. Artificial Analysis’s independent coding-agent suite puts Sol just 0.17 points behind Opus while completing the average task in 10.2 minutes rather than 23.6. These results don’t guarantee anything about your repository, but they suggest Sol is very good at navigating a shell, editing across files, and finishing long engineering tasks.

The more interesting feature is Programmatic Tool Calling. Sol can write small programs inside the Responses API to coordinate tools and filter their output. Instead of feeding every line from a test run back through the model, the program can retain only the failing tests or relevant log entries. That reduces round trips and keeps tool noise out of the working context.

OpenAI also exposes standard and pro reasoning modes, effort levels up to max, and a multi-agent beta. These knobs make Sol unusually configurable, but they make cost comparisons harder. A $30-per-million output rate says little if pro mode or a four-agent run consumes several times the tokens.

My default pick would be Sol if you already use Codex. Its native integration is established, its terminal results are strong, and it has had enough public runtime for obvious launch problems to start surfacing. Its weakness is price: output costs twice as much as Kimi K3, and requests above 272K input tokens are billed at $10 input and $45 output per million tokens for the whole request. Opus and Kimi keep their standard rates through the advertised context window.

Kimi K3: the cheapest API with unusual edges

Kimi K3 is the technically interesting release. It has 2.8 trillion total parameters, activates 16 of 896 experts, accepts text, images, and video, and is scheduled to publish its full weights on July 27. Moonshot recommends at least 64 accelerators for deployment, so “open” doesn’t mean “running on the workstation under your desk.”

At $3 per million uncached input tokens and $15 per million output tokens, K3 costs half as much as Sol on output and 40% less than Opus. Automatic cache hits reduce input to $0.30 per million tokens. Moonshot claims its API sees a cache-hit rate above 90% on coding workloads, although that figure comes from Moonshot rather than an independent audit.

Moonshot has built Kimi’s clearest differentiation around work that combines code with visual feedback. Its native multimodal model can inspect a running frontend, a game, or a CAD render and revise the code from what it sees. Moonshot’s launch examples include GPU kernel optimisation, compiler development, chip design, and long-running scientific implementations. They are impressive demonstrations, but they are also curated vendor examples. They show the shape of the capability, not its average success rate.

The most useful part of Moonshot’s announcement is its limitations section. K3 can become unstable if a harness doesn’t preserve its thinking history, or if a session switches to K3 halfway through. Moonshot recommends Kimi Code for that reason. It also warns that the model can be excessively proactive, making decisions on the user’s behalf when requirements are ambiguous.

That is a real concern for coding agents. “Kept going without asking” sounds productive until the model chooses a migration strategy, changes an API contract, or burns an hour implementing the wrong interpretation. Moonshot recommends explicit boundaries in the system prompt or AGENTS.md.

K3 is worth testing first for high-volume implementation, visual development, and teams that want eventual control over the weights. It is not automatically the cheapest model to operate privately, and its harness requirements make dropping it into an existing agent less straightforward than its OpenAI-compatible API suggests.

Claude Opus 5: the strongest launch, with the thinnest evidence

Opus 5’s first numbers are excellent. It leads Frontier-Bench at 43.5%, compared with Sol’s 34.4%. Artificial Analysis puts it first on broad intelligence, narrowly first on its coding-agent index, and effectively level with Sol on Terminal-Bench. Anthropic charges $5/$25 per million tokens: the same input price as Sol and $5 less on output.

Anthropic’s description of the improvement is more useful than “smarter.” The company says Opus 5 is better at checking its work, finding root causes instead of surface fixes, and continuing through long, underspecified tasks. Early-access reports from Cursor, Cognition, JetBrains, Replit, and other partners repeatedly mention difficult debugging, cleaner diffs, and better judgment before editing.

Those reports are directionally encouraging. They are not the same as independent developers living with the model for a month.

Opus 5 launched on July 24. Two days is enough to run a benchmark suite; it isn’t enough to discover whether the model develops recurring bad habits in large repositories, how stable it is under sustained Claude Code use, or whether its apparent thoroughness turns into expensive overreach. The launch-day Hacker News discussion already captures the tension: one commenter sees a model inventing a computer-vision workaround as valuable persistence, while another sees a model that should have asked permission before burning tokens.

There is another reason to wait for more evidence. Artificial Analysis found that Opus 5 improved factual accuracy over Opus 4.8 but answered more often when uncertain, producing a 50% hallucination rate on its AA-Omniscience evaluation. The benchmark calculates that as incorrect answers divided by incorrect, partial, and unattempted answers. It is not a claim that half of Opus’s responses, code, or answered questions are wrong. It does suggest that stronger agency and confidence still need supervision.

For now, Opus 5 looks promising for difficult debugging, architecture, code review, and vague tasks where judgment matters more than raw throughput. It may become the best daily coding model. Anyone making that claim today is extrapolating from two days of public use and a controlled early-access programme.

How to compare them on your own codebase

Don’t give each model one prompt and judge the prettiest answer. Coding agents are stochastic systems wrapped in different tools. A useful evaluation needs repeated tasks and a definition of done.

Start with five repository-level jobs:

1. Fix a bug with a known root cause and regression test.
2. Add a feature that touches the database, backend, UI, and tests.
3. Review a deliberately flawed pull request without editing it.
4. Reproduce a page from a screenshot and test it at two viewport sizes.
5. Investigate an intermittent failure from logs without being told which file is responsible.

Run each task at least three times. Use the model’s native harness first — Codex, Kimi Code, and Claude Code — because that is the product developers actually use. Then repeat the most important task in a shared harness if you need to separate model behaviour from tooling.

Record more than pass or fail:

model: claude-opus-5
effort: high
task: cross-layer-feature
runs: 3
measure:
  - tests_passed
  - human_review_minutes
  - unnecessary_files_changed
  - tool_calls
  - wall_clock_seconds
  - input_tokens
  - output_tokens
  - total_cost
  - asked_before_irreversible_action

The most revealing field is usually human_review_minutes. A cheap model that creates a plausible patch in ten minutes can be more expensive than Opus if a senior engineer spends an hour finding its shortcuts. Conversely, paying flagship rates for routine migrations and test generation is wasteful if Kimi produces equally reviewable work.

Use the same acceptance tests, but don’t force identical tool traces. Sol’s programmatic tool calls, Kimi’s preserved thinking history, and Claude Code’s native context handling are part of their practical performance. Removing those advantages creates a cleaner model experiment and a less useful purchasing decision.

Which one should you use?

Choose GPT-5.6 Sol today for long terminal sessions, complex implementation, and teams already invested in Codex. It has the strongest combination of public evidence and native agent infrastructure.

Choose Kimi K3 for visual coding or high-volume workloads. It may also suit teams that need open weights, once Moonshot publishes them and the licence can be checked. Budget for integration work, and constrain its authority explicitly.

Trial Claude Opus 5 for difficult debugging, architecture, and code review. Its first results justify the test. They don’t yet justify replacing every production workflow. Give the wider developer community a couple of weeks to find the failure modes that launch benchmarks miss.

A routed setup makes more sense than model loyalty: Kimi for inexpensive exploration and visual implementation, Sol for autonomous terminal work, and Opus for the hard review or root-cause task. All three are close enough that repository fit now matters more than brand.

If you’re making a production choice, rerun the evaluation in late August. There should be substantially more public evidence about Opus by then. Kimi’s weights and technical report may also be available if Moonshot meets its announced schedule.