This benchmark expects models to not only do the happy path, but also edge cases, even though the ticket they give to the model doesn’t specify if. This to align more to real world tickets. Usually with benchmarks it’s the other way around: only implement what’s asked. So I welcome this type of benchmark because this is how I use the models.
Also, handing over the work to AI robs you of learning and understanding. Let’s say you write a ticket for a feature and that description is good enough that an AI agent can implement it from start to finish. The AI will then discover things while implementing the feature, and will use those learning to make the feature work. That learning will then be discarded once the AI is done. No one will be able to partake or share the knowledge with others. Sure, some of it can be saved in form of comments, but not everything.
I don’t see why they just don’t allow a smaller model to answer the question while letting the bigger one vet it. The vetting can be asynchronous and can be delivered after a few seconds (if it’s an easy query). If it’s a hard query, the UI can show the answer is currently being vetted or something.
What surprised me is that caffeine gets converted to other compounds that _also_ binds to the same receptors thus causing you to feel less tired. And that compound has its own half life. Now I’m just guessing, but it could be that one is more affected by one of the byproducts so to speak than the caffeine itself.
Why conscious thought as described in the article was thought to be signs of a delirious person was because that’s where you could find it in real life, is my guess. If someone blabbing about without a listener, they’re insane. So to describe someone’s inner thought would render them insane to the reader. But with psychology came greater understanding and value of the inner thoughts and now a character no longer was deemed insane by having innate thoughts.
Finally an alternative to the big dogs that a company can use. People have been asking for a way to run the Chinese models from a trusted provider. Here GitHub delivered!
The performance, if we trust the benchmarks, put it at Sonnet 4.6.
Microsoft needs to offer cheaper option since they change to token based billing. GPT-5.4 used to be x1 for yearly subscriber but now it cost 6x. i run out the premium request for just couple prompts. Github copilot for $10 used to be the best value since you get all the US AI labs model for cheap.
> The performance, if we trust the benchmarks, put it at Sonnet 4.6.
I don't trust these benchmarks. I used a number of times Kimi K2.7 and I was disappointed. It would run in circles for things that Claude would do in one shot. However, my usage was via Ollama cloud, and I have no idea if they serve the actual model or a quantized version, and it was the quantization that degraded the performance.
The great news, in my opinion, is the precedent. If Microsoft is now serving Kimi K2.7, then very soon they might start serving GLM 5.2, and that is indeed a very competitive model.
Check your harness. I use Kimi K2.6 for a lot of stuff with OpenCode and omp and it's extremely effective. I'm gonna try 2.7, but it should be capable model based on what I've seen with previous models.
> People have been asking for a way to run the Chinese models from a trusted provider
I'm going to be called a chiller again, but at this point I don't care as it is relevant. Synthetic runs their own models for a reasonable price, GLM5.2 & Kimi K2.7-Code included.
The Vibe CLI is really bad on Windows, sure they don’t officially support it, so can’t blame them, but a FYI for anyone wanting to try it. It can’t get find and replace right.
Oh man, this was the release I wanted to link, as it has a new feature (tiled follow link) that I actually started using right away. A new browser feature I find useful didn’t happen that often for me, so I got excited.
reply