Reflections on AI benchmarks, agent architecture, memory systems, and engineering reality.
Last week I ran Weft, my homegrown memory layer, through LongMemEval and it scored 69.0% overall, 72.1% task-averaged. I was pleased. I shouldn't have been. The number was a lie, and the system that produced it was destroying the test data and calling it a win.
Anthropic's Advisor Strategy promises near-Opus intelligence at near-Sonnet cost. A server-side tool that pairs a cheap executor model with an expensive advisor. I benchmarked it in task decomposition.
I was recently watching a video about vibecoding where they said that you really needed to use a model for a year before you got to the point where you could trust it. So I decided to make a CMS for my blog.
I used AI for the better part of 2025 because I had to understand it for work. Here is why that changed in 2026 and what I learned about treating prompts as instruction manuals.