Technical Director & Maker. Thoughts on autonomous agents, AI evaluation, technical SEO, and engineering systems.https://mediumroast.dev/en-ushttps://mediumroast.dev/blog/we-were-not-beating-longmemeval/https://mediumroast.dev/blog/we-were-not-beating-longmemeval/Last week I ran Weft, my homegrown memory layer, through LongMemEval and it scored 69.0% overall, 72.1% task-averaged. I was pleased. I shouldn't have been. The number was a lie, and the system that produced it was destroying the test data and calling it a win.Sun, 03 May 2026 00:00:00 GMThttps://mediumroast.dev/blog/model-testing/https://mediumroast.dev/blog/model-testing/Anthropic's Advisor Strategy promises near-Opus intelligence at near-Sonnet cost. A server-side tool that pairs a cheap executor model with an expensive advisor. I benchmarked it in task decomposition.Thu, 09 Apr 2026 00:00:00 GMThttps://mediumroast.dev/blog/making-a-cms/https://mediumroast.dev/blog/making-a-cms/I was recently watching a video about vibecoding where they said that you really needed to use a model for a year before you got to the point where you could trust it. So I decided to make a CMS for my blog.Sun, 25 Jan 2026 00:00:00 GMThttps://mediumroast.dev/blog/starting-with-ai/https://mediumroast.dev/blog/starting-with-ai/I used AI for the better part of 2025 because I had to understand it for work. Here is why that changed in 2026 and what I learned about treating prompts as instruction manuals.Sun, 25 Jan 2026 00:00:00 GMT