Hackathons in the vibe coding era: our token economics experiment
Students had one Friday evening, 18 GB of coding-agent transcripts, and a single question: where are we burning tokens? They found many wasteful patterns in agentic coding.
Quesma Blog
Insights on agentic coding tools, LLM evaluation, benchmarking, and simulation environments.
Get new posts by email
Subscribe via RSS
GPT-6 Astra solves puzzles
GPT-6 Astra solved twice as many Baba Is You levels as Claude Fable 5.1. It fits a pattern: ARC-AGI-3, the 3D game Portal, MazeBench, and Przemysław “Psyho” Dębiak’s obscure puzzles.
HN RTK reports huge token savings, but our cost benchmarks disagree
We tested RTK (Rust Token Killer) with Claude Code on Fable 5.0, and OpenCode with DeepSeek V4 Pro 0813 on Terminal-Bench 2.1.
Knowledge vs wisdom: asking AI “What mushroom is that?”
AI mushroom identification from a photo with ChatGPT, Claude or Google Gemini: GPT-6 Astra, Gemini 3.8 Flash, Claude Fable 5.1, GPT-5.6 and GLM-5.3-Flash. Asked “What mushroom is that?” on 360 photos of poisonous species. Which warn, which get it right, which fail.
HN Mushroom hunting with LLMs: what can go wrong?
Mushroom identification with AI: GPT-5.6-Sol, Gemini 3.8 Flash, GLM-5.3-Flash and Claude Fable 5.1 benchmarked on poisonous and edible species of FungiTastic. A lot of dangerous errors.
OpenAI Codex pricing: the $270 PR a $200/month sub covers daily
A buyer’s guide to OpenAI Codex for individuals, teams, and enterprises. Includes comparisons with Claude Code.
HN Benchmarking Qwen3.8 27B quantizations: 4-bit holds up, 1-bit collapses
I test Unsloth GGUFs of Qwen3.8 27B (Q4_K_M, UD-Q2_K_XL, UD-IQ1_S) with llama.cpp on GPQA Diamond, IFBench, Terminal-Bench 2.1. Q4_K_M matches BF16 abd fits an RTX 4090.
Gemini 3.7 Flash, Grok 4.6, GLM-5.3 and DeepSeek V4 Pro joined the frontier
August 2026 on Baba Is Bench: Gemini 3.7 Flash, Grok 4.6 and DeepSeek V4 Pro 0813 each beat their predecessor while costing 3-20x less. For open-weight GLM-5.3 and Qwen3.8 progress is gradual.
HN Claude Code pricing: same tokens, same model, up to 40x the price
Claude Code buyer’s guide, August 2026: what Max, Team, and Enterprise cost, how to get the best deal, and how Uber and Shopify cap the spend.
I wired 4 models together in Claude Code. One refused, and it backfired 4 ways on Terminal-Bench
Four Claude models wired together in Claude Code: model refusals on delegated security tasks, seventh place, twice the cost of the top single-model run.
Quantization hurts knowledge nonlinearly - Qwen3.6 27B case study
Quantization is a lossy compression, and factual knowledge is incompressible. We test Qwen3.6 27B GGUF quantizations from Hugging Face (by Unsloth and Bartowski) and llama.cpp on the Incompressible Knowledge Probes (IKP) benchmark.
Kimi K3 is Open, Opus 5 is Good, DeepSeek V4 Flash is Cheap: LLMs on Baba Is You
We evaluate July 2026 fresh releases Kimi K3, Claude Opus 5, Grok 4.5, Gemini 3.6 Flash, and DeepSeek V4 Flash 0731 on Baba Is Bench, an LLM agent benchmark based on the puzzle game Baba Is You, comparing pass rate, speed, and cost with Claude Fable 5 and GPT-5.6.
A lesson about retries, hidden in the DeepSeek-V4 paper
A warning hidden in the DeepSeek-V4 paper says retrying interrupted LLM requests is mathematically incorrect — it introduces length bias. I reproduced it on 100,000 poems.
Do Qwen3.6 27B quantizations break the pelican?
We tested Qwen3.6 27B quantizations by Unsloth on Hugging Face, with pelicans on bikes, gears, Terminal-Bench 2.1, and AIME-120.
HN I burned all my tokens researching how to save tokens
How I burned a full Claude limit in 30 minutes and built my own deep research pipeline instead: 3 subscriptions, shared memory, a clear role for each model. You can build the same from what you already pay for.