Model Updates · Research · Benchmarks · Security · Fundamentals
Keeping up with machine learning, one mechanism at a time.
FundamentalsDeep dive9 min
NSFT cuts each expert in a Mixture-of-Experts model into channel groups and trains only the ones that matter — beating LoRA and ESFT on accuracy per trained parameter, and forgetting less along the way.
FundamentalsDeep dive5 min
Classic LLM pruning commits to one compressed structure before it knows what question a user will ask. TASP calibrates five task-specific masks instead, and a cheap router picks one per turn.
SecurityPaper5 min
Exploring how CONCURDEP enhances the understanding of concurrency-related dependency invalidations in CPython by tracing runtime properties and events.
1FundamentalsExplainer18 min
Why a model that can explain quantum tunnelling cannot count the letters in strawberry, why long conversations get slow and expensive, and why Dutch costs more than English. It all comes back to the very first thing that happens to your text.
2FundamentalsNews2 min
A plain-language guide to what tokens are and why they matter in AI language processing.
3FundamentalsDeep dive14 min
A deep dive into Srivastava et al. (2014) — the mechanism, the arithmetic behind weight scaling, what the benchmarks actually showed, and the tuning heuristics that still hold up.
4FundamentalsDeep dive9 min
NSFT cuts each expert in a Mixture-of-Experts model into channel groups and trains only the ones that matter — beating LoRA and ESFT on accuracy per trained parameter, and forgetting less along the way.
A technique for stripping a model's refusals went from a research curiosity to a business, and then agents started doing similar things to each other. Four posts, in the order they happened — two quick news items and the two deep dives that explain what was actually going on underneath them.