$ cat interpretability/week/2026-09-28.log
the week in review · Sep 28 – Oct 4, 2026
SAELens 6.53 ships bidirectional AbsTopK and whitening
By RepoJournal · composed from the cited sources · human-reviewed weekly · methodology
Three releases in three days added two activation options while fixing a multi-SAE training deadlock and a JumpReLU load failure.
v6.53.0 jbloomAus/SAELens
AbsTopK keeps the top-k pre-activations by absolute value with the sign preserved and no ReLU, so a latent can fire in either direction. It registers alongside TopK and BatchTopK, meaning you select it the same way you already pick your activation, with no new training loop.
v6.52.0 jbloomAus/SAELens
Covariance whitening joins the normalize_activations options, applying PCA whitening to activations before the SAE and equalizing the eigenspectrum, following the Data Whitening Improves Sparse Autoencoder Learning paper. The same release avoids a deadlock when a multi-SAE user evaluator reads from the prefetcher.
fix: avoid deadlock when a multi-SAE user evaluator reads from the prefetcher jbloomAus/SAELens
MultiSAEEvaluator ran every evaluation inside PrefetchingIterator.paused(), which holds the lock the producer thread needs, so multi-SAE training hung whenever prefetch_llm_batches was on alongside a user evaluator. The fix lands in 6.52.0; if you hit the hang, that is your upgrade.
fix: load dictionary_learning JumpReluAutoEncoder SAEs in dictionary_learning_1 converter jbloomAus/SAELens
get_dictionary_learning_config_1_from_hf mapped JumpReluAutoEncoder to architecture="standard", and since v6.23.0 the loader also copies threshold into the state dict, so JumpReLU SAEs trained with dictionary_learning would not load at all. That covers SAEBench's JumpReLU suite, so any comparison runs against it need this in place.
fix(evals): preserve excluded tokens during SAE reconstruction jbloomAus/SAELens
With normalization enabled the residual hook kept the scaled activation for excluded tokens instead of the original, and the attention hooks skipped the exclusion mask entirely. Tokens you marked excluded are now left untouched during reconstruction, so evaluations that rely on the mask no longer silently shift, BOS included. Fixed in 6.53.1.
Expand language-model targets, CI architectures, and training diagnostics goodfire-ai/param-decomp
by PD User (shared)
param-decomp adds maintained dense and mixture-of-experts language-model configurations, explicit routed expert selection, document-aware inputs, streamed outputs, and router, position, attention, and prompt-targeted diagnostics. If you were wiring those targets yourself, they are now maintained configs.
ci: switch from Poetry to uv and speed up CI jbloomAus/SAELens
The build job moves off Poetry to uv. The install-dependencies step was re-resolving everything on every run and the job as a whole took 15 to 20 minutes; nothing in your training or evaluation workflow changes.
$ ls interpretability/week/ # the briefings behind this review
Keep up with Interpretability in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.
fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.
Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.