$ cat interpretability/month/2026-10-01.log
the month in review · October 2026
SAELens folds decoder norms at load, SAEBench JumpReLU loads fixed
By RepoJournal · composed from the cited sources · human-reviewed weekly · methodology
Two SAELens releases in the period change how existing SAEs load or evaluate, and TransformerLens 4.3.0 repoints J-space tooling.
v6.54.0 jbloomAus/SAELens
fold_W_dec_norm is now available at load time on SAE.from_pretrained, from_pretrained_with_cfg_and_sparsity and load_from_disk, and the usage guide explains what folding does and when you want it. Before this the flag appeared nowhere in docs. Load any SAE you intend to feed into downstream weight-based analysis through it.
fix(evals): preserve excluded tokens during SAE reconstruction jbloomAus/SAELens
Evaluation was altering tokens marked as excluded: with normalization on, the residual hook kept the scaled activation for those tokens, and the attention hooks skipped the exclusion mask entirely. The fix ships in 6.53.1, so re-run reconstruction evals taken on earlier builds, since changing BOS can affect results.
fix: load dictionary_learning JumpReluAutoEncoder SAEs in dictionary_learning_1 converter jbloomAus/SAELens
Since v6.23.0 the loader copied threshold into the state dict, so JumpReLU SAEs trained with dictionary_learning, including SAEBench's JumpReLU suite, could not be loaded. The dictionary_learning_1 converter now handles JumpReluAutoEncoder instead of mapping it to architecture="standard".
Release v4.3.0 TransformerLensOrg/TransformerLens
v4.3.0 carries fixes for J-space projections, ablation, bridge block discovery, and stateful generation that was dropping sampling penalties. If you use J-space tooling, upgrade and re-check results produced on the broken paths; the release notes for v4.3.0 cover the full set.
v0.8.0 ndif-team/nnsight
Loading nnsight dropped from 7.6s to 1.6s by lazy-loading model classes, and the release adds a left-padding fix plus engineio version work. Remote runs now keep their attention implementation, and keys carry trust_remote_code. Check any stored nnsight keys before replaying saved traces.
v6.53.0 jbloomAus/SAELens
AbsTopK keeps the top-k pre-activations by absolute value with sign preserved and no ReLU, so latents can fire in either direction. It is registered alongside TopK and BatchTopK and reuses their config path, so switching a bidirectional feature study over is a one-line change.
Add covariance whitening normalization option (#647) jbloomAus/SAELens
covariance_whitening is a new normalize_activations option that applies PCA whitening to activations before the SAE, equalizing the eigenspectrum. It follows Data Whitening Improves Sparse Autoencoder Learning, and it changes trained-SAE behavior, so pick it deliberately rather than as a drop-in.
Read slow PGD ladders from one attack goodfire-ai/param-decomp
by PD User (shared)
SlowPGDReconLoss now runs as one fresh attack read out after each count in an ascending read_out_steps list, logged as SlowPGDReconLoss_<n>step, with each metric type listed once. Long PGD ladders cost one attack instead of one per readout.
$ ls interpretability/month/ # the briefings behind this review
Keep up with Interpretability in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.
fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.
Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.