$ cat interpretability/week/2026-10-05.log
the week in review · Oct 5 – Oct 11, 2026
TransformerLens 4.2.0 fixes Cohere bridge and optimizer parity
By RepoJournal · composed from the cited sources · human-reviewed weekly · methodology
The release also adds tie-aware sparse probe metrics, while SAELens moves decoder-norm folding to load time.
v4.2.0 TransformerLensOrg/TransformerLens
Out now. The bridge for `CohereForCausalLM` and `Cohere2ForCausalLM` now rotates Q and K with the models' own RoPE convention; the notes say these models "do not match Hugging Face through the bridge" because Cohere's `CohereRotaryEmbedding` returns cos/sin with every frequency repeated twice. If you hook Cohere models, your activations should now line up with the HF reference.
fix: keep synthetic output biases out of the optimizer TransformerLensOrg/TransformerLens
Wrapping a model with no output bias still added one as a trainable parameter, which for DistilGPT2 handed the optimizer 50,257 extra values to update during validation. That parity gap is closed in 4.2.0, so training runs against hooked models should now match the unwrapped baseline.
feat: fold decoder norms at load time via fold_W_dec_norm=True jbloomAus/SAELens
`fold_W_dec_norm=True` is now available at load time on `SAE.from_pretrained`, `SAE.from_pretrained_with_cfg_and_sparsity` and `SAE.load_from_disk`, with the usage guide explaining what folding does and when you want it. You no longer need a separate post-load step to get a folded decoder.
v6.54.0 jbloomAus/SAELens
SAELens 6.54.0 is the first release carrying load-time decoder-norm folding. Adopt it if your pipeline currently folds after loading; nothing else in the release changes how you call the loaders.
Read slow PGD ladders from one attack goodfire-ai/param-decomp
by PD User (shared)
`SlowPGDReconLoss` can now be read out from a single fresh attack after each step count in a strictly ascending `read_out_steps` list, logged as `SlowPGDReconLoss_<n>step`, while fast probes log as `PGDReconLoss_<n_steps>step`. It cuts the eval cost of slow ladders to one attack run instead of one per configuration.
fix(bridge): rotate Cohere and Cohere2 with their own RoPE convention TransformerLensOrg/TransformerLens
This is the change behind the Cohere bridge fix in 4.2.0, and it only affects Cohere and Cohere2 models loaded through the bridge. Models with standard RoPE conventions are untouched.
Prep for release v4.2.0 – Contribution follow ups TransformerLensOrg/TransformerLens
Alongside: the release follow-up work and the slow-test trimming were internal test and coverage changes, and neither changes how you call the library.
$ ls interpretability/week/ # the briefings behind this review
Keep up with Interpretability in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.
fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.
Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.