$ cat interpretability/week/2026-10-19.log
the week in review · Oct 19 – Oct 25, 2026
TransformerLens 4.3.0 fixes J-space, ablation, bridge discovery
By RepoJournal · composed from the cited sources · human-reviewed weekly · methodology
SAELens 6.54.6 forwards model kwargs to reconstruction evals, and nnsight carries attention implementation and trust_remote_code on remote requests.
Release v4.3.0 TransformerLensOrg/TransformerLens
The release carries the three corrections operators were most likely to hit: sparse J-space projections no longer drop a selected direction to the float32 pinv rank cutoff, ablation_hooks removes several tokens' lens vectors jointly instead of one at a time, and block enumeration finds architectures that register their block lists under non-standard names. The release notes also list a stateful...
fix(jacobian_lens): ablate token directions jointly TransformerLensOrg/TransformerLens
Lens vectors of related tokens are far from orthogonal, so removing them one at a time reintroduced a component along directions already ablated. Ablation hooks now project the given token directions out jointly, which changes numbers if you measured multi-token ablation on 4.2.x.
fix(bridge): discover non-standard block lists in _enumerate_blocks TransformerLensOrg/TransformerLens
Block discovery walked a fixed tuple of attribute names, so an architecture registering its lists anywhere else enumerated zero blocks with no error. Raven/Huginn registers its lists under other names; check whether your bridge results on unusual architectures were silently empty.
fix: forward model kwargs to reconstruction evaluation jbloomAus/SAELens
run_evals(model_kwargs=...) was forwarding options to sparsity and variance evaluation but dropping them from the CE/KL reconstruction path, so an attention_mask changed one forward pass and left reconstruction on the model's default mask. Reconstruction metrics now see the same masking as the rest of the run.
feat: carry the attention implementation with every remote request ndif-team/nnsight
A TransformersModel loaded with attn_implementation="eager" returned attention weights locally and None remotely, because the request named only the model and the server ran its deployed default, sdpa on NDIF. Every remote request now names the implementation, so captured attention weights should match your local setup.
feat: carry trust_remote_code in the model key ndif-team/nnsight
The HuggingFace model key now records whether the checkpoint's own code may run, joining repo_id, revision, and task. Checkpoints built without that option produce an unchanged key, so existing remote caches stay valid.
tracing: keep an earlier invoke's rebind when a later one pushes ndif-team/nnsight
Inside a session, a trace with two or more invokes undid a rebind made by an earlier invoke. If you assigned activations inside one invoke and read them in a later one, that value now survives.
fix: store pretrained SAE paths as str so the HF sync check runs again jbloomAus/SAELens
Alongside: SAELens restored the nightly HF sync check, red on all 413 retained runs since 2025-09-02 because pretrained SAE paths were stored as non-str objects, and param-decomp split its repository into project directories and now evaluates on fixed, row-shuffled eval sets.
$ ls interpretability/week/ # the briefings behind this review
Keep up with Interpretability in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.
fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.
Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.