$ the-wire · showcase
nnsight 0.8.2 fixes silent wrong values under vLLM tensor parallelism, SAELens splices skip transcoders correctly
By RepoJournal · Filed · About Interpretability · Composed from the cited sources · methodology
Two corrections that change what your numbers mean: nnsight 0.8.2 stops vLLM from returning wrong values without raising, and SAELens now applies skip connections when it splices skip transcoders, including every Gemma Scope 2 *_affine transcoder.
v0.8.2 ndif-team/nnsight
The bug-fix release frees GPU memory that stopped traces, failed backward passes and edits kept allocated, and closes several cases where vLLM returned wrong values without raising under tensor parallelism and with concurrent requests. It also pins vLLM to 0.27 and httpx below 1.0 and supports tensor parallelism on transformers 5.21, so the vLLM section of the notes is required reading if you t...
fix: apply the skip connection when HookedSAETransformer splices in a skip transcoder jbloomAus/SAELens
_SAEWrapper has called encode and decode directly since #637, so SkipTranscoder's x @ W_skip.T, added in forward() outside both, was dropped at splice time. Your Gemma Scope 2 *_affine transcoders, loaded as JumpReLUSkipTranscoder, now carry their skip connection, which means reconstructions and eval numbers from earlier runs of those transcoders were computed without it.
feat: add model_class_name="TransformerBridge" for loading models jbloomAus/SAELens
load_model accepts model_class_name="TransformerBridge", running the Hugging Face model directly through TransformerLens's TransformerBridge.boot_transformers (transformer-lens 3.0+) while keeping TransformerLens hook names, so configs like hook_name="blocks.1.hook_resid_post" work unchanged. Like HookedTransformer.from_pretrained_no_processing, it uses unprocessed weights.
fix: skip excluded tokens in the zero-ablation baseline jbloomAus/SAELens
The zero-ablation reference used to zero every position, BOS included, while the SAE replacement kept original activations at excluded tokens, so the two runs behind ce_loss_score and kl_div_score no longer touched the same positions. Excluded tokens are now skipped in both, and ce_loss_score values from earlier evals are not comparable to new ones.
docs: say a block can see one step past a stop on vLLM ndif-team/nnsight
Because vLLM's async scheduling is on by default, a request ending on a stop token or stop string runs one more forward that a block under tracer.iter or tracer.all() sees but the output does not hold; on gpt2 a stop token gave 7 tokens and 8 block steps, a stop string 1 token and 3 steps, while max_tokens was exact. The team documented it rather than disabling it, so size your block loops acco...
Action items
- → Upgrade nnsight to 0.8.2 before your next vLLM trace run, and recheck results produced under tensor parallelism, LoRA... ndif-team/nnsight [immediate]
