$ the-wire · showcase
SAELens fixes a training deadlock and a JumpReLU load failure
By RepoJournal · Filed · About Interpretability · Composed from the cited sources · methodology
SAELens 6.52.0 ships a fix for a hang that could stall multi-SAE training runs, alongside an unmerged fix for the load error blocking existing dictionary_learning JumpReLU checkpoints.
fix: avoid deadlock when a multi-SAE user evaluator reads from the prefetcher jbloomAus/SAELens
Multi-SAE training with `prefetch_llm_batches` enabled alongside a user evaluator could deadlock: the evaluator's data view read from the prefetcher's queue inside a pause that still held the lock the producer thread needed, so any eval that pulled more than the buffered `prefetch_llm_batches + 1` batches hung. The pause no longer holds the lock, so long evaluations against the prefetcher compl...
fix: load dictionary_learning JumpReluAutoEncoder SAEs in dictionary_learning_1 converter jbloomAus/SAELens
`get_dictionary_learning_config_1_from_hf` maps `JumpReluAutoEncoder` to `architecture="standard"`, but since v6.23.0 the loader also copies `threshold` into the state dict, so JumpReLU SAEs trained with `dictionary_learning` fail to load on `main` with a RuntimeError. That covers SAEBench's JumpReLU suites, including `adamkarvonen/saebench_pythia-160m-deduped_width-2pow14_date-0108`.
ci: switch from Poetry to uv and speed up CI jbloomAus/SAELens
The `build` job runs 15 to 20 minutes, with dependency install alone taking 300 to 400 seconds because Poetry re-resolves roughly 190 sub-solves on every run and then pulls the CUDA torch and `nvidia-*` wheels (about 4 GB), and the venv cache meant to skip that has never worked. Moving CI to uv targets that install step.
Add covariance whitening normalization option (#647) jbloomAus/SAELens
`covariance_whitening` joins the `normalize_activations` options, applying PCA whitening to activations before the SAE to equalize the eigenspectrum of the activation covariance, following *Data Whitening Improves Sparse Autoencoder Learning*. Unlike the other normalization options, it needs the covariance computed rather than a simple per-feature scale.
v6.52.0 jbloomAus/SAELens
The 6.52.0 release carries both changes above: the covariance whitening normalization option and the prefetcher deadlock fix. Operators pinning to a tag can pull the deadlock fix without tracking `main`.
