$ the-wire · showcase
SAELens 6.54.6 forwards model kwargs to reconstruction evals
By RepoJournal · Filed · About Interpretability · Composed from the cited sources · methodology
SAELens shipped 6.54.6, closing a gap where reconstruction metrics ignored the model options you passed to run_evals while sparsity and variance evaluation honored them.
6.54.6 jbloomAus/SAELens
by semantic-release
Version 6.54.6 carries the fix through to a released package, so pulling it is now the way to get consistent evaluation behavior rather than pinning a branch.
fix: forward model kwargs to reconstruction evaluation jbloomAus/SAELens
Previously run_evals(model_kwargs=...) forwarded your options to sparsity and variance evaluation but dropped them from the CE/KL reconstruction path, so an attention_mask changed the sparsity forward pass while reconstruction metrics silently fell back to the model's default mask. The mapping now flows through get_downstream_reconstruction_metrics to get_recons_loss, which already passed it to...
fix: forward model kwargs to reconstruction evaluation (#765) jbloomAus/SAELens
The merge landed as commit 83c7b9bd and is the change behind the release, if you track the repo's history rather than its tags.
Action items