Tag: mechanistic-interpretability
-
Foundations of Machine Cognition
Physical mechanisms, mathematical boundary conditions, and architectural trade-offs in Foundations of Machine Cognition.
-
Sparse Autoencoder Features
Disentangling polysemantic neuron superposition into interpretable, atomic feature directions.
-
Semantic Vector Manifolds
Analyzing latent space curvature, semantic clustering, and geometric superposition in scaled representations.
-
Kv Cache Bottlenecks
Managing memory fragmentation, context window growth, and high-throughput serving via virtual paging.
-
Flashattention Kernel Tiling
How IO-aware SRAM tiling eliminates high-bandwidth memory roundtrips in scaled transformer architectures.
-
Memory Bandwidth Wall
Analyzing arithmetic intensity, roofline model boundaries, and interconnect saturation in tensor accelerators.
-
Selective State Space
Hardware-efficient selective state transitions, parameter gating, and sub-quadratic sequence modeling.
-
Quadratic Attention Scaling
Physical mechanisms, mathematical boundary conditions, and architectural trade-offs in Quadratic Attention Scaling.
-
Induction Head Circuits
How two-layer attention subcircuits implement in-context pattern replication and prefix matching.
-
Machine Cognition Architecture
Physical mechanisms, mathematical boundary conditions, and architectural trade-offs in Machine Cognition Architecture.