IMAXING Latent Space

Drop the dye where the thought enters — watch where it surfaces

One strike, imaged: the emitter fires at L7, and one route stands lit in the fog.

What this is

A language model computes an answer along some path through its own latent space — but which path? IMAXING renders a prompt as a volume: token trajectories climbing the layer axis, surrounded by the fog of a thousand contrast prompts. A dye emitter injects signal at a chosen feature, and the trace that lights up — through attribution edges and sparse-autoencoder checkpoints — is the route the thought actually took.

The technical shape

Scene: token strands over transformer layers · x/z = principal components · y = depth
Context fog: diverse-text and sibling-prompt trajectories, depth-faded
Instruments: dye emitters · SAE bottles (top features/layer) · attribution edges
Method: baseline vs. perturbed runs, per-token, per-layer
Surface: interactive volume — orbit, zoom, hover any strand for its prompt position

The deep end · full technical outline

Outline v2 · expanding

The scene

A “strike” is one prompt run rendered as a volume: x and z are the first two principal components of the residual stream (PC1 typically carries ~65% of variance), y is transformer depth. Each token is a colored strand; solid strands are the baseline run, dashed are the perturbed run. BOS is excluded from view so the content cluster fills the volume.

Fog — the contrast field

The focus prompt is surrounded by fog: trajectories from diverse-text strikes and from structural siblings of the focus prompt, depth-faded so they read as context rather than subject. Fog is what makes “this path is special” a visible claim instead of an assertion.

The dye protocol

A dye emitter injects signal at a chosen layer/feature (e.g. L7, feature 8598, magnitude 4.0) and relative ‖dye‖ is traced per token per layer on a fixed colorbar. Sparse-autoencoder “bottles” mark the top features per layer; attribution edges (|corr| thresholded) connect strand segments across layers. The lit route is the answer to “which path carried the thought?”

Currents, not circuits

The founding objection: mechanistic interpretability calls its objects “circuits,” which implies fixed topology, on/off switches, engineered structure. Activation vectors do not behave like wire — they behave like fluid: pressure differentials, continuous flow, emergent paths. IMAXING is the hydrodynamic reading of the circuits literature — tracer-based interpretability. If the medium is a fluid, you do not enumerate its wiring; you drop dye in it and watch where it surfaces.

Bottles

A fluid the size of a model is untraceable, so shrink it: reduce the space to small matrices — bottles. Put an emitter in each bottle, fire a dye stream, watch where the medium takes it. One traceable current at a time instead of the entire fog at once. The SAE checkpoints in the volume are exactly these bottles: the places where the flow is small enough to read.

Perturbation maps and fingerprints

Rotate the emitter slightly — a minimal input perturbation — and watch the stream shift. Do it per bottle, combine the snapshots, run at refresh rate: a dynamic map of the entire current topology built from individually traceable streams. And because the fog geometry is fixed while the paths are not, different models draw different streams from the same emitter position — a model identified by its interference pattern rather than by its weights.