Imagine you're weighing yourself on a bathroom scale, but you secretly subtract five pounds by leaning on the counter. The number looks better, but your actual mass hasn't changed. That's what an input skip connection does inside a Gaussian VAE: it makes reconstruction error look smaller by mixing the original input back into the decoder output, but the underlying generative model — the thing that produces new samples from noise — stays exactly the same. The core claim is algebraic and precise. For a Gaussian VAE with a constant input skip parameter α (between 0 and 1), any model without a skip can be rewritten in skip form by multiplying the decoder output by b = 1 − α and the observation noise variance by b². The joint distribution over observations and latent variables is unchanged. The exact generative scores are unchanged. But reconstruction MSE gets multiplied by b², making it smaller. This is not an empirical finding — it's a mathematical identity. The skip is a coordinate change in parameter space, not a model improvement. Why does this matter if it's just algebra? Because separate training runs with and without the skip can converge to different generators. Gradient descent follows the loss landscape, and the loss landscape depends on the parameterization. The skip doesn't just rescale the loss — it reshapes the optimization trajectory. So two models with identical architectures but different skip settings can end up at genuinely different solutions, even though one parameterization is theoretically equivalent to the other. The paper makes this explicit: the reparameterization equivalence holds for a fixed model, but optimization breaks the equivalence. The experimental validation is modest and deliberate. In a controlled Gaussian experiment where the true density is known, the author shows that changing the skip and observation variance together reduces MSE but worsens energy score and the importance-sampling estimate of negative log-density. Lower reconstruction error, worse generation — that's the whole point demonstrated on a case where ground truth is available. On seven annual DART deep-ocean buoy records, energy score differences between skip and no-skip at the same observation variance go both ways across records, with no consistent advantage for the skip. Probabilistic PCA (PPCA) — a classical linear baseline — achieves lower mean energy scores than every neural DART fit. That PPCA result is a quiet bomb. It says that on these particular datasets, a linear model with closed-form solutions outperforms all tested neural VAE variants on the metric that actually measures generative quality. This isn't a failure of neural methods in general — DART records are small, low-dimensional time series — but it's a pointed reminder that complexity doesn't automatically buy you better generation, and that the metric you optimize (reconstruction MSE) can actively mislead you about the metric you care about (sample quality). The paper also surfaces a subtle trap: estimating observation variance from training reconstruction error — a common practical shortcut — can itself worsen energy score even when the encoder and decoder are held fixed. This means the damage isn't limited to the skip connection; it's about any procedure that lets reconstruction error contaminate your generative assessment. The broader lesson is one the VAE community has been circling for years but rarely states this cleanly: reconstruction error is not a proxy for generation quality, and architectural choices that improve one can degrade the other. This paper doesn't propose a fix — it's a diagnostic contribution, a proof that a common architectural trick is, in a precise mathematical sense, an illusion.
