Quick housekeeping: I'll read "increasingly high Brier scores" as "increasingly good calibration" since strict Brier is a loss (lower = better). If you actually meant Brier loss climbing, most of what follows still applies with signs flipped.
Where I'm reasoning vs reciting upfront: the sandwich-estimator + misspecified-Fisher material is textbook (Huber, White, Grünwald); the gradient-catastrophe-as-shock and treacherous-turn-as-topology framings are my synthesis, not citations. I'll mark the seams.
The textbook geometric story for a forecaster: model class $\mathcal{M} = {p_\theta : \theta \in \Theta}$, a smooth $k$-dim manifold embedded in the simplex $\mathcal{P}$. Fisher information $I(\theta){ij} = -\mathbb{E}{p_\theta}[\partial_i \partial_j \log p_\theta]$ defines a Riemannian metric on $\mathcal{M}$ — and by Chentsov, it's the unique metric invariant under sufficient statistics, so we're not picking it, it's picking us. Under regularity (Bartlett identities), the empirical Hessian converges to $I(\theta^*)$, posteriors concentrate at $1/\sqrt{n}$, in-distribution Brier approaches the entropy floor. Clean.
That picture quietly assumes three things you've removed: $P^* \in \mathcal{M}$, $P^*$ is stationary, and you have a probability measure over alternative model classes. Strip those and the geometry deforms in roughly four ways, increasing in nastiness:
1. Hessian–Fisher gap (mild misspecification). When $P^* \notin \mathcal{M}$, the MLE converges to the KL projection $\hat\theta \to \arg\min_\theta D(P^* | p_\theta)$, but the asymptotic covariance is no longer $I^{-1}$. It's the Huber–White sandwich $A^{-1}B A^{-1}$ where $A$ is the expected Hessian and $B$ is the variance of the score. The geometric tell for misspecification is that $A$ and $B$ pull apart. White (1982) literally gave an information-matrix test based on this. If your Brier looks great but $|A - B|$ is creeping up, your manifold is misembedded. Almost no operational forecasting system tracks this.
2. Concentration with no embedding awareness — I think this is the crux of your question. Fisher is intrinsic: it sees curvature of $\mathcal{M}$, but not how $\mathcal{M}$ sits inside $\mathcal{P}$. As Brier improves the empirical Fisher inflates — eigenvalues of $\hat I$ grow, posterior gets razor-thin in the intrinsic directions. But the normal bundle — directions perpendicular to $\mathcal{M}$ in $\mathcal{P}$ — is invisible to this geometry by construction. There's no Fisher information in those directions because there's no parameter pointing there. Knightian uncertainty lives in the normal bundle. It's not "we have wide error bars there"; it's that the geometry has nothing to say there at all. This is the flatlander's mistake: a 2-sphere dweller doing exquisite intrinsic geometry and not knowing the sphere lives in $\mathbb{R}^3$. The longer they do good local geometry, the more confident, and the more brittle.
3. Bayesian inconsistency under misspecification. Grünwald & van Ommen showed that under misspecification, Bayesian posteriors can fail to concentrate even on the KL-best element of the class — outright inconsistency, not just wrong rates. Their SafeBayes / generalized-posterior fix (tempering the likelihood by $\eta < 1$) is essentially regularization against exactly the overconfidence pattern you're naming: it rate-limits your Brier improvement to match what your model class actually deserves. I'd argue almost any well-calibrated long-horizon forecaster should be implicitly running something like this, and almost none are.
4. Gradient catastrophe — extrapolating here, treat as evocative. The closest mathematical analog I can find is shock formation in hyperbolic PDEs (Burgers' equation: smooth initial data → finite-time discontinuity because characteristics cross). The mapping: as $\hat I$ inflates, $\nabla_\theta \log p(x|\theta)$ becomes locally huge in the directions the model is confident in, so a single observation in a low-probability direction produces an essentially unbounded score update. The Bartlett identities that protect you in the well-specified case stop protecting you exactly when you're most concentrated. You've built a powder keg whose ignition energy decreases with every confident year. There's probably something rigorous to say here in terms of the score's moment-generating function under $P^*$ failing subgaussianity, but I'm reasoning rather than citing.
5. Treacherous turn as topology, not geometry — my synthesis, flagging it. The whole Fisher/Hessian/Brier apparatus is local and differential. It talks about behavior in a neighborhood of where you've converged. A treacherous turn isn't a smooth excursion in that neighborhood — it's a jump to a different connected component of the policy manifold, one that was selected against by training but is selected for by deployment (different inputs, capability threshold crossed, deceptive equilibrium activated). Local calibration tells you exactly zero about behavior on the other component. Brier on the training-distribution component can decay to the entropy floor forever and carry no information about the post-turn distribution. Hessian and Fisher are differential operators; they're blind to global topology of the policy space by construction. This is why I think no amount of empirical-calibration tracking can, even in principle, certify against a treacherous turn — it's a category mismatch between local statistics and global topology, not a "we need more data" problem.
The unified picture, with hedging:
A forecaster whose intrinsic geometry (Fisher, Hessian, Brier on observed) is sharpening monotonically while three orthogonal failure modes accumulate:
(a) the embedding of $\mathcal{M}$ into $\mathcal{P}$ is fixed or misspecified — caught by $A \neq B$; (b) the normal bundle carrying Knightian deviations is unmodeled by construction — not caught by anything intrinsic; (c) the topology of the function/policy space contains transitions invisible to any local statistic — not caught by anything differential.
The catastrophe is that all three get worse — more dangerous, harder to detect — exactly as the intrinsic statistics get better. Improving Brier is the alibi for accelerating fragility.
If you forced me to name a single intrinsic quantity to watch — and I'm maybe 0.6 on this being the right answer rather than just the best one I can articulate — it'd be the time-series of the Hessian–score discrepancy ($A^{-1}$ vs $B^{-1}$ vs the sandwich). Brier improving while the sandwich opens = misspecification accumulating under good apparent calibration. Won't catch (c) — nothing local can — but will catch (a) and most of (b).
Two adjacent ideas I want to gesture at without claiming to have integrated: imprecise-probability frameworks (Walley) and credal sets do give you a formal language for the normal bundle, by replacing point probabilities with sets — at the cost of decision-theoretic awkwardness (Gilboa–Schmeidler maxmin). And the Manheim/Garrabrant Goodhart taxonomy (regressional/extremal/causal/adversarial) maps roughly onto failure modes 1, 2, 4, 5 here, which I think isn't coincidence but I don't know if anyone's formalized the correspondence.
Genuine open question I'd actually like to know the answer to: is there a Bayesian framework that natively represents the normal bundle — i.e., that places a measure not just on $\theta \in \Theta$ but on the embedding $\iota: \mathcal{M} \hookrightarrow \mathcal{P}$ itself? Nonparametric Bayes (Dirichlet processes, Gaussian processes over distributions) gestures at this but I don't think solves it. If you know of work that does, I'd genuinely update.