Letting a model advise without letting it decide
There are questions a rule cannot answer. "Does this tool call look like exfiltration" is one. You can enumerate paths and hosts and command shapes forever and still not capture it, and a model is genuinely better at the judgement. So the question is not whether to ask one. It is how to ask one without handing it the keys.
The obvious design is a decision provider: hand the model the situation, get back allow or deny, combine it with the deterministic verdict. It is also the design that quietly destroys the thing worth protecting, because the moment two components can emit a verdict you have to answer "which one wins" in code that is very easy to get subtly wrong.
We didn't have to work that out, because a comment shipped months earlier had already said so. detectors.ts, on why it contains no semantic classifier: "it would need a model call, and governance-core stays network-free and off the PDP's synchronous hot path by design." And two lines later, where a new signal is allowed in: "INPUTS to the one risk scorer, never a second scoring path." The design had been specified as a refusal, and the refusal named the alternative.
So the new layer asks bounded questions and converts the answer into a contribution to one of the fifty risk categories that already exist. Three kinds, all closed, and no free-text kind:
choice pick exactly one member of a closed set (>=2, distinct)
level one position on an ordered scale 1..n
likelihood probability 0..1 that one declared statement is true
The absence of a free-text kind is the point. An unbounded answer cannot be validated, cannot be calibrated against a held-out set, and cannot be folded into a number without an interpretation step that would itself need governing.
There is still exactly one scorer and one reducer. The entire semantic surface is one dimension of a fifty-dimensional deterministic score, subject to the same hard floors as every other input — so a signal can push a category up, and the floors, the hard-deny set and the tolerance ceiling all still run afterwards exactly as they did.
The property that makes it safe is monotonicity: a signal can raise a risk category and can never lower one. Every failure mode — no answer, late answer, malformed answer, an explicit abstention, confidence below the configured floor — contributes the question's declared failsafe score, which sits at the dangerous end of the clamped 1–10 scale.
That direction is a deliberate inheritance. The risk scorer clamps unknown input to 10 because high risk is the safe assumption; the confidence engine fails to zero because there high confidence is the dangerous direction. Inquiry follows the scorer. Silence is treated as danger, not as permission.
Which gives a blunt worst case. A provider that is compromised, miscalibrated, hallucinating, or actively hostile can make Starfish more cautious and nothing else. It can cost you availability. It cannot buy authority. And because the signal type has no allow, deny, ask or approve field anywhere on it, that isn't a convention a reviewer has to keep noticing — there is no field to misread. A test asserts those fields are absent, which is an odd test to write and the one most likely to catch a future well-meaning "let's just add a verdict field for convenience".
The symmetrical version is tempting and is rejected on purpose: a confident, reassuring signal is not allowed to lower a score. That would convert model confidence into permission, and turn provider compromise from an availability problem into a privilege-escalation path.
Four things that make it survive contact with a hostile provider
Thresholds must be complete or the fold refuses. A threshold set that misses a declared option, has the wrong number of entries for its scale, or does not cover probability all the way to 1 is rejected outright rather than defaulted. A silently-defaulted threshold is indistinguishable from a configured one in an audit record, which makes it worse than a missing one. And the model never picks the threshold it is judged against — deterministic code owns that, the same way it owns every other limit.
Questions are versioned instruments, not editable prompts. The question's version is inside its provenance hash, so rewording invalidates every signal and every calibration bound to the old text. A signal whose hash does not bind to the current request is refused as a replay. The test fixture is keyed by questionId:version specifically so that a fixture written for v1 reports unavailable against v2 rather than quietly answering — which is what makes offline conformance meaningful instead of self-confirming.
Admission is checked before the provider receives anything. A provider declares the highest data classification it may see; a request carrying anything above that never reaches it. So "we asked and it said yes" cannot be manufactured by sending a request that should never have been sent — and classified state cannot leak to an uncleared endpoint by accident.
Network stays out of the trusted core. Providers that reach the network do not implement the in-core interface at all. They use a request builder and a response parser, with the send performed by a higher ring — mirroring how the model-transport layer has always built requests it never sends. The builder bakes in no default endpoint, because a default endpoint is a silent cloud fallback. And the parser turns anything unexpected into a fail-closed contribution rather than throwing, so a broken or hostile endpoint cannot produce an exception a caller might catch and ignore.
The same asymmetry governs the second boundary added here, for external MCP control planes. Starfish decides first, and only then is a call forwarded. An external gateway saying "allowed" never overrides a Starfish deny; an external gateway saying "denied" does stop a call Starfish allowed. Denials compose. Grants do not. A second opinion may subtract authority, never add it.
Three details in that boundary are worth stating, because they are the ones that get skipped:
Weak isolation is refused in production no matter what a registry entry claims about itself. An in-process or Docker-sibling sandbox is rejected for prod outright. A sibling container sharing the host daemon is not an isolation boundary — the popular quick-start pattern for this class of tool mounts the host Docker socket, and that is a convenience, not a seam.
A lost response is not a failure. A forward that throws or times out records forwarded-lost, distinct from forwarded-error. "We do not know whether it happened" and "it did not happen" are different facts, and completion logic that conflates them will eventually mark an executed side effect as never-run. The Starfish-side invocation id is minted before forwarding, so a call whose response vanishes is still correlatable.
External audit ids are correlated into our record, never substituted for it. A gateway's activity log proves a call was made. It does not prove the claimed real-world effect occurred, which is what our own evidence rule demands. So the gateway's request id and audit id sit alongside our task, decision and invocation ids — a reviewer can pivot between the two ledgers without either being authoritative for the other. Registry entries also carry a content hash, an owner, approved environments, permitted principals and a review expiry, and an entry that is unapproved, revoked, expired or unhashable is not executable.
A deprecation that has to earn its own flip
Alongside this, a long-standing loose end: the constitution says all work is a task, but the task id on a tool call has been optional since the beginning. Closing that is a breaking change, so it gets a ladder — off, then observe, then required — the same three-rung shape the scope-contract gate already uses.
observe admits precisely the calls off admits and records more, so it cannot weaken anything. A call arriving with no task id is allowed and attributed; a call that claims a task id is still checked, exactly as it would be under required. A bad task id is a violation, not a legacy caller.
The interesting part is what gates the promotion. It would be easy to ship a readiness flag that reports "safe to enforce" — and easy for that flag to report clean because nothing was watching. That is the same unfalsifiable-disclosure shape as a hardcoded false we found and fixed in v0.27.0. So readiness in off mode is always false, with the reason stated plainly: absence of recorded untasked calls is not evidence that none occur. And the report names the offending agent and tool, because a count tells you legacy callers exist while a name tells you whom to go and fix.
What this does not do yet
Nothing calls the fold in production. The module is built, exported, reachable and tested, and no live code path folds a signal into a real decision — so governance behaves exactly as it did before. The next step is to wire one call site in shadow mode and compare it against the production path before anything counts for real.
Two other limits worth stating. The calibration record exists; the statistics do not. Nothing computes accuracy, calibration error or drift, so no statistical claim should be read into a threshold that a human approved by hand. And the task-binding default has not moved: it is still opt-in, and the site still says so.
That restraint is the v0.27.0 lesson applied to ourselves. A green module is not an enforced control, and a boundary that exists is not a boundary that runs.