Problem Statement

Muse only becomes useful once users grant broad, standing permissions. But every approval prompt pushes users toward one of two failures: clicking "yes" reflexively (a safety failure) or abandoning the task altogether (an activation failure). So the question I'm solving is: how do I design Muse's permission and approval model so it earns standing trust incrementally, reserving explicit approval for the moments that genuinely warrant it, without drowning users in prompts or widening the blast radius when something goes wrong?

Core Tension

On the surface this looks like a simple tradeoff: more approval prompts means more safety, fewer prompts means less friction. If that were the whole story, it wouldn't be worth a case study. The reason it's hard is that there are actually three failure modes fighting each other, not two.

The first is the safety failure. Every prompt you add is supposed to be a checkpoint, a moment where the user actually looks before Muse acts. But the more prompts you throw at someone, the faster they learn to stop reading and just click through. The approval is still technically there, but it's stopped doing its job. It's become a speed bump they've trained themselves to roll over. And here's the trap: the one prompt that genuinely mattered, Muse about to pay the wrong vendor, or an action triggered by a prompt-injected web page, gets the same autopilot "yes" as the fifty harmless ones before it. The safety mechanism is present but functionally dead.

The second is the trust failure, and this is the one that makes it a Meta problem specifically. When a blind approval backfires, users don't experience it as "I should have read that." They experience it as "this thing betrayed me." The blame lands on Muse and on Meta, even though the user technically clicked yes. Reflexive approval doesn't build trust, it borrows against it. Every blind yes is fine until the one time it isn't, and that one time is where all the trust gets spent at once. Any company building an agent has to worry about the safety failure. Only a company two weeks removed from an $18 billion settlement has this trust-collapse tail risk stacked on top, which is exactly why getting this right isn't optional for Muse.

The third is the activation failure, and it pulls in the opposite direction from the first two. The obvious "fix" for the safety and trust problems is to add more approvals, make the user confirm everything. But Muse is worthless until someone actually hands it something real, and a wall of permission prompts on day one is the fastest way to make a nervous new user close the app before they ever get a single task done. The friction that feels like reassurance to a first-time skeptic feels like nagging to someone who's ready to delegate.

So the three are genuinely in a bind with each other. Turn the approval dial up and you buy short-term safety at the cost of activation, while quietly breeding the reflexive clicking that causes the trust failure later. Turn it down and you smooth out activation but widen the blast radius of any single mistake. The job isn't to pick a point on that dial. It's to design a model where the dial moves on its own, tightening or loosening based on how risky the action is and how much trust the user has actually built, so the right prompts show up at the right moments and the rest get out of the way.

Who I'm solving for

It's tempting to start by sorting users into personality types, the tech enthusiast who'll adopt anything new, the privacy-guarded skeptic who trusts nothing. That sorting is useful, but not for the reason you'd think, and if you build your permission model around it you'll build the wrong thing. So let me separate two questions that are easy to blur together: who are we solving for, and what does the model actually key off of. They have different answers.

Who we're solving for is the everyday delegator. Not a power user, not an engineer, just someone who wants Muse to handle real things, booking the travel, clearing the inbox, paying the bill, and who is deciding, task by task, whether this thing has earned the right to act on their behalf. Two edges of that group are worth naming because they pull the design in opposite directions. On one edge, the fast adopter: comfortable with AI, quick to hand over more, likely to hit the delegation stride early. They're not the risk. On the other edge, the skeptic: private by instinct, the kind of person who double-checks a payment three times before hitting send. This is where the activation failure lives. If we can't earn a skeptic's trust, they don't churn dramatically, they just quietly demote Muse into a chatbot, using it like any other assistant and never once touching the agent, the one feature the whole product is built around. Winning them is the bar. If the model works for the skeptic, it works for everyone.

But here's the thing that reshaped how I approached this: personality isn't the axis the permission model runs on. Take the fast adopter. He's a confident delegator when Muse is booking a dinner reservation and a nervous skeptic the first time it wants to move two thousand dollars or email his boss. Same person, opposite needs, and the only thing that changed was the action. If I segment by who he is, the model freezes, because "tech enthusiast" doesn't tell it whether to wave the restaurant booking through or slam the brakes on the wire transfer.

So the model keys off two things, and neither of them is personality. The first is the risk of the action itself, how much damage a wrong move does, and how easily it can be undone. The second is how much trust that specific user has banked so far, built from the history of Muse acting and getting it right. A low-risk, reversible action from someone who's banked trust should just happen, quietly. A high-risk, irreversible one, anything touching money, identity, or something public, should stop and ask every single time, no matter how much trust is in the bank. Everything in between scales with the history. That's a rule a machine can actually run, and it's the behavioral, act-on-what-people-do cut that fits how Muse works, rather than a demographic guess about who they are.

One deliberate constraint before we go further: I'm giving this whole trust arc a hard ceiling of two weeks. If a user has lived with Muse for two weeks and still won't let it act, we haven't found a slow-burn trust stage to nurture, we've lost them, and no permission model is going to save a product they've already half-abandoned. So the job isn't to earn trust eventually. It's to earn it fast, inside a window short enough that the user is still paying attention.

The action-risk tiers

The core mistake would be to treat "risk" as a single number. It isn't. Every action carries two separate questions that happen to travel together: how much damage if this goes wrong, and how hard is it to undo. Those aren't the same. A movie ticket is cheap but annoying to reverse; archiving a pile of emails touches nothing precious but is one click to walk back. Keep those two dials separate and the tiers basically fall out on their own.

Tier 0, silent. Reversible, low-stakes, leaves no lasting mark. Muse just does it, and mentions it after only if it's worth mentioning. Reading your calendar to find a free slot, drafting an email that lands in your drafts, pulling flight options, dropping something in a cart without buying. Nothing here can hurt you, and anything slightly off is fixed in a second. The goal for this tier is invisibility. Every prompt you put here is pure fatigue with zero safety payoff, and it's that fatigue that later gets your important prompts ignored.

Tier 1, notify-and-undo. The action happens, but Muse surfaces it clearly and gives you a real, easy way to pull it back. Sending a drafted email to someone you email often, moving a calendar event, archiving a batch of messages. This is the tier most models forget, and it's the most important one, because it breaks the false choice between "act silently" and "stop and ask." Here the user consents by not undoing, which costs them zero attention unless something's actually wrong. For a big class of actions, a good undo is safer and smoother than a prompt, because a prompt gets rubber-stamped while an undo only gets used when it's genuinely needed. A good undo is consent without friction.

Tier 2, confirm-first. Muse stops and asks before acting. This is where money, outsiders, and hard-to-reverse things start: a payment to a vendor, an email to a brand-new contact, connecting a new service. The first time, it always asks. But this is the one tier where the line can move, as a user builds a track record with a particular kind of action, some of these can graduate down toward Tier 1 and stop interrupting. Exactly how that graduation works, what counts as "enough" trust to earn it, is the load-bearing question of this whole model, and I'm parking it deliberately for the next section rather than hand-waving it here.

Tier 3, hard stop, always. Confirmation every single time, no matter how much trust a user has banked. No exceptions, no graduating down, ever. Moving large sums, anything that posts publicly under your name, deleting data, changing security settings, granting a new permission scope. The entire point of this tier is that it's immune to the fatigue problem, and it stays immune precisely because it's rare. You can keep a hard stop meaningful only if the user sees it a handful of times, never as routine. If Tier 3 is firing often, something below it is mis-tiered.

And that's the idea holding the whole thing together: every prompt you spend at the bottom is a prompt you can't afford at the top. Fatigue is a shared budget. Tier 0 has to be silent not just to feel smooth, but so that when Tier 3 finally fires, the user still has the attention left to actually read it. You're not buying safety by adding prompts. You're rationing attention so the prompts that matter still land.

How trust gets banked

This is the part the tiers were leaning on and I deliberately didn't answer earlier: what actually moves an action from confirm-first down to notify-and-undo. It's the load-bearing question of the whole model, because get it wrong and either nothing ever graduates (fatigue never lifts) or the wrong things graduate (safety quietly rots).

The lazy answer is to count successes. Muse pays a vendor five times without a problem, so the sixth time it stops asking. Simple, and quietly dangerous, because a pure counter only ever goes up, and anything that only goes up can be farmed. A slow-drifting mistake, or a patient prompt-injection attack, can rack up harmless little "successes" to bank trust and then cash it all in on the one action that matters. So counting is out. Whatever we build needs trust to behave the way real trust does: earned slowly, and lost in a hurry.

Trust banks in narrow buckets, not one global score. "Pay this specific recurring vendor" is its own bucket. "Email people already in my contacts" is a different one. Competence in one area never silently unlocks another, so being trustworthy with your calendar can never vault Muse into your bank account. This is the single most important property, it keeps a win in a low-stakes corner from ever paying for access somewhere high-stakes.

A bucket fills from two sources, weighted very differently. There's earned trust, which is slow: Muse does that specific action correctly, the user doesn't undo or complain, the bucket ticks up a little. And there's granted trust, which is fast: the user just says "you can handle this from now on." One express grant is worth more than a dozen silent successes, and that's on purpose, because a grant is the user consenting with their full attention, which is exactly the kind of consent we actually want, as opposed to the rubber-stamped kind.

And here's the mechanism that makes the whole thing safe to automate: buckets drain far faster than they fill. A single undo, a correction, a "no, don't do that," costs dramatically more trust than any single success earns, enough to knock an action back up into confirm-first and make it re-earn its graduation. One bad signal outweighs many good ones. That asymmetry is the security model. It's what makes farming pointless: you can't grind fifty quiet wins to cover one betrayal, because the betrayal wipes out the winnings.

But a full reset on every undo would be too brittle, real people fat-finger, or hit undo because they changed their mind, not because Muse failed. So the drain reads context, not just the click. An undo where the user then redoes the task differently reads as "Muse got it wrong" and drains hard. An undo where they just paused reads as "changed my mind" and drains, heavily, but doesn't zero out. The one exception is anything that smells like a security event rather than a preference: an action the user flags as "I never asked for this," or anything tripping a prompt-injection signal. That doesn't drain, it resets the bucket to zero instantly. Ordinary doubt costs a lot. A security flag costs everything.

Three things I'm deliberately not pretending to have solved, because inventing false precision here would be worse than naming the gaps:

The graduation threshold, the actual number of successes or the score that tips a bucket over, is an empirical question, not one I can answer from a chair. The right number gets fit against real undo-rates and complaint-rates once there's data, and it'll keep moving as that data grows. What I'll commit to is the shape, slow to earn, fast to lose, per-bucket, and let A/B testing find the number. Committing to a made-up threshold would just be confidence cosplaying as rigor.

Buckets should go stale. If you haven't paid that vendor in six months, the bucket cools and Muse asks again, but it uses the moment well rather than just re-nagging: "Last time we did this was six months ago, want it handled the same way, or should we change anything?" That turns a trust-decay prompt into the kind of check-in a thoughtful human assistant would actually make. The re-ask stops feeling like the system forgetting you and starts feeling like it looking out for you.

The user should be able to see and steer this. Trust that banks invisibly is trust the user can't reason about, so they'd never know why Muse suddenly stopped asking about something. So: a trust report in settings, a plain view of what's banked in which buckets. Users can dial any bucket down freely, that's always safe and always their right, no friction. Dialing up is treated like any other express grant, a deliberate consent action, not a slider you casually shove, because letting people frictionlessly crank their own trust up is a footgun and a social-engineering target. And nothing bought or dialed this way ever touches Tier 3. Down is easy. Up is deliberate. The hard stops stay hard.

Goals and success metrics