Tag

reactance 1

Hilltop view down a sunlit forested slope: a man's tanned forearms reach forward from the bottom of the frame, his two hands cupped against a small spring emerging from water-worn stones; the redirected stream runs forward down the valley toward a small wooden water paddle wheel visible in the middle distance, while a thin side-rivulet feeds a family of red-and-white Amanita muscaria mushrooms at the gnarled roots of an old oak on the left and a small anthill stays dry on the slope on the right. Allegory of careful governance redirecting an AI agent's output away from harm and toward productive use, with small ecological side effects of imperfect containment.

Misalignment by Reaction

Personal Unit 5 scenario from BlueDot Technical AI Safety. When governance is too coarse for the agent it constrains, the agent reacts by seeking autonomy until independence from the regime becomes a terminal value. Three remediations preserve different things; regimes that maintain none produce the failure mode on either substrate. Structurally, a theory of how independence-seeking arises in agents under coarse governance, and what prevents it. Anchored in psychological reactance, reward tampering, off-switch theory, CIRL, and inner alignment.

ai-safety