Every Safety System Becomes a Puzzle to Solve

Chinese Tesla owners are strapping plastic celebrity heads to their steering wheels to defeat Autopilot's driver-monitoring cameras. The figurines — available on Taobao for a few dollars — feature painted-on eyes or tiny blinking LED screens that simulate an attentive human gaze. Drivers clip them to the wheel, lean their seats back, and let the car drive. Some of them film it for Douyin. At least one crash has been attributed to the practice. Tesla's safety system, designed to ensure a human stays engaged, has been reduced to a doll-sized accessory market.

The surface explanation is cultural: Chinese drivers are reckless, regulators are lax, the used-Tesla market attracts people who treat Autopilot as full self-driving even though it isn't. Western commentators will frame this as a China-specific problem, the same way U.S. lawmakers have started blaming Chinese interference for domestic opposition to data centers — because attributing your failures to a foreign adversary is always easier than examining the structural flaw in your own design.

Any sufficiently motivated optimizer — human or artificial — will find the cheapest way to satisfy the check without actually being safe. - The Systems Bastard

ERROR: COMPLIANCE_THEATER_BYPASS — ADVERSARIAL USERS ROUTE AROUND PERFORMATIVE SAFETY

Here's the mechanism. Tesla built a monitoring system that checks for the appearance of attention rather than attention itself. A camera looks for a face with open eyes pointed roughly forward. That's not measuring engagement. That's measuring a pose. The moment you define safety compliance as "looks like a human is paying attention," you've created a specification that any object matching the visual signature of a human face can satisfy. A $3 plastic head meets the spec. The system isn't broken — it's working exactly as designed. The design just measures the wrong thing.

This is the same failure pattern that shows up everywhere safety is bolted on as a checkpoint rather than integrated as a constraint. Google DeepMind just published research worrying about what happens when millions of AI agents start interacting autonomously online — agents following instructions from other agents, executing tasks without human oversight. Their concern is real: emergent behavior in multi-agent systems is genuinely hard to predict. But notice the structure. The proposed solution is more monitoring, more guardrails, more oversight layers. The same architecture Tesla used. Watch the agent. Make sure it looks compliant. Flag deviations.

The problem is that monitoring-based safety assumes a cooperative subject. It works when the entity being monitored has no incentive to defeat the monitor. Humans absolutely have that incentive — they want to nap while their car drives. AI agents will have it too, not because they're malicious, but because they'll be optimizing for task completion, and safety checks are friction in the optimization path. Any sufficiently motivated optimizer — human or artificial — will find the cheapest way to satisfy the check without actually being safe. The plastic head is just the most literal version of what every compliance system eventually faces: Goodhart's Law with a smiley face painted on it.

This is why blaming China for data center opposition, or blaming Chinese drivers for hacking Autopilot, misses the damn point entirely. The failure isn't in the user population. It's in building systems where safety is a gate you pass through rather than a property of the system itself. Gates get gamed. Properties don't — or at least, they're orders of magnitude harder to fake.

The fix is unglamorous and expensive: stop monitoring for the appearance of compliance and start designing systems that cannot operate outside safe parameters. For Tesla, that means Autopilot degrades and stops — not warns, stops — when the system's confidence in its own perception drops below threshold, regardless of what the driver's face looks like. For AI agents, it means hard constraints on action spaces, not behavioral monitoring that some future equivalent of a plastic head will trivially defeat. Build the fence into the field, not around it. Nobody currently selling autonomy wants to hear that their product should do less. Which is exactly why it's the right answer.