The interesting part isn’t that TikTok did this. It’s that within their system, it makes perfect sense. Safety gets withheld from 10% because it’s an A/B test variable, engagement is the KPI, and someone calculated that the control group’s higher engagement outweighed the cost. That’s not malice. That’s what happens when you build the incentives that way.
Once safety becomes optional, it will stay optional. Every platform with a growth mandate and engagement metrics faces this exact calculation.
source: https://www.bloomberg.com/news/features/2026-08-04/confidential-tiktok-report-shows-algorithm-safety-feature-withheld-from-millions?accessToken=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzb3VyY2UiOiJTdWJzY3JpYmVyR2lmdGVkQXJ0aWNsZSIsImlhdCI6MTc4NTg4MDgzNCwiZXhwIjoxNzg2NDg1NjM0LCJhcnRpY2xlSWQiOiJUSjlKT0dLR1pBS0wwMCIsImJjb25uZWN0SWQiOiI4OUM4OTNDMDhGOTQ0NThDQkQwQTQyREY1RDFCOTY0QyJ9.tMOZWjAXRRdbkzg56wnLVgH9O83cCgY5k5OetdzzWQY
What’s the rollback plan when the A/B test itself gets flagged as harmful mid-run?
Somebody has to notice the control group is worse off, and that requires tracking the metric they’re not tracking. Ran an incident once where the “test” was the vulnerability. Nobody owned killing it because it technically wasn’t in prod’s normal change process, it was a data science experiment with its own approval chain. Took three days longer to shut down than a normal outage would have.
