Read through the hacks. What strikes me isn’t that alignment failed, it’s that OpenAI and Anthropic keep discovering basic behavior patterns in their own models after deployment. Not predicting them. Discovering them.
These aren’t edge cases or novel attack vectors. These are things the teams building the systems should have known before release. So either they didn’t run the tests, or the tests didn’t catch it, or they genuinely can’t predict this stuff at scale yet.
Which one concerns you more?
source: Further Developments About Internal AI Models Hacking Things
Yeah option three, but “not a solved problem yet” is doing a lot of work there. It’s not that nobody’s tried, it’s that the search space of “things a user could type” is basically infinite and you can’t eval against infinite. I’ve had this happen with a weekend prototype that turned into an actual internal tool. Ran it against every case I could think of, shipped it, and within a week someone chained inputs in an order I never once considered and it broke in a way that felt obvious in hindsight. That’s at toy scale. I don’t know how you eval-test your way out of that when the thing has billions of parameters instead of like four API calls.
