Beware the siren song of AI self-governance - alignment requires community ownership

It’s very hard right now to hear voices of dissent in the economy of souls that is AI in this moment. There is so much, so fast, and so much money and power beckoning us forward. Yet, a significant challenge for humanity is coming closer to the light: AI alignment has not been solved.

Perhaps more importantly, a recognition is emerging that AI systems are being shaped around the goals and incentives of the companies building them, not the people whose creations have been extracted, whose livelihoods are being disrupted, or whose families have been harmed, including in Tumbler Ridge.

Alignment cannot happen when the answer to 'aligned with whom' is - tech companies business goals.

“It's sort of an open secret that we don't really have a good solution to the alignment problem right now.” — Daniel Kokotajlo, AI safety researcher and former OpenAI employee. (Two years ago.)

And then yesterday, from Donald Trump, standing with some of the most powerful AI executives in the world declared an agreement around self-policing for the largest AI players in the world.

“I think I'm seeing tremendous self-policing. And they understand that they have to self-police.” - CTV News article

This is the siren song I want us to challenge: the companies shaping the problem also get to define and solve it for us.

It doesn't always sound like self-policing. It sounds like responsible AI principles, "human-centered AI policies", safety frameworks, alignment programs and guardrails. It sounds responsible. Some of it may even be useful.

But if the same companies building and profiting from these systems also define what harm is, what acceptable risk looks like, what alignment means, and whether their interventions worked, that isn't accountability. It's self-governance without community and user alignment - and that's why we're in this position.

AI companies have known about these risks. They have built anyway. They have deployed anyway. We are living with the harms now.

When I started the CHAOSS AI Alignment Group for OSS, I did so because I believe grassroots communities are a mechanism for change and that alignment with these humans is critical. It was discouraging how often alignment with communities has dismissed as unimportant; how many times I have been dismissed as 'not understanding alignment as a ML engineering problem and nothing else'.

Now that bad things are starting to happen - we are finally hearing about alignment in exactly these human terms: whose values, whose interests, whose harms, and who gets to decide.

We cannot outsource those decisions to the corporations building and profiting from AI.

To that end, I am going to be focusing more on this type of effort; I may or may not write as much in the next little while as it feels like its more of a 'rolling up your sleeves' moment - and I feel like raising the alarm is now less important than working on grassroots efforts to counter.

Note: We have at least one metric model over in CHAOSS AI Alignment Working Group for what human RLF would look like beyond corporate walls. Maybe we can get to that soon too.