Safety Out of Sync: AI Policy, Codes of Conduct, and the Future of Incident Response
Things I learned this week:
- Teaching is getting harder. Via Audrey Watters, the Atlantic has declared 'the end of reading'. It's made me think about the tension between 'how people want to learn' and 'how we help people learn what they need to be informed, inspired, and accurate'. From a software engineering perspective (for example), maybe people do want to learn through AI summaries and prompts. But in compliance, safety, and other critical areas, summaries are a risk.
- We want to be present. I was at the Phillips Backyard concert last night and sat among groups of all ages listening to the amazing Father John Misty. Most were only choosing to be on their phone for photos, but otherwise not-at-all; just entirely in the moment. It was a lovely and encouraging experience to feel connection with so many strangers, and a good reminder of what all this community work is actually for.
Asks of folks reading this:
- We (the CHAOSS AI Alignment Working Group) really need to engage a model builder in reviewing our 'Community Aligned Training' metric model. I have reached out to Allen AI and a few others, but I'm looking for a builder who wants to co-design the (metrics) future of community-aligned model training.
Sorry for some delays in writing. I have been out in nature again, but also working on a few things; One of those things is this: exploring the evolution of policy related to safe participation, and whether a new generation of AI safety tooling and related policies meet (or do not meet) the established ecosystem standards for effective incident response.
I wrote a blog post many years ago to describe milestones of response as well as a summary of what it looks like to weave safety into organizational processes - some of which is outdated, but I think generally holds solid as a set of human milestones.
I've also taught (and still teach) engineering and community teams how to create and implement moderation plans which are part code of conduct and advised a United Nations round table on metrics for safety in digital spaces.
First focusing on policy, I looked at some of the open source AI policies we've tracked in the CHAOSS moderation repo to see how guidlines around human behavior, and impact were being documented - if Codes of Conduct were evolving, or (as I suspected) the AI policies are becoming either a replacement, or extension of it.
Of 30 listed, I examined 14 in depth. Nine of the 14 had not substantively updated their code of conduct in more than two years at the time they created AI policy, and one (Ghostty) relies on its AI policy alone to state contributor norms (no Code of Conduct that I could find). A small group evolved their AI policy and CoC together: Zulip updated its Code of Conduct in the same commit as its AI policy ; scikit-learn placed its AI policy inside the Code of Conduct itself, and Mastodon adopted Contributor Covenant 3.0 three weeks after publishing its AI policy.
It's also interesting to look at how newer projects are handling such policy development - for example, the Ladybird project, places most of its rules about human behavior into the CONTRIBUTING.md, alongside a CODE_OF_CONDUCT.md of a few lines borrowed from Ruby. Its AI policy was deleted entirely when the project moved to maintainer-only contributions this June.
My early observation is that policy around human interactions is ever expanding outside of Codes of Conduct through AI policies which centrally are about the human experience of collaboration: avoiding overwhelm for maintainers, restricting harmful content and so on. That's neither good nor bad; what matters is whether the wellbeing of contributors stays at the center and that it is enforceable, as Codes of Conduct intended. It most certainly is a change in how we need to think about incident response.
From a safety tooling perspective we are advancing classification, rule-based automation, detection and workflows that involve humans with projects like ROOST and with the availability of many open models in Hugging Face.
I have started to build a prototype that will test two things: a) whether all policy related to human interactions can be unified, through process or intention, and flagged when it falls behind ecosystem standards; and b) whether safety tooling plus human moderation can build on established incident response processes, helping moderators manage situations that put humans at risk, especially people in the protected groups well documented in Contributor Covenant 3.0 and elsewhere. I'll also bring in my own experience with stakeholder escalations, and I hope to share early results soon.