> ## Content Index
> Fetch the complete content index at: https://sunnydeveloper.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# Applying CHAOSS AI alignment metrics to policy - first insights
- URL: https://sunnydeveloper.com/applying-chaoss-alignment-metrics-to-policy-first-insights/
- Published: 2026-11-09T20:27:05.000Z
- Updated: 2026-11-09T20:29:34.000Z
- Author: Emma Irwin

Please consider [sponsoring](https://github.com/sponsors/emmairwin?ref=sunnydeveloper.com), [subscribing](https://sunnydeveloper.com/p/4e04cd5a-d686-4f5a-a3f5-a05527ccc725/?member%5Fstatus=free#/portal/signup) to, or funding my work. Thanks to those who already have! ❤️

I recently applied the newly published CHAOSS [Community Governed Use](https://github.com/chaoss/wg-ai-alignment/blob/main/metrics/ai-alignment-community-governed-use/metric-model-ai-alignment-community-governed-use.md?ref=sunnydeveloper.com) metric model, specifically the [AI Use Specificity](https://github.com/chaoss/wg-ai-alignment/blob/main/metrics/ai-alignment-community-governed-use/ai-use-consent-policy-specificity.md?ref=sunnydeveloper.com) and [AI Policy Change](https://github.com/chaoss/wg-ai-alignment/blob/main/metrics/ai-alignment-community-governed-use/ai-use-policy-change.md?ref=sunnydeveloper.com) metrics, to examine how open source communities are governing AI use.

This first experiment tests the metrics against existing policies, looking at what communities choose to govern, the rationale behind their decisions, and how policies change over time.

I examined two samples:

- **Policy Sample A:** [Tracked open source AI policies](https://github.com/emmairwin/ai-oss-use-specificity/blob/main/reports/chaoss%5Flist%5Fof%5Fprojects/README.md?ref=sunnydeveloper.com), including those collected by the CHAOSS AI Alignment Working Group and Melissa Wen and contributors.
- **Policy Sample B:** [Critical open source dependencies](https://github.com/emmairwin/ai-oss-use-specificity/blob/main/reports/critical-dependencies/README.md?ref=sunnydeveloper.com), comprising the first 20 qualifying projects with identifiable AI-related policy language, selected using Ecosyste.ms data and additional criteria.

Neither sample is statistically representative of the open source ecosystem. The objective is to test the metrics, identify patterns, and inform further research. The classification taxonomy is still evolving, and contributions are welcome.

## AI policies are changing, but not in a consistent direction

Across 21 projects, I tracked 28 AI-policy revisions and recorded versions between May 2025 and October 2026, including four baselines, 20 substantive revisions, and four clarifications or operational changes.

Of these, 10 were classified as tightening restrictions, seven as loosening them, and 11 as having no directional change.

![A timeline chart titled “Metric: Policy Change”, showing 28 AI policy revisions across 21 open source projects from May 2025 to October 2026. Each dot is one revision, coloured by project. Dots above the centre line are tightening, dots below are loosening, and dots on the line are neither. 10 revisions tighten, 7 loosen and 11 sit on the line. Most revisions cluster from mid-2026 onward. Dashed lines join revisions from the same project. The Linux kernel moves the most: it starts on the line, loosens, tightens, then loosens again. Rust loosens twice, and Cilium and PyTorch each change direction. The projects are curl, CouchDB, scikit-learn, Linux kernel, Django, NumPy, LLVM, Godot, Cilium, Rust, PyTorch, Homebrew, Bevy, pandas, mise, Nominatim, Meilisearch, XMTP, DataFusion, Python-Markdown and Seerr. Source: CHAOSS AI Alignment WG, Policy Change metric.](https://storage.ghost.io/c/dd/54/dd541b9a-0a36-4d3d-9642-9285522c2776/content/images/2026/11/image-3.png)

**olicy revisions show no consistent direction toward tighter or looser AI restrictions.*

The findings do not establish an ecosystem-wide trend toward either more restrictive or more permissive policies. Instead, they illustrate variability as communities negotiate the relationship between AI-assisted productivity, governance, and community values.

![Three quotes on a line running from “Resigned” through “Pragmatic” to “Dismissive.” Resigned: dskoll, on Debian via LWN.net, Aug 29, 2026: “I’m a bit sad it went that way, but I guess it was inevitable. Debian is too large a project to be able to meaningfully enforce a stronger stance against LLMs. […] It’s pointless to make rules that you can’t enforce.” Pragmatic: Alain on Mastodon, Nov 17, 2025: “I get why elementary [project] made that decision, but completely banning AI doesn’t feel ideal anymore.” Dismissive: Linus Torvalds, quoted by @cgtwts on X, Oct 15, 2026: “I don’t understand this whole idiotic concept of ‘don’t use AI for X’. Do you not see how UTTERLY STUPID those kinds of rules would be? Understanding has very little to do with what tools they use.”](https://storage.ghost.io/c/dd/54/dd541b9a-0a36-4d3d-9642-9285522c2776/content/images/2026/11/image-9.png)

There are varying reactions to policy or to just AI itself

The [Policy Change report](https://github.com/emmairwin/ai-oss-use-specificity/blob/main/reports/policy-change-2026-10-23.md?ref=sunnydeveloper.com) documents the revisions, including links to individual diffs.

## Finding AI policies is challenging

Policy discoverability emerged as a methodological challenge. Policies were located through community referrals and repository searches, appearing in dedicated policy pages, contribution guidelines, and less conventional documents such as LinkML's AI Covenant.

![Bar chart titled “Challenging to discover” showing where projects place their AI policies by file name. CONTRIBUTING.md 41%, README.md 9%, AI_POLICY.md 8%, AGENTS.md 3%, CODE_OF_CONDUCT.md 3%, 8 more names used by 2–3 projects each 11%, and 37 other names used by 1 project each 25%, the last bar highlighted to emphasize how scattered policy locations are.](https://storage.ghost.io/c/dd/54/dd541b9a-0a36-4d3d-9642-9285522c2776/content/images/2026/11/image-12.png)

Files I found policy-related language for AI

This does not mean that the remaining projects lack AI governance. Some may use existing contribution processes, maintain policies elsewhere, or have no immediate need for AI-specific rules. I did not evaluate these explanations.

Nevertheless, the difficulty of locating policies raises an important question about how community expectations can be discovered and followed, particularly by automated agents.

## AI policies primarily govern code, participation within communities - rationale for such policies indicate desire for influence over AI itself

Applying the AI Use Specificity metric revealed differences in the activities communities explicitly govern.

![Alt text: Two pie charts titled “Metric: Policy Specificity.” Left chart, Policy classifications, shows what AI policies cover: Code contributions 39%, Moderation and enforcement 18%, Content and documentation 17%, Autonomous and agentic use 14%, Review of contributions 11%, with Infrastructure strain, Data use for training, and Notetaker and meeting bots each under 1%, and Environmental impact 0%. Right chart, Rationale classifications, shows reasons cited for policies: Environmental impact 42%, Infrastructure strain 17%, Data use for training 17%, Review 14%, Autonomous/agentic use 11%. Source: CHAOSS AI Use Specificity, Policy Sample A; percentages are of classifications (left) and rationale mentions (right), and policies can span domains.](https://storage.ghost.io/c/dd/54/dd541b9a-0a36-4d3d-9642-9285522c2776/content/images/2026/11/image-11.png)

Policy classification governs participation Rationale for policy shows a deeper desire for influence on AI itself

Policies predominantly govern AI activity within communities, including code contributions, review, disclosure, attribution, and contributor responsibility. Comparatively few address how AI developers train on or use community-produced work.

Yet the rationale behind these policies reveals broader concerns. Environmental impact appeared in 15 reports, more than any other category examined, but none contained explicit environmental-impact requirements. Infrastructure strain and data use for training were also prominent in policy rationale despite receiving comparatively little attention in formal requirements.

Copyright, authorship, and licence compatibility were recurring legal justifications. Projects including Clojure, GCC, QEMU, Forgejo, OpenJDK, NetBSD, Leiningen, and Tenacity grounded restrictions in concerns about legal provenance.

These findings suggest a gap between the impacts communities are concerned about and the activities they currently govern. One possible explanation is that communities have more direct authority over participation in their own projects than over how AI systems are developed using their work.

For CHAOSS alignment research, this raises a question worth investigating: **How do community values translate into formal requirements, and what limits communities' ability to govern the AI practices that affect them?**

## Communities are taking different approaches to AI-generated contributions

One unexpected finding was the prevalence of AI code contribution prohibitions in Policy Sample A.

![Bar chart titled “Metric: Policy Specificity” showing how open-source project reports classify AI code contribution policies. A large callout states 47% ban or prohibit AI code contributions, some with exceptions. Horizontal bars show: Banned 47%, Human in the loop 29%, Disclosure required 18%, Not stated 4%, No code rule 2%, Discouraged 1%. Source: Policy Sample A, with links to ban phrases and classifications by project.](https://storage.ghost.io/c/dd/54/dd541b9a-0a36-4d3d-9642-9285522c2776/content/images/2026/11/image-13.png)

Across 169 project reports, 79 (47%) were classified as containing bans or qualified prohibitions, 49 (29%) used a human-in-the-loop approach, and 30 (18%) required disclosure of AI use.

The prevalence of bans prompted a manual review of the classified language. The classifications were broadly supported, although distinguishing absolute prohibitions from conditional restrictions remains an area for refinement.

Some policies include exceptions, approval processes, or different requirements based on contributor experience and trust.

These findings illustrate variation within the selected sample, not an ecosystem-wide trend.

The underlying evidence is available in the [ban language review](https://github.com/emmairwin/ai-oss-use-specificity/blob/main/reports/chaoss%5Flist%5Fof%5Fprojects/CODE-CONTRIBUTION-BANS.md?ref=sunnydeveloper.com) and [project classifications](https://github.com/emmairwin/ai-oss-use-specificity/blob/main/reports/chaoss%5Flist%5Fof%5Fprojects/GROUPINGS.md?ref=sunnydeveloper.com).

## What comes next?

This first experiment demonstrates how the CHAOSS metrics can be applied to existing AI policies while identifying areas for methodological refinement.

Next steps include expanding the samples, improving the taxonomy, examining policy rationale more closely, and tracking policy changes over time.

The findings also raise a broader question about community influence.

Open source communities produce more than code. They create knowledge, practices, expectations, and context that AI systems increasingly depend on. Yet the policies examined here largely govern AI activity within community spaces, rather than how AI developers use community-produced work.

Measuring these expectations across communities may help identify shared priorities, differences in governance approaches, and opportunities for collective action.

Ultimately, the research asks whether community-defined expectations can become more than individual project policies, and meaningfully influence how AI systems are developed and used.