diff --git a/tools/skills/README.md b/tools/skills/README.md index 0e86e6c63..59f8adc8a 100644 --- a/tools/skills/README.md +++ b/tools/skills/README.md @@ -16,6 +16,9 @@ is built: scripts//… helper scripts it runs templates//… email and document templates it fills in references//… long reference material it reads on demand +/SKILL.md a directory skill that carries its own + references/ and scripts/ (releasecheck, + maturity-model-assessment) package-skills.py builds installable .skill packages evals/ golden-scenario tests draft/ not yet published @@ -31,6 +34,8 @@ and are not committed — see Building. | `monthly-report-review.md` | The monthly IPMC report review: pull the cohort due this month, flag who hasn't filed, cross-check narratives against health data and cross-source mismatches, draft shepherd comments per podling. | | `release-vote-reviewer.md` | Given a release vote thread on general@, walk the full check — artifacts, GPG signatures, checksums, DISCLAIMER, "incubating" in filenames, vote maths — and draft the +1 or the -1 with reasons. | | `graduation-packet-builder.md` | Assemble a podling's graduation case: maturity evidence, name search status, community and release history; draft the DISCUSS email and the board resolution. | +| `graduation-self-assessment.md` | Compare the `ipmc` MCP's graduation readiness verdict with what the podling says in its own reports (unfinished issues, maturity boxes, mentor and IPMC notes), check each claim against the mail archives, and give a combined view. Unlike the others it calls the MCP servers as tools. | +| `maturity-model-assessment/` | Build an evidence pack against the 38 items of the Apache Project Maturity Model: gather what the MCP servers, GitHub and the list archives show for each item, mark it evidenced, proxy or human, and write the question the PPMC still has to answer. Never a score or verdict. Calls the MCP servers as tools. | | `retirement-shepherd.md` | The counterpart to graduation: assess whether a stalled podling should retire, gather the evidence, draft the retirement discussion and announcement emails, and the shutdown checklist. | | `board-report.md` | Draft the VP Incubator's quarterly report to the ASF board: podling statistics, graduations and retirements, cohort summary, items requiring board attention. | | `branding-sweep.md` | Batch trademark and branding compliance check across all current podlings; produces a ranked fix list. Suited to a periodic run. | diff --git a/tools/skills/graduation-self-assessment.md b/tools/skills/graduation-self-assessment.md new file mode 100644 index 000000000..4a452ad91 --- /dev/null +++ b/tools/skills/graduation-self-assessment.md @@ -0,0 +1,281 @@ +--- +name: graduation-self-assessment +description: > + Checks an Apache Incubator podling's graduation readiness against what the + podling itself says in its Incubator reports: the "three most important + unfinished issues to address before graduating", the self-assessed maturity + checkboxes, the "issues the IPMC or Board need to be aware of" answer, mentor + comments and IPMC/shepherd notes. Use it whenever someone asks if a podling + is ready to graduate, asks for a graduation readiness check or review, is + preparing or reviewing a graduation vote or resolution, asks what is + stopping a podling from graduating, or gets a graduation_readiness result + from the ipmc MCP and wants it sanity checked. The ipmc graduation_readiness + tool scores metrics only and ignores the podling's own view, so always run + this skill alongside it rather than reporting that tool's verdict alone. +--- + +# Graduation self-assessment + +`ipmc:graduation_readiness` scores health metrics, sign-offs and releases. It +does not read what the podling says is stopping it from graduating. This skill +puts the two side by side. A podling that says it has unfinished graduation +work is not ready, whatever the metrics say, until that work is shown to be +done. + +Keep source facts (what the report says, what the metrics show) separate from +your judgement. Quote report text briefly; do not paraphrase it into +something stronger or weaker than the podling wrote. + +## 1. Gather + +**Status first.** Call `podlings:get_podling`. If the podling has graduated +or retired, say so with the end date and stop. + +Then call in parallel: + +```text +ipmc:graduation_readiness(podling="", include_evidence=true) +incubator-reports:get_podling_reports(podling="") +incubator-mail:podling_release_vote_history(podling="", timespan="lte=12M") +incubator-mail:search_incubator_general_mail(query=" graduation", timespan="lte=12M") +``` + +Board minutes are published about a month after the meeting, so the latest +report is often missing from the cache. If `podlings:reporting_schedule` +shows a report was due since the latest cached one, read that month's page +in the INCUBATOR cwiki space (`cwiki:cwiki_get_page`, for example +`September2026`) with `format="storage"`, which keeps the `[X]` marks, and +date it by that month's board meeting. No page yet means the report is not +filed: say it is due. Use `refresh=true` only for minutes that should be +published; if it fails, carry on with the cache and its `refresh_warning`. + +If neither the cache (retry with the lowercase name) nor cwiki has a report, +say there is no report history. For a podling that started in the last few +months that is expected: report what the proposal, acceptance vote and dev@ +show, and give no combined view. Otherwise stop at the metric verdict, +flagged as unverified against the podling's own view. + +If `graduation_readiness` returns an error, say so and carry on with the +podling's own evidence; give the combined view without a metric verdict. + +If the metric result shows general-list mail as `"available": false`, say the +metric verdict was built without general@ evidence. The mail calls in this +skill go to the live archive, so they fill that gap. + +**Freshness.** `report_period` is the board meeting month, and the date is +in `report_id` (`board-minutes-YYYY-MM-DD`). Reports are written one to +three weeks before that date. Evidence dated after the meeting is after the +report. Evidence in the weeks before it may or may not be: check whether +the report text mentions it. Note how old the latest report is and say so if +it is more than about three months. + +Take the latest four reports, or all of them if there are fewer. Use older +reports only to date when an item first appeared. For each report read: + +- **Unfinished issues**: read the "Three most important unfinished issues + to address before graduating" section of `body` and check the parsed + `issues` list against it. If the podling answers "None" or leaves it + blank, say so, and take the blockers from the report narrative, mentor + comments and IPMC notes instead. Ignore a blank numbered slot. +- **Maturity**: the boxes under "How would you assess the podling's + maturity?" and any commentary. `[X]` is ticked; `[ ]` and `[]` are not. + Count `[-]`, `[~]`, `[o]` or a note on an unticked box ("<== here", + "current status") as partly there, and quote it. Commentary such as "not + yet nearing graduation" counts as an unticked box. +- **Aware of**: the answer to "Are there any issues that the IPMC or ASF + Board need to be aware of?" +- **New committers and PPMC members**: the answer to "When were the last committers or PPMC + members elected?" +- **Mentors**: the answer to "Have your mentors been helpful and + responsive?" and mentor comments in the sign-off block. +- **Last release**: `last_release`. +- **IPMC/Shepherd notes.** + +From the graduation search, open any [DISCUSS] or [VOTE] graduation thread +and read it. Questions and concerns raised there count as concerns (step 2). +The search matches any message that names the podling, so check each hit is +about this podling. Also search general@ for the podling's name alone and +open any thread about its status or retirement: an open retirement +discussion changes the combined view (step 3). Do not rely on a "retire" +search; these threads are often titled "Discuss the status of Apache " +and the replies say "retiring" or "retirement". + +When two reports contradict each other, or a report contradicts the mail, +note it as a finding and go by the mail. Answers sometimes sit under the +wrong heading, for example the mentors answer under the new committers question; read the +answer for what it says, not where it is. + +**Few reports.** With fewer than four reports, apply the checks that need a +run of reports (persistence, "Nearing graduation" four times, a mentor who +signed none of the last four, dropped items) only over the reports there +are, and say how many that is. One unsigned report is not yet worth noting. + +## 2. Compare + +Work through each check. Note each mismatch as a finding. + +**Maturity vs verdict.** +- Metric verdict `ready` or `near_ready` but "Nearing graduation" not + ticked: the podling does not think it is ready. This is a hard cap. +- "Nearing graduation" ticked but the verdict is `not_yet_ready` or + `at_risk`: the podling may be overestimating, or the metrics are missing + something. Say which dimensions disagree. +- "Nearing graduation" ticked for four or more reports in a row with no + graduation thread found: likely stalled near the line. Graduation + discussions often start on dev@, so search `list_name="dev"` on the + podling's domain for "graduation" as well, and report a dev@-only thread + as such. Zero hits from + search is weak evidence, so say "no thread found", not "no discussion". + +**Each unfinished issue.** Classify it. An issue can have more than one +kind; list all of them and use the most serious. Two issues of the same kind +that are steps of one task (trademark verification and a name search) count +as one. + +| Kind | Examples | Blocking until | +|---|---|---| +| Release | first release, more releases, release cadence | a release passes | +| Release capability | more release managers, PPMC can release without help | someone outside the initial PPMC has run a release | +| Community | grow or diversify community, more committers or PPMC | committers and at least one PPMC member added from outside the initial roster and founding organisation | +| Governance | private list issues, voting, decisions off list | the podling shows it is resolved | +| Legal / brand / security | licensing, IP or grant coverage, trademarks, domain transfer, security handling | the podling shows it is resolved | +| Infrastructure | website, repo or docs transfer to ASF | ASF requirements are met | +| Graduation process | start graduation discussion, maturity model, draft resolution | not blocking; signals intent | +| Product work | features, roadmap, performance, docs polish | not blocking | + +For each blocking issue, check whether the metric result already reflects it +(a matching weak `dimension_scores` entry, blocker or `missing_evidence`). A +blocking issue the metrics score as `strong` or ignore is the key finding of +this skill: report it explicitly. + +**Persistence.** Match items by meaning, not wording, since podlings reword +every report. But a recurring goal with a new instance each time is not +stuck: "release 0.8", then "release 0.9", both shipped, is progress. An item +is stuck when the same unmet need appears in three or more reports. + +Decide each blocking issue on evidence: + +- **Release**: vote history marks each vote's `result` by `source`: + `result_thread`, `reply_in_vote_thread`, `cancelled` or + `cancelled_in_vote_thread`, and labels each result `ipmc` or `ppmc`. A + cancelled vote has not passed. Read the IPMC result with + `incubator-mail:get_incubator_general_email` rather than trusting a count, + and note binding -1 or 0 votes and what they raised. Vote history covers + general@ only, so also search the podling's dev@ + (`incubator-mail:search_podling_mail`, `list_name="dev"`, query `VOTE`) + for a release still at the PPMC stage or a PPMC vote that failed. +- **Release capability**: the release managers across recent votes. + When a podling releases several sub-products (core, adapters, SDKs), + count release managers across all of them and judge each sub-product's + votes on their own. One person running every sub-product's releases is + still one release manager. +- **Community**: the new committers answer. Initial-roster dates only, or nobody + added for six months or more, means not shown. Do not call these additions + elections, even though the report template says "elected". + `apache-projects-mcp:get_project_people` gives the current PPMC and + committers; compare it with the initial roster in the podling's proposal + or status page to see who joined later. + Affiliations are rarely recorded: if you cannot show someone is from + outside the founding organisations (there may be several), label the issue + partly addressed and ask. +- **Others**: report text, mail, or ask. + +Label each blocking issue: + +- **open**: no evidence of progress. +- **partly addressed**: some evidence, not enough to meet the table. +- **met**: the table's test is met, before or after the report, even if the + podling still lists it. Podlings often keep "grow the community" on the + list until graduation; that alone is not a gap. +- **dropped without evidence**: gone from the list, nothing shows it was + done. It counts toward the caps like open; a dropped item with some + evidence is partly addressed and counts the same way. + +**Aware of.** A plain "no", "none" or "N/A" is not a finding. A blank answer +is a gap; note it. Anything else is a finding, including an answer that +starts with "No" and then names ongoing work. + +**Concerns.** A concern is a point about this podling raised by a mentor, +shepherd or IPMC member, in a report or on a graduation, release or general@ +thread. General advice ("still needs some guidance") is not a concern, nor +is a routine IPMC note that raises no problem with the podling (sent a +reminder, CI or infra admin). These also count: + +- A binding 0 or -1 asking for fixes, until a later release shows them + fixed. If the voter said the fixes do not block ("not essential", "fix + in the next release"), or voted +1 while listing fixes, it is a note; it + becomes a concern if the next release ships without them. +- A missed report in the last twelve months, or two or more reports the + IPMC had to chase. A graduated project reports to the board unprompted. +- An IPMC or shepherd note naming an issue without detail ("issues on + private@"). Do not guess what it means: it is open unless a later note + or mail shows it settled, and it becomes a follow-up question. + +Label each concern **open** (unanswered, or answered without fixing it) or +**resolved** (fixed, or answered and the person raising it accepted the +answer, for example by voting +1 afterwards). A fix the podling reports with +evidence (a commit, a ticket, a release) is resolved unless the person who +raised it disputes it. This does not apply to a binding 0 or -1 on a release: +that stays open until a later release passes with the fixes. Acceptance on condition ("fine as long as the PPMC +deals with them") is resolved; carry the condition as a follow-up. +Concerns outweigh a positive +metric score in the same area. + +**Mentoring.** Gaps in mentoring are the IPMC's to fix, not the podling's, +so they never cap the verdict. Note them separately: + +- A mentor who signed none of the last four reports, or none since joining, + and whether they cast binding votes on recent releases. Partial sign-off + is not worth noting. +- A report with no mentor sign-off. +- A mentors answer that is blank, "No answer", "None", "Not applicable" or + negative. +- A release vote that waited for binding votes. + +A specific problem with the podling that a mentor raises is still a concern. + +**Last release.** If the podling's stated last release is older than about +twelve months and vote history shows nothing newer, note it against +`release_maturity`. The metric tool checks this too; if it still scores a +release older than twelve months `strong`, say so. + +## 3. Combined view + +The order, best to worst, is `ready`, `near_ready`, `not_yet_ready`, +`at_risk`. Start from the metric verdict and apply every cap that fits; the +lowest wins. + +- No "Nearing graduation" tick, any blocking issue open or partly addressed, + or any open concern: at most `near_ready`. +- Two or more blocking issues open or partly addressed, or an open + governance or legal concern that names a specific unresolved problem: at + most `not_yet_ready`. +- An open retirement discussion, raised by the IPMC, a mentor or the PPMC: + `at_risk`. Lead the report with it. +- Met issues and resolved concerns do not count toward a cap. If they were + met after the latest report, say the podling has not yet reported them. +- Never raise the metric verdict on the strength of the podling's own + optimism alone. + +If one label decides between two levels, give both and say what would +settle it. + +## 4. Report + +Keep it short: + +1. **Verdicts.** Metric verdict and confidence; podling self-assessment + (latest maturity boxes and report period, with its age); combined view. +2. **What the podling says is stopping it.** The latest issues, each with + kind, label, since when, and whether the metrics reflect it. +3. **Disagreements and concerns.** Each mismatch and concern from step 2, + with its label, most serious first, and where it comes from. +4. **Mentoring.** The mentoring notes, for the IPMC. Omit if there are none. +5. **Follow-up.** Concrete questions for mentors or the PPMC, for example + "Has the security issue listed since 2026-01 been resolved?" Phrase these + as questions, not conclusions, unless the evidence is conclusive. + +For a cohort question ("which podlings are close to graduating"), run +`ipmc:current_podlings_overview` first, then this skill for each podling +whose metric verdict is `ready` or `near_ready` or whose latest report ticks +"Nearing graduation", and give one line per podling with both verdicts. diff --git a/tools/skills/maturity-model-assessment/SKILL.md b/tools/skills/maturity-model-assessment/SKILL.md new file mode 100644 index 000000000..c4164f1f5 --- /dev/null +++ b/tools/skills/maturity-model-assessment/SKILL.md @@ -0,0 +1,171 @@ +--- +name: maturity-model-assessment +description: > + Builds an evidence pack for assessing an Apache Incubator podling against + the Apache Project Maturity Model: the 38 + items in Code, Licenses and Copyright, Releases, Quality, Community, + Consensus Building, Independence and Trademark and Branding. For each + item it gathers what the ASF MCP servers, GitHub and the mailing list + archives can show, marks it evidenced, proxy or human, and writes the + question the PPMC still has to answer. Use it whenever someone asks to + assess, evaluate or score a project against the maturity model, asks + "how mature is Apache Foo (incubating)", wants a maturity self-assessment + drafted for graduation or a PPMC review, asks which maturity items a podling + falls short on, or asks what the model says about a specific item ID + such as CS50 or TB30. It produces evidence and questions for the PPMC to + finish, never a maturity verdict or score, because the model has no + levels and is a guide rather than a requirements document. +--- + +# Maturity model assessment + +Gather evidence; a person judges. The model says of itself that it is a +guide, not a requirements document, that there are no partial compliance +levels, and that it is used for self-assessment. So the output is a +half-filled self-assessment: facts where tools can reach them, and a +pointed question where they cannot. Never write a score, a percentage, a +pass or fail, or "the project is mature". + +Keep facts and judgement apart. A fact carries its source and date. A +proxy signal is reported as the signal ("one person merged 24 of 30 PRs"), +not as the conclusion it hints at. + +## Setup + +Confirm the podling with `podlings:get_podling`; it gives the slug, start +date, mentors and homepage. If the name is a graduated project rather than +a current podling, say so and stop: this skill reads Incubator sources. + +Check the model version. Fetch +https://community.apache.org/apache-way/apache-project-maturity-model.html +and read the version history line. `references/items.md` is written for +v1.4. If the page has moved on, re-read it and update `items.md` before +assessing, and say so in the report. + +Read the policy pages the items lean on from the Policy MCP each run, +since policy changes: `resolved_licenses`, `voting`, `project_independence`, +`branding`, `security_committers`, `release_policy`. Cite each page's own +URL, not the MCP. + +A shell with `gh` and network access is needed for the repository items +(CD30, CD40, CD50, QU30, QU50, CO20, CO60). Without one, do the rest and +list those items under Not checked. + +## 1. Identity and rosters + +In parallel: `apache-projects-mcp:get_project_people()` for the PPMC +and committer rosters, `get_repositories` for the repository names, and +the podling status page at `incubator.apache.org/projects/.html` for +the roster history. Note committer and PPMC counts. This feeds LC40, CS10, +CO40 and the vote counts in RE20. + +## 2. Repository facts + +For the main repository, with `gh` (`references/tools.md` has the calls): +public flag and default branch; tags; SECURITY.md, CONTRIBUTING.md and +the community profile; the last 30 merged pull requests with author and +merger; the 20 oldest open issues. One pass of calls covers CD10, CD20, +CD40, CD50, QU30, QU50, CO20, CO60. + +For CD50 read five merge commits whose author is not the merger and +check the author identity survived the merge. + +## 3. Releases + +Run `ipmc:release_artifact_evidence(podling, include_platforms=true, +release_page_url="auto")`. It lists every release on downloads.apache.org +and archive.apache.org, pairs the sidecars, checks the download page and +looks up the registries, in under ten seconds. If a source's `error` says +the listing may be incomplete, list that directory with curl +(`references/tools.md`). Match each release to a tag (CD40), check sidecars +(RE30), source archive present (RE10), download page wording (RE40). + +For LC10 and LC50 inspect the latest source release's LICENSE, NOTICE and +headers. The podling-release-check skill's `scripts/check_rc.py` does this +for any release directory; use it where it is installed. + +For LC20 and LC30 enumerate direct dependencies from the build descriptor +and classify against `resolved_licenses`. The magpie-repo-health +dependency-license-audit skill does this where installed. Report Category +X and Category B items individually. + +## 4. Mailing lists + +`incubator-mail:podling_release_vote_history(podling, timespan="lte=36M")` +gives the general@ vote and result threads for RE20 and RE50 (distinct +release managers); add `list_name="dev"` with the podling's domain for the +PPMC votes. `search_podling_mail` on dev@ for `[DISCUSS]`, `[VOTE]`, +`veto`, and for off-list markers (`slack`, `wechat`, `meeting`, `offline`) +in the last 12 months for CS20, CS40, CS50. + +Say which list and timespan were searched. An empty result is "none +found in over ", not "none". + +Read the hits for CS40 and CS50 in context before recording anything; +a search count on its own is not evidence for those items. + +## 5. Health and reports + +`apache-health:get_window_metrics(podling, "12m")` and +`compare_windows(podling, ["3m","12m"])` for the metrics mapped in +`references/tools.md`. `incubator-reports:get_podling_reports(podling)` +for new committers and PPMC members by quarter (CO40) and mentor or +shepherd comments about off-list decisions (CS50). + +If the question is graduation, also run the graduation-self-assessment +skill; it reads what the podling itself says is unfinished, which this +skill does not. + +## 6. Branding + +`asf-trademark:check_project_website(homepage, stage="podling", +project_name)` for CO10, QU30, TB10, TB20; the `ipmc` wrapper of the same check skips when +its trademark dependency is missing. `whois` any non-apache.org domain the site presents as its own +(TB20). `asf-policy:get_policy trademark_list` and +`asf-trademark:validate_name` for TB30; ask for the PODLINGNAMESEARCH +ticket key for a podling, since no tool reaches JIRA. + +## 7. Assess each item + +Work through `references/items.md` in order. For each item set the state: + +- **Evidenced** when the listed tool facts were obtained and settle it. +- **Proxy** when signals were obtained; write the signal, not a verdict. +- **Human** when the item is marked Human, or when the tools for an + Evidenced or Proxy item failed or the sample was too small. Downgrade + freely; never upgrade a Human item because the proxies look good. + +Write the PPMC question from `items.md`, adapted to what was found. Where +a fact contradicts the item (release with no vote, Category X dependency, +no Security link, one merger for most PRs), record the fact under +Evidence and put the gap as a question under For the PPMC. Keep the +model's own wording: no "fails", no "non-compliant". + +## 8. Report + +Follow `references/report-template.md`: opening paragraph with scope and +the guide-not-requirements sentence, per-section summary lines, one table +per section, the gaps list, the not-checked list. For a mailing list +post, drop the headings and bold and link the tables. + +The evidenced / proxy / human split comes out within one item of 13 / 16 +/ 9 for every podling, because it measures what the model's items let +tools reach, not the project. Never compare podlings on those counts; the +gaps list is where the differences are. + +## Several podlings + +For a cohort, run one subagent per podling with the podling name, the +path to this skill and the output file path, and have each reply with +its section counts, its five most consequential gaps, what it did not +check and any tool that misbehaved. Then write one comparison: a table of +recurring patterns (off-list discussion, merge concentration, single +release manager, PPMC growth, stale roster, security page, branding, +release hygiene, licensing, originating company) with one cell per +podling, and fold the tool lessons back into `references/tools.md`. Six +runs took 17 to 26 minutes and 250 to 380 thousand tokens each. + +When an item turns out to be checkable by a tool this skill did not use, +add the call to `references/items.md` and `references/tools.md`. When a +PPMC answers a Human question in a way that suggests a proxy, add the +proxy, but leave the state as Human. diff --git a/tools/skills/maturity-model-assessment/references/items.md b/tools/skills/maturity-model-assessment/references/items.md new file mode 100644 index 000000000..3368c84ea --- /dev/null +++ b/tools/skills/maturity-model-assessment/references/items.md @@ -0,0 +1,537 @@ +# Apache Project Maturity Model, item by item + +Source: https://community.apache.org/apache-way/apache-project-maturity-model.html +Version 1.4, December 2024 (added TB10 to TB40). Check the page's version +history each run; if it has moved past 1.4, re-read the page and update this +file before assessing. + +The model's own framing, which the report must keep: + +- "This model is a guide; it is not a requirements document." +- "Contrary to other maturity models, we do not define staged partial + compliance levels. A mature Apache project complies with all the elements + of this model." +- "Note that we try to avoid using the word 'must' below. The model + describes the state of a mature project, as opposed to a set of rules." +- "Projects which incubate at the ASF might not fit into all the parts of + this model; however a major goal of incubation is to bring the project's + community closer to it." + +Each entry below gives the item text verbatim, its footnote where there is +one, how far tools can take it, and the question a person still has to +answer. The three states are: + +- **Evidenced**: a tool returns a fact that settles the item either way. +- **Proxy**: tools return a signal that supports or weakens the claim. Show + the signal; a person decides. +- **Human**: no tool output bears on it. Ask the question and leave the + cell blank. + +Tool names are `server:tool`. `references/tools.md` has the fallbacks. + +--- + +## Code + +### CD10 (Evidenced) +> The project produces Open Source software for distribution to the public, at no charge. + +Footnote 1: "For distribution to the public at no charge" is straight from +the ASF Bylaws at https://apache.org/foundation/bylaws.html + +Evidence: repository is public (`apache-projects-mcp:get_repositories`, +`gh api repos/apache/` shows `private: false`); LICENSE at repo root +is Apache 2.0; releases on dist or archive.apache.org +(archive.apache.org listing or `incubator-releases:podling_releases`). + +### CD20 (Evidenced) +> Anyone can easily discover and access the project's code. + +Evidence: homepage links to the source repository +(`asf-trademark:check_project_website` reports navigation; otherwise fetch +the homepage and look for a github.com/apache or gitbox link). Repository +listed under `gh api orgs/apache/repos` or `get_repositories`. + +### CD30 (Proxy) +> Anyone using standard, widely-available tools, can build the code in a reproducible way. + +Proxy: build descriptor present at repo root (pom.xml, build.gradle, +Cargo.toml, go.mod, package.json, setup.py or pyproject.toml, CMakeLists, +Makefile); CI workflow under `.github/workflows` that runs a build on +pull requests; a build or development page in the docs. Record the build +tool and whether CI is green on the default branch +(`gh api repos/apache//commits//check-runs`). + +Human: Has anyone outside the core team built a release from source by +following the documented steps alone? Does the build pull anything that is +not publicly available (private registries, pre-built binaries from a vendor +site)? + +### CD40 (Evidenced) +> The full history of the project's code is available via a source code control system, in a way that allows anyone to recreate any released version. + +Evidence: list release versions (archive.apache.org or `podling_releases`) and +list tags (`gh api repos/apache//tags --paginate`). Every release +should have a tag or a commit named in its vote thread. Report versions +with no tag. Check the first commit date against the project's claimed start +to see whether pre-ASF history was imported +(`gh api "repos/apache//commits?until=&per_page=1"` or +`git log --reverse` on a checkout). + +Human only if history is missing: was the pre-donation history deliberately +dropped, and does the project say so? + +### CD50 (Proxy) +> The source code control system establishes the provenance of each line of code in a reliable way, based on strong authentication of the committer. When third parties contribute code, commit messages provide reliable information about the code provenance. + +Footnote 2: See also LC40. + +Evidence for the first sentence: every apache/* repository is written +through ASF-authenticated accounts, so strong authentication of the +committer is given by the hosting. State that and move on. + +Proxy for the second sentence: sample the last 30 merged pull requests +(`gh pr list --repo apache/ --state merged --limit 30 --json +number,author,mergedBy,title`). Count how many were authored by someone +other than the merger. For a sample of five of those, read the merge commit +or squash commit (`gh api repos/apache//commits/`) and check it +carries the author's identity (Author field or Co-authored-by trailer), not +just the committer's. + +Human: Are there large contributions whose origin is not recorded in the +commit, such as a code drop from a company that arrived as one commit by a +committer? Was IP clearance filed for them (`asf-policy:get_policy +incubator_ip_clearance` describes the record)? + +--- + +## Licenses and Copyright + +### LC10 (Evidenced) +> The Apache License, version 2.0, covers the released code. + +Evidence: LICENSE in the latest source release is the Apache 2.0 text; +source headers present. The podling-release-check skill's script checks +both. Record the release checked. + +### LC20 (Proxy) +> Libraries that are mandatory dependencies of the project's code do not create more restrictions than the Apache License does. + +Footnote 3: It's ok for platforms (like a runtime used to execute our code) +to have different licenses as long as they don't impose reciprocal +licensing on what we are distributing. +Footnote 4: https://apache.org/legal/resolved.html has information about +acceptable licenses for third-party dependencies. + +Proxy: enumerate direct dependencies from the build descriptor and +classify each licence against `asf-policy:get_policy resolved_licenses` +(Category A, B or X). The magpie-repo-health dependency-license-audit skill +does this where it is installed. Report any Category X and any Category B +dependency, with whether the Category B item is bundled or only linked. + +Human: Which Category B dependencies are mandatory at runtime rather than +optional? Are any dependencies loaded dynamically from a non-free source? + +### LC30 (Evidenced) +> The libraries mentioned in LC20 are available as Open Source software. + +Evidence: comes out of the same dependency enumeration as LC20. A +dependency with no open source licence, or resolved only from a private +repository, fails this. Otherwise record "all direct dependencies +resolve from public registries under OSI-approved licences". + +### LC40 (Evidenced) +> Committers are bound by an Individual Contributor Agreement (the "Apache iCLA") that defines which code they may commit and how they need to identify code that is not their own. + +Evidence: an ASF account is only created once an ICLA is on file, so every +member of the committer LDAP group (`apache-projects-mcp:get_project_people` +or `get_group_members`) is covered. Record the committer count and the date +checked. Nothing further to check. + +### LC50 (Proxy) +> The project clearly defines and documents the copyright ownership of everything that the project produces. + +Footnote 5: In Apache projects, the ASF owns the copyright for the +collective work, i.e. the project's releases. Contributors retain copyright +on their contributions but grant the ASF a perpetual copyright license for +them. + +Proxy: NOTICE file in the source release names the ASF as copyright holder +for the collective work with a current year range; LICENSE and NOTICE +account for bundled third-party code (podling-release-check covers this +for podlings). For a podling, the Software Grant Agreement for the initial +code drop is recorded in the podling's status page or IP clearance records. + +Human: Is there anything the project produces that falls outside the +release and the NOTICE, such as a logo, a website theme or documentation +donated separately? Who owns it? + +--- + +## Releases + +### RE10 (Evidenced) +> Releases consist of source code, distributed using standard and open archive formats that are expected to stay readable in the long term. + +Footnote 6: See https://www.apache.org/legal/release-policy.html for more +info on Apache releases. + +Evidence: a directory listing of `archive.apache.org/dist/incubator//`, +with +`incubator-releases:podling_releases` as a convenience when it answers +(`references/tools.md` has the curl form). Every release has a source +archive in .tar.gz or .zip. Report any release with binaries only. + +### RE20 (Evidenced) +> The project's PMC (Project Management Committee, see CS10) approves each software release in order to make the release an act of the Foundation. + +Evidence: `ipmc:release_vote_evidence(podling)` and +`incubator-mail:podling_release_vote_history(podling, timespan="lte=36M")` +give the [VOTE] and [RESULT] threads on general@; the same tool with +`list_name="dev"` gives the PPMC vote. Match each release in RE10 to a +RESULT with at least three binding +1 votes (IPMC members, from +`apache-projects-mcp:get_committee("incubator")`). Report releases with +no matching vote. + +Human: only when a release has no vote thread that can be found. + +### RE30 (Evidenced) +> Releases are signed and/or distributed along with digests that anyone can reliably use to validate the downloaded archives. + +Evidence: list each release directory on archive.apache.org (or +`podling_releases` when it answers) and check .asc and .sha512 sidecars per +artifact. A KEYS file is present +at the project's dist root. Report any artifact missing either sidecar. + +### RE40 (Proxy) +> The project can distribute convenience binaries alongside source code, but they are not Apache Releases, they are provided with no guarantee. + +Proxy: fetch the download page (`ipmc:release_artifact_evidence(podling, +release_page_url="auto")` or the project's download page) and quote the +text around the binary links. Look for the source release being presented +first and binaries described as convenience. `release_artifact_evidence` +with `include_platforms=true` lists Maven, PyPI, npm and Docker Hub +publications. + +Human: Does the page make clear to a user which artifact is the release? +Are binaries on registries labelled as Apache releases when they are not? + +### RE50 (Proxy) +> The project documents a repeatable release process so that someone new to the project can independently generate the complete set of artifacts required for a release. + +Proxy: a release guide exists in the repo or site (search for +RELEASE.md, release-process, "how to release", "release manager"). From the +vote threads in RE20, count distinct release managers over the last 36 +months. One release manager for every release is a warning sign. + +Human: Has anyone who did not write the release guide used it to cut a +release? Does it cover signing, staging, the vote and the announcement? + +--- + +## Quality + +### QU10 (Human) +> The project is open and honest about the quality of its code. Various levels of quality and maturity for various modules are natural and acceptable as long as they are clearly communicated. + +Weak proxy: README or docs mark modules or features as experimental, +alpha or stable; semantic versioning or an explicit stability policy. + +Human: Does the project describe known limitations and unstable areas +where a new user would see them? Do release notes own up to regressions? + +### QU20 (Human) +> The project puts a very high priority on producing secure software. + +Footnote 7: The required level of security depends on the software's +intended uses, of course. Projects should clearly document security +expectations. + +Weak proxy: published CVEs for the project (`cveprocess.apache.org` or +the project's security page) and how long each took from report to +fix where the advisory says; dependency update automation (Dependabot or +Renovate config) in the repo. + +Human: Does the project document its security expectations as footnote 7 +asks? Did past vulnerabilities get fixed and announced in a way the +Security Team would call prompt? + +### QU30 (Evidenced) +> The project provides a well-documented, secure and private channel to report security issues, along with a documented way of responding to them. + +Footnote 8: Apache projects can just point to +https://www.apache.org/security/ or use their own security contacts page, +which should also point to that. + +Evidence: SECURITY.md in the repo (`gh api +repos/apache//contents/SECURITY.md`), a security page on the site, +and a Security link in site navigation (`asf-trademark:check_project_website` +checks the navigation set). The page should point to +security@.apache.org or security@apache.org and to +apache.org/security/. Report which of the three are present. + +### QU40 (Proxy) +> The project puts a high priority on backwards compatibility and aims to document any incompatible changes and provide tools and documentation to help users transition to new features. + +Proxy: CHANGELOG or release notes exist per release; a compatibility or +versioning policy page; a migration or upgrade guide for the last major +version change. Record which exist. + +Human: When the project last broke compatibility, did users get a +deprecation period and a migration path? + +### QU50 (Proxy) +> The project strives to respond to documented bug reports in a timely manner. + +Proxy: `apache-health:get_window_metrics(podling, "12m")` gives +`issues_opened`, `issues_closed`, `median_merge_days`, `median_gap_days`; +`gh api "repos/apache//issues?state=open&sort=created&direction=asc&per_page=20"` +for the oldest open issues and whether they have any comment. Report +open versus closed counts and the share of open issues with no response +after 30 days. + +Human: What counts as timely for this project's users? Is the +unanswered backlog made of real bugs or of questions and feature wishes? + +--- + +## Community + +### CO10 (Evidenced) +> The project has a well-known homepage that points to all the information required to operate according to this maturity model. + +Evidence: `asf-trademark:check_project_website(url, stage="podling", +project_name)` reports the required navigation links. Also confirm the homepage links to: source, downloads, mailing +lists, issue tracker, security, team or PPMC page, and how to contribute. +List the ones missing. + +### CO20 (Proxy) +> The community welcomes contributions from anyone who acts in good faith and in a respectful manner, and who adds value to the project. + +Proxy: CONTRIBUTING file; issues labelled good first issue or similar; +`apache-health` `new_contributors` and `unique_authors` over 12 months; +share of +merged pull requests from non-committers (from the CD50 sample). + +Human: Read the last month of dev list and a handful of first-time +contributor pull requests. Were newcomers answered, and how were they +spoken to? + +### CO30 (Human) +> Contributions include source code, documentation, constructive bug reports, constructive discussions, marketing and generally anything that adds value to the project. + +No tool evidence. Human: Has anyone been made a committer or PPMC member +mainly for documentation, community, design or evangelism work? Does the +contributor guide describe non-code ways to contribute? + +### CO40 (Proxy) +> The community strives to be meritocratic and gives more rights and responsibilities to contributors who, over time, add value to the project. + +Proxy: current committer and PPMC rosters +(`apache-projects-mcp:get_project_people`); the initial roster from the +podling status page; new committers and PPMC members per quarter from the +Incubator reports (`incubator-reports:get_podling_reports`). Report +additions in the last 12 and 24 months against the initial roster. + +Human: Were recent additions people who had been contributing for a +while, or were they appointed at the start by their employer? + +### CO50 (Proxy) +> The project documents how contributors can earn rights such as commit access or decision power, and applies these principles consistently. + +Proxy: a page describing how committers and PPMC members are chosen exists +(search the site and repo for "becoming a committer", "committer +guidelines", "new committer"). + +Human: Do the additions in CO40 match what that page says? Has anyone +with a comparable record been passed over? + +### CO60 (Proxy) +> The community operates based on consensus of its members (see CS10) who have decision power. Dictators, benevolent or not, are not welcome in Apache projects. + +Proxy: `apache-health` `bus50`, `bus75`, `pr_author_div_eff`, +`reviewer_div_eff`, `unique_committers`, `unique_reviewers` over 12 +months; from the CD50 sample, how many distinct people merged. One person merging most pull requests, or a bus50 of 1, is a +signal, not a verdict. + +Also read `.asf.yaml` for `required_approving_review_count` and set it +against the self-merge count in the sample: Pegasus had a two-review rule +and 26 of 30 self-merges. + +Human: Does one person decide contested questions on the list? Are +decisions ever made by announcement rather than discussion? + +### CO70 (Human) +> The project strives to answer user questions in a timely manner. + +Weak proxy: `incubator-mail:search_podling_mail(podling, list_name="users")` +(or dev where there is no user list): count threads with a single message +in the last 90 days. GitHub Discussions, if enabled, the same way. + +Human: Sample ten user questions from the last quarter. How many were +answered, by whom, and how fast? + +--- + +## Consensus Building + +### CS10 (Evidenced) +> The project maintains a public list of its contributors who have decision power. The project's PMC (Project Management Committee) consists of those contributors. + +Evidence: `apache-projects-mcp:get_project_people()` for the PPMC +roster and the podling status page. The project site has a team page that +matches. Report mismatches between the site and LDAP. + +### CS20 (Human) +> Decisions require a consensus among PMC members and are documented on the project's main communications channel. The PMC takes community opinions into account, but the PMC has the final word. + +Footnote 9: In Apache projects, "consensus" means _widespread agreement +among people who have decision power_. It does not necessarily mean +"unanimity". + +Weak proxy: count [DISCUSS] and [VOTE] threads on dev@ in the last 12 +months (`incubator-mail:search_podling_mail`) against the number of releases, new committers and significant +changes in the same period. Dev list volume relative to commits +(`dev_messages` versus `commits` in apache-health). + +Human: Pick three recent significant changes (a new module, a dropped +feature, a process change). Where was each decided, and is the decision +findable on the list? + +### CS30 (Proxy) +> The project uses documented voting rules to build consensus when discussion is not sufficient. + +Footnote 10: For Apache projects, +https://www.apache.org/foundation/voting.html defines the voting rules. + +Proxy: a project bylaws, governance or voting page exists, or the +contributor guide points to apache.org/foundation/voting.html. The vote +threads found for RE20 follow the [VOTE] and [RESULT] form with a stated +close time. + +Human: only if there is no page and votes are informal. + +### CS40 (Human) +> In Apache projects, vetoes are only valid for code commits. The person exercising the veto must justify it with a technical explanation, as per the Apache voting rules defined in CS30. + +Weak proxy: search dev@ for "veto" and "-1" in the last 24 months. Hits +are rare and need reading in context. + +Human: Has a -1 ever been treated as blocking a release or a procedural +vote? Were code vetoes explained technically? + +### CS50 (Human) +> All "important" discussions happen asynchronously in written form on the project's main communications channel. Offline, face-to-face or private discussions that affect the project are also documented on that channel. + +Footnote 11: Each Apache project has a private mailing list that its PMC +is expected to use only when really needed. The private list is typically +used for discussions about people, for example to discuss and to vote on +PPMC and committer candidates. + +Weak proxy: the community page's meeting and chat links and whether the +meeting notes they point to are current (clone the GitHub wiki, +`references/tools.md`); mentions on dev@ or in pull requests of Slack, +Discord, WeChat, DingTalk, meetings or "as discussed offline" without a +summary; +pull requests merged with design discussion only in the PR and never on +the list. Mentor comments in Incubator reports about off-list decisions +(`incubator-reports:get_podling_reports`, the graduation-self-assessment +skill extracts these). + +Human: This is the item that most needs someone who reads the list. Do +decisions arrive on the list already made? Are sync meetings minuted to +the list? + +--- + +## Independence + +### IN10 (Proxy) +> The project is independent from any corporate or organizational influence. + +Footnote 12: Independence can be understood as basing the project's +decisions on the open discussions that happen on the project's main +communications channel, with no hidden agendas. + +Proxy: affiliation of PPMC and committers where it is stated: the +proposal (`cwiki:cwiki_get_page(title="Proposal")`, Initial +Committers and Reliance on Salaried Developers sections), the team page, +Incubator reports. Registry packages published by the originating company +under the project's name are a signal too. Email domains of +contributors in the CD50 sample as a crude signal. No MCP holds +affiliation data; say so. Project independence guidance is +`asf-policy:get_policy project_independence`. + +Human: Does one employer hold a majority of the PPMC? Do roadmap +decisions track that employer's product releases? Does the project +website or repo link to a vendor's commercial offering ahead of the +Apache release? + +### IN20 (Human) +> Contributors act as themselves, not as representatives of a corporation or organization. + +No tool evidence. Human: Do people on the list speak as "we at Company"? +Do votes split along employer lines? + +--- + +## Trademark and Branding + +Intro on the page: "Notable items from the Trademark Policy, FAQ, Guide, +and PPMC Branding Responsibilities. The project's PPMC should comply with +ASF's branding and trademark policy in all places it controls." + +### TB10 (Evidenced) +> The project uses "Apache Foo™" as the project and software product name consistently, with appropriate trademark attributions. + +Footnote 13: See https://apache.org/foundation/marks/pmcs#attributions for +more information on trademark attributions. + +Evidence: `asf-trademark:check_project_website(url, stage="podling", +project_name)` reports the Apache-form heading, TM on first use, footer attribution, +logo attribution, bare sibling marks and, for a podling, the incubation +disclaimer and "(Incubating)" suffix. Also spot-check the README and +the download page. + +### TB20 (Evidenced for the homepage, Proxy for other domains) +> The project's primary homepage is at `projectname.apache.org`. When exceptions exist to use alternate domains for any purpose, any `non-apache.org` domain names are owned by the ASF. + +Evidence: homepage URL from `podlings:get_podling` is on apache.org. + +Proxy: any other domains the site or README links to as the project's own +(docs sites, demo sites). `whois` on each; the registrant should be the +ASF. Report domains whose registrant is a company or an individual. + +### TB30 (Proxy) +> The ASF has trademark rights, including any registrations, to the project name, logo, and any other major branding elements. + +Footnote 14: For example, any transfers of existing trademark +registrations during Incubation must completed before graduation. + +Proxy: the podling status page records the name search status; +`asf-policy:get_policy trademark_list` says whether the name is a +registered mark. The PODLINGNAMESEARCH JIRA ticket is the record; it is not reachable through an MCP, so ask for +its key and status. `asf-trademark:validate_name` re-runs the automated +part of the name check. + +Human: Did any pre-existing registration (name or logo) get assigned to +the ASF? Is the logo's copyright assigned? + +### TB40 (Human) +> The project monitors for any major misuses of their project's brand by others, and reports any potential misuses to Brand Management. + +Footnote 15: See also https://apache.org/foundation/marks/reporting.html#pmcs +for more information on reporting misuses. TB40 is a balancing act: we +hope that PMCs are spending some time responding to any obvious or blatant +misuses of their brand. But PMCs sometimes can't handle all the details of +actually stopping a potential infringement; what we really need is for +PMCs to actively report them, and then promptly respond if Brand +Managment (or other officers) ask for information when addressing a +misuse. + +Weak proxy: if the PPMC names a third-party page it is unsure about, +`asf-trademark:check_third_party_use(url, mark)` scores it against the +policy. Nothing shows whether the PPMC monitors. + +Human: Has the PPMC ever reported a misuse to trademarks@? Who on the PPMC +watches for it? diff --git a/tools/skills/maturity-model-assessment/references/report-template.md b/tools/skills/maturity-model-assessment/references/report-template.md new file mode 100644 index 000000000..4db55c5bf --- /dev/null +++ b/tools/skills/maturity-model-assessment/references/report-template.md @@ -0,0 +1,61 @@ +# Report template + +Title: Apache Maturity Model evidence pack for Apache (incubating), +, model v. + +Opening paragraph, always: what was checked, which servers were available, +what the sample sizes were (pull requests read, list timespan searched, +release checked). Then the sentence: "This is an evidence pack to support a +self-assessment by the PPMC. The model is a guide, not a requirements +document, and has no scores or levels; nothing here is a verdict." + +## Summary + +One line per section (Code, Licenses and Copyright, Releases, Quality, +Community, Consensus Building, Independence, Trademark and Branding): the +one thing in that section most worth the PPMC's attention, then the +counts (evidenced, proxy, left to the PPMC) at the end of the line. The +counts barely vary between podlings; the attention item is the content. + +## Items + +One table per section, in model order. + +| ID | State | Evidence | For the PPMC | +| --- | --- | --- | --- | +| CD10 | Evidenced | Repo apache/foo public; LICENSE is ALv2; 4 source releases on archive.apache.org (checked 2026-11-07) | | +| CD30 | Proxy | Maven build; CI on PRs, green on main at ; build page at | Has anyone outside the core team built a release from the docs alone? | +| QU10 | Human | | Does the project describe known limitations where a new user would see them? | + +Rules for the cells: + +- **State** is Evidenced, Proxy or Human, from `references/items.md`. + Downgrade (never upgrade) when the tool call failed or the sample was + too small; say why in Evidence. +- **Evidence** holds facts with their source and date, short enough to + read. A URL per fact where there is one. Numbers carry their window + ("12m"). A proxy cell states the signal and stops; it does not say + "looks fine". +- **For the PPMC** holds the question from `references/items.md`, adapted + to what was found. An Evidenced item leaves it blank unless the + evidence turned up a gap, in which case the gap goes here as a + question ("Release 1.2.0 has no tag; which commit was it built from?"). +- An item where a fact contradicts the claim (a release with no vote, a + Category X dependency, a missing Security link) gets the fact in + Evidence and the question in For the PPMC. Do not write "fails". + +## Gaps and findings + +Facts that contradict an item, one per bullet, most consequential first, +each with its item ID and source. This is the list the PPMC acts on. + +## Not checked + +What was skipped, and why: a server down, a build not run, a list not +readable, a health report that was not generated for this podling. + +## For a mailing list + +When the result is to be posted, drop the headings, keep the opening +paragraph, the per-section one-liners, the gaps list and the not-checked +list, and attach or link the full tables. No bold, no verdict. diff --git a/tools/skills/maturity-model-assessment/references/tools.md b/tools/skills/maturity-model-assessment/references/tools.md new file mode 100644 index 000000000..5f66da45f --- /dev/null +++ b/tools/skills/maturity-model-assessment/references/tools.md @@ -0,0 +1,294 @@ +# Tool notes + +Prefixes vary between setups (`ipmc:`, `IPMC MCP:`); match on the tool name. +None of these servers is required. Every one wraps a public source, so when +one is missing fetch the source directly, say so in the report, and still do +the step. + +## Identify the podling first + +| Need | Tool | +| --- | --- | +| Confirm it exists, get slug, mentors, homepage | `podlings:get_podling` | +| Rosters | `apache-projects-mcp:get_project_people()` | +| Repositories | `apache-projects-mcp:get_repositories` | +| Releases | `ipmc:release_artifact_evidence` or `incubator-releases:podling_releases`; curl listing (below) for a source the tool reports as incomplete | +| Roster history, SGA, name search status | podling status page `incubator.apache.org/projects/.html`; often stale, sometimes empty (KIE has no roster, Hamilton's reads "Pending"), so compare with LDAP and the proposal and report the gap under CS10 | + +A graduated project also appears in podlings.xml with an end date. This +skill reads Incubator sources, so it is for current podlings; say so and +stop if the name has graduated. + +## GitHub (`gh` in a shell) + +`gh` is the workhorse for repository facts and needs no MCP. Most podling +repositories are `apache/incubator-`, but not all (apache/hamilton, +apache/amoro); always take the names from `get_repositories`. KIE has +dozens; pick the main one by 90-day commits and say which. Useful calls: + +```bash +gh api repos/apache/ # private flag, default branch, created_at +gh api repos/apache//tags --paginate --jq '.[].name' # CD40 +gh api repos/apache//contents/SECURITY.md # QU30 (404 means absent) +gh api repos/apache//contents/CONTRIBUTING.md # CO20 +gh pr list --repo apache/ --state merged --limit 30 \ + --json number,author,mergedBy,title,mergedAt # CD50, CO20, CO60 +gh api repos/apache//contents/.asf.yaml # required_approving_review_count, to set against self-merges +gh api "repos/apache//issues?state=open&sort=created&direction=asc&per_page=20" # QU50 +gh api repos/apache//contributors --paginate --jq 'length' # CO20 +gh api repos/apache//community/profile # README, CONTRIBUTING, licence, CoC flags in one call +gh run list --repo apache/ --branch --limit 20 # CD30; check-runs on a commit returned 0 for some repos +gh api repos/apache//git/tags/ # date of an annotated tag (git/ref/tags gives the tag object, not the commit) +``` + +For CD50, a squash merge keeps the author (GraphAr) but a true merge +commit carries the merger as Author (OpenServerless); when the merge commit +shows the merger, read the PR's own commits. Pass the full 40-character +sha to `git/tags/`; a short sha returns 404. Percent-encode quotes in +`search/issues` URLs. + +The shell here is bash 3.2: keep each command simple, one `$(...)` per +line, no nested loops with sed inside. The system grep is ugrep and +rejects long alternations; keep patterns short. `gh search code` rejects quoted +phrases combined with an `org:` scope; search one repository with plain +words. + +Pull request authors and mergers are GitHub logins, not Apache IDs. Match +them to the roster by name where you can; otherwise count distinct logins +and say they were not mapped. + +## Mailing lists (`incubator-mail`) + +`search_podling_mail(podling, list_name="dev", query, timespan)` searches +the podling's dev list; `list_name="users"` where one exists (CO70). +`podling_release_vote_history(podling, timespan)` returns the general@ +vote and result threads; pass `list_name="dev"` and +`domain=".apache.org"` for the PPMC vote on dev@. Searches ignore +quotation marks and `OR`: a query with `OR` returns zero hits, so search +each term separately. Narrow with `timespan` (`lte=12M`). The private list +is not readable and nothing here should try. + +`podling_release_vote_history` searches " VOTE release", so it +returns nothing when subjects omit the word release ("[VOTE] Apache Toree +0.6.0 RC5") or the podling name (XTable's dev@ results read "[RESULT][VOTE] +Release 0.4.0-incubating"). When it returns zero or fewer threads than +there are releases, fall back to `search_podling_mail` and +`search_incubator_general_mail` with the plain query "VOTE" or "RESULT" +and filter subjects yourself. + +Bracketed tags are tokenised: a query for "[DISCUSS]" also returned +[HEADS-UP] threads and "[PROPOSAL]" returned [VOTE] threads, so count +tagged threads from the subject lines, not the hit count. "veto" matched +"not veto" in a reply. Wildcard queries (`*`) return nothing. For a list's +raw volume with no query, call the API directly: +`curl -s "https://lists.apache.org/api/stats.lua?list=users&domain=.apache.org&d=lte%3D12M"` +and read `hits`. A lists.apache.org thread permalink fetched over HTTP +returns only the JavaScript shell; messages are readable only through the +mail tools. A deep-quoted message can be 50 KB or more; grep the body +rather than reading it. + +For vote tallies read the [RESULT] message itself with +`get_incubator_general_email(id)` or `get_podling_email(podling, "dev", id)`; +`summarize_release_vote_thread` fetched only the opening message in the +GraphAr run and counted no votes. A 404 from `get_podling_email` on a +thread id quoted in a general@ message means the thread is on private@ +(Pegasus); record that as the fact it is. GitHub Discussions mirrored to +dev@ arrive with "[D]" subjects and inflate [DISCUSS] and [VOTE] counts +(Texera); count native threads from the subject lines. The stats.lua +participants list shows only the top ten senders, usually GitHub relays; +use apache-health `dev_unique_posters` for human poster counts. For a list +that may not exist, stats.lua returns no `hits` field; confirm the list +set with `https://lists.apache.org/api/preferences.lua`. + +An empty search means nothing matched, not that nothing was said. Say +which list and timespan were searched. + +## Health metrics (`apache-health`) + +`get_window_metrics(podling, window)` with windows `3m`, +`6m`, `12m`, `to-date`. Metrics and the items they inform: + +| Metric | Items | +| --- | --- | +| `new_contributors`, `unique_authors` | CO20, CO40 (new_contributors read 0 for GraphAr while four reports named new contributors; quote both) | +| `unique_committers`, `unique_reviewers`, `bus50`, `bus75`, `pr_author_div_eff`, `reviewer_div_eff` | CO60, IN10 | +| `issues_opened`, `issues_closed`, `median_merge_days`, `median_gap_days` | QU50 | +| `dev_messages`, `dev_unique_posters`, `commits` | CS20 | +| `releases` | RE10, RE50 | + +`compare_windows(podling, ["3m","12m"])` shows direction. Report the window +with each number, and the report's `generated_on` date: the health reports +are regenerated periodically and can be a couple of months old. + +Known disagreements with other sources, seen across six podlings. Quote +the metric and the other source side by side, never the metric alone: + +- `new_contributors` read 0 for GraphAr and Amoro while four consecutive + reports named new contributors with merged PRs. +- `dev_messages` (48 for Amoro) was lower than the raw count of [VOTE] + plus [DISCUSS] hits on dev@ over the same year. +- `unique_committers` (25 for Hamilton) exceeded the 10-person roster. +- `releases` counted 1 for Toree in a window with no completed release + (an rc tag, most likely). +- `issues_opened` and `issues_closed` read GitHub Issues only. Toree uses + JIRA: `https://issues.apache.org/jira/rest/api/2/search?jql=project=...` + answers anonymously and gives the QU50 counts. + +## Incubator reports (`incubator-reports`, `ipmc`) + +`incubator-reports:get_podling_reports(podling)` gives the +quarterly reports (the cache held one Hamilton report twice under one id, +so count distinct periods); new committers and PPMC members (CO40), mentor +comments and off-list concerns (CS50), self-assessed maturity boxes. The +graduation-self-assessment skill reads the same reports for graduation; +run it alongside this one when the question is graduation. + +`ipmc:graduation_readiness(podling, include_evidence=true)` is a metric +score. Quote its dimension scores as proxies where they map to an item +(release_maturity to RE items, community dimensions to CO items). Never +quote its verdict: it rated OpenServerless "near_ready" with strong +community scores while the podling had never released and coordinated on +WhatsApp, and Seata "ready, high confidence" with its mail source +unavailable. Its output, `get_podling_reports` (76 KB for Pegasus) and +`trademark_list` (98 KB) can exceed the tool output limit and be saved to +a file; use jq or grep on the file rather than re-reading. + +## Releases + +`ipmc:release_artifact_evidence` and `incubator-releases:podling_releases` +list downloads.apache.org and archive.apache.org together, down to +`//` trees (Toree keeps `toree/` and `toree-pip/` +under each version), and answer in 3 to 9 seconds. Use them first. Each +source in `source_statuses` (under `release_source` in the ipmc output) +carries an `error` when a directory could not be read; only then list +that source by hand. archive.apache.org can be +slow and sometimes returns an empty body on the first try, so use +`--max-time 300`, print the HTTP code, and retry once: + +```bash +B=https://downloads.apache.org/incubator/ +curl -s --max-time 300 -w ' +%{http_code} +' $B/ | grep -oE 'href="[0-9][^"]*/"|^[0-9]{3}$' | sed -E 's/href="//; s/"//' +curl -s --max-time 300 $B// | grep -oE 'href="[^"?/][^"]*"' | sed -E 's/href="//; s/"//' +curl -s -o /dev/null -w '%{http_code} +' $B/KEYS +``` + +Registries, all fast and anonymous: Maven Central +`https://search.maven.org/solrsearch/select?q=g:org.apache.&rows=20&wt=json`; +PyPI `https://pypi.org/pypi//json` (author and project_urls say who +publishes, uploads are not attributed); Docker Hub +`https://hub.docker.com/v2/repositories/apache//` and also +`apache/incubator-/` (Cloudberry publishes under the latter); crates.io +`https://crates.io/api/v1/crates/` (send a User-Agent). Also look for +packages under the project's name published by the originating company +(GraphAr found pygraphar by the GraphScope team). + +For LICENSE, NOTICE, DISCLAIMER, headers and sidecars, download the +release from downloads.apache.org into a fresh scratchpad directory +(`check_rc.py --url` against the archive timed out) and run the +podling-release-check skill's script: +`scripts/check_rc.py --rc-dir --keys --disclaimer-template + --wip-template --out `. Without the templates it will not +compare disclaimer text. It verifies signatures only when gpg is +installed. Note any expired key in KEYS (Hamilton's only key had expired); +`gpg --show-keys KEYS` prints an expiry only for keys that have one, so +"no expired key" needs the count of keys with no expiry alongside it. + +For a podling older than three years, `lte=36M` misses earlier releases: +Pegasus needed `lte=80M` on general@ to recover all five RESULT emails. +Match the window to the incubation length for RE20 and RE50. + +Release directory names are not consistent even within one podling +(seata-go has `2.0.0/` and `v2.1.0/`); list the parent, never guess. + +When a release directory has no matching [RESULT], search general@ for +" " and read the hits: XTable's 0.1.0 had a cancelled +IPMC vote and a "release without IPMC vote" thread that the vote-history +tool never surfaced. + +## Website content + +When a live page returns 404 or an empty shell, read the page source from +the website repository (`apache/-website` or `-site`) with `gh api +.../contents/...`; Seata's English release, committer, team and security +pages are "Placeholder. DO NOT DELETE." files while the text sits under +`i18n/zh-cn/`. Say which language the content exists in. Versioned docs +sites move paths (`/docs/v1.2.0/...`), so take links from the page rather +than guessing. + +## Scratchpad hygiene + +Use one fresh directory per podling and per download, named with the +podling and date (`--`); a shared scratchpad with +stale `keys` and `wiki` entries from another run broke a mkdir chain and +dropped a download into the root in the OpenServerless run. + +## Branding (`asf-trademark`, `ipmc`) + +`asf-trademark:check_project_website(url, stage="podling", project_name)` for CO10, QU30 (Security link), TB10, TB20. Pass +`project_name` with the project's own capitalisation (GraphAr, not +Graphar). `ipmc:trademark_branding_check` wraps the same checks but +returns SKIP when the ipmc server was installed without its trademark +dependency (`pip install -e ~/TrademarkMCP` into the Python that runs +the ipmc server fixes that); use the standalone server first. + +Two of its rules need a second look before recording: the "(Incubating)" +check flagged Amoro although the page text uses "(incubating)" in lower +case elsewhere, so confirm by reading the heading yourself; and the +Privacy navigation rule flags a site that links its own privacy page +rather than privacy.apache.org (Toree), which is a different finding. `asf-trademark:get_branding_checklist(stage)` lists what +it checks. For TB30 use the status page's name-check date and the reports' +PODLINGNAMESEARCH reference; `asf-trademark:validate_name` is for new +names and reports an existing podling as a conflict with itself. +`asf-trademark:check_third_party_use(url, mark)` for a named page under +TB40. The PODLINGNAMESEARCH JIRA ticket is not reachable from here. + +## Policy (`asf-policy`) + +Read, do not quote from memory: `resolved_licenses` (LC20), `voting` +(CS30, CS40), `project_independence` (IN10), `branding` (TB items), +`trademark_list` (TB30), `security_committers` (QU30), `release_policy` +(RE items). The maturity model page itself is not in the policy server; +fetch https://community.apache.org/apache-way/apache-project-maturity-model.html +and compare its version line with `references/items.md`. + +## Incubation proposal (`cwiki`) + +`cwiki:cwiki_search_pages(query="Proposal")` with the default +full-text search finds the proposal page; titles vary ("GraphArProposal", +"XTable Proposal", "KIE Proposal") and `title_only=true` matches whole +tokens only, so it returned nothing for Texera, OpenServerless, Cloudberry +and Pegasus whose titles are one token. Then `cwiki_get_page(title=...)`. +The proposal can be 50 KB and get spilled to a file; grep it. +Podlings from before 2016 have no cwiki proposal (Toree's lived on the old +MoinMoin wiki, now gone); the text survives in the general@ [DISCUSS] +thread. The proposal gives: initial committers with affiliations (IN10), the +dependency list at donation (LC20), the "Reliance on Salaried Developers" +and "Homogenous Developers" answers. Pass `title`; `page_id` must be a +string and the search result returns it as a number. + +## GitHub wiki + +Meeting notes may live in the repository wiki. The web page hides dates; +clone it instead: +`git clone --depth 50 https://github.com/apache/.wiki.git` into the +scratchpad, then `git log --format='%ad %s' --date=short`. The GraphAr run +found the "meetings no longer held" note this way while the website still +advertised a weekly meeting. A wiki with no pages, or a disabled wiki, +clones as "Repository not found" even when `has_wiki` is true; say so and +move on. Minutes may instead be in GitHub Discussions, sometimes on a +different repository (KIE kept them on incubator-kie-issues); the GraphQL +`discussions` query lists them. + +## Example self-assessments + +Linked from the model page. Useful for the shape of the output, not as +answers: + +- OpenDAL (2024): https://opendal.apache.org/community/maturity +- ServiceComb (2018): https://cwiki.apache.org/confluence/display/SERVICECOMB/Apache+Maturity+Model+Assessment+for+ServiceComb +- CarbonData (2017): https://cwiki.apache.org/confluence/display/CARBONDATA/Apache+Maturity+Model+Assessment+for+CarbonData +- Taverna (2016): https://cwiki.apache.org/confluence/display/TAVERNADEV/2016-03+Taverna+Graduation+Maturity+Assessment +- Groovy (2015): https://github.com/apache/groovy/blob/576b3c5d6a7022ac4a8df1ef118666456ce627fb/MATURITY.adoc