From b962e6e8eec5473219025f8089dae9dfd02dfb09 Mon Sep 17 00:00:00 2001 From: Justin Mclean Date: Sun, 8 Nov 2026 20:41:23 +1100 Subject: [PATCH] Describe tool quirks without naming podlings The tool notes used results from single runs on named podlings as examples. They go stale, so the notes now describe each quirk on its own. --- .../references/items.md | 4 +- .../references/tools.md | 122 ++++++++---------- 2 files changed, 58 insertions(+), 68 deletions(-) diff --git a/tools/skills/maturity-model-assessment/references/items.md b/tools/skills/maturity-model-assessment/references/items.md index 3368c84ea..cd14001ae 100644 --- a/tools/skills/maturity-model-assessment/references/items.md +++ b/tools/skills/maturity-model-assessment/references/items.md @@ -355,8 +355,8 @@ months; from the CD50 sample, how many distinct people merged. One person mergin signal, not a verdict. Also read `.asf.yaml` for `required_approving_review_count` and set it -against the self-merge count in the sample: Pegasus had a two-review rule -and 26 of 30 self-merges. +against the self-merge count in the sample: a two-review rule alongside +mostly self-merges is a finding. Human: Does one person decide contested questions on the list? Are decisions ever made by announcement rather than discussion? diff --git a/tools/skills/maturity-model-assessment/references/tools.md b/tools/skills/maturity-model-assessment/references/tools.md index 5f66da45f..6d63fef76 100644 --- a/tools/skills/maturity-model-assessment/references/tools.md +++ b/tools/skills/maturity-model-assessment/references/tools.md @@ -13,7 +13,7 @@ the step. | Rosters | `apache-projects-mcp:get_project_people()` | | Repositories | `apache-projects-mcp:get_repositories` | | Releases | `ipmc:release_artifact_evidence` or `incubator-releases:podling_releases`; curl listing (below) for a source the tool reports as incomplete | -| Roster history, SGA, name search status | podling status page `incubator.apache.org/projects/.html`; often stale, sometimes empty (KIE has no roster, Hamilton's reads "Pending"), so compare with LDAP and the proposal and report the gap under CS10 | +| Roster history, SGA, name search status | podling status page `incubator.apache.org/projects/.html`; often stale, sometimes empty (no roster, or a roster that reads "Pending"), so compare with LDAP and the proposal and report the gap under CS10 | A graduated project also appears in podlings.xml with an end date. This skill reads Incubator sources, so it is for current podlings; say so and @@ -22,9 +22,9 @@ stop if the name has graduated. ## GitHub (`gh` in a shell) `gh` is the workhorse for repository facts and needs no MCP. Most podling -repositories are `apache/incubator-`, but not all (apache/hamilton, -apache/amoro); always take the names from `get_repositories`. KIE has -dozens; pick the main one by 90-day commits and say which. Useful calls: +repositories are `apache/incubator-`, but some are `apache/`; +always take the names from `get_repositories`. A podling can have dozens; +pick the main one by 90-day commits and say which. Useful calls: ```bash gh api repos/apache/ # private flag, default branch, created_at @@ -41,8 +41,8 @@ gh run list --repo apache/ --branch --limit 20 # CD30; check-r gh api repos/apache//git/tags/ # date of an annotated tag (git/ref/tags gives the tag object, not the commit) ``` -For CD50, a squash merge keeps the author (GraphAr) but a true merge -commit carries the merger as Author (OpenServerless); when the merge commit +For CD50, a squash merge keeps the author but a true merge commit +carries the merger as Author; when the merge commit shows the merger, read the PR's own commits. Pass the full 40-character sha to `git/tags/`; a short sha returns 404. Percent-encode quotes in `search/issues` URLs. @@ -69,8 +69,8 @@ each term separately. Narrow with `timespan` (`lte=12M`). The private list is not readable and nothing here should try. `podling_release_vote_history` searches " VOTE release", so it -returns nothing when subjects omit the word release ("[VOTE] Apache Toree -0.6.0 RC5") or the podling name (XTable's dev@ results read "[RESULT][VOTE] +returns nothing when subjects omit the word release ("[VOTE] Apache +0.6.0 RC5") or the podling name (a dev@ result reading "[RESULT][VOTE] Release 0.4.0-incubating"). When it returns zero or fewer threads than there are releases, fall back to `search_podling_mail` and `search_incubator_general_mail` with the plain query "VOTE" or "RESULT" @@ -89,12 +89,11 @@ rather than reading it. For vote tallies read the [RESULT] message itself with `get_incubator_general_email(id)` or `get_podling_email(podling, "dev", id)`; -`summarize_release_vote_thread` fetched only the opening message in the -GraphAr run and counted no votes. A 404 from `get_podling_email` on a -thread id quoted in a general@ message means the thread is on private@ -(Pegasus); record that as the fact it is. GitHub Discussions mirrored to -dev@ arrive with "[D]" subjects and inflate [DISCUSS] and [VOTE] counts -(Texera); count native threads from the subject lines. The stats.lua +`summarize_release_vote_thread` can fetch only the opening message and +count no votes. A 404 from `get_podling_email` on a thread id quoted in a +general@ message means the thread is on private@; record that as the fact +it is. GitHub Discussions mirrored to dev@ arrive with "[D]" subjects and +inflate [DISCUSS] and [VOTE] counts; count native threads from the subject lines. The stats.lua participants list shows only the top ten senders, usually GitHub relays; use apache-health `dev_unique_posters` for human poster counts. For a list that may not exist, stats.lua returns no `hits` field; confirm the list @@ -110,7 +109,7 @@ which list and timespan were searched. | Metric | Items | | --- | --- | -| `new_contributors`, `unique_authors` | CO20, CO40 (new_contributors read 0 for GraphAr while four reports named new contributors; quote both) | +| `new_contributors`, `unique_authors` | CO20, CO40 (new_contributors can read 0 while the reports name new contributors; quote both) | | `unique_committers`, `unique_reviewers`, `bus50`, `bus75`, `pr_author_div_eff`, `reviewer_div_eff` | CO60, IN10 | | `issues_opened`, `issues_closed`, `median_merge_days`, `median_gap_days` | QU50 | | `dev_messages`, `dev_unique_posters`, `commits` | CS20 | @@ -120,45 +119,43 @@ which list and timespan were searched. with each number, and the report's `generated_on` date: the health reports are regenerated periodically and can be a couple of months old. -Known disagreements with other sources, seen across six podlings. Quote -the metric and the other source side by side, never the metric alone: - -- `new_contributors` read 0 for GraphAr and Amoro while four consecutive - reports named new contributors with merged PRs. -- `dev_messages` (48 for Amoro) was lower than the raw count of [VOTE] - plus [DISCUSS] hits on dev@ over the same year. -- `unique_committers` (25 for Hamilton) exceeded the 10-person roster. -- `releases` counted 1 for Toree in a window with no completed release - (an rc tag, most likely). -- `issues_opened` and `issues_closed` read GitHub Issues only. Toree uses - JIRA: `https://issues.apache.org/jira/rest/api/2/search?jql=project=...` +Known disagreements with other sources. Quote the metric and the other +source side by side, never the metric alone: + +- `new_contributors` can read 0 while consecutive reports name new + contributors with merged PRs. +- `dev_messages` can be lower than the raw count of [VOTE] plus + [DISCUSS] hits on dev@ over the same year. +- `unique_committers` can exceed the roster, since it counts commit + authors. +- `releases` can count a release vote that never completed. +- `issues_opened` and `issues_closed` read GitHub Issues only. For a + podling on JIRA, `https://issues.apache.org/jira/rest/api/2/search?jql=project=...` answers anonymously and gives the QU50 counts. ## Incubator reports (`incubator-reports`, `ipmc`) `incubator-reports:get_podling_reports(podling)` gives the -quarterly reports (the cache held one Hamilton report twice under one id, -so count distinct periods); new committers and PPMC members (CO40), mentor +quarterly reports (the cache can hold one report twice under one id, so +count distinct periods); new committers and PPMC members (CO40), mentor comments and off-list concerns (CS50), self-assessed maturity boxes. The graduation-self-assessment skill reads the same reports for graduation; run it alongside this one when the question is graduation. `ipmc:graduation_readiness(podling, include_evidence=true)` is a metric score. Quote its dimension scores as proxies where they map to an item -(release_maturity to RE items, community dimensions to CO items). Never -quote its verdict: it rated OpenServerless "near_ready" with strong -community scores while the podling had never released and coordinated on -WhatsApp, and Seata "ready, high confidence" with its mail source -unavailable. Its output, `get_podling_reports` (76 KB for Pegasus) and -`trademark_list` (98 KB) can exceed the tool output limit and be saved to +(release_maturity to RE items, community dimensions to CO items), and its +`graduation_guide` entries (name search, IPMC release result, committers or +PPMC members added) as evidence with their sources. Never quote its verdict: +the maturity model has no levels. Its output, `get_podling_reports` and +`trademark_list` can exceed the tool output limit and be saved to a file; use jq or grep on the file rather than re-reading. ## Releases `ipmc:release_artifact_evidence` and `incubator-releases:podling_releases` list downloads.apache.org and archive.apache.org together, down to -`//` trees (Toree keeps `toree/` and `toree-pip/` -under each version), and answer in 3 to 9 seconds. Use them first. Each +`//` trees, and answer in 3 to 9 seconds. Use them first. Each source in `source_statuses` (under `release_source` in the ipmc output) carries an `error` when a directory could not be read; only then list that source by hand. archive.apache.org can be @@ -180,10 +177,9 @@ Registries, all fast and anonymous: Maven Central PyPI `https://pypi.org/pypi//json` (author and project_urls say who publishes, uploads are not attributed); Docker Hub `https://hub.docker.com/v2/repositories/apache//` and also -`apache/incubator-/` (Cloudberry publishes under the latter); crates.io +`apache/incubator-/`; crates.io `https://crates.io/api/v1/crates/` (send a User-Agent). Also look for -packages under the project's name published by the originating company -(GraphAr found pygraphar by the GraphScope team). +packages under the project's name published by the originating company. For LICENSE, NOTICE, DISCLAIMER, headers and sidecars, download the release from downloads.apache.org into a fresh scratchpad directory @@ -192,29 +188,26 @@ podling-release-check skill's script: `scripts/check_rc.py --rc-dir --keys --disclaimer-template --wip-template --out `. Without the templates it will not compare disclaimer text. It verifies signatures only when gpg is -installed. Note any expired key in KEYS (Hamilton's only key had expired); +installed. Note any expired key in KEYS; `gpg --show-keys KEYS` prints an expiry only for keys that have one, so "no expired key" needs the count of keys with no expiry alongside it. -For a podling older than three years, `lte=36M` misses earlier releases: -Pegasus needed `lte=80M` on general@ to recover all five RESULT emails. +For a podling older than three years, `lte=36M` misses earlier releases. Match the window to the incubation length for RE20 and RE50. Release directory names are not consistent even within one podling -(seata-go has `2.0.0/` and `v2.1.0/`); list the parent, never guess. +(`2.0.0/` next to `v2.1.0/`); list the parent, never guess. When a release directory has no matching [RESULT], search general@ for -" " and read the hits: XTable's 0.1.0 had a cancelled -IPMC vote and a "release without IPMC vote" thread that the vote-history -tool never surfaced. +" " and read the hits: a cancelled IPMC vote or a +"release without IPMC vote" thread may not show in the vote-history tool. ## Website content When a live page returns 404 or an empty shell, read the page source from the website repository (`apache/-website` or `-site`) with `gh api -.../contents/...`; Seata's English release, committer, team and security -pages are "Placeholder. DO NOT DELETE." files while the text sits under -`i18n/zh-cn/`. Say which language the content exists in. Versioned docs +.../contents/...`; English pages can be "Placeholder. DO NOT DELETE." +files while the text sits under another language such as `i18n/zh-cn/`. Say which language the content exists in. Versioned docs sites move paths (`/docs/v1.2.0/...`), so take links from the page rather than guessing. @@ -222,23 +215,22 @@ than guessing. Use one fresh directory per podling and per download, named with the podling and date (`--`); a shared scratchpad with -stale `keys` and `wiki` entries from another run broke a mkdir chain and -dropped a download into the root in the OpenServerless run. +stale `keys` and `wiki` entries from another run can break a mkdir chain +and drop a download into the root. ## Branding (`asf-trademark`, `ipmc`) `asf-trademark:check_project_website(url, stage="podling", project_name)` for CO10, QU30 (Security link), TB10, TB20. Pass -`project_name` with the project's own capitalisation (GraphAr, not -Graphar). `ipmc:trademark_branding_check` wraps the same checks but +`project_name` with the project's own capitalisation. `ipmc:trademark_branding_check` wraps the same checks but returns SKIP when the ipmc server was installed without its trademark dependency (`pip install -e ~/TrademarkMCP` into the Python that runs the ipmc server fixes that); use the standalone server first. Two of its rules need a second look before recording: the "(Incubating)" -check flagged Amoro although the page text uses "(incubating)" in lower -case elsewhere, so confirm by reading the heading yourself; and the +check can flag a page whose text uses "(incubating)" in lower case +elsewhere, so confirm by reading the heading yourself; and the Privacy navigation rule flags a site that links its own privacy page -rather than privacy.apache.org (Toree), which is a different finding. `asf-trademark:get_branding_checklist(stage)` lists what +rather than privacy.apache.org, which is a different finding. `asf-trademark:get_branding_checklist(stage)` lists what it checks. For TB30 use the status page's name-check date and the reports' PODLINGNAMESEARCH reference; `asf-trademark:validate_name` is for new names and reports an existing podling as a conflict with itself. @@ -257,12 +249,11 @@ and compare its version line with `references/items.md`. ## Incubation proposal (`cwiki`) `cwiki:cwiki_search_pages(query="Proposal")` with the default -full-text search finds the proposal page; titles vary ("GraphArProposal", -"XTable Proposal", "KIE Proposal") and `title_only=true` matches whole -tokens only, so it returned nothing for Texera, OpenServerless, Cloudberry -and Pegasus whose titles are one token. Then `cwiki_get_page(title=...)`. +full-text search finds the proposal page; titles vary ("Proposal", +" Proposal") and `title_only=true` matches whole tokens only, so it +returns nothing for one-token titles. Then `cwiki_get_page(title=...)`. The proposal can be 50 KB and get spilled to a file; grep it. -Podlings from before 2016 have no cwiki proposal (Toree's lived on the old +Podlings from before 2016 have no cwiki proposal (they lived on the old MoinMoin wiki, now gone); the text survives in the general@ [DISCUSS] thread. The proposal gives: initial committers with affiliations (IN10), the dependency list at donation (LC20), the "Reliance on Salaried Developers" @@ -274,12 +265,11 @@ string and the search result returns it as a number. Meeting notes may live in the repository wiki. The web page hides dates; clone it instead: `git clone --depth 50 https://github.com/apache/.wiki.git` into the -scratchpad, then `git log --format='%ad %s' --date=short`. The GraphAr run -found the "meetings no longer held" note this way while the website still -advertised a weekly meeting. A wiki with no pages, or a disabled wiki, +scratchpad, then `git log --format='%ad %s' --date=short`. This can find a "meetings no +longer held" note while the website still advertises a meeting. A wiki with no pages, or a disabled wiki, clones as "Repository not found" even when `has_wiki` is true; say so and move on. Minutes may instead be in GitHub Discussions, sometimes on a -different repository (KIE kept them on incubator-kie-issues); the GraphQL +different repository; the GraphQL `discussions` query lists them. ## Example self-assessments