Get reviews from GitHub pull requests

This skill is utilized when requesting reviews from GitHub pull requests.

1. Get the reviews.
2. Organize the reviews.
3. Help to resolve the reviews.

Get the reviews

To get the reviews from a GitHub pull request, you can use the GitHub API. Check the gh CLI tool is installed and authenticated. gh auth status can be used to check the authentication status. If gh isn't installed, try installing it by apt install gh. If authentication is not set up, tell the contributor to run gh auth login to authenticate with GitHub.

Use the GraphQL API to fetch the reviews for a specific pull request. Check fetch_reviews.sh to fetch the reviews and save them in a JSON file:

• Fill the variables in the command and run the command to fetch the reviews:

  # Omit the new lines
  PR_NUMBER= NUMBER_OF_PR_COMMENTS= NUMBER_OF_REVIEWS= \
  NUMBER_OF_THREADS= NUMBER_OF_COMMENTS_PER_THREAD= LAST_CURSOR= \
  bash .agents/skills/get-reviews/fetch_reviews.sh
  

  • PR_NUMBER: The number of the pull request to fetch reviews from.
  • NUMBER_OF_PR_COMMENTS: The number of PR comments to fetch.
  • NUMBER_OF_REVIEWS: The number of reviews to fetch.
  • NUMBER_OF_THREADS: The number of review threads to fetch.
  • NUMBER_OF_COMMENTS_PER_THREAD: The number of comments per review thread to fetch.
  • LAST_CURSOR: The cursor for incremental fetches. Optional.

• For incremental fetches, save pageInfo.endCursor from the previous fetch and pass it as after: $LAST_CURSOR (a base64 cursor, not a review thread node ID) so the query returns only new review threads.

• Use jq to filter the reviews and information if necessary.

The fetched JSON files in plans/{PR_NUMBER}/fetched/ contain the raw data of PR comments, reviews, and review threads (with pullRequestReview back-references on thread comments). When more context is needed later (e.g., to resolve which review a thread belongs to, or to check the original body of a comment), refer back to these files instead of re-fetching.

Organize the reviews

After fetching the PR and its reviews, organize the reviews. plans directory in the root is a good place to store them. plans/ is already ignored in .gitignore, so don't check that it is ignored.

• plans/{PR_NUMBER}/index.md: The main file for the PR.
  • The body of the PR
• plans/{PR_NUMBER}/reviews/{REVIEW_ID}.md: The file for each review which is not resolved.
  • After applying or dismissing the review, move the file to plans/{PR_NUMBER}/reviews/resolved/{REVIEW_ID}.md.
  • If the review file is too long, move the content to plans/{PR_NUMBER}/reviews/{REVIEW_ID}/index.md, and separate the content into multiple files in the same directory. In this case, after resolving the review, move the whole directory to plans/{PR_NUMBER}/reviews/resolved/{REVIEW_ID}/.
• plans/{PR_NUMBER}/reviews/resolved/{REVIEW_ID}.md: The file for each review which is resolved.

Don't use the first comment of the review thread as the review ID. The ID of the review thread starts with “PRRT_”. Use the first comment ID of the review thread only on the link.

Review threads aren't the only place that asks for changes. A PR-level comment (in comments of the fetched JSON, whose ID starts with “IC_”) or the body of a review (in reviews of the fetched JSON, whose ID starts with “PRR_”) can also point out things to fix. When such a comment or review body requests modifications, organize it as its own review file alongside the review-thread files, using the same directory layout and naming rules:

• Use the node ID of the PR comment (“IC_…”) or the review (“PRR_…”) as the {REVIEW\_ID} in the file path.
• If a PR comment or review body only contains approval, general impressions, questions without a modification request, or other content with nothing to fix, skip it and do not create a file for it.
• If only a part of a longer PR comment or review body requests changes, create a file only for that request and quote the relevant excerpt in the file so the context is preserved.

The format of review files should be as review.md. Read the review and draft the format based on the relevant information. The files should be written in the contributor's language. But the title of the item in the file (e.g., “Summary”, “Judgement”, “Plans”) should be in English for consistency.

Empty the space between the “Title” and the “Summary” sections.

All related information with the review should be stored in plans/{PR_NUMBER}/reviews/{REVIEW_ID}.md or the files in plans/{PR_NUMBER}/reviews/{REVIEW_ID}/.

After organizing the reviews, show the links to the files to the contributor.

Help to resolve the reviews

The agent's role in this step is to help the contributor resolve the reviews, not to resolve them on the agent's own authority. The final judgement always belongs to the contributor; the agent's job is to surface the information needed to decide, and then to carry out whatever the contributor decides.

For each unresolved review, give the contributor what they need to judge it, rather than judging on their behalf:

• Add relative links to the code or documentation the review points at.
• Summarize the relevant facts, the trade-offs, and any available options.
• Where the facts are clear, the agent may include a suggested judgement, but it must be clearly marked as a suggestion and never substitute for the contributor's decision.

Let the contributor read the review files, decide the judgement and the plans for each review, and update the review files if necessary. Do not apply or dismiss anything until the contributor has decided. After the contributor decides the judgement and the plans, apply or dismiss the reviews based on the files.

Categorize the reviews and the plans, and apply them at once by category. After applying the review, use /commit skill to commit the changes. The commit message should include the related review links:

https://github.com/fedify-dev/fedify/pull/{PR_NUMBER}#discussion_r{REVIEW_THREAD.COMMENTS[0].DATABASE_ID}

Because these changes are produced with agent assistance, the commit message must also carry the Assisted-by: AGENT_NAME:MODEL_VERSION trailer required by AI_POLICY.md, in addition to the review links.

After committing the changes, update the review file to include the commit hash and the comment section. If the review is dismissed, update the review file to include the reason for dismissing and the comment section.

If the Comments are written only in the contributor's language, provide an English translation and have the contributor review it. If they are written in both languages, check for any discrepancies between the two. If differences exist between the two versions, review them based on the facts and revise the English version to match the content in the contributor's language.

Post all review comments in English, even if the file written in the contributor's native language. The comments should be polite and constructive.

Before pushing commits, posting comments, or marking any review as resolved, the contributor must verify the actual applied changes and the relevant test/check results (e.g., mise run check and the related tests). Per AI_POLICY.md, AI-created work must be fully verified with human use; do not post comments or mark reviews resolved for changes whose correctness is only hypothetical. The agent prepares everything for this verification but leaves the verification itself to the contributor.

After resolving the reviews, pushing commits, posting comments, and updating the reviews as resolved, move the review files to plans/{PR_NUMBER}/reviews/resolved. Before moving the files, check status and comments of the PR from GitHub. Use fetch_reviews.sh, but instead of fetching all reviews and attributes, fetch only the necessary attributes to check the review status and comments.