Reads the diff of each open pull request, has AI note the obvious problems, and posts them as a comment.
Uses
GitHub
OpenAI
AI first-pass review on open pull requests
Run every hour.Run every hour.
Set repositoryOwner to blank.Set repositoryOwner to blank.Set repositoryName to blank.Set repositoryName to blank.Set repositoryReadToken to blank.Set repositoryReadToken to blank.Set reviewedLabel to first-pass-reviewed.Set reviewedLabel to first-pass-reviewed.Set maxPullRequestsPerRun to 5.Set maxPullRequestsPerRun to 5.Set maxFilesPerReview to 20.Set maxFilesPerReview to 20.Set maxDiffCharactersPerFile to 3000.Set maxDiffCharactersPerFile to 3000.Set githubHeaders to a record of Authorization, Accept, and X-GitHub-Api-Version.Set githubHeaders to a record of Authorization, Accept, and X-GitHub-Api-Version.
Join a list of [https://api.github.com/repos/, repositoryOwner, /, repositoryName] with nothing and save it as repositoryApiUrl.Join a list of [https://api.github.com/repos/, repositoryOwner, /, repositoryName] with nothing and save it as repositoryApiUrl.Join a list of [repositoryApiUrl, /pulls] with nothing and save it as openPullRequestsUrl.Join a list of [repositoryApiUrl, /pulls] with nothing and save it as openPullRequestsUrl.
Start an empty list and save it as openPullRequests.Start an empty list and save it as openPullRequests.
If a step below fails, save the error as pullRequestListErrorIf a step below fails, save the error as pullRequestListErrorTry
Change openPullRequests to <expr>.body.Change openPullRequests to <expr>.body.On error
Stop and report a problem: a list of [Could not read the open pull requests from GitHub: , pullRequestListError.message with openPullRequestsUrl replaced by the GitHub API address, Check the repository details and the token in the 'repositoryReadToken' step.] joined by nothing.Stop and report a problem: a list of [Could not read the open pull requests from GitHub: , pullRequestListError.message with openPullRequestsUrl replaced by the GitHub API address, Check the repository details and the token in the 'repositoryReadToken' step.] joined by nothing.Set reviewedThisRun to 0.Set reviewedThisRun to 0.
For each pullRequest in openPullRequests:For each pullRequest in openPullRequests:
If reviewedThisRun is greater than or equal to maxPullRequestsPerRunIf reviewedThisRun is greater than or equal to maxPullRequestsPerRunYes
Show in the output: a list of [Reviewed , maxPullRequestsPerRun, pull requests, which is this run's limit. The rest keep their place in the queue and are picked up on the next run.] joined by nothing.Show in the output: a list of [Reviewed , maxPullRequestsPerRun, pull requests, which is this run's limit. The rest keep their place in the queue and are picked up on the next run.] joined by nothing.Stop repeating.Stop repeating.No
No steps
Set pullNumber to pullRequest.number.Set pullNumber to pullRequest.number.If pullNumber is noIf pullNumber is noYes
Show in the output: Skipped a pull request that GitHub returned without a number, so there was nothing to comment on..Show in the output: Skipped a pull request that GitHub returned without a number, so there was nothing to comment on..Skip to the next item.Skip to the next item.No
No steps
If pullRequest.draft is yesIf pullRequest.draft is yesYes
Show in the output: a list of [Pull request #, pullNumber, is still a draft, so it was left alone.] joined by nothing.Show in the output: a list of [Pull request #, pullNumber, is still a draft, so it was left alone.] joined by nothing.Skip to the next item.Skip to the next item.No
No steps
Set alreadyReviewed to false.Set alreadyReviewed to false.For each pullRequestLabel in pullRequest.labels:For each pullRequestLabel in pullRequest.labels:If pullRequestLabel.name is reviewedLabelIf pullRequestLabel.name is reviewedLabelYes
Change alreadyReviewed to true.Change alreadyReviewed to true.Stop repeating.Stop repeating.No
No steps
If alreadyReviewed is yesIf alreadyReviewed is yesYes
Skip to the next item.Skip to the next item.No
No steps
Join a list of [repositoryApiUrl, /pulls/, pullNumber, /files] with nothing and save it as changedFilesUrl.Join a list of [repositoryApiUrl, /pulls/, pullNumber, /files] with nothing and save it as changedFilesUrl.Start an empty list and save it as changedFiles.Start an empty list and save it as changedFiles.If a step below fails, save the error as changedFilesErrorIf a step below fails, save the error as changedFilesErrorTry
Change changedFiles to <expr>.body.Change changedFiles to <expr>.body.On error
Show in the output: a list of [Could not read the changed files for pull request #, pullNumber, : , changedFilesError.message with changedFilesUrl replaced by the GitHub API address] joined by nothing.Show in the output: a list of [Could not read the changed files for pull request #, pullNumber, : , changedFilesError.message with changedFilesUrl replaced by the GitHub API address] joined by nothing.If changedFiles is noIf changedFiles is noYes
Show in the output: a list of [Pull request #, pullNumber, has no changed files this workflow can read, so it was left for a human.] joined by nothing.Show in the output: a list of [Pull request #, pullNumber, has no changed files this workflow can read, so it was left for a human.] joined by nothing.Skip to the next item.Skip to the next item.No
No steps
Count the items in changedFiles and save it as changedFileCount.Count the items in changedFiles and save it as changedFileCount.Take the first maxFilesPerReview items of changedFiles and save it as filesToReview.Take the first maxFilesPerReview items of changedFiles and save it as filesToReview.Count the items in filesToReview and save it as reviewedFileCount.Count the items in filesToReview and save it as reviewedFileCount.Start an empty list and save it as diffBlocks.Start an empty list and save it as diffBlocks.For each changedFile in filesToReview:For each changedFile in filesToReview:Set changedFileName to blank.Set changedFileName to blank.If a step below fails, save the error as fileNameErrorIf a step below fails, save the error as fileNameErrorTry
Trim changedFile.filename and save it as changedFileName.Trim changedFile.filename and save it as changedFileName.On error
Change changedFileName to blank.Change changedFileName to blank.If changedFileName is emptyIf changedFileName is emptyYes
Show in the output: a list of [Skipped a changed file in pull request #, pullNumber, because GitHub returned it without a filename.] joined by nothing.Show in the output: a list of [Skipped a changed file in pull request #, pullNumber, because GitHub returned it without a filename.] joined by nothing.Skip to the next item.Skip to the next item.No
No steps
Set changedFilePatch to blank.Set changedFilePatch to blank.If a step below fails, save the error as filePatchErrorIf a step below fails, save the error as filePatchErrorTry
Trim changedFile.patch and save it as changedFilePatch.Trim changedFile.patch and save it as changedFilePatch.On error
Change changedFilePatch to blank.Change changedFilePatch to blank.Set diffBlockBody to (GitHub sent no text diff for this file - it may be binary, a pure rename, or too large to display.).Set diffBlockBody to (GitHub sent no text diff for this file - it may be binary, a pure rename, or too large to display.).If changedFilePatch isn't emptyIf changedFilePatch isn't emptyYes
Shorten changedFilePatch to maxDiffCharactersPerFile characters and save it as diffBlockBody.Shorten changedFilePatch to maxDiffCharactersPerFile characters and save it as diffBlockBody.No
No steps
Add a list of [File: , changedFileName, "Add a list of [File: , changedFileName,
, diffBlockBody] joined by nothing to the end of diffBlocks and save it as diffBlocks.If diffBlocks is noIf diffBlocks is noYes
Show in the output: a list of [None of the changed files in pull request #, pullNumber, could be read, so it was left for a human.] joined by nothing.Show in the output: a list of [None of the changed files in pull request #, pullNumber, could be read, so it was left for a human.] joined by nothing.Skip to the next item.Skip to the next item.No
No steps
Set coverageNote to blank.Set coverageNote to blank.If changedFileCount is greater than reviewedFileCountIf changedFileCount is greater than reviewedFileCountYes
Join a list of [This pull request changes , changedFileCount, files. This first pass read only the first , reviewedFileCount, of them, in the order GitHub lists them - the rest have not been looked at.] with nothing and save it as coverageNote.Join a list of [This pull request changes , changedFileCount, files. This first pass read only the first , reviewedFileCount, of them, in the order GitHub lists them - the rest have not been looked at.] with nothing and save it as coverageNote.No
Join a list of [This first pass read all , changedFileCount, changed files.] with nothing and save it as coverageNote.Join a list of [This first pass read all , changedFileCount, changed files.] with nothing and save it as coverageNote.Join a list of ["You are giving a pull request a first-pass read for a human reviewer who has not opened it yet.Join a list of [You are giving a pull request a first-pass read for a human reviewer who has not opened it yet.
, Repository: , repositoryOwner, /, repositoryName,
Pull request #, pullNumber, : , pullRequest.title,
Opened by: , pullRequest.user.login,
, coverageNote,
The changed files and their diffs follow. Lines starting with + or - are the change itself.
, diffBlocks joined by a blank line,
Write short markdown bullet points about what is plainly visible in the diff above:
- error handling that looks missing, such as an unchecked failure or a swallowed exception
- names that do not say what the thing is
- new behaviour that arrives without a test
- anything else obviously risky, such as a hardcoded secret or an unbounded loop
Group the points by file and quote the identifier or the line you mean. Make at most eight points, most important first. If nothing stands out, reply with the single sentence: Nothing obvious stood out in this first pass. Do not approve the pull request, do not say LGTM, and do not suggest merging - a human decides that. Do not comment on anything that is not in the diff above.] with nothing and save it as reviewPrompt.Set reviewText to blank.Set reviewText to blank.If a step below fails, save the error as reviewErrorIf a step below fails, save the error as reviewErrorTry
Trim the response text from the OpenAI “Ask OpenAI” step and save it as reviewText.Trim the response text from the OpenAI “Ask OpenAI” step and save it as reviewText.On error
Show in the output: a list of [Could not get a first-pass review for pull request #, pullNumber, : , reviewError.message] joined by nothing.Show in the output: a list of [Could not get a first-pass review for pull request #, pullNumber, : , reviewError.message] joined by nothing.If reviewText is emptyIf reviewText is emptyYes
Show in the output: a list of [The model returned nothing for pull request #, pullNumber, , so no comment was posted and it stays unlabelled for the next run.] joined by nothing.Show in the output: a list of [The model returned nothing for pull request #, pullNumber, , so no comment was posted and it stays unlabelled for the next run.] joined by nothing.Skip to the next item.Skip to the next item.No
No steps
Join a list of ["### Automated first-pass reviewJoin a list of [### Automated first-pass review
, coverageNote,
, reviewText,
This is a first pass, meant to give a human reviewer somewhere to start. It is not an approval, and this workflow never approves, merges or closes anything. Remove the ', reviewedLabel, ' label from this pull request to have it looked at again.] with nothing and save it as commentBody.GitHubComment on issue pullNumber in repositoryOwner/repositoryName.Comment on issue pullNumber in repositoryOwner/repositoryName.commentBodyIf a step below fails, save the error as labelErrorIf a step below fails, save the error as labelErrorTry
GitHubAdd label reviewedLabel to issue pullNumber in repositoryOwner/repositoryName.Add label reviewedLabel to issue pullNumber in repositoryOwner/repositoryName.On error
Show in the output: a list of [Commented on pull request #, pullNumber, but could not add the ', reviewedLabel, ' label, so the next run will comment on it again. Add the label by hand to stop that: , labelError.message] joined by nothing.Show in the output: a list of [Commented on pull request #, pullNumber, but could not add the ', reviewedLabel, ' label, so the next run will comment on it again. Add the label by hand to stop that: , labelError.message] joined by nothing.Add 1 to reviewedThisRun.Add 1 to reviewedThisRun.Show in the output: a list of [First-pass review finished. Commented on , reviewedThisRun, pull request(s).] joined by nothing.Show in the output: a list of [First-pass review finished. Commented on , reviewedThisRun, pull request(s).] joined by nothing.