SFDX GitHub Code Review - Part 1

TL; DR;

Building a GitHub Action to do code review on a Pull Request using Salesforce Code Analyzer CLI plugin. (previously known as Code Scanner).

Check out the current version on GitHub or jump down a few paragraphs to skip the intro waffling. I originally wrote this post at the end of 2021 and I couldn’t bring myself to ditch it.

The Problem

I always felt that very few Salesforce developers pay enough attention to feedback from static analysis tools like Apex PMD and ESLint. That’s why anytime I am somehow “in charge” of quality I try to make this feedback as in-your-face as possible. Most teams I’ve worked with used Pull Requests and reviewed them (at least to some extent). Having identified issues highlighted right there in code diff view I feel is the best way to make sure people definitely see the feedback. It’s also a good way to not have to be the one who points it out all the time.

Having the extensions installed and using the inbuilt features in VS Code is also very important of course. PR time is quite late after all.

Alternatives and Previous Efforts

I worked on a team where we used GitLab. The Code Quality widget there was really nice and I built an action that would convert a static analysis report into the right format and upload it. We moved to GitHub before I managed to test it though.

One year later I found myself working with BitBucket Cloud long enough to have a go at it again. I didn’t find any other way than running Code Scanner and then parsing the output into a format expected by the BitbucketCloud API’s endpoint to create annotations. The transformation was a bit trickier than last time, but I managed with the combination of jq and some bash string replacements.

(not sharing code publicly as I wasn’t on my own time)

Back to GitHub

In my current team we’re using GitHub again. I’m not actually in charge this time, but.. yea. We have a SonarCloud plan available and its PR Decoration comment would have been good enough, if it worked. Somehow, because the org is primarily linked to Azure DevOps, it doesn’t play nice with GitHub though.

So I looked into running SFDX Code Scanner again. Quickly I found you can just upload a SARIF formatted report to GitHub (the scanner supports that) and it will annotate the code for you. Except that the repository has to be public or you need the Enhanced Security Licence. We don’t have that of course.

And that’s why I found myself building a GitHub Action to create a PR Review over API with the outputs of the SFDX Code Scanner. It should be just 1 API call with an array of reviewComments that will appear in the diff as annotations. Then I also want to define a threshold severity for the violations to decide if the Review is REQUEST_CHANGES (found violations with threshold severity), COMMENT (only lower severity violations) or APPROVE (no violations at all).

The Plan

The Scanner json formatted report contains an array of files each with its own array of violations. I didn’t want to exercise my (almost non-existent) skills with jq too much so I found it a lot easier to generate a csv report (which is flat) and convert it to json using this neat Python one liner I found on stackoverflow. Then it was quite easy to look back at my previous attempts with BitBucket and polish off the report with jq to have a valid array of Review Comments.

Both jq and python are preinstalled in the normal Ubuntu image in GitHub Actions. This means I just need to install the SFDX CLI and Code Analyzer plugin and my environment is fully ready.

This is the plan for the Action

Install SFDX CLI and Code Scanner plugin
Run the analysis exporting into CSV
Convert CSV into JSON
Alter the JSON to match Review Comments format
Check violations to find highest severity
Create PR Review via API

Proper Git Diff

First of all the limiting of the scanner –target attribute. It’s quite wasteful to be analysing the full repository when we’re only interested in what’s changed in the Pull Request. git diff works fine, but just like plugging in a USB2.0 it took a few tries to settle on the right comparison commit order. Branch..Base or Base..Branch (that one).

From my past attempts I remember an important problem with this. The Pull Request branch often falls behind the base. Diff from base..branch will then include the newer commits on the base branch too, but in reverse. And it just gets confusing.

With the help of Stack Overflow I’ve found a neat solution that finds the commit at which the branch diverged from base. Check this out, including piping the results together in a comma separated list. Bash is really black magic.

git diff --name-only --diff-filter=MCRA $(git merge-base --fork-point origin/${{ github.base_ref }} origin/${{ github.head_ref }})..origin/${{ github.head_ref }} ${{ inputs.source_path }} | paste -sd "," -

Transformations with jq

After the mentioned conversion of the report from csv to JSON I use this jq query to get rid of the attributes I don’t need and rename those I do to match the API names.

jq -c 'map({path: ("." + .File | split("${{ github.event.repository.name }}")[2])), position: (.Line)|tonumber, side: "RIGHT", body: (.Description + " (" + .Engine + "-" + "Severity: " + .Severity + ")")})' report.json`

The path was absolute and the review comment needed to have a relative path. Scanner report contained the project folder name twice in the path for some reason so the solution for me was to use the split function in jq on the repo name. This resulted in 3 array items (before first repo name, slash between the two instances of the name and finally the rest of the path). Using the 3rd and putting a “.” before it worked the treat.

Passing Values between Action Steps

- name: Read Comments File
    id: getcomments
    run: echo "::set-output name=comments::$(cat comments.json)"

Passing the JSON array as an attribute to another action required me to load it as a Step output. When the JSON was “pretty” formatted though it wasn’t reading in properly. So I had to use the -c argument in the jq query to flatten it into a single line. I was then able to pass it in like this:

'${{ steps.getcomments.outputs.comments }}'

Calling GitHub API

Making the API call using the octokit/request-action was really easy. I thought I was done. Life is never that simple of course. It’s not allowed to create review comments for lines outside the PR diff. The API won’t just ignore them (somewhat understandably). I am limiting the scanner to just changed files, but of course that’s not enough. There can be issues in unchanged parts of the files.

- name: 'Create Review'
     uses: octokit/request-action@v2.x
     with:
         route: POST /repos/{repo}/pulls/{pull_number}/reviews
         repo: ${{ github.repository }}
         pull_number: ${{ github.event.number }}
         body: 'SFDX Code Review'
         event: 'COMMENT'
         comments: '${{ steps.getcomments.outputs.comments }}'
     env:
         GITHUB_TOKEN: ${{ inputs.github_token }}

Disillusion

Worse still, the review comment must specify a position in the diff and not the line number. Looking at the description of how to calculate position made me want to cry (remember, I thought I was nearly done). Quick google didn’t find any ready-made solution.

So I just changed the jq query to make all issues go to position 1 and decided I’ll have to regroup and come back again later. You win this one world..