Extending Lift

Custom Tool APIs are interface specifications about program arguments, standard input and standard output formats a tool must support in order to interoperate with Lift. The configuration reference presents how to configure your repository so that each analysis will invoke such a tool.

Execution Environment

Each tool runs in an Ubuntu 20.04 environment specifically for analysis of either the source or destination branches of a pull request. Shell scripts are used in examples below because they are a common platform supported, but there is no requirement to use shell scripts. More complex tools might benefit from a shell wrapper that verifies the environment before invoking the analysis tool.

For each issue discovered by a tool it should emit a tool note specifying the type of the issue, a detailed message, and location including file and line.

The Android SDK will be available on the file system accessible under $ANDROID_HOME.

When Lift runs your tool the issues it discovers are categorized into those that are introduced (appearing only in the source branch), fixed (appearing only in the destination branch), and preexisting (appearing in both branches). Only introduced issues are propagated to the user by way of a pull request comment. All tool results are available to the user on the console.

APIv1

Bring Your Own (BYO) tools are called with arguments of a repository directory, commit hash, and a command. In addition, they must output a JSON value specific to the command. These tools is called once on the whole repository and are expected to analyze every relevant file in the repository.

API Commands

The command and their associated standard outputs are:

  • version: Always print the number  1  on standards out and return with exit success (zero).
  • applicable: Perform any needed analysis of the repository to determine if this tool is applicable (ex: a tool might only be applicable to particular lanugages). Print the string  true  or  false  to standard out dependingly and return with exit success.
  • run: Evaluate the repository and emit JSON results, detailed below.

The  run  command must emit JSON of:

[ { "type"        : <string>,
    "message"     : <string>,
    "file"        : <string>,
    "line"        : <int>,
    "details_url" : <optional string>,
  }
]

The tool is always passed arguments of repository directory, branch of interest, and command. An example of running the three commands is:

$ ./tool.sh /lift/code/path ce44de9a version
1
$ ./tool.sh /lift/code/path ce44de9a applicable
true
$ ./tool.sh /lift/code/path ce44de9a run
[{"type" : "Invalid configuration keyword",
  "message": "The keyword 'customTols' is not a valid configuration value.",
  "file": "configs/config.toml",
  "line": 12,
  "details_url": null
 },
 {"type" : "Invalid configuration keyword",
  "message": "The keyword 'riika' is not a valid configuration value.",
  "file": "configs/config.toml",
  "line": 11
 }
]

The tool must always exit successfully, returning exitcode 0.

BYO API Hello World

Let’s make a hello-world example of a Lift custom tool at work. We’ll use a shell script to case over the commands and take appropriate action. As a framework let's start with:

#!/usr/bin/env bash
# dir=$1 but it is not needed
commit=$2
cmd=$3 

# ... to be filled in ... 

if [[ "$cmd" = "run" ]] ; then 
  run 
fi 
if [[ "$cmd" = "applicable" ]] ; then 
  applicable 
fi 
if [[ "$cmd" = "version" ]] ; then 
  version 
fi

Now we’ll fill in functions for each command. The  version  command is the simplest as there is no decision needing made. Just echo  1  out:

function version() {
    echo 1
}

For applicability let’s always run - this tool isn’t specific to configuration files, languages, times of day, or any other variable.

function applicable() {
    echo "true"
}

Before tackling  run, recall that Lift runs on both the source and destination branches before classifying the bugs as introduced/fixed/prexisting. Only introduced bugs are sent as comments. Therefore we need to do something branch specific and not just emit a tool note saying “hello world”. Our hello world tool will emit issues including the commit hash:

function run() {
    echo "[{ \"type\": \"Hello World\", \
            \"message\": \"We are analyzing commit $commit\", \
            \"file\": \"N/A\", \
            \"line\": 0, \
            \"details_url\": \"https://help.sonatype.com/lift/extending-lift\" \
          }]"
}

Suggested Development Process

We suggest you keep development local, and not push scripts to upstream repositories or test via Lift servers until after ensuring functionallity locally. You should start with docker, access to the Ubuntu 20.04 image, and a shell. You can develop entirely locally and with a fast feedback loop by working directly inside of the docker container.

First we will look at the raw commands you can use to test your tool. After that we’ll write our own tool and verify it operates using the same method.

Development Process Steps

Create a test repository (or many) that is fast to analyze and sufficiently exercises your tool. Commit your tool, or a wrapper, in the repository such as at .lift/tool.sh. Also commit a .lift/config.toml with the entry customTools = [ ".lift/tool.sh" ].

Once setup, you can test the tool by executing the commands manually inside the container:

docker run --rm -it -v /path/to/the/repository:/code ubuntu:20.04 bash
cd /code
# execute any setup script
[[ -x ".lift/setup.sh" ]] && ./.lift/setup.sh
./.lift/tool.sh $(pwd) $(git rev-parse HEAD) version
./.lift/tool.sh $(pwd) $(git rev-parse HEAD) applicable
./.lift/tool.sh $(pwd) $(git rev-parse HEAD) run

Before concluding development you can verify the tool conforms with Lift’s API with the check-lift-api tool:

docker run --rm -it -v /path/to/the/repository:/code ubuntu:20.04 bash
cd /code
curl -LO https://help.sonatype.com/lift/files/78578749/78578750/1/1623180836097/check-lift-api.sh
chmod +x check-lift-api.sh
./check-lift-api.sh

N.B. Make sure you test your tool on a repository that doesn’t have any findings. Make sure the standard output is still a JSON list such as  [ ].

Development Process by Example

As an example, let's develop a Bash based tool that complains about commited files which are not JSON. This is only for example; your tool can be written in any language and do any desired deep inspection of the code.

First we will create our test workspace:

mkdir lift-test-project ; cd lift-test-project
echo "{ }" > empty_dictionary.json
echo "[ ]" > empty_list.json
echo '""' > empty_string.json
echo '{ "number":  42, "string" : "nuh-uh", "meaning of life" : nill }' > complex.json
git init
git add *.json
git commit -m 'json files'

Then we’ll create the tool:

mkdir .lift
cat <<EOF >.lift/config.toml
customTools = [ ".lift/tool.sh" ]
EOF
cat <<EOF >.lift/tool.sh
#!/usr/bin/env bash
function run() {
    for i in \$(git ls-files | grep -v lift) ; do
        if ! jq \$i >/dev/null 2>/dev/null ; then
            echo "[ { \"type\" : \"bad json\", \"file\" : \"\$i\", \"line\": 1,\
                      \"message\" : \"JSON is many things, but it is not this.\",\
                      \"details_url\": null } ]"
            exit 0
        fi
    done
    echo "[ ]" ; exit 0
}

[[ "\$3" = "version" ]] && echo "1"
[[ "\$3" = "applicable" ]] && echo "true"
[[ "\$3" = "run" ]] && run
[[ -z "\$3" ]] && echo '{ "version" : 1, "name" : "json-verifier" }'
EOF
chmod +x .lift/tool.sh
git add .lift/*
git commit -m "lift tool"

Now we can test the tool manually and iterate, fixing the tool as appropriate:

./.lift/tool.sh $(pwd) $(git rev-parse HEAD) version
./.lift/tool.sh $(pwd) $(git rev-parse HEAD) applicable
./.lift/tool.sh $(pwd) $(git rev-parse HEAD) run

And finally test the tool with the script:

docker run --rm -it -v $(pwd):/code ubuntu:20.04 bash
cd /code
apt update && apt install -y curl jq git
curl -LO https://help.sonatype.com/lift/files/78578749/78578750/1/1623180836097/check-lift-api.sh
chmod +x check-lift-api.sh
./check-lift-api.sh .lift/tool.sh $(pwd)

This tool is incomplete, such as outputting at most one message, but it does conform to the Lift API and passes the verification.