Using Webhooks: IQ & Slack Integration

Using our Webhooks documentation in the IQ Space as a starting point, we want to show you how you can take that information and make something useful out of it. This article shows you how to build a webhook and deploy it to a Serverless  framework like AWS Lambda. The Serverless function will consume an IQ policy evaluation event and push a message about it to Slack.

We chose Serverless because we found a lot of value in this integration, but we don't want the overhead of an operations team managing it all. The Serverless framework allows AWS to handle management of the application, and only accrues charges when the requests come in and out. In other words—it spins up our application and handles the request, then turns down when it's not needed. Serverless is perfectly suited for an example like this because we're simply mapping an object that IQ Server provides into something that Slack expects.

Crib Notes

All project files are available on GitHub courtesy of Sonatype Integrations Product Owner, Justin Young.

Make sure you have npm and IQ Server installed and that you have access to Slack as an admin. We'll use the following tools and frameworks to build the integration:

We also made a video that shows you how to quickly use the project files to set up a test integration. Check it out below:

 

Get Started

We found a great blog on deploying a REST API using Serverless, Express, and Node.js. The following steps are taken from that post, and provide a starting point for getting your Serverless project set up.

First we want to make a directory and call it something like iq-slack-integration.

Open the directory and use npm init to create a new npm project.
This auto-inits all the properties in the package json. We also want to go ahead and change the license to Apache V2 (Apache-2.0).

Get the Express framework and Serverless package with npm install express serverless-http.
This installs Express web framework and Serverless-http package to handle the interface between your Node.js app and AWS's Lambdas and the API Gateway.

We now have Express and Serverless installed. Next it's time to make the entry point.

Make an index.js (this is taken directly from the Serverless Blog):

// index.js

const serverless = require('serverless-http');
const express = require('express')
const app = express()

app.get('/', function (req, res) {
  res.send('Hello World!')
})

module.exports.handler = serverless(app);

We want to deploy it with a serverless.yml file. (this is also taken directly from the Serverless Blog):

# serverless.yml

service: my-express-application

provider:
  name: aws
  runtime: nodejs6.10
  stage: dev
  region: us-east-1

functions:
  app:
    handler: index.handler
    events:
      - http: ANY /
      - http: 'ANY {proxy+}'

For this demo, we're going to run our project offline. To do this, we want to install the Serverless-offline plugin with npm install --save-dev serverless-offline.

And then add the plugin to our serverless.yml:

plugins:
	-serverless-offline

Run sls offline start

Check that localhost is listening at http://localhost:3000

Write the Webhook Receiver

Next up, we're going to write something that will consume our webhook and map it into something that's useful like a plain text message for Slack. We're starting with the webhook consumption because we know the structure of the IQ policy evaluation event and have some examples of how to work with it.

Evaluation Mapper Test

As a company we value test driven development, so we're going to go ahead and make some tests for this project. We're going to use Mocha JS test framework and Chai assertion library to write tests that will help us develop without having to spin everything up.

Install Mocha with npm install --save-dev mocha.

Install Chai with npm install --save-dev chai.

In your project folder, make a test folder and then create a file in it called policy-evaluation-mapper-test.js. This file will contain the test for the policy evaluation mapper that converts the policy evaluation webhook into a Slack message.

We're going to look at the IQ Webhooks page on Sonatype Help to get an idea of what this object could look like. We'll grab a copy of the example application evaluation payload:

const { assert } = require('chai');
const mapper = require('../src/policy-evaluation-mapper');

function getPayload(criticalComponentCount) {
    return {
	    'applicationEvaluation': {
    	    'policyEvaluationId': 'debceb1d-9209-485d-8d07-bd5390de7ef5',
       		'stage': 'build',
	        'ownerId': '6a454175-f55d-4d33-ba44-90ac3af2e8b8',
        	'evaluationDate': '2015-05-05T23:40:12Z',
        	'affectedComponentCount': 10,
	        'criticalComponentCount': criticalComponentCount,
	        'severeComponentCount': 5,
	        'moderateComponentCount': 3,
	        'outcome': 'fail'
    	}
	};
}

We also want the message mapper to map to a payload and assert that the message is “ IQ Policy Eval: 2 Critical". If there is no critical violations, we'll write “ IQ Policy Eval: Congrats Critical Violation Free!!!!!".

describe('policy-evaluation-mapper', () => {
  it('should convert policy evaluation webhook to slack message', () => {
    const message = mapper.map(getPayload(2));

    assert.equal(message, 'IQ Policy Eval: 2 Critical Violations');
  });

  it('should convert policy evaluation webhook to slack message, no critical', () => {
    const message = mapper.map(getPayload(0));

    assert.equal(message, 'IQ Policy Eval: Congrats Critical Violation Free!!!!!');
  });
});

Mapper.js

Now that we have a test written, we want to go ahead and make our mapper. In your project, make a folder called src and then create a new file called policy-evaluation-mapper.js

The mapper will be successful if the tests don't fail, so we'll implement logic to create the critical violation message if there are critical violations and a congrats message if there are none.

const mapper = {};

mapper.map = (payload) => {
  const trailingMessage = payload.applicationEvaluation.criticalComponentCount
      ? `${payload.applicationEvaluation.criticalComponentCount} Critical Violations`
      : 'Congrats Critical Violation Free!!!!!';
  return `IQ Policy Eval: ${trailingMessage}`;
};

module.exports = mapper;

Our evaluation mapper will now take our payload and return some text. We can run the tests with ./node_modules/mocha/bin/mocha or npm test with the correct configuration. Doing so shows our first tests passing! Next, we need to verify the webhook signature and ensure that the source came from where we thought it did.

Validator Test

In the tests folder, make a file called webhook-signature-validator-test.js

In here, we want to describe the webhook and what it does (validate a well formed body with signature). We also want to make a “fake" webhook with some headers, a body, a secret, and assert that our validator validates the request and secret. We can use openssl to determine what the signature should be given the body and our secret key:

echo -n '{"field":"value"}' |  openssl sha1 -hmac "secret"

Finally, we should also check that the validator invalidates a well formed body with an incorrect signature:  

const { assert } = require('chai');
const validator = require('../src/webhook-signature-validator');

describe('webhook-signature-validator', () => {
  const request = {
    headers: {
      'x-nexus-webhook-signature': '42f0ce462b5631af387660d11c7b93a1d8ae209d'
    },
    body: {
      'field': 'value'
    }
  };

  it('should validate a well formed body with signature', () => {
    const secret = 'secret';

    assert.isTrue(validator.validate(request, secret));
  });

  it('should invalidate a well formed body with incorrect signature', () => {
    const secret = 'notsecret';

    assert.isFalse(validator.validate(request, secret));
  });
});

Validator.js

Now that we have tests, we can write our validator. We want to make a new file in our src folder called something like webhook-signature-validator.js.

Back in the webhooks doc example consumer, there's a signature that is compared against a secret key and the body. All requests are going to look the same from IQ server whether it's a policy evaluation or something else. This validator will take in the request and the secret key and ensure the request is authentic.

const crypto = require('crypto');

const validator = {};

validator.validate = (request, secretKey) => {
  const body = request.body;
  const signature = request.headers['x-nexus-webhook-signature'];
  const jsonBody = JSON.stringify(body);
  const hmacDigest = crypto.createHmac('sha1', secretKey).update(jsonBody).digest('hex');
  return signature === hmacDigest;
};

module.exports = validator;

Again, running the tests verify that our validator is working as intended. Now that we have our validator and mapper, the next step is getting our webhook and spitting it out in Slack.

Write the Slack Integration

Looking at Slack for more information, we found the Slack Developer Kit for Node.js. This kit is great because they have something that makes building apps with Node.js incredibly simple. To post a message in Slack, we need to get a Slack token, make a new web client with that token, and then set a channel and post the message. 

Install the Slack Developer Kit with npm install @slack/client.

Make a new channel in Slack called something like "webhook-test."

Go to the Incoming Webhooks custom integration page (need an Admin account for this). Click Add Configuration and select to post to iq-test-channel and then click Add Incoming Webhook integration.

Copy the incoming Slack webhook URL and paste it in serverless.yml that we created in the Get Started section.

environment:
      SLACK_WEBHOOK_URL: 'https://hooks.slack.com/services/000000000/000000000/00000000000000000000000'

Index.js

Now that we have our incoming Webhook URL from Slack, we're going to add our validator to index.js

We'll set the validator request secret key as an environmental variable and call it IQ_SECRET_KEY. If the validation fails, we don't want to do anything so we'll just send a 401 Auth Error.

We also need our message. We wrote our mapper already, so the message will be the mapper and the payload is the body of the request. This basically means that we have our message and then we send it.

The Express web application framework we're using is very lightweight and you can add modules to it. One of the modules we need to add when expecting to get json data is a bodyParser. Without this, the body will never exist, and we want the application to use the bodyParser to take json data.

const serverless = require('serverless-http');
const express = require('express');
const bodyParser = require('body-parser');
const app = express();
const {IncomingWebhook} = require('@slack/client');
const validator = require('./src/webhook-signature-validator');
const mapper = require('./src/policy-evaluation-mapper');

app.use(bodyParser.json({strict: false}));

app.post('/', function(req, res) {
  if (!validator.validate(req, process.env.IQ_SECRET_KEY)) {
    res.status(401).send({error: 'Not authorized'});
    return;
  }
  const iqNotification = new IncomingWebhook(process.env.SLACK_WEBHOOK_URL);
  const message = mapper.map(req.body);
  iqNotification.send(message, (error, resp) => {
    if (error) {
      console.error(error);
      res.status(500).send({error: error});
      return;
    }
    res.send('');
  });
});

module.exports.handler = serverless(app);

Serverless.yml

We've already copied the incoming Slack webhook constructor into our serverless.yml file. Next we'll add the secret environmental variable. The Slack Developer Kit article has secrets in the environmental variables of the servers, but for this example we'll put them in the serverless.yml file. In a real world scenario, we'd want to configure them on our cloud provider and not store them in source control.

service: iq-slack-integration

provider:
  name: aws
  runtime: nodejs6.10
  stage: dev
  region: us-east-1

functions:
  app:
    handler: index.handler
    events:
      - http: ANY /
      - http: 'ANY {proxy+}'
    environment:
      SLACK_WEBHOOK_URL: 'https://hooks.slack.com/services/000000000/000000000/00000000000000000000000'
      IQ_SECRET_KEY: 'secret'

plugins:
  - serverless-offline


In general you don't want environmental variables where people can see it! We're just doing this for development, but you should use correct process.

Make a Webhook in IQ server

Start up your instance of IQ Server. We're using version 1.44 (or newer) to make use of features that get you up and running quickly, like a sandbox application with the reference policies already loaded. 

In the Admin menu, click Webhooks, and make new webhook.

If we look at our Serverless offline app, it is listening on http://localhost:3000 so that's our webhook URL.

The secret key is whatever you set.

We want an Application Evaluation event type.

Click Create.

Run an Evaluation and Cross your Fingers

Go back to the Org & Policies area of IQ. Select the Sandbox App and evaluate a binary (for example WebGoat).

The analysis runs and if the secret key was validated correctly... TA DA! You will see the incoming webhook message in your Slack channel. We got 11 critical violations with the Webgoat application:

Next Steps

In this example, we're running in the offline mode. If we were running this in production, our next steps would be to deploy it by setting up our Amazon configuration locally (or on a CI server), using Serverless to deploy, and then having the function available at any time. Serverless allows you to deploy it by adding some configuration with AWS keys and environment configured with AWS credentials. Once it's all set up, deploying is as easy as running sls deploy. It'll deploy to Amazon and set up the API gateway and Lambdas so that you have the post method available. Remember to check out the Deploy a REST API using Serverless, Express and Node.js blog from Serverless for more information.

At Sonatype, we are impressed with AWS Lambda and all the functions of their service because they allow you to build integrations with very little overhead. We also like what the team at Serverless has built, and chose their framework because it's portable—we can write some code and push it up to AWS or even a multicloud deployment. It also facilitates deployment and development by taking some of the heavier getting started stuff and managing it in the framework.