Creating a Webhook Listener

As described in our Setting Up Webhooks guide, testing webhooks using RequestBin.com is a great way to get started with using and understanding webhooks. It provides temporary URLs that you can register through the ShipEngine dashboard or by using the v1/environment/webhooks endpoint.

When using ShipEngine in a production environment, you will need to provide a more robust web service to host your own custom endpoints. This guide walks you through creating a simple web application that exposes an endpoint you can use to subscribe to ShipEngine track webhooks.

We will be developing a web application that listens for webhook requests, often called a webhook listener. Our application will run a web server and listen for HTTP POST requests on a specified endpoint This example is written in JavaScript/ NodeJS and uses the Express web framework.

Requirements

To follow along with the steps in this guide, you will need to do the following:

Now that you have the source code available, we will walk through and explain it line by line.

Install Dependencies

We will be using several external NPM packages in our web application, such as Express. These are known as dependencies. We could install each package individually by running npm install <package name>. However, it is customary to provide a package.json file in the root of your project that lists all the project's dependencies as well as other identifying information about the application. This facilitates installation since all dependencies for all files in the project are listed in a central location and can all be installed at once with a single command.

When you create a new NodeJS project, you can run npm init and answer a series of questions to generate a default package.json file. We have included a complete package.json file in the repository.

Open a terminal in the node-webhook-listener-track directory and run the following command: npm install.

This command installs the dependencies we listed in thepackage.json file and creates a package.json.lock file to nail down the specific versions installed since the package.json file allows us to specify minimum versions.

Import Dependencies

We've now installed all the dependencies required for this application, and we are ready to take a look at the code, which resides in the index.js file.

We start by importing the tools and frameworks mentioned above, as well as a few others, at the top of our file. The code below includes all the packages needed by this application. We will be using express, which is a web framework that provides the web server we are using. We will configure express to use the body-parser package so that we can access the data in the requests sent to the endpoint. The other packages are used in other endpoints in this application.

1
2
const express = require('express');
const bodyParser = require('body-parser');

Create the Web Application

At this point, we have specified and installed dependencies, and we are ready to create our web application and define endpoints. When you configure webhooks in ShipEngine, you provide a URL to which ShipEngine will send an HTTP POST request with a JSON payload whenever a particular event occurs. We recommend that you create an individual endpoint for each type of webhook you wish to subscribe to and limit traffic on those endpoints to ShipEngine webhook traffic.

The code below creates an instance of an express web server and assigns it to the app variable. We then configure our express application to use the body-parser package. Since we called the variable bodyParser in the import statement above, that's how we reference it in the code below. The last line of code starts the server, listening on port 3000 for incoming requests. This line is customarily the last one in the script. We will be filling in our endpoint implementation in the space between.

1
2
3
4
5
6
7
8
9
const app = express();
app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());

...

let server = app.listen(3000, function() {
  console.log('Listening on port %d', server.address().port);
});

Create the /track Endpoint

We now have a very simple web application called app, but we have not defined any endpoints for our application. In this example, we are going to create a /track endpoint to use for tracking event webhooks. One of the benefits of developing and hosting your own web application is that you can programmatically trigger other events to occur once you receive the webhook, which we'll demonstrate in our example endpoints.

Before we attempt to implement the /track endpoint, let's take a look at the structure of the message we expect to receive. This is an example of the payload the /track endpoint will receive whenever a tracking event webhook is fired.

Sample Track Webhook Payload

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
{
  "resource_url": "https://api.shipengine.com/v1/tracking?carrier_code=usps&tracking_number=9400111298370264401222",
  "resource_type": "API_TRACK",
  "data": {
    "label_url": null,
    "tracking_number": "9400111298370264401222",
    "status_code": "IT",
    "carrier_detail_code": null,
    "status_description": "In Transit",
    "carrier_status_code": "NT",
    "carrier_status_description": "Your package is moving within the USPS network and is on track to be delivered the expected delivery date. It is currently in transit to the next facility.",
    "ship_date": "2020-06-30T16:09:00",
    "estimated_delivery_date": "2020-07-06T00:00:00",
    "actual_delivery_date": null,
    "exception_description": null,
    "events": [
      {
        "occurred_at": "2020-07-02T00:00:00Z",
        "carrier_occurred_at": "2020-07-02T00:00:00",
        "description": "In Transit, Arriving On Time",
        "city_locality": "",
        "state_province": "",
        "postal_code": "",
        "country_code": "",
        "company_name": "",
        "signer": "",
        "event_code": "NT",
        "event_description": "In Transit, Arriving On Time",
        "carrier_detail_code": null,
        "status_code": null,
        "latitude": null,
        "longitude": null
      },
      {
        "occurred_at": "2020-06-30T20:09:00Z",
        "carrier_occurred_at": "2020-06-30T16:09:00",
        "description": "Shipment Received, Package Acceptance Pending",
        "city_locality": "VERSAILLES",
        "state_province": "KY",
        "postal_code": "40383",
        "country_code": "",
        "company_name": "",
        "signer": "",
        "event_code": "TM",
        "event_description": "Shipment Received, Package Acceptance Pending",
        "carrier_detail_code": null,
        "status_code": null,
        "latitude": 37.8614,
        "longitude": -84.6646
      }
    ]
  }
}

You can see that the example payload above contains a status_code property, which you can use to find out where the package is in the delivery process. You can use the status_description for an explanation of the status_code. For example, the status_code 'DE' has a status_description of 'Delivered'.

This payload also contains a tracking_number, which you should be able to use to locate the customer and their contact data from within your application - not your webhook listener application but the application you used to create the label you want to receive tracking events for.

The code below is an example implementation of an endpoint that listens for HTTP POST requests. We do this by making a call to the app.post method of our express server instance. The first parameter we pass to the app.post method is the path we want to use for the endpoint, in this case /track. The second parameter is a callback function.

If you need more context on asynchronous programming and callback functions, check out this reference. The app.post method is asynchronous, meaning that the program does not wait for this call to return before moving to the next line of code in the file. Instead, we pass a callback function as the second parameter to tell the app.post method what to do when it completes. In this case, we define the callback function directly in the call to app.post. This is known as an anonymous function.

Once the endpoint receives a request, it invokes the callback function. In this case, we extract useful data from the payload that is sent from the webhook. We can look at the status_code to determine that the package has been delivered. We can use the tracking_number to look up the customer's contact information and send an email informing them that their package has been delivered, adding a personal touch to your service. We have access to this data on the request object because we configured the application to use the body-parser package.

Finally, we send a 200 HTTP status code to terminate the call and return program control to the main application. Whenever ShipEngine sends a tracking event webhook to your /track endpoint, this code will be called.

1
2
3
4
5
6
7
8
9
10
11
12
13
app.post('/track', (req, res) => {

  let trackingNumber = req.body.data.tracking_number;
  let statusCode = req.body.data.status_code;

  if(statusCode === 'DE') { // Package was delivered

    // Use the trackingNumber to get the contact
    // info associated with this package from your backend system.
    // Email the user that the package was delivered.
  }
  res.sendStatus(200);
});

Putting It All Together

We have now performed the following:

The complete script for the /track endpoint is included below. If you cloned our code-samples repository, you will find this example application, in the node-webhook-listener-track directory.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
const express = require('express');
const bodyParser = require('body-parser');

const app = express();
app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());

const API_KEY = YOUR_API_KEY;

app.post('/track', (req, res) => {

  let trackingNumber = req.body.data.tracking_number;
  let statusCode = req.body.data.status_code;

  if(statusCode === 'DE') { // Package was delivered

    // Use the trackingNumber to get the contact
    // info associated with this package from your backend system.
    // Email the user that the package was delivered.
  }
  res.sendStatus(200);
});

let server = app.listen(3000, function() {
  console.log('Listening on port %d', server.address().port);
});

Running the Webhook Listener

We have now written a web application that exposes a single endpoint. Let's run it locally to test it out. If you are developing in Visual Studio Code, follow these steps to run your application:

Testing the Webhook Listener

Your application is now running! But let's test it out before we try to use it to receive webhooks. Follow these steps to test that your application is working.

Validation

We copied and pasted the sample payload to use in the test request to our web application because we wrote the endpoint to expect certain properties in the request and we didn't provide any error handling. Before using the application in production, you will need check that the message is what you expect and only attempt to access those properties if you receive the correct message. Furthermore, you can check that requests received on your endpoint are coming from ShipEngine by inspecting the headers from within your endpoint. All requests coming from ShipEngine will have the user-agent header set to ShipEngine/v1.

Authentication

You may also wish to add security to your webhooks by using Basic Authentication. This would require you to supply the username and password directly in the URL as specified in the example below. It would prevent any traffic from reaching your endpoint that did not include the valid username and password in the URL.

Copy

HTTPcURLJavaScriptNodeJSPHPPythonRubyC#JavaGoSwiftObjective-CPowerShell

1
2
3
4
5
6
7
8
9
POST /v1/environment/webhooks HTTP/1.1
Host: api.shipengine.com
API-Key: __YOUR_API_KEY_HERE__
Content-Type: application/json
{
  "url": "https://username:[email protected]",
  "event": "batch"
}

Getting to Production

At this point, we have created a simple web application that listens for webhook requests on the /track endpoint. We tested it locally by sending a sample webhook payload using Postman. There are additional steps you will need to take before you can configure ShipEngine to use the endpoints exposed by this application. Namely, you will need to host your application and make it publicly accessible so that ShipEngine can reach it.

If you have an existing web application that integrates with ShipEngine, then you should be familiar with the steps required to host your webhook listener application. If not, you'll probably need to start by registering a domain name for your application. You will then need to select a cloud provider, such as AWS, Azure, or GCP to host your application and provide DNS services for your application.

You might also use a web hosting service that handles domain registration, hosting, security, and other facets of web hosting for you.