Punching firewalls with Mandrill webhooks

(blogarhythm ~ Fire Cracker - Ellegarden)

Mandrill is the transactional email service by the same folks who do MailChimp. I've written about it before, in particular how to use the mandrill-rails gem to simplify inbound webhook processing.

Mandrill webhooks are a neat, easy way for your application to respond to various events, from recording when users open email, to handling inbound mail delivery.

That all works fine if your web application lives on the public internet i.e. Mandrill can reach it to post the webhook. But that's not always possible: your development/test/staging environments for example; or perhaps production servers that IT have told you must be "locked down to the max".

Mandrill currently doesn't offer an official IP whitelist, so it's not possible to use firewall rules to just let Mandrill servers in. Mandrill does provide webhook authentication (supported by the mandrill-rails gem), but that solves a different problem: anyone can reach your server, but you can distinguish the legitimate webhook requests.

I thought I'd share a couple of techniques I've used to get Mandrill happily posting webhooks to my dev machine and servers behind firewalls.

Using HAProxy to reverse-proxy Mandrill Webhooks

If you have at least one internet-visible address, HAProxy is excellent for setting up a reverse-proxy to forward inbound Mandrill Webhooks to the correct machine inside the firewall. I'm currently using this for some staging servers so we can run real inbound mail scenarios.

Here's a simple scenario:

• gateway.mydomain.net - your publicly-visible server, with HAProxy installed and running OK
• internal/192.168.0.1 - a machine on an internal network that you want to receive webooks posted to 192.168.0.1:8080/inbox

Say the gateway machine already hosts http://gateway.mydomain.net, but we want to be able to tell Mandrill to post it's webhooks to http://gateway.mydomain.net/inbox_internal, and these (and only these) requests will be forwarded to http://192.168.0.1:8080/inbox.

Here are the important parts of the /etc/haproxy/haproxy.cfg used on the gateway machine:

global
  #...
 
defaults
  mode http                                # enable http mode which gives of layer 7 filtering
  #...

# this is HAProxy listening on http://gateway.mydomain.net
frontend app *:80                          
  default_backend webapp                   # set the default server for all requests
  # next we define a rule that will send requests to the internal_mandrill backend instead if the path starts with /inbox_internal
  acl req_mandrill_inbox_path path_beg /inbox_internal 
  use_backend internal_mandrill if req_mandrill_inbox_path 

# define a group of backend servers for the main web app
backend webapp                             
  server app1 127.0.0.1:8001             
 
# this is where we will send the Mandrill webhooks
backend internal_mandrill                  
  reqirep     ^([^\ ]*)\ /inbox_internal(.*) \1\ /inbox\2 
  server int1 192.168.0.1:8080  # add a server to this backend

Obviously the path mapping is optional (but neat to demonstrate), and I've left out all the normal HAProxy options like balancing, checks and SSL option forwarding that you might require in practice, but are not relevant to the issue at hand.

Job done! Our internal server remains hidden behind the firewall, but Mandrill can get through to it by posting webhooks to http://gateway.mydomain.net/inbox_internal.

Tunneling to dev machines with ngrok

For development, we usually don't want anything so permanent. There are quite a few services for tunneling to localhost, mainly with developers in mind. Lately I've been using ngrok which is living up to it's name - it rocks! Trival to setup and works like a dream. Say I'm developing a Rails app:

# run app locally (port 3000)
rails s
# run ngrok tunnel to port 3000
ngrok 3000

Once started, ngrok will give you http and https addresses that will tunnel to port 3000 on your machine. You can use these addresses in the Mandrill webhook and inbound domains configuration, and they'll work as long as you keep your app and ngrok running.