Reverse engineering X-Twilio-Signature

Here's what I learned while reverse engineering the X-Twilio-Signature header to emulate requests in Paw.

Reverse engineering X-Twilio-Signature

While setting up a Paw project to emulate webhooks sent by Twilio for incoming SMS messages, I ran into a problem generating the X-Twilio-Signature header. Calculating the header is a multi-step process and relies on a lot of the HTTP request data. Paw has a HMAC-SHA function, but taking the input URL, sorting the form data joning the form data without field separators, and then appending the sorted form data... it doesn't do that.

I would have to calculate the header by hand, manually adjusting it every time I change. Or I could write an extension to calculate the header for me. As a software developer, I obviously chose to shave the yak write the extension.

Of course, what seems simple at first is never quite as easy as one might think. I followed Twilio's docs, but for some reason I was still getting a different header value than the request I captured. Here's what I learned.

Takeaway #1: Form data isn't URL encoded

When Twilio calculates the signature, their input data isn't URL encoded. You can see this in their security docs. Look at the URL with the To and From phone numbers:

The phone numbers weren't URL encoded before being used to calculate the X-Twilio-Signature header.

Since Paw does URL encode the phone numbers in the extension context, that needs to be reversed when calculating the input data for the signature header.

Correct: +18005551212 (+ is correct)
Wrong: %2B18005551212 (%2B is incorrect)

Takeaway #2: Seriously, don't URL encode the form data

After fixing the two phone numbers in my request URL, the HMAC-SHA1 function still didn't give me the correct output. The docs didn't have a body field, but what I learned about the URL encoding had me take another look at the other form data. Paw had URL encoded the phone numbers, and it had replaced spaces in the Body field with plus characters. Unfortunately, Javascript's decodeURIcomponent function didn't fix this, so I tried doing it by hand. And it worked.

Correct body: Hello world (Use an actual space character: )
Wrong body: Hello+world (Don't replace whitespace with a +)

And that was it. Suddenly the header I was calculating exactly matched the one from the request I saved. Problem solved. Extension working. Back to my actual project.