Typeform Security API and Django: Not Verifiying Hash Correctly - django

Question

I am trying to use Typeform's security for their webhooks. This involves
1) Receiving the signed packets and extracting the signature
2) Getting the body of the requst
3) Creating a hash with a secret key on the payload
4) Matching the hash with the received signature
My web framework is Django (Python based). I am following the example at the TypeForm link here: https://developer.typeform.com/webhooks/secure-your-webhooks/.
For the life of me, I can't figure out what's going on. I've tried it in both Python and Ruby, and I can't get the hash right. I call a Ruby script from Python to match the output, but they are different and neither work. Does anyone have any insight? I'm starting to think that it might have something to do with the way that Django sends request bodies. Does anyone have any input?
Python implementation:
import os
import hashlib
import hmac
import base64
import json
class Typeform_Verify:
# take the request body in and encrypt with string
def create_hash(payload):
# convert the secret string to bytes
file = open("/payload.txt", "w")
# write to a payload file for the ruby script to read later
file.write(str(payload))
# access the secret string
secret = bytearray(os.environ['DT_TYPEFORM_STRING'], encoding="utf-8")
file.close()
# need to have the ruby version also write to a file
# create a hash with payload as the thing
# and the secret as the key`
pre_encode = hmac.new(secret,
msg=payload, digestmod=hashlib.sha256).digest()
post_encode = base64.b64encode(pre_encode)
return post_encode
# another approach is to make a ruby script
# that returns a value and call it from here
def verify(request):
file = open("/output.txt", "w")
# check the incoming hash values
received_hash = request.META["HTTP_TYPEFORM_SIGNATURE"]
# create the hash of the payload
hash = Typeform_Verify.create_hash(request.body)
# call ruby script on it
os.system(f"ruby manager/ruby_version.rb {received_hash} &> /oops.txt")
# concatenate the strings together to make the hash
encoded_hash = "sha256=" + hash.decode("utf-8")
file.write(f"Secret string: {os.environ['DT_TYPEFORM_STRING']}\n")
file.write(f"My hash : {encoded_hash}\n")
file.write(f"Their hash : {received_hash}\n")
file.close()
return received_hash == encoded_hash
Ruby script (called from Python)
require 'openssl'
require 'base64'
require 'rack'
def verify_signature(received_signature, payload_body, secret)
hash = OpenSSL::HMAC.digest(OpenSSL::Digest.new('sha256'), secret, payload_body)
# the created signature
actual_signature = 'sha256=' + Base64.strict_encode64(hash)
# write created signature to the file
out_file = File.new("/output.txt", "a")
out_file.write("Ruby output: ")
out_file.write(actual_signature)
out_file.close()
return 500, "Signatures don't match!" unless Rack::Utils.secure_compare(actual_signature, received_signature)
end
# MAIN EXECUTION
# get the hash from the python scriupt
received_hash = ARGV[0]
# read the content of the file into the f array
# note that this is the json payload from the python script
f = IO.readlines("/payload.txt")
# declare the secret string
secret = "SECRET"
# call the funtion with the recieved hash, file data, and key
result = verify_signature(received_hash, f[0], secret)
Code output:
Typeform hash: sha256=u/A/F6u3jnG9mr8KZH6j8/gO+Uny6YbSYFz7+oGmOik=
Python hash: sha256=sq7Kl2qBwRrwgGJeND6my4UPli8rseuwaK+f/sl8dko=
Ruby output: sha256=BzMxPZGmxgOMeJ236eAxSOXj85rEWI84t+6CtQBYliA=

Answer 1

UPDATED First see this github article as the one you referred to may be based on it.
The idea is that your requests should be signed. Here is a more basic pure ruby example which should illustrate how this should work.
# test.rb
ENV['SECRET_TOKEN'] = 'foobar'
require 'openssl'
require 'base64'
require 'rack'
def stub_request(body)
key = ENV['SECRET_TOKEN']
digest = OpenSSL::Digest.new('sha256')
hmac_signature = OpenSSL::HMAC.hexdigest(digest, key, body)
{ body: body, hmac_signature: hmac_signature }
end
def verify_signature(payload_body, request_signature)
digest = OpenSSL::Digest.new('sha256')
hmac = OpenSSL::HMAC.hexdigest(OpenSSL::Digest.new('sha256'), ENV['SECRET_TOKEN'], payload_body)
if Rack::Utils.secure_compare(request_signature, hmac)
puts "They match"
else
puts "They don't match"
end
puts "request_signature: #{request_signature}"
puts " hmac: #{hmac}"
puts " body: #{payload_body}"
end
request = stub_request(ARGV[0])
verify_signature(request[:body], request[:hmac_signature])
Now to test this, just run:
ruby test.rb 'this is some random body string'
Here is a Python version of the same code. But this is vulnerable to timing attack vulnerability. There is probably a Python equivalent somewhere to mitigate this but I didn't do the research to find it. It shouldn't be hard to write something like the Ruby Rack version here in Python if your server doesn't already have something like it.
#test.py
import sys
import hashlib
import binascii
import hmac
import base64
KEY = 'foobar'
def stub_request(body):
key = bytes(KEY, 'utf-8')
body_bytes = bytes(body, 'utf-8')
hmac_signature = hmac.new(key,
msg=body_bytes, digestmod=hashlib.sha256).digest()
return {'body': body, 'hmac_signature': hmac_signature}
def verify_signature(payload_body, request_signature):
key = bytes(KEY, 'utf-8')
hmac_sig = hmac.new(key, msg=bytes(payload_body,'utf-8'), digestmod=hashlib.sha256).digest()
if hmac_sig == request_signature:
print("They match")
else :
print("They don't match")
print(f"request_signature: {binascii.hexlify(request_signature)}")
print(f" hmac: {binascii.hexlify(hmac_sig)}")
print(f" body: {payload_body}")
return request_signature
body = sys.argv[-1]
request = stub_request(body)
verify_signature(request['body'], request['hmac_signature'])

Answer 2

I ended up figuring it out. The Python implementation I had worked fine. The problem was in how I was saving the secret string. Apparently, environment variables in Python will not allow characters like $ or *. My Ruby implementation started working when I hardcoded my secret into the code, which led me to believe that the problem was in how I was saving the secret string. I recommend the Python implementation to anyone trying to do this kind of authentication. Cheers!