I have passwords stored in web2py using SHA 512 algorithm. I am now migrating the models to django and hence need a way to hash passwords in django using SHA 512 in the same way as web2py does so that I can authenticate the old users with the same passwords.Please suggest some way.
According to this post a Python snippet to recreate the convention used in web2py would be the following:
from hashlib import md5
import hmac
hmac_key = '<your secret key>'
password = 'insecure'
thehash = hmac.new(hmac_key, password).hexdigest()
print thehash
web2py uses hmac (which is your secret + the plaintext of the user's password) as the final hash and not just a straight MD5/SHA hash (depending on your settings). So you would just need to swap out MD5 for SHA in the above example to get things working on your end. But this implementation is all you would need to implement in your new application to make them cross compatible as long as the secret key is the same.
According to the docs the hash is stored in the following format:
<algorithm>$<salt>$<hash>
so if there is a salt used then it's stored with the hash making it easy to grab the salt for use in your new application. The dollar signs make it easy to parse each value.
algo, salt, hash = password_hash.split("$")
UPDATE: I pulled the below code from the web2py source but what you need to do is update the variable hmac_key with the value that you have set for auth.settings.hmac_key. Hopefully when you run (after you update the hmac_key variable) this the hashes should match.
import hashlib
import hmac
from hashlib import sha512
h="sha512$b850ed44943b861b$c90901439983bce7fd512592b20d83f8e654632dee51de515773e70eabe609f62cebec64fed4df03acd54e6a627c9291e70fdf3a89996ffa796897c159e95c11"
algo,salt,hash = h.split("$")
print "crypted hash: %s"%hash
pwd = "pawan123"
##get this value from auth.settings.hmac_key
hmac_key = ""
def get_digest(value):
"""
Returns a hashlib digest algorithm from a string
"""
if not isinstance(value, str):
return value
value = value.lower()
if value == "md5":
return md5
elif value == "sha1":
return sha1
elif value == "sha224":
return sha224
elif value == "sha256":
return sha256
elif value == "sha384":
return sha384
elif value == "sha512":
return sha512
else:
raise ValueError("Invalid digest algorithm: %s" % value)
#hashed = simple_hash(self.password, key, salt, digest_alg)
def simple_hash(text, key='', salt='', digest_alg='md5'):
"""
Generates hash with the given text using the specified
digest hashing algorithm
"""
if not digest_alg:
raise RuntimeError("simple_hash with digest_alg=None")
elif not isinstance(digest_alg, str): # manual approach
h = digest_alg(text + key + salt)
elif digest_alg.startswith('pbkdf2'): # latest and coolest!
iterations, keylen, alg = digest_alg[7:-1].split(',')
return pbkdf2_hex(text, salt, int(iterations),
int(keylen), get_digest(alg))
elif key: # use hmac
digest_alg = get_digest(digest_alg)
h = hmac.new(key + salt, text, digest_alg)
else: # compatible with third party systems
h = get_digest(digest_alg)()
h.update(text + salt)
return h.hexdigest()
print "result hash: %s"%simple_hash(pwd, hmac_key, salt, "sha512")
I think your best solution is to write an auth backend that will authenticate the User against the web2py base, then ask him to change or confirm his password and build a new Django auth-passwords.
The hole idea of crypto-hashing passwords is that you or any hacker can't see them if you have access to the database.
Here is the Django documentation on writing an authentication backend.
Related
I am trying to use Typeform's security for their webhooks. This involves
1) Receiving the signed packets and extracting the signature
2) Getting the body of the requst
3) Creating a hash with a secret key on the payload
4) Matching the hash with the received signature
My web framework is Django (Python based). I am following the example at the TypeForm link here: https://developer.typeform.com/webhooks/secure-your-webhooks/.
For the life of me, I can't figure out what's going on. I've tried it in both Python and Ruby, and I can't get the hash right. I call a Ruby script from Python to match the output, but they are different and neither work. Does anyone have any insight? I'm starting to think that it might have something to do with the way that Django sends request bodies. Does anyone have any input?
Python implementation:
import os
import hashlib
import hmac
import base64
import json
class Typeform_Verify:
# take the request body in and encrypt with string
def create_hash(payload):
# convert the secret string to bytes
file = open("/payload.txt", "w")
# write to a payload file for the ruby script to read later
file.write(str(payload))
# access the secret string
secret = bytearray(os.environ['DT_TYPEFORM_STRING'], encoding="utf-8")
file.close()
# need to have the ruby version also write to a file
# create a hash with payload as the thing
# and the secret as the key`
pre_encode = hmac.new(secret,
msg=payload, digestmod=hashlib.sha256).digest()
post_encode = base64.b64encode(pre_encode)
return post_encode
# another approach is to make a ruby script
# that returns a value and call it from here
def verify(request):
file = open("/output.txt", "w")
# check the incoming hash values
received_hash = request.META["HTTP_TYPEFORM_SIGNATURE"]
# create the hash of the payload
hash = Typeform_Verify.create_hash(request.body)
# call ruby script on it
os.system(f"ruby manager/ruby_version.rb {received_hash} &> /oops.txt")
# concatenate the strings together to make the hash
encoded_hash = "sha256=" + hash.decode("utf-8")
file.write(f"Secret string: {os.environ['DT_TYPEFORM_STRING']}\n")
file.write(f"My hash : {encoded_hash}\n")
file.write(f"Their hash : {received_hash}\n")
file.close()
return received_hash == encoded_hash
Ruby script (called from Python)
require 'openssl'
require 'base64'
require 'rack'
def verify_signature(received_signature, payload_body, secret)
hash = OpenSSL::HMAC.digest(OpenSSL::Digest.new('sha256'), secret, payload_body)
# the created signature
actual_signature = 'sha256=' + Base64.strict_encode64(hash)
# write created signature to the file
out_file = File.new("/output.txt", "a")
out_file.write("Ruby output: ")
out_file.write(actual_signature)
out_file.close()
return 500, "Signatures don't match!" unless Rack::Utils.secure_compare(actual_signature, received_signature)
end
# MAIN EXECUTION
# get the hash from the python scriupt
received_hash = ARGV[0]
# read the content of the file into the f array
# note that this is the json payload from the python script
f = IO.readlines("/payload.txt")
# declare the secret string
secret = "SECRET"
# call the funtion with the recieved hash, file data, and key
result = verify_signature(received_hash, f[0], secret)
Code output:
Typeform hash: sha256=u/A/F6u3jnG9mr8KZH6j8/gO+Uny6YbSYFz7+oGmOik=
Python hash: sha256=sq7Kl2qBwRrwgGJeND6my4UPli8rseuwaK+f/sl8dko=
Ruby output: sha256=BzMxPZGmxgOMeJ236eAxSOXj85rEWI84t+6CtQBYliA=
UPDATED First see this github article as the one you referred to may be based on it.
The idea is that your requests should be signed. Here is a more basic pure ruby example which should illustrate how this should work.
# test.rb
ENV['SECRET_TOKEN'] = 'foobar'
require 'openssl'
require 'base64'
require 'rack'
def stub_request(body)
key = ENV['SECRET_TOKEN']
digest = OpenSSL::Digest.new('sha256')
hmac_signature = OpenSSL::HMAC.hexdigest(digest, key, body)
{ body: body, hmac_signature: hmac_signature }
end
def verify_signature(payload_body, request_signature)
digest = OpenSSL::Digest.new('sha256')
hmac = OpenSSL::HMAC.hexdigest(OpenSSL::Digest.new('sha256'), ENV['SECRET_TOKEN'], payload_body)
if Rack::Utils.secure_compare(request_signature, hmac)
puts "They match"
else
puts "They don't match"
end
puts "request_signature: #{request_signature}"
puts " hmac: #{hmac}"
puts " body: #{payload_body}"
end
request = stub_request(ARGV[0])
verify_signature(request[:body], request[:hmac_signature])
Now to test this, just run:
ruby test.rb 'this is some random body string'
Here is a Python version of the same code. But this is vulnerable to timing attack vulnerability. There is probably a Python equivalent somewhere to mitigate this but I didn't do the research to find it. It shouldn't be hard to write something like the Ruby Rack version here in Python if your server doesn't already have something like it.
#test.py
import sys
import hashlib
import binascii
import hmac
import base64
KEY = 'foobar'
def stub_request(body):
key = bytes(KEY, 'utf-8')
body_bytes = bytes(body, 'utf-8')
hmac_signature = hmac.new(key,
msg=body_bytes, digestmod=hashlib.sha256).digest()
return {'body': body, 'hmac_signature': hmac_signature}
def verify_signature(payload_body, request_signature):
key = bytes(KEY, 'utf-8')
hmac_sig = hmac.new(key, msg=bytes(payload_body,'utf-8'), digestmod=hashlib.sha256).digest()
if hmac_sig == request_signature:
print("They match")
else :
print("They don't match")
print(f"request_signature: {binascii.hexlify(request_signature)}")
print(f" hmac: {binascii.hexlify(hmac_sig)}")
print(f" body: {payload_body}")
return request_signature
body = sys.argv[-1]
request = stub_request(body)
verify_signature(request['body'], request['hmac_signature'])
I ended up figuring it out. The Python implementation I had worked fine. The problem was in how I was saving the secret string. Apparently, environment variables in Python will not allow characters like $ or *. My Ruby implementation started working when I hardcoded my secret into the code, which led me to believe that the problem was in how I was saving the secret string. I recommend the Python implementation to anyone trying to do this kind of authentication. Cheers!
if I create a hash using django's django.contrib.auth.hashers.make_password of the same string I get different hash every time. I don't understand how is this legal because as far as I know, hash functions must generate the same hash every time since by definition its a function. What am I missing?
from django.contrib.auth.hashers import make_password
password = "helloworld"
h1 = make_password(password)
h2 = make_password(password)
print h1, h2
h1 = u'pbkdf2_sha256$20000$Tr6NV5MewXYl$X+sezT6WRqBwYmJR/RZmZHLP6/l6ntSaBke0RKU1/v0='
h2 = u'pbkdf2_sha256$20000$05rEmxChtXlI$NdZGfTKH+kqt1viuFng3GmvBp6eJcsstxV4JcDlBGIs='
I suspect that different algorithms are used to hash every time and hence the hash is also different. Am I correct?
You see different results because of the salt. In simple words Django add some random string to the password before hashing to get different values even for same password. This makes rainbaw tables attack are useless. Actually what you see in DB is not plain hash value, it's structure in following format: <algorithm>$<iterations>$<salt>$<hash>
Each time you use make_password, the password is hashed with a different salt. Django stores the salt with the hashed password. You can then use check_password to check the password later.
from django.contrib.auth.hashers import check_password, make_password
password = "helloworld"
h1 = make_password(password)
check_password(password, h1) # returns True
check_password("incorrect", h1) # returns False
Read the docs on how Django stores passwords for more info.
from cryptography.fernet import Fernet
import base64
# Put this somewhere safe!
key = Fernet.generate_key()
f = Fernet()
token = f.encrypt(b"A really secret message. Not for prying eyes.")
token
print f.decrypt(token)
How can I generate my own key instead of fernet.genrate_key()?
The implementation shows how this is done:
return base64.urlsafe_b64encode(os.urandom(32))
So to generate your own you'll want to generate 32 cryptographically secure random bytes and then urlsafe base64 encode them. Of course, since generate_key already does this you should probably just call that unless you need to generate the key outside of your Python process.
In Bash, you can do:
dd if=/dev/urandom bs=32 count=1 2>/dev/null | openssl base64
Source: The Ruby implementation of Fernet
In fernet a key can be generated using one of fernet's Key Derivation Functions
One of the functions provided by fernet is the 'Password Based Key Derivation Function 2'.
An example that uses PBKDF2HMAC can be found at Using Passwords with Fernet. This is discussed in git issue #1333 of pyca/cryptography, maebert points out that the example uses salt=os.urandom(16) and will generate a new key from a password each time the kdf class is constructed with a different salt value.
If you need to use a custom key derivation function look at source code for kdf
and pbkdf2
to have an example of a class that implements the KeyDerivationFunction interface.
A class that matches its signature and implements the interface should be able to be dropped in as a custom key derivation function.
Here is how to do this using a passcode, unsalted. Note this method does not generate a very secure key:
from cryptography.fernet import Fernet
import base64, hashlib
def gen_fernet_key(passcode:bytes) -> bytes:
assert isinstance(passcode, bytes)
hlib = hashlib.md5()
hlib.update(passcode)
return base64.urlsafe_b64encode(hlib.hexdigest().encode('latin-1'))
Usage:
passcode = '249524.405925.606329'
key = gen_fernet_key(passcode.encode('utf-8'))
fernet = Fernet(key)
data_in = "SOME DATA"
cypher_text = fernet.encrypt(data_in.encode('utf-8'))
decr_data = fernet.decrypt(cypher_text).decode('utf-8')
print(f"passcode: {passcode}")
print(f"data_in: {data_in}")
print(f"key: {key}")
print(f"cypher_text: {cypher_text}")
print(f"decr_data: {decr_data}")
Output:
passcode: 249524.405925.606329
data_in: SOME DATA
key: b'NWRmMTk3ZWUwY2RjNjA3NWY4NzQ2NmQyOGRkYzczMmM='
cypher_text: b'gAAAAABit-SGVb4JMb-AWCetN-T029YzxQBkRou3fQElSY0zidJbM7M5w5TeJzIacyMaFycmUxFPYrSDgDnOrhC0OggtJ_xDMw=='
decr_data: SOME DATA
The python Cyrptography Fernet documentation has a note on generating a secure key using a password and salt:
import base64
import os
from cryptography.fernet import Fernet
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
password = b"password"
salt = os.urandom(16)
kdf = PBKDF2HMAC(
algorithm=hashes.SHA256(),
length=32,
salt=salt,
iterations=390000,
)
key = base64.urlsafe_b64encode(kdf.derive(password))
f = Fernet(key)
token = f.encrypt(b"Secret message!")
token
# b'...'
f.decrypt(token)
# b'Secret message!'
Note, the salt must be stored along with the passcode in order to re-generate the same key; which makes this feel a bit redundant, although it does add a layer of obfuscation. Its likely best just to use Fernet.generate_key() and store that.
I'm trying to do some authentication from my previous django web app in node. I got PBKDF2-sha256 working but I'm not able to get the BCryptSHA256PasswordHasher working in Node. I tried the following:
var Bcrypt = require('bcrypt');
var sha256 = require('sha256');
var pass = sha256("test password")
// from django ("bcrypt_sha256$$2b$12$mUg9hoKn0tt2/VwWaNb6Euie4.jtQjfU6.CY1pT0EH8GPORqAsh66")
var hash = "$2b$12$mUg9hoKn0tt2/VwWaNb6Euie4.jtQjfU6.CY1pT0EH8GPORqAsh66"
Bcrypt.compare(pass, hash, function (err, isMatch) {
if (err) {
return console.error(err);
}
console.log('do they match?', isMatch);
});
Is there something i'm missing with the above? I'm taking the sha256 of the password and testing with bcrypt. The corresponding code in Django is below:
def verify(self, password, encoded):
algorithm, data = encoded.split('$', 1)
assert algorithm == self.algorithm
bcrypt = self._load_library()
# Hash the password prior to using bcrypt to prevent password truncation
# See: https://code.djangoproject.com/ticket/20138
if self.digest is not None:
# We use binascii.hexlify here because Python3 decided that a hex encoded
# bytestring is somehow a unicode.
password = binascii.hexlify(self.digest(force_bytes(password)).digest())
else:
password = force_bytes(password)
# Ensure that our data is a bytestring
data = force_bytes(data)
# force_bytes() necessary for py-bcrypt compatibility
hashpw = force_bytes(bcrypt.hashpw(password, data))
return constant_time_compare(data, hashpw)
UPDATE
I have no idea why, but when I change the salt slightly to the following:
var hash = "$2a$12$mUg9hoKn0tt2/VwWaNb6Euie4.jtQjfU6.CY1pT0EH8GPORqAsh66"
everything works! I changed the 2b to 2a at the beginning. Why is this working and the other isn't? Is there something i'm missing?
From the excellent Passlib library:
ident (str) – Specifies which version of the BCrypt algorithm will be used when creating a new hash. Typically this option is not needed,
as the default ("2a") is usually the correct choice. If specified, it
must be one of the following:
"2" - the first revision of BCrypt, which suffers from a minor security flaw and is generally not used anymore. "2a" - some
implementations suffered from a very rare security flaw. current
default for compatibility purposes.
"2y" - format specific to the crypt_blowfish BCrypt implementation, identical to "2a" in all but name.
"2b" - latest revision of the official BCrypt algorithm (will be default in Passlib 1.7).
I'm using Django to manage a Postgres database. I have a value stored in the database representing a city in Spain (Málaga). My Django project uses unicode strings for everything by putting from __future__ import unicode_literals at the beginning of each of the files I created.
I need to pull the city information from the database and send it to another server using an XML request. There is logging in place along the way so that I can observe the flow of data. When I try and log the value for the city I get the following traceback:
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in position 1: ordinal not in range(128)
Here is the code I use to log the values I'm passing.
def createXML(self, dict):
"""
.. method:: createXML()
Create a single-depth XML string based on a set of tuples
:param dict: Set of tuples (simple dictionary)
"""
xml_string = ''
for key in dict:
self.logfile.write('\nkey = {0}\n'.format(key))
if (isinstance(dict[key], basestring)):
self.logfile.write('basestring\n')
self.logfile.write('value = {0}\n\n'.format(dict[key].decode('utf-8')))
else:
self.logfile.write('value = {0}\n\n'.format(dict[key]))
xml_string += '<{0}>{1}</{0}>'.format(key, dict[key])
return xml_string
I'm basically saving all the information I have in a simple dictionary and using this function to generate an XML formatted string - this is beyond the scope of this question.
The error I am getting had me wondering what was actually being saved in the database. I have verified the value is utf-8 encoded. I created a simple script to extract the value from the database, decode it and print it to the screen.
from __future__ import unicode_literals
import psycopg2
# Establish the database connection
try:
db = psycopg2.connect("dbname = 'dbname' \
user = 'user' \
host = 'IP Address' \
password = 'password'")
cur = db.cursor()
except:
print "Unable to connect to the database."
# Get database info if any is available
command = "SELECT state FROM table WHERE id = 'my_id'"
cur.execute(command)
results = cur.fetchall()
state = results[0][0]
print "my state is {0}".format(state.decode('utf-8'))
Result: my state is Málaga
In Django I'm doing the following to create the HTTP request:
## Create the header
http_header = "POST {0} HTTP/1.0\nHost: {1}\nContent-Type: text/xml\nAuthorization: Basic {2}\nContent-Length: {3}\n\n"
req = http_header.format(service, host, auth, len(self.xml_string)) + self.xml_string
Can anyone help me correct the problem so that I can write this information to the database and be able to create the req string to send to the other server?
Am I getting this error as a result of how Django is handling this? If so, what is Django doing? Or, what am I telling Django to do that is causing this?
EDIT1:
I've tried to use Django's django.utils.encoding on this state value as well. I read a little from saltycrane about a possible hiccup Djano might have with unicode/utf-8 stuff.
I tried to modify my logging to use the smart_str functionality.
def createXML(self, dict):
"""
.. method:: createXML()
Create a single-depth XML string based on a set of tuples
:param dict: Set of tuples (simple dictionary)
"""
xml_string = ''
for key in dict:
if (isinstance(dict[key], basestring)):
if (key == 'v1:State'):
var_str = smart_str(dict[key])
for index in range(0, len(var_str)):
var = bin(ord(var_str[index]))
self.logfile.write(var)
self.logfile.write('\n')
self.logfile.write('{0}\n'.format(var_str))
xml_string += '<{0}>{1}</{0}>'.format(key, dict[key])
return xml_string
I'm able to write the correct value to the log doing this but I narrowed down another possible problem with the .format() string functionality in Python. Of course my Google search of python format unicode had the first result as Issue 7300, which states that this is a known "issue" with Python 2.7.
Now, from another stackoverflow post I found a "solution" that does not work in Django with the smart_str functionality (or at least I've been unable to get them to work together).
I'm going to continue digging around and see if I can't find the underlying problem - or at least a work-around.
EDIT2:
I found a work-around by simply concatenating strings rather than using the .format() functionality. I don't like this "solution" - it's ugly, but it got the job done.
def createXML(self, dict):
"""
.. method:: createXML()
Create a single-depth XML string based on a set of tuples
:param dict: Set of tuples (simple dictionary)
"""
xml_string = ''
for key in dict:
xml_string += '<{0}>'.format(key)
if (isinstance(dict[key], basestring)):
xml_string += smart_str(dict[key])
else:
xml_string += str(dict[key])
xml_string += '<{0}>'.format(key)
return xml_string
I'm going to leave this question unanswered as I'd love to find a solution that lets me use .format() the way it was intended.
This is correct approach (problem was with opening file. With UTF-8 You MUST use codecs.open() :
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import codecs
class Writer(object):
logfile = codecs.open("test.log", "w", 'utf-8')
def createXML(self, dict):
xml_string = ''
for key, value in dict.iteritems():
self.logfile.write(u'\nkey = {0}\n'.format(key))
if (isinstance(value, basestring)):
self.logfile.write(u'basestring\n')
self.logfile.write(u'value = {0}\n\n'.format( value))
else:
self.logfile.write(u'value = {0}\n\n'.format( value ))
xml_string += u'<{0}>{1}</{0}>'.format(key, value )
return xml_string
And this is from python console:
In [1]: from test import Writer
In [2]: d = { 'a' : u'Zażółć gęślą jaźń', 'b' : u'Och ja Ci zażółcę' }
In [3]: w = Writer()
In [4]: w.createXML(d)
Out[4]: u'<a>Za\u017c\xf3\u0142\u0107 g\u0119\u015bl\u0105 ja\u017a\u0144</a><b>Och ja Ci za\u017c\xf3\u0142c\u0119</b>'
And this is test.log file:
key = a
basestring
value = Zażółć gęślą jaźń
key = b
basestring
value = Och ja Ci zażółcę