I have two servers running django. I'll call one server my "logging" server and another my "client" server. The client server wants to log a message with the logging server by passing over a username, password, and message over the internet. With my current implementation I'm hitting an error when trying to decrypt the encrypted message, username, and password that was sent over the wire. It looks like I should be trying to decrypt a "byte string" according to the pycrypto documentation, but I can't seem to create a byte string correctly since I haven't been able to get around this problem. Also, it feels like my implementation is taking me down a rabbit hole of security vulnerabilities and codec confusion. Is there a package that I should look at which already implements this type of functionality? If so what would that implementation look like?
client:
from Crypto.Hash import MD5
from Crypto.PublicKey import RSA
from base64 import b64decode
import urllib2
import urllib
#I realize recreating the hash everytime is slow. I just included it here for simplicity.
logger_public_signature_message = "I am a client :)"
logger_public_signature_hash = MD5.new(logger_public_signature_message).digest()
client_private_key = #private key
logger_public_key = #public key
client_private = RSA.importKey(client_private_key)
client_public = client_private.publickey()
logger_public = RSA.importKey(logger_public_key)
message = "my message"
username = "user"
password = "password"
encrypted_message = logger_public.encrypt(message, "ignored_param")
encrypted_username = logger_public.encrypt(username, "ignored_param")
encrypted_password = logger_public.encrypt(password, "ignored_param")
signature = client_private.sign(logger_public_signature_hash, '')
params = { "message": encrypted_message, "username": encrypted_username, "password": encrypted_password, "signature": signature }
url_encoded_params = urllib.urlencode(params)
url = 'http://localhost:8000/url/to/logger/'
req = urllib2.Request(url, url_encoded_params)
logger:
from Crypto.Hash import MD5
from Crypto.PublicKey import RSA
from base64 import b64decode
def log(request):
#I realize recreating the hash everytime is slow. I just included it here for simplicity.
logger_public_signature_message = "I am a client :)"
logger_public_signature_hash = MD5.new(logger_public_signature_message).digest()
client_public_key = #client public key
logger_private_key = #logger private key
client_public = RSA.importKey(client_public_key)
logger_private = RSA.importKey(logger_private_key)
p = request.POST
encrypted_message = urllib2.unquote(p["message"])
encrypted_username = urllib2.unquote(p["username"])
encrypted_password = urllib2.unquote(p["password"])
signature = urllib2.unquote(p["signature"])
#I'm hitting exceptions when trying to decrypt the encrypted messages.
#The exceptions are: "ValueError: Message too large" I think the issue is that
#I'm trying to decrypt a base64 string where I should be trying to decrypt a byte
#string from reading the documentation. But I haven't been able I guess to correctly
#create a byte string because I can't get it to work.
decrypted_message = logger_private.decrypt(encrypted_message.encode("base64"))
decrypted_username = logger_private.decrypt(encrypted_username.encode("base64"))
decrypted_password = logger_private.decrypt(encrypted_password.encode("base64"))
verified = client_public.verify(logger_public_signature_hash, signature)
I think you are putting a lot of effort into stuff, that doesn't need to be handled by Django.
Here is what I would usually do:
Use HTTPS, as transport encryption layer
Use HTTP Basic Auth. Basic auth is implemented in urllib2 as well as requests.
But there is an even better solution: Django REST framework
It provides you will a full blown REST API including different authentication solutions.
If you need any help, setting up one of these options, let me know and I'll add an example.
May it be that you would use sentry for logging? Of course if it isn't task for training.
I look at the sentry since it been django application, and now it surely is excellent production-ready solution.
We're using it in banking-sphere software development.
You are very close to decrypting the values on the server. The result of the encryption on the client is a tuple. When you urllib2.unquote the items on the server, you then recreate tuples from them.
For example:
>>> c = public.encrypt('Hello', "ignored")
>>> c
('3\xae0\x1f\xd7\xe4b\xd4\xf1\xf4\x88!Be\xff!\x1e\xda\x82\x10\x9bRy\x0c\xa0v\xed\x84\xf9\xe35\xc6QG\xcf\xb7\x1b\xea\x9fe\t\x9b\x8d\xd6\xf3\x8cw\xde\x17\xb5\xf7\x9a+\x84i%#\x8a\xdf\xf4\xdd\xc8wY',)
which in your code you pack into params like this:
>>> params = { "message" : c }
>>> params
{'message': ('3\xae0\x1f\xd7\xe4b\xd4\xf1\xf4\x88!Be\xff!\x1e\xda\x82\x10\x9bRy\x0c\xa0v\xed\x84\xf9\xe35\xc6QG\xcf\xb7\x1b\xea\x9fe\t\x9b\x8d\xd6\xf3\x8cw\xde\x17\xb5\xf7\x9a+\x84i%#\x8a\xdf\xf4\xdd\xc8wY',)}
>>> urllib.urlencode(params)
'message=%28%273%5Cxae0%5Cx1f%5Cxd7%5Cxe4b%5Cxd4%5Cxf1%5Cxf4%5Cx88%21Be%5Cxff%21%5Cx1e%5Cxda%5Cx82%5Cx10%5Cx9bRy%5Cx0c%5Cxa0v%5Cxed%5Cx84%5Cxf9%5Cxe35%5Cxc6QG%5Cxcf%5Cxb7%5Cx1b%5Cxea%5Cx9fe%5Ct%5Cx9b%5Cx8d%5Cxd6%5Cxf3%5Cx8cw%5Cxde%5Cx17%5Cxb5%5Cxf7%5Cx9a%2B%5Cx84i%25%40%5Cx8a%5Cxdf%5Cxf4%5Cxdd%5Cxc8wY%27%2C%29'
I would guess that urllib2.unquote(p["message"]) returns this (but I did not try this):
"('3\\xae0\\x1f\\xd7\\xe4b\\xd4\\xf1\\xf4\\x88!Be\\xff!\\x1e\\xda\\x82\\x10\\x9bRy\\x0c\\xa0v\\xed\\x84\\xf9\\xe35\\xc6QG\\xcf\\xb7\\x1b\\xea\\x9fe\\t\\x9b\\x8d\\xd6\\xf3\\x8cw\\xde\\x17\\xb5\\xf7\\x9a+\\x84i%#\\x8a\\xdf\\xf4\\xdd\\xc8wY',)"
then you can recreate the tuple at the server like this (m is the unquoted message):
>>> from ast import literal_eval
>>> literal_eval(m)
('3\xae0\x1f\xd7\xe4b\xd4\xf1\xf4\x88!Be\xff!\x1e\xda\x82\x10\x9bRy\x0c\xa0v\xed\x84\xf9\xe35\xc6QG\xcf\xb7\x1b\xea\x9fe\t\x9b\x8d\xd6\xf3\x8cw\xde\x17\xb5\xf7\x9a+\x84i%#\x8a\xdf\xf4\xdd\xc8wY',)
once you have the tuple back, you can decrypt it:
>>> private.decrypt(literal_eval(m))
'Hello'
It would be better to find a vetted and standard mechanism to do this rather than roll your own. For example, in your scheme, I could capture different messages between the client and server, and then mix and match messages and username/password pairs, making it appear that the messages came from different users.
However, with just this minor tweak (recreating the tuples from the "unquoted" strings) your code should decrypt just fine.
Related
Good evening everyone I had to implement the twilio module in my django project and it works well, the problem is that it does not allow you to send sms to phone numbers other than mine
I think sample code is like the following. You need set the two environment variables TWILIO_ACCOUNT_SID and TWILIO_AUTH_TOKEN.
import os
from twilio.rest import Client
account_sid = os.environ['TWILIO_ACCOUNT_SID']
auth_token = os.environ['TWILIO_AUTH_TOKEN']
client = Client(account_sid, auth_token)
message = client.messages.create(
body="your message",
from_='+15017122661',
to='+15558675310'
)
print(message.sid)
I am making an app in flutter which uses Google sign-in. I also have a Django backend linked to the app and I want to verify the user in the Django backend. I found many solutions on the internet but none is working. Probably I am messing up somewhere.
I tried using python-jose for verification and here is the code:
from jose import jwt
import urllib.request, json
token = '<token recieved using await user.getIdToken in flutter>'
target_audience = "<tried projectid/appid>"
certificate_url = 'https://www.googleapis.com/robot/v1/metadata/x509/securetoken#system.gserviceaccount.com'
response = urllib.request.urlopen(certificate_url)
certs = response.read()
certs = json.loads(certs)
print(certs)
user = jwt.decode(token, certs, algorithms='RS256',
audience=target_audience)
I also tried oauth2client, the code is here:
from oauth2client import crypt
import urllib.request, json
certificate_url = 'https://www.googleapis.com/robot/v1/metadata/x509/securetoken#system.gserviceaccount.com'
target_audience = 'tried projectid/appid'
response = urllib.request.urlopen(certificate_url)
certs = response.read()
certs = json.loads(certs)
print(certs)
crypt.MAX_TOKEN_LIFETIME_SECS = 30 * 86400
idtoken = 'token received from await user.getIdToken()'
crypt.verify_signed_jwt_with_certs(idtoken, certs, target_audience)
I also tried firebase_admin for python:
import firebase_admin
from firebase_admin import credentials
from firebase_admin import auth
cred = credentials.Certificate('<firebase service accounts private key>')
default_app = firebase_admin.initialize_app(cred)
token = 'token from flutter'
verifyied =auth.verify_id_token(id_token=token)
Just to check whether the firebase_admin library itself is working or not, I passed the userid to server from the app and tried deleting the user using firebase_admin and I could do that. But for some reason I am unable to verify the token.
Thanks for the help.
I have also faced the same issue.
Case:
Initially: I was printing auth token in vscode console and was verifying in terminal.
It gave me the error: token length cannot be 1 more than % 4.
I tried verifying the token from jwt.io and it was seemingly correct.
Actual Reason for the issue:
The console output of vscode (in my case windows 7 and 64 bit). Is limited to 1064 characters for a line.
Although the actual length of token is supposed to be 1170 characters.
Workaround Solution:
Print the substring in the vscode console and the join them in python shell to verify.
Answering my own question. The problem was that my server was not actually deployed, so, I was copying the token printed in vscode console when the user logs in and pasting it into the python code and trying to verify the token. Turns out it doesn't work that way.
I hosted my Django app and passed the token in a post request and then tried to verify the token and it worked.
You can refer the solutions here if you are stuck :
https://coders-blogs.blogspot.com/2018/11/authenticating-user-on-backend-of-your.html
I want to have a lambda calling a Sagemaker instance in another region. If both are in the same region, everything works fine. If they are not, I get the following error:
The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
The Canonical String for this request should have been
'POST
/endpoints/foo-endpoint/invocations
host:runtime.sagemaker.us-east-1.amazonaws.com
x-amz-date:20180406T082536Z
host;x-amz-date
1234567890foobarfoobarfoobarboofoobarfoobarfoobarfoobarfoobarfoo'
The String-to-Sign should have been
'AWS4-HMAC-SHA256
20180406T082536Z
20180406/us-east-1/sagemaker/aws4_request
987654321abcdeffoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarf'
I use aws-requests-auth (0.4.1) with boto3 (1.5.15 - updating to 1.7.1 didn't change anything, changelog) like this:
import requests
from aws_requests_auth.aws_auth import AWSRequestsAuth
auth = AWSRequestsAuth(aws_access_key=config['AWS']['ACCESS_KEY'],
aws_secret_access_key=(
config['AWS']['SECRET_ACCESS_KEY']),
aws_host=config['AWS']['HOST'],
aws_region=config['AWS']['REGION'],
aws_service=config['AWS']['SERVICE'])
payload = {'foo': 'bar'}
response = requests.post(post_url,
data=json.dumps(payload),
headers={'content-type': 'application/json'},
auth=auth)
printing auth only gives <aws_requests_auth.aws_auth.AWSRequestsAuth object at 0x7f9d00c98390>.
Is there a way to print the "Canonical String" mentioned in the error message?
(Any other ideas how to fix this are appreciated as well)
A work-around for the asked question:
req = requests.request('POST', 'http://httpbin.org/get')
req.body = b''
req.method = ''
print(auth.get_aws_request_headers(req,
aws_access_key=auth.aws_access_key,
aws_secret_access_key=auth.aws_secret_access_key,
aws_token=auth.aws_token))
The problem is not solved, though. And now I wonder what the first argument of auth.get_aws_request_headers is.
First time poster, and I'm not really a developer, so perspective is always appreciated :)
Objective:
I am attempting to put (or patch) a json.dumps(mergedFile) into firebase as one payload without firebase auto creating indexes (0, 1, etc..) in front of each object
Problem statement:
I am submitting the following json object into the /testObject path:
[{"test1":"226.69"},{"test2":"7.48"}]
In firebase the response is stored as:
[
{
"testObject": {
0: {
"test1": "226.69"
},
1: {
"test2": "7.48"
}
}
}
]
Background:
The total # of items in the payload of the data I need to store is
just over 5000
If I parse each object via a for loop the data is written as
expected, however, it initiates a new request for each itteriation of
the loop and has a large overhead impact compared to just
dumping one large object in one request.
Here is my Code:
import json
import requests
import xml.etree.ElementTree as ET
def get_data():
try:
print 'hampsters are running...'
# OFFLINE TESTING
sourceFile = 'response.xml'
tree = ET.parse(sourceFile)
root = tree.getroot()
for symbol in root.iter('symbol'):
company = symbol.attrib['company']
location = symbol.attrib['location']
destinationData = {company: location}
mergedFile.append(destinationData)
print('downlaoding the info was a success! :)')
except:
print 'Attempt to download information did not complete successfully :('
def patch_data():
try:
print 'attempting to upload info to database...'
data = json.dumps(mergedFile)
print data
try:
req = requests.put(url, data=data, headers=headers)
req.raise_for_status()
except requests.exceptions.HTTPError as e:
print e
print req.json()
print 'upload to database complete!'
except:
print 'Attempt to upload information did not complete successfully :('
if __name__ == "__main__":
mergedFile = []
auth = "*****"
databaseURL = 'https://*****.firebaseio.com'
headers = {"auth": auth, "print": "pretty"}
# headers = {"auth": auth, "print": "pretty", "Accept": "text/event-stream"}
requestPath = '/testObject.json?auth=' + auth
url = databaseURL + requestPath
get_data()
patch_data()
I feel like its storing an array, but I'm leveraging data = json.dumps(mergedFile) before the put request. Do I have a mis-understanding of how json.dumps works? Based on the output before the request I feel it looks good. I'm also leveraging the requests python module... is this converting the data to an array?
Any insight anyone could provide would be greatly appreciated!
Regards,
James.
The Firebase Database stores arrays as regular key-value pairs, with the keys being numbers. So what you see is the expected behavior.
There are many reasons why Firebase recommends against storing arrays in the database. A few can be found in these links:
Best Practices: Arrays in Firebase
Proper way to store values array-like in Firebase
Firebase documentation on structuring data
Other questions about arrays in Firebase
this answer on arrays vs sets
I'm working on a project where I, among other things, need to read the message in e-mails from my google account. I came up with a solution that works but wonder if there are any simpler ways?
The first part of the code is pretty standard to get access to the mailbox. But I post it so you can see what I did to get it to work.
SCOPES = 'https://www.googleapis.com/auth/gmail.modify'
CLIENT_SECRET ='A.json'
store =file.Storage('storage.json')
credz=store.get()
flags = tools.argparser.parse_args(args=[])
if not credz or credz.invalid:
flow = client.flow_from_clientsecrets(CLIENT_SECRET,SCOPES)
if flags:
credz = tools.run_flow(flow, store, flags)
GMAIL = build('gmail','v1',http=credz.authorize(Http()))
response = GMAIL.users().messages().list(userId='me',q='').execute()
messages = []
if 'messages' in response:
messages.extend(response['messages'])
print len(messages)
while 'nextPageToken' in response:
page_token = response['nextPageToken']
response = service.users().messages().list(userId='me', q=query,pageToken=page_token).execute()
messages.extend(response['messages'])
FromMeInd=0
for message in messages:
ReadMessage(GMAIL,'me',message['id'])
It is this part that I'm more interested to imporve. Is there any other way to more directly get the message with python and the gmail-api. I've looked through the api documentation but could not get any more efficient way to read it.
def ReadMessage(service,userID,messID):
message = service.users().messages().get(userId=userID, id=messID,format='full').execute()
decoded=base64.urlsafe_b64decode(message['payload']['body']['data'].encode('ASCII'))
print decoded
You can get the body as raw and then parse it using the standard Python email module
According to the official API: https://developers.google.com/gmail/api/v1/reference/users/messages/get:
import email
message = service.users().messages().get(userId='me', id=msg_id,
format='raw').execute()
print 'Message snippet: %s' % message['snippet']
msg_str = base64.urlsafe_b64decode(message['raw'].encode('ASCII'))
mime_msg = email.message_from_string(msg_str)
You'll get a mime message with a payload containing mime parts, e.g. plain text, HTML, quoted printable, attachments, etc.