I want to avoid store personal information in database (no last names, no email). This is my approach to achieve it:
Delegate authentication to social networks authentication service ( thanks to python-social-auth )
Change python-social-auth pipeline to anonymize personal information.
Then I replaced social_details step on pipeline by this one:
#myapp/mypipeline.py
def social_details(strategy, response, *args, **kwargs):
import md5
details = strategy.backend.get_user_details(response)
email = details['email']
fakemail = unicode( md5.new(email).hexdigest() )
new_details = {
'username': fakemail[:5],
'email': fakemail + '#noreply.com',
'fullname': fakemail[:5],
'first_name': details['first_name'],
'last_name': '' }
return {'details': new_details }
settings.py
SOCIAL_AUTH_PIPELINE = (
'myapp.mypipeline.social_details',
'social.pipeline.social_auth.social_uid',
...
The question:
Is this the right way to get my purpose?
Looks good.
I'm doing something similar to anonymize IP addresses (hash them).
Related
I am trying to write an API test in Python for my web service. I would like to avoid using the password of the test user from my AWS Cognito pool. My strategy for this, and let me know if there's a better way here, is to require that the API test be run with Cognito admin privileges. Then use the boto3 library to get the JWT AccessToken for the user which I will add to the header of every request for the API test.
The documentation doesn't seem to give me a way to get the AccessToken. I'm trying to use this here: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/cognito-idp.html#CognitoIdentityProvider.Client.admin_initiate_auth
admin_initiate_auth needs one of three auth modes. USER_PASSWORD_AUTH requires the password, USER_SRP_AUTH requires a client secret, CUSTOM_AUTH requires a secret hash. I'm hoping to find a way to write this script so that I just need to have the right IAM privileges and not need to check in a public test user password.
Or... I guess... be told that this is not a great way to be doing this and that another way is more appropriate. The end goal is to have an API black box test for a service that is secured by Cognito.
For my own project, I was also thinking a similar strategy to test Cognito-protected APIs.
I think making a temporary user with a random password for each test run is a fair approach.
To create a user from command line, I think there are simpler cognito API calls, which are sign-up and admin-confirm-sign-up provided in cognito-idp CLI tool. With this, you can skip the steps to resolve the challenges and the user is ready to use.
If you want to use boto3, here is a simple function to create a new user:
def create_user(username: str, password: str,
user_pool_id: str, app_client_id: str) -> None:
client = boto3.client('cognito-idp')
# initial sign up
resp = client.sign_up(
ClientId=app_client_id,
Username=username,
Password=password,
UserAttributes=[
{
'Name': 'email',
'Value': 'test#test.com'
},
]
)
# then confirm signup
resp = client.admin_confirm_sign_up(
UserPoolId=user_pool_id,
Username=username
)
print("User successfully created.")
Then, to obtain JWT,
def authenticate_and_get_token(username: str, password: str,
user_pool_id: str, app_client_id: str) -> None:
client = boto3.client('cognito-idp')
resp = client.admin_initiate_auth(
UserPoolId=user_pool_id,
ClientId=app_client_id,
AuthFlow='ADMIN_NO_SRP_AUTH',
AuthParameters={
"USERNAME": username,
"PASSWORD": password
}
)
print("Log in success")
print("Access token:", resp['AuthenticationResult']['AccessToken'])
print("ID token:", resp['AuthenticationResult']['IdToken'])
If the API test must be secured using Cognito, you're always going to need some kind of password. The best way I can think of to avoid storing it is to create a temporary user before running the test suite, and then delete it when finished.
You can use AdminCreateUser to accomplish this. Generate a new password at runtime and pass it as the temporary password for the user, along with SUPRESS specified for MessageAction. The temporary password is good for one login, which is all you need in this use case. Then you can run AdminInitiateAuth with the ADMIN_NO_SRP_AUTH auth mode, specifying your generated password. Cleanup with AdminDeleteUser after the tests have finished.
To expand on #xlem's answer and #mmachenry's comment with an example:
Using the Cognito client of AWS SDK
"AdminCreateUser" flow will create the user.
"AdminSetPassword" will clear the FORCE_PASSWORD_CHANGE state
"AdminInitiateAuth" will return the token
#pytest.fixture()
def given_a_new_user_token(request):
client = boto3.client("cognito-idp")
user_pool_id = "eu-west-2_xxxxxxxxx"
username = f"test_user-{uuid.uuid4().hex}"
temp_pwd = f"{uuid.uuid4().hex}"
ci_test_client_id, secret_hash = get_cognito_secrets(username)
user_response = client.admin_create_user(
UserPoolId=user_pool_id,
Username=username,
UserAttributes=[
{"Name": "email", "Value": f"{username}#example.com"},
],
TemporaryPassword=temp_pwd,
ForceAliasCreation=False,
MessageAction="SUPPRESS",
DesiredDeliveryMediums=[
"EMAIL",
],
)
assert user_response["ResponseMetadata"]["HTTPStatusCode"] == 200
def delete_user():
client.admin_delete_user(
UserPoolId=user_pool_id,
Username=username,
)
request.addfinalizer(delete_user)
set_pwd_response = client.admin_set_user_password(
UserPoolId=user_pool_id,
Username=username,
Password=temp_pwd,
Permanent=True
)
assert set_pwd_response["ResponseMetadata"]["HTTPStatusCode"] == 200
auth_info = client.admin_initiate_auth(
UserPoolId=user_pool_id,
ClientId=ci_test_client_id,
AuthFlow="ADMIN_NO_SRP_AUTH",
AuthParameters={
"USERNAME": username,
"PASSWORD": temp_pwd,
"SECRET_HASH": secret_hash,
},
)
assert auth_info["ResponseMetadata"]["HTTPStatusCode"] == 200
return auth_info["AuthenticationResult"]["AccessToken"]
def get_cognito_secrets(username: str) -> Tuple[str, str]:
ci_test_client_id = "client_id_xxxxxxx"
ci_test_client_secret = "client_secret_xxxxx"
# convert str to bytes
key = bytes(ci_test_client_secret, 'latin-1')
msg = bytes(username + ci_test_client_id, 'latin-1')
new_digest = hmac.new(key, msg, hashlib.sha256).digest()
secret_hash = base64.b64encode(new_digest).decode()
return ci_test_client_id, secret_hash
I have an AWS Organization and I create member accounts for every new project I make. Since I have control over all of the accounts I use the same email account for all of those, using the account-name+project-name#gmail.com pattern.
This means that I get the same marketing email for every new account I create. I know I can unsubscribe manually, but since I create the member accounts through the CLI I was wondering if there is a way to automatically unsubscribe (or avoid being subscribed) through the SDK.
I've looked in the AWS Organizations SDK documentation, particularly around create-account but haven't found anything relevant.
apparently there is no solution from AWS for this. The only place I found is this. Which involves manual intervention.
Doing this on Organization ID would be a good option.
meanwhile, I wrote this as a workaround.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import requests
from bs4 import BeautifulSoup
GET_URL = 'https://pages.awscloud.com/communication-preferences'
POST_URL = 'https://pages.awscloud.com/index.php/leadCapture/save2'
s = requests.Session()
def fetch(url, data=None):
if data is None:
return s.get(url).content
return s.post(url, data=data).content
def get_form_id():
forms = BeautifulSoup(fetch(GET_URL), 'html.parser').findAll('form')
for form in forms:
fields = form.findAll('input')
for field in fields:
if field.get('name') == 'formid':
return field['value']
email_id = 'testing#example.com'
formid = get_form_id()
form_data = {'Email': email_id, 'Unsubscribed': 'yes', 'formid': formid, 'formVid': formid}
r = fetch(POST_URL, data=form_data)
print(r)
AWS recently changed this web form again breaking things for the existing unsubscribe function I had written. They added 2 new required form data fields: checksum and checksumFields. The checksum field is a sha256 hash of a concatenation of all checksumFields values (single string joined with a ,).
Below is my unsubscribe function written in python.
NOTE: I like #samtoddler's example using beautiful soup to dynamically lookup the formid vs hard-coding it in as a function input var like I have.
def unsubscribe_aws_mkt_emails(email,
url='https://pages.awscloud.com/index.php/leadCapture/save2',
form_id=34006,
lp_id=127906,
sub_id=6,
munchkin_id='112-TZM-766'):
'''
Unsubscribes email from AWS marketing emails via HTTPS POST
'''
sha256_hash = hashlib.sha256()
# Data fields used to calculate the payload SHA256 checksum value
checksum_fields = [
'FirstName',
'LastName',
'Email',
'Company',
'Phone',
'Country',
'preferenceCenterCategory',
'preferenceCenterGettingStarted',
'preferenceCenterOnlineInPersonEvents',
'preferenceCenterMonthlyAWSNewsletter',
'preferenceCenterTrainingandBestPracticeContent',
'preferenceCenterProductandServiceAnnoucements',
'preferenceCenterSurveys',
'PreferenceCenter_AWS_Partner_Events_Co__c',
'preferenceCenterOtherAWSCommunications',
'PreferenceCenter_Language_Preference__c',
'Title',
'Job_Role__c',
'Industry',
'Level_of_AWS_Usage__c',
]
headers = {
'content-type': 'application/x-www-form-urlencoded; charset=UTF-8',
'user-agent': 'Mozilla/5.0',
}
data_dict = {
'Email': email,
'preferenceCenterCategory': 'no',
'preferenceCenterGettingStarted': 'no',
'preferenceCenterOnlineInPersonEvents': 'no',
'preferenceCenterMonthlyAWSNewsletter': 'no',
'preferenceCenterTrainingandBestPracticeContent': 'no',
'preferenceCenterProductandServiceAnnoucements': 'no',
'preferenceCenterSurveys': 'no',
'PreferenceCenter_AWS_Partner_Events_Co__c': 'no',
'preferenceCenterOtherAWSCommunications': 'no',
'Unsubscribed': 'yes',
'UnsubscribedReason': 'I already get email from another account',
'unsubscribedReasonOther': 'I already get email from another account',
'zOPEmailValidationHygiene': 'validate',
'formid': form_id,
'formVid': form_id,
'lpId': lp_id,
'subId': sub_id,
'munchkinId': munchkin_id,
'lpurl': '//pages.awscloud.com/communication-preferences.html?cr={creative}&kw={keyword}',
'_mkt_trk': f'id:{munchkin_id}&token:_mch-pages.awscloud.com-1644428507420-99548',
'_mktoReferrer': 'https://pages.awscloud.com/communication-preferences',
'checksumFields': ','.join(checksum_fields),
}
# calculated via js: f.checksum=v("sha256").update(s.join("|")).digest("hex")
# src = https://pages.awscloud.com/js/forms2/js/forms2.min.js
sha256_hash.update('|'.join([data_dict.get(v, '') for v in checksum_fields]).encode())
data_dict['checksum'] = sha256_hash.hexdigest()
data = parse.urlencode(data_dict).encode()
req = request.Request(url, data=data, headers=headers, method='POST')
resp = request.urlopen(req)
While I was looking into this problem, I stumbled across a different endpoint that was buried in the HTML of the unsubscribe page. It seems like it was once used for browsers with JavaScript disabled (although the page just redirects in that situation, so it's never actually used).
This endpoint works exactly the same way as it does in #samtoddler's answer, but the endpoint is https://pages.awscloud.com/index.php/leadCapture/save3. I've tested submitting a POST request to that endpoint with the following form data, and it seemed to work:
{
"email": "test#gmail.com",
"Unsubscribed": "yes",
"formid": "34006",
"formVid": "34006",
"munchkinId": "112-TZM-766"
}
Wouldn't be a bad idea to use beautiful soup or something similar to get the form ID and Munchkin ID dynamically, but they haven't changed in 2 years so I feel fairly safe hard-coding them.
I'm testing login function.
def setUpClass(cls):
super(BasePage_loggedin, cls).setUpClass()
cls.selenium = WebDriver()
cls.client = Client()
cls.user_1 = MyUser.objects.create_user(username='myself',password='12345')
cls.client.login(username=cls.user_1.username, password=cls.user_1.password)
# create session cookie:
session = SessionStore()
session[SESSION_KEY] = cls.user_1.pk
session[BACKEND_SESSION_KEY] = settings.AUTHENTICATION_BACKENDS[0]
session[HASH_SESSION_KEY] = cls.user_1.get_session_auth_hash()
session.save()
# Finally, create the cookie dictionary
cookie = {
'name': settings.SESSION_COOKIE_NAME,
'value': session.session_key,
'secure': False,
'path': '/',
}
# add the session cookie
cls.selenium.get('{}'.format(cls.live_server_url))
cls.selenium.add_cookie(cookie)
cls.selenium.refresh()
cls.selenium.get('{}'.format(cls.live_server_url))
So I can pass the login page, but then, when I do request.user to check the data for this user, it's an AnonymousUser
When you're creating the user that way - I believe it has to do with the password. Setting the password to a string like that doesn't do what you think it would do.
You could create the user like that - then add this after the user creation but before the login:
cls.user_1.set_password('12345')
cls.user_1.save()
Then login the user with something like this:
cls.client.login(username=cls.user_1.username, password='12345')
I believe it has something to do with the hashing of the password or something along those lines - it's been a while since I stumbled around with it, but I remember having the exact same issue as you.
Something like this should work:
cls.selenium = WebDriver()
cls.client = Client()
cls.user_1 = MyUser.objects.create_user(username='myself',password='12345')
cls.user_1.set_password('12345')
cls.user_1.save()
cls.client.login(username=cls.user_1.username, password='12345')
Please check your settings.py and try below codes.
REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': (
'rest_framework.authentication.SessionAuthentication', # needed only up to the test env
'rest_framework.authentication.TokenAuthentication',
)
}
I have configured django social auth's to take from google only e-mail, but google shows this screen alerting app user that gender, date of birth, picture, language will be collect:
My django-social-auth config is as follow:
WHITE_LISTED_DOMAINS = [ 'some_domain', ]
GOOGLE_WHITE_LISTED_DOMAINS = WHITE_LISTED_DOMAINS
SOCIAL_AUTH_EXTRA_DATA = False
#LOGIN_ERROR_URL = '/login-error/' Not set
#SOCIAL_AUTH_DEFAULT_USERNAME = 'new_social_auth_user' Not set
#GOOGLE_CONSUMER_KEY = '' Not set
#GOOGLE_CONSUMER_SECRET = '' Not set
#GOOGLE_OAUTH2_CLIENT_ID = '' Not set
#GOOGLE_OAUTH2_CLIENT_SECRET = '' Not set
SOCIAL_AUTH_USERNAME_IS_FULL_EMAIL = False
SOCIAL_AUTH_PROTECTED_USER_FIELDS = ['email',]
INSTALLED_APPS = (
'django.contrib.auth',
...
'social_auth',
)
How can I do to avoid this google message?
EDITED
I have move to GoogleOauth2 auth and inherit and change google backend:
from social_auth.backends.google import *
GOOGLE_OAUTH2_SCOPE = ['https://www.googleapis.com/auth/userinfo.email',]
class GoogleOAuth2(BaseOAuth2):
"""Google OAuth2 support"""
AUTH_BACKEND = GoogleOAuth2Backend
AUTHORIZATION_URL = 'https://accounts.google.com/o/oauth2/auth'
ACCESS_TOKEN_URL = 'https://accounts.google.com/o/oauth2/token'
REVOKE_TOKEN_URL = 'https://accounts.google.com/o/oauth2/revoke'
REVOKE_TOKEN_METHOD = 'GET'
SETTINGS_SECRET_NAME = 'GOOGLE_OAUTH2_CLIENT_SECRET'
SCOPE_VAR_NAME = 'GOOGLE_OAUTH_EXTRA_SCOPE'
DEFAULT_SCOPE = GOOGLE_OAUTH2_SCOPE
REDIRECT_STATE = False
print DEFAULT_SCOPE #<------ to be sure
def user_data(self, access_token, *args, **kwargs):
"""Return user data from Google API"""
return googleapis_profile(GOOGLEAPIS_PROFILE, access_token)
#classmethod
def revoke_token_params(cls, token, uid):
return {'token': token}
#classmethod
def revoke_token_headers(cls, token, uid):
return {'Content-type': 'application/json'}
But google still ask for profile data, profile is still in scope:
https://accounts.google.com/o/oauth2/auth?response_type=code&scope=https://www.googleapis.com/auth/userinfo.email+https://www.googleapis.com/auth/userinfo.profile&redirect_uri=...
Runs fine if I modify by hand social-auth code instead inherit:
def get_scope(self):
return ['https://www.googleapis.com/auth/userinfo.email',]
What is wrong with my code?
That's because the default scope used on google backend is set to that (email and profile information), it's defined here. In order to avoid that you can create your own google backend which just sets the desired scope, then use that backend instead of the built in one. Example:
from social_auth.backends.google import GoogleOAuth2
class SimplerGoogleOAuth2(GoogleOAuth2):
DEFAULT_SCOPE = ['https://www.googleapis.com/auth/userinfo.email']
Those who don't know how to add in AUTHENTICATION_BACKENDS, if using the way Omab suggested you need to add newly defined backend in your setting.py file:
AUTHENTICATION_BACKENDS = (
'app_name.file_name.class_name', #ex: google_auth.views.SimplerGoogleOAuth2
# 'social_core.backends.google.GoogleOAuth2', # comment this as no longer used
'django.contrib.auth.backends.ModelBackend',
)
To know how to create the class SimplerGoogleOAuth2 check Omab's answer.
I'm using django-allauth for my authentication system. I need that when the user sign in, the profile module get populated with the provider info (in my case facebook).
I'm trying to use the pre_social_login signal, but I just don't know how to retrieve the data from the provider auth
from django.dispatch import receiver
from allauth.socialaccount.signals import pre_social_login
#receiver(pre_social_login)
def populate_profile(sender, **kwargs):
u = UserProfile( >>FACEBOOK_DATA<< )
u.save()
Thanks!!!
The pre_social_login signal is sent after a user successfully
authenticates via a social provider, but before the login is actually
processed. This signal is emitted for social logins, signups and when
connecting additional social accounts to an account.
So it is sent before the signup is fully completed -- therefore this not the proper signal to use.
Instead, I recommend you use allauth.account.signals.user_signed_up, which is emitted for all users, local and social ones.
From within that handler you can inspect whatever SocialAccount is attached to the user. For example, if you want to inspect Google+ specific data, do this:
user.socialaccount_set.filter(provider='google')[0].extra_data
UPDATE: the latest development version makes this a little bit more convenient by passing along a sociallogin parameter that directly contains all related info (social account, token, ...)
Here is a Concrete example of #pennersr solution :
Assumming your profile model has these 3 fields: first_name, email, picture_url
views.py:
#receiver(user_signed_up)
def populate_profile(sociallogin, user, **kwargs):
if sociallogin.account.provider == 'facebook':
user_data = user.socialaccount_set.filter(provider='facebook')[0].extra_data
picture_url = "http://graph.facebook.com/" + sociallogin.account.uid + "/picture?type=large"
email = user_data['email']
first_name = user_data['first_name']
if sociallogin.account.provider == 'linkedin':
user_data = user.socialaccount_set.filter(provider='linkedin')[0].extra_data
picture_url = user_data['picture-urls']['picture-url']
email = user_data['email-address']
first_name = user_data['first-name']
if sociallogin.account.provider == 'twitter':
user_data = user.socialaccount_set.filter(provider='twitter')[0].extra_data
picture_url = user_data['profile_image_url']
picture_url = picture_url.rsplit("_", 1)[0] + "." + picture_url.rsplit(".", 1)[1]
email = user_data['email']
first_name = user_data['name'].split()[0]
user.profile.avatar_url = picture_url
user.profile.email_address = email
user.profile.first_name = first_name
user.profile.save()
If you are confused about those picture_url variable in each provider. Then take a look at the docs:
facebook:
picture_url = "http://graph.facebook.com/" + sociallogin.account.uid + "/picture?type=large" Docs
linkedin:
picture_url = user_data['picture-urls']['picture-url'] Docs
twitter:
picture_url = picture_url.rsplit("_", 1)[0] + "." + picture_url.rsplit(".", 1)[1] Docs And for the rsplit() take a look here
Hope that helps. :)
I am doing in this way and taking picture (field) url and google provider(field) as an example.
socialaccount_obj = SocialAccount.objects.filter(provider='google', user_id=self.user.id)
picture = "not available"
if len(socialaccount_obj):
picture = socialaccount_obj[0].extra_data['picture']
make sure to import : from allauth.socialaccount.models import SocialAccount
There is an easier way to do this.
Just add the following to your settings.py. For example, Linked in...
SOCIALACCOUNT_PROVIDERS = {
'linkedin': {
'SCOPE': [
'r_basicprofile',
'r_emailaddress'
],
'PROFILE_FIELDS': [
'id',
'first-name',
'last-name',
'email-address',
'picture-url',
'public-profile-url',
]
}
The fields are automatically pulled across.