Decrypting SAML2 response using pysaml2 Python module - django

I am integrating my app with okta to have single sign on. Okta will be passing some user information in SAML response which I need to use in my application.
Hence, we decided to encrypt the saml response(xml) at IDP using my server(apache) public key.
Now I am trying to decrypt the saml2 response so that I can get the attributes.
My applications uses
Python 3.5
Django 1.11
pysaml2 python module
I am using below to validate/parse the saml2 response coming from okta
https://github.com/fangli/django-saml2-auth
If the saml response is not encrypted, I am able to process the response and able to get the user identity and user attributes from it.
However once it is encrypted at okta end with my server public key, I am not able to decrypt with my private key.
The saml setting I have at my application is below :
saml_settings = { 'metadata': {
"local": [ metadat_xml
],
},
'service': {
'sp': {
'endpoints': {
'assertion_consumer_service': [
(acs_url, BINDING_HTTP_REDIRECT),
(acs_url, BINDING_HTTP_POST),
(https_acs_url, BINDING_HTTP_REDIRECT),
(https_acs_url, BINDING_HTTP_POST)
],
},
'allow_unsolicited': True,
'authn_requests_signed': False,
'logout_requests_signed': True,
'want_assertions_signed': True,
'want_response_signed': False,
},
},
'key_file': "mykey.key", # private part
'cert_file': "mykey.crt", # public part
'xmlsec_binary': '/usr/bin/xmlsec1',
'encryption_keypairs': [{
'key_file': 'mykey.key',
'cert_file': 'mykey.crt',
}]
}
if 'ENTITY_ID' in settings.SAML2_AUTH:
saml_settings['entityid'] = settings.SAML2_AUTH['ENTITY_ID']
#print('entity id ' , settings.SAML2_AUTH['ENTITY_ID'])
if 'NAME_ID_FORMAT' in settings.SAML2_AUTH:
saml_settings['service']['sp']['name_id_format'] = settings.SAML2_AUTH['NAME_ID_FORMAT']
# NOTE-'NAME_ID_FORMAT is set to None above
spConfig = Saml2Config()
spConfig.load(saml_settings)
spConfig.allow_unknown_attributes = True
saml_client = Saml2Client(config=spConfig)
return saml_client
Then I have
saml_client = _get_saml_client(get_current_domain(r))
resp = r.POST.get('SAMLResponse', None)
authn_response = saml_client.parse_authn_request_response(resp, entity.BINDING_HTTP_POST )
This auth_response object is not returning me anything when the message is encrypted.
In logs I see below error
GbHvkJJM0WIsPYFGtiQ/0n+ux0tV/z/OKpT1AqEE74iRVHEHD7omP41iY/c4=
</ns3:CipherValue></ns3:CipherData><ns3:ReferenceList><ns3:DataReference
URI="#_648cdbd139564492f0bdfe4fbbda92f6" /></ns3:ReferenceList>
</ns3:EncryptedKey></ns1:EncryptedAssertion></ns0:Response>
2018-04-30 18:21:09,232 [DEBUG] sigver saml2.sigver decrypt(): Decrypt input
len: 15187
2018-04-30 18:21:09,233 [DEBUG] sigver saml2.sigver _run_xmlsec(): xmlsec
command: /usr/bin/xmlsec1 --decrypt --privkey-pem
/private.pem --id-attr:ID EncryptedKey --output /tmp/tmp7rt7g95u.xml
/tmp/tmpkhxwo8s4
2018-04-30 18:21:09,247 [DEBUG] sigver saml2.sigver _run_xmlsec(): xmlsec
p_out:
2018-04-30 18:21:09,247 [DEBUG] sigver saml2.sigver _run_xmlsec(): xmlsec
p_erryy:
func=xmlSecXPathDataExecute:file=xpath.c:line=273:obj=unknown:
subj=xmlXPtrEval:error=5:libxml2 library function
failed:expr=xpointer(id('_841612fffac65343e73f8913eeecfb30'))
func=xmlSecXPathDataListExecute:file=xpath.c:line=373:obj=unknown:
subj=xmlSecXPathDataExecute:error=1:xmlsec library function failed:
func=xmlSecTransformXPathExecute:file=xpath.c:line=483:
obj=xpointer:subj=xmlSecXPathDataExecute:error=1:xmlsec library function
failed:
func=xmlSecTransformDefaultPushXml:file=transforms.c:
line=2411:obj=xpointer:subj=xmlSecTransformExecute:error=1:xmlsec library
function failed:
func=xmlSecTransformCtxExecute:file=transforms.c:line=1302:
obj=unknown:subj=xmlSecTransformCtxXmlExecute:error=1:xmlsec library
function failed:
func=xmlSecKeyDataRetrievalMethodXmlRead:file=keyinfo.c:line=1178:
obj=retrieval-method:subj=xmlSecTransformCtxExecute:error=1:xmlsec library
function failed:
func=xmlSecKeyInfoNodeRead:file=keyinfo.c:line=114:obj=retrieval-method:
subj=xmlSecKeyDataXmlRead:error=1:xmlsec library function
failed:node=RetrievalMethod
func=xmlSecKeysMngrGetKey:file=keys.c:line=1349:obj=unknown:
subj=xmlSecKeyInfoNodeRead:error=1:xmlsec library function
failed:node=KeyInfo
func=xmlSecEncCtxEncDataNodeRead:file=xmlenc.c:line=957:
obj=unknown:subj=unknown:error=45:key is not found:
func=xmlSecEncCtxDecryptToBuffer:file=xmlenc.c:line=715:
obj=unknown:subj=xmlSecEncCtxEncDataNodeRead:error=1:xmlsec library function
failed:
func=xmlSecEncCtxDecrypt:file=xmlenc.c:line=623:
obj=unknown:subj=xmlSecEncCtxDecryptToBuffer:error=1:xmlsec library function
failed:
Error: failed to decrypt file
Error: failed to decrypt file "/tmp/tmpkhxwo8s4"
I am not sure why xmlsec1 command is failing and what I am missing here.
I have tried decrypting with my private key(self signed private key) here
https://www.samltool.com/decrypt.php
and it works
Could you please help me here and let me know what I am not doing correctly?

You need to add
saml_settings['id_attr_name'] = 'Id'
The default id attr is ID, but Okta uses Id. See xmlsec FAQ for more details.

Related

AWS sagemaker endpoint received client (400) error

I've deployed a tensorflow multi-label classification model using a sagemaker endpoint as follows:
predictor = sagemaker_model.deploy(initial_instance_count=1, instance_type="ml.m5.2xlarge", endpoint_name='testing-2')
It gets deployed and works fine when I invoke it from the Sagemaker Jupyter instance:
sample = ['this movie was extremely good']
output=predictor.predict(sample)
output:
{'predictions': [[0.00370046496,
4.32942124e-06,
0.00080883503,
9.25126587e-05,
0.00023958087,
0.000130862]]}
However, I am unable to send a request to the deployed endpoint from other notebooks or sagemaker studio. I'm unsure of the request format.
I've tried several variations in the input format and still failed. The error message is as below:
sagemaker error
Request:
{
"body": {
"text": "Testing model's prediction on this text"
},
"contentType": "application/json",
"endpointName": "testing-2",
"customURL": "",
"customHeaders": [
{
"Key": "sm_endpoint_name",
"Value": "testing-2"
}
]
}
Error:
Error invoking endpoint: Received client error (400) from primary with message "{ "error": "Failed to process element:
0 key: text of 'instances' list. Error: INVALID_ARGUMENT: JSON object: does not have named input: text" }".
See https://us-west-2.console.aws.amazon.com/cloudwatch/home?region=us-west-2#logEventViewer:group=/aws/sagemaker/Endpoints/testing-2
in account 793433463428 for more information.
Is there any way to find out exactly how the model expects the request format to be?
Earlier I had the same model on my local system and the way I tested it was using this curl request:
curl -s -H 'Content-Type: application/json' -d '{"text": "what ugly posts"}' http://localhost:7070/sentiment
And it worked fine without any issues.
I've tried different formats and replaced the "text" key inside body with other words like "input", "body", nothing etc.
Based on your description above, I assume you are deploying the TensorFlow model using the SageMaker TensorFlow container.
If you want to view what your model expects as input you can use the saved_model CLI:
1
├── keras_metadata.pb
├── saved_model.pb
└── variables
├── variables.data-00000-of-00001
└── variables.index
!saved_model_cli show --all --dir {"1"}
After you have confirmed the input name above you can invoke the endpoint as follows:
import json
import boto3
client = boto3.client('runtime.sagemaker')
data = {"instances": ['this movie was extremely good']}
response = client.invoke_endpoint(EndpointName=<EndpointName>,
Body=json.dumps(data))
response_body = response['Body']
print(response_body.read())
The same payload can then also be used in Studio when invoking the endpoint.

Custom "statusMsg" not working as expected

After adding the following adaptive authentication script, if the username is not according to the format I should be getting the custom status message saying, Access Denied, invalid username format. But instead I get the default status message, Something went wrong during the authentication process. Please try signing in again.
function onLoginRequest(context) {
executeStep(1, {
onSuccess: function(context) {
var user = context.currentKnownSubject;
if(user!= null && user.username != null && !user.username.equals('')) {
Log.info("username: " + user.username);
} else {
sendError('',{'status':'AUTHENTICATION USERNAME ERROR', 'statusMsg': 'Access denied, invalid username format.'});
}
}
});
}
In addition, I get the following error in the wso2carbon.log file as well.
TID: [-1234] [authenticationendpoint] [2022-10-05 15:44:12,715] [37951f7d-8240-48d4-ad4f-1d4c8a6a3ec4] ERROR {org.wso2.carbon.identity.application.authentication.endpoint.util.AuthContextAPIClient} - Sending GET request to URL : https://dev.wso2istemp.com/api/identity/auth/v1.1/data/AuthenticationError/0b0efc37-819d-4b39-85b2-517126c3c9cb, failed. java.io.IOException: Server returned HTTP response code: 401 for URL: https://dev.wso2istemp.com/api/identity/auth/v1.1/data/AuthenticationError/0b0efc37-819d-4b39-85b2-517126c3c9cb
...
org.wso2.carbon.identity.application.authentication.endpoint.util.AuthContextAPIClient.getContextProperties(AuthContextAPIClient.java:70)
at org.apache.jsp.retry_jsp._jspService(retry_jsp.java:194)
...
org.wso2.carbon.ui.filters.cache.ContentTypeBasedCachePreventionFilter.doFilter(ContentTypeBasedCachePreventionFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
...
org.wso2.carbon.identity.application.authentication.endpoint.util.filter.AuthenticationEndpointFilter.doFilter(AuthenticationEndpointFilter.java:190)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
...
The <IS_HOME>/repository/conf/deployment.toml configurations for [server] are as follows.
[server]
hostname = "dev.wso2istemp.com"
node_ip = "127.0.0.1"
base_path = "https://$ref{server.hostname}:${carbon.management.port}"
What is the reason for the following issue in the wso2carbon.log and why the custom status message is not shown properly?
When the adaptive authentication script is running, the values are passed to the Identity Server(IS) encrypted. In above case, the encrypted data chunk is passed to the IS,
https://dev.wso2istemp.com/api/identity/auth/v1.1/data/AuthenticationError/24e56d99-9494-4989-a3e2-4008b73ebd9b
with the last segment of the URL is being the data chunk. When the server tries to get that data chunk with a GET request, java.io.IOException: Server returned HTTP response code: 401 for URL which is the code for unauthorized is thrown. Since that data chunk is not properly received, the default status message is shown instead of the custom status message. The steps to resolve this are given below.
First clarify whether the dev.wso2istemp.com which you were using is not mapped with the localhost in /etc/hosts file.
Go to <IS_HOME>/repository/conf/deployment.toml file and check for the following configuration [identity.auth_framework.endpoint] and check whether the mutual SSL is set to false via mutual_ssl_manager_enabled=false
If so, enable that by commenting the mutual_ssl_manager_enabled=false, since using mutual SSL is recommended for IS. If you go to the <IS_HOME>/repository/resources/conf/default.json file, you can notice that the default value for mutual_ssl_manager_enabled is true
In a multi-node situation, the above error can occur if the mutual SSL is not enabled, If that does not resolves the issue, then you have to check whether the internal_hostname has been set properly so that the internal API calls are being sent properly.
To do that, if you have not added the following configuration to <IS_HOME>/repository/conf/deployment.toml file, you can check whether it has been properly applied to the <IS_HOME>/repository/conf/identity/identity.xml by checking for <ServerHostName>localhost</ServerHostName>
[server]
internal_hostname="localhost"
If you are using a multi-node deployment, this localhost value should be added to the SAN for the certificate when the certificate is generated (-ext SAN=dns:localhost)
keytool -genkey -alias newcert -keyalg RSA -keysize 2048 -keystore newkeystore.jks -dname "CN=dev.wso2istemp.com, OU=Is,O=Wso2,L=SL,S=WS,C=LK" -storepass mypassword -keypass mypassword -ext SAN=dns:localhost
But if you are using a single node you can add the following configuration to the deployment.toml and check whether it resolves things. (In single node case the internal_hostname should be similar to hostname)
[server]
hostname = "dev.wso2istemp.com"
internal_hostname = "dev.wso2istemp.com"
If that is not working[https://github.com/wso2/product-is/issues/11878] then go to <IS_HOME>/repository/deployment/server/webapps/authenticationendpoint/WEB-INF/web.xml and uncomment the following commented snippet.
<!--context-param>
<param-name>AuthenticationRESTEndpointURL</param-name>
<param-value>https://localhost:9443/api/identity/auth/v1.1/</param-value>
</context-param-->
This might route the internal_hostname being reflected on the request since the internal API calls are blocked when hostname of the server being replaced instead of the internal_hostname for the internal API calls(https://dev.wso2istemp.com/api/identity/auth/v1.1/data/AuthenticationError/24e56d99-9494-4989-a3e2-4008b73ebd9b).

AWS Glue - Kafka Connection using SASL/SCRAM

I am trying to create an AWS Glue Streaming job that reads from Kafka (MSK) clusters using SASL/SCRAM client authentication for the connection, per
https://aws.amazon.com/about-aws/whats-new/2022/05/aws-glue-supports-sasl-authentication-apache-kafka/
The connection configuration has the following properties (plus adequate subnet and security groups):
"ConnectionProperties": {
"KAFKA_SASL_SCRAM_PASSWORD": "apassword",
"KAFKA_BOOTSTRAP_SERVERS": "theserver:9096",
"KAFKA_SASL_MECHANISM": "SCRAM-SHA-512",
"KAFKA_SASL_SCRAM_USERNAME": "auser",
"KAFKA_SSL_ENABLED": "false"
}
And the actual api method call is
df = glue_context.create_data_frame.from_options(
connection_type="kafka",
connection_options={
"connectionName": "kafka-glue-connector",
"security.protocol": "SASL_SSL",
"classification": "json",
"startingOffsets": "latest",
"topicName": "atopic",
"inferSchema": "true",
"typeOfData": "kafka",
"numRetries": 1,
}
)
When running logs show the client is attempting to connect to brokers using Kerberos, and runs into
22/10/19 18:45:54 INFO ConsumerConfig: ConsumerConfig values:
sasl.mechanism = GSSAPI
security.protocol = SASL_SSL
security.providers = null
send.buffer.bytes = 131072
...
org.apache.kafka.common.errors.SaslAuthenticationException: Failed to configure SaslClientAuthenticator
Caused by: org.apache.kafka.common.KafkaException: Principal could not be determined from Subject, this may be a transient failure due to Kerberos re-login
How can I authenticate the AWS Glue job using SASL/SCRAM? What properties do I need to set in the connection and in the method call?
Thank you

python, google cloud platform: unable to overwite a file from google bucket: CRC32 does not match

I am using python3 client to connect to google buckets and trying to the following
download 'my_rules_file.yaml'
modify the yaml file
overwrite the file
Here is the code that i used
from google.cloud import storage
import yaml
client = storage.Client()
bucket = client.get_bucket('bucket_name')
blob = bucket.blob('my_rules_file.yaml')
yaml_file = blob.download_as_string()
doc = yaml.load(yaml_file, Loader=yaml.FullLoader)
doc['email'].clear()
doc['email'].extend(["test#gmail.com"])
yaml_file = yaml.dump(doc)
blob.upload_from_string(yaml_file, content_type="application/octet-stream")
This is the error I get from the last line for upload
BadRequest: 400 POST https://storage.googleapis.com/upload/storage/v1/b/fc-sandbox-datastore/o?uploadType=multipart: {
"error": {
"code": 400,
"message": "Provided CRC32C \"YXQoSg==\" doesn't match calculated CRC32C \"EyDHsA==\".",
"errors": [
{
"message": "Provided CRC32C \"YXQoSg==\" doesn't match calculated CRC32C \"EyDHsA==\".",
"domain": "global",
"reason": "invalid"
},
{
"message": "Provided MD5 hash \"G/rQwQii9moEvc3ZDqW2qQ==\" doesn't match calculated MD5 hash \"GqyZzuvv6yE57q1bLg8HAg==\".",
"domain": "global",
"reason": "invalid"
}
]
}
}
: ('Request failed with status code', 400, 'Expected one of', <HTTPStatus.OK: 200>)
why is this happening. This seems to happen only for ".yaml files".
The reason for your error is because you are trying to use the same blob object for both downloading and uploading this will not work you need two separate instances... You can find some good examples here Python google.cloud.storage.Blob() Examples
You should use a seperate blob instance to handle the upload you are trying with only one...
.....
blob = bucket.blob('my_rules_file.yaml')
yaml_file = blob.download_as_string()
.....
the second instance is needed here
....
blob.upload_from_string(yaml_file, content_type="application/octet-stream")
...

Hyperledger chaincode does not get current user metadata

Currently I'm working with Hyperledger chaincode and trying to get at least any info regarding current user who invokes/queries chaincode. For some reason chaincode example asset_management.go results in an error "ERRO 031 Got error: Invalid admin certificate. Empty." I have security.enabled and security.privacy set to true and Membership services running. I've enrolled "admin".
Here are the lines in the code where it happens
// Set the admin
// The metadata will contain the certificate of the administrator
adminCert, err := stub.GetCallerMetadata()
if err != nil {
myLogger.Debug("Failed getting metadata")
return nil, errors.New("Failed getting metadata.")
}
if len(adminCert) == 0 {
myLogger.Debug("Invalid admin certificate. Empty.")
return nil, errors.New("Invalid admin certificate. Empty.")
}
Do you have any ideas how to make the chaincode return any data for stub.GetCallerMetadata() ?
"Metadata" should be provided in your deploy command, an example of "deploy" for asset_management_with_roles:
curl -XPOST -d ‘{“jsonrpc": "2.0", "method": "deploy", "params": {"type": 1,"chaincodeID": {"path": "github.com/hyperledger/fabric/examples/chaincode/go/asset_management_with_roles","language": "GOLANG"}, "ctorMsg": { "args": ["init"] }, "metadata":[97, 115, 115, 105, 103, 110, 101, 114] ,"secureContext": "assigner"} ,"id": 0}' http://localhost:7050/chaincode
In this command "metadata" contains utf-8 encoded string “assigner”. This string will be saved in a ledger and only user with such role will be able to execute “assign” function in smart contract.
"asset_management" example expects that you will provide certificate in metadata field. In order to obtain certificate you can use step 9 described in related question: How is running the asset_management.go different from running a simple chaincode like chaincode_example02.go