"ValueError: Invalid control character" using Python OAuth client - python-2.7

I am trying to load data from a google spreadsheet into a postgres database.
The problem is when I am trying to authenticate my credentials, I get the following error:
File "/usr/local/lib/python2.7/dist-packages/oauth2client/_openssl_crypt.py", line 117, in from_string
pkey = crypto.load_privatekey(crypto.FILETYPE_PEM, parsed_pem_key)
OpenSSL.crypto.Error: [('PEM routines', 'PEM_read_bio', 'no start line')]
I have followed all steps in Using OAuth2 for Authorization, enabled the API and created a service account from which I got a .json file containing the necessary keys and authentication elements.
The way I am trying to authenticate is the following:
json_key = json.load(open('gdoc.json'),strict=False)
creds = SignedJwtAssertionCredentials(json_key['client_email'], json_key['private_key'], scope)
login = gspread.authorize(creds)
Perhaps the issue comes from the parameter in the json load strict=False, the problem is if I remove it, get the error:
File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Invalid control character at: line 5 column 38 (char 174)
I saw a couple other forums and they suggest using the strict=False parameter since there are non escaped \n inside the json key file.
This is a copy of the .json key file:
{
"type": "service_account",
"project_id": "geometric-shine-118101",
"private_key_id": "xxx",
"private_key": "-----BEGIN PRIVATE KEY-----\nxxx\n-----END PRIVATE KEY-----\n",
"client_email": "dataload#geometric-shine-118101.iam.gserviceaccount.com",
"client_id": "117076930343404252458",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://accounts.google.com/o/oauth2/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/dataload%40geometric-shine-118101.iam.gserviceaccount.com"
}

"private_key": "-----BEGIN PRIVATE KEY-----\nxxx\n-----END PRIVATE KEY-----\n",
"client_email": "dataload#geometric-shine-118101.iam.gserviceaccount.com",
Copy and paste the actual values into single quotes (.e.g. ''). It worked for me.
'dataload#geometric-shine-118101.iam.gserviceaccount.com'
'-----BEGIN PRIVATE KEY-----\nxxx\n-----END PRIVATE KEY-----\n'

Related

AWS SecretsManager works in Eclipse, can't connect to Service Endpoint in ColdFusion

I have the following class written in Java using Eclipse on my Amazon EC2 instance.
import java.nio.ByteBuffer;
import com.amazonaws.auth.*;
import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.services.secretsmanager.*;
import com.amazonaws.services.secretsmanager.model.*;
public class SMtest {
public static void main(String[] args) {
String x = getSecret();
System.out.println(x);
}
public SMtest()
{
}
public static String getSecret() {
String secretName = "mysecret";
String endpoint = "secretsmanager.us-east-1.amazonaws.com";
String region = "us-east-1";
String result = "";
//BasicAWSCredentials awsCreds = new BasicAWSCredentials("mypublickey", "mysecretkey");
AwsClientBuilder.EndpointConfiguration config = new AwsClientBuilder.EndpointConfiguration(endpoint, region);
AWSSecretsManagerClientBuilder clientBuilder = AWSSecretsManagerClientBuilder.standard()
.withCredentials(new InstanceProfileCredentialsProvider(false));
//.withCredentials(new AWSStaticCredentialsProvider(awsCreds));
clientBuilder.setEndpointConfiguration(config);
AWSSecretsManager client = clientBuilder.build();
String secret;
ByteBuffer binarySecretData;
GetSecretValueRequest getSecretValueRequest = new GetSecretValueRequest()
.withSecretId(secretName).withVersionStage("AWSCURRENT");
GetSecretValueResult getSecretValueResult = null;
try {
getSecretValueResult = client.getSecretValue(getSecretValueRequest);
} catch(ResourceNotFoundException e) {
System.out.println("The requested secret " + secretName + " was not found");
} catch (InvalidRequestException e) {
System.out.println("The request was invalid due to: " + e.getMessage());
} catch (InvalidParameterException e) {
System.out.println("The request had invalid params: " + e.getMessage());
}
if(getSecretValueResult == null) {
result = "";
}
// Depending on whether the secret was a string or binary, one of these fields will be populated
if(getSecretValueResult.getSecretString() != null) {
secret = getSecretValueResult.getSecretString();
result = secret;
//System.out.println(secret);
}
else {
binarySecretData = getSecretValueResult.getSecretBinary();
result = binarySecretData.toString();
}
return result;
}
}
When I execute it from within Eclipse, it works just fine. When I compile the class to use it in ColdFusion (2021) from the same EC2 with the following code:
<cfscript>
obj = CreateObject("java","SMtest");
obj.init();
result = obj.getSecret();
</cfscript>
<cfoutput>#result#</cfoutput>
I get a "Failed to connect to service endpoint" error. I believe I have all the IAM credentials set up properly since it is working in straight Java. However, when I change the credentials to use the Basic Credentials with my AWS Public and Secret Key (shown in comments in the code above), it works in both Java and ColdFusion.
I created an AWS Policy that manages the Secrets Manager permissions. I also added AmazonEC2FullAccess. I also tried to create a VPC Endpoint, but this had no effect.
Why would it be working in Java and not in ColdFusion (which is based on Java) ? What roles/policies would I have to add to get it to work in ColdFusion when it is already working in Java?
STACK TRACE ADDED:
com.amazonaws.SdkClientException: Failed to connect to service endpoint:
at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:100)
at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.getToken(InstanceMetadataServiceResourceFetcher.java:91)
at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.readResource(InstanceMetadataServiceResourceFetcher.java:69)
at com.amazonaws.internal.EC2ResourceFetcher.readResource(EC2ResourceFetcher.java:66)
at com.amazonaws.auth.InstanceMetadataServiceCredentialsFetcher.getCredentialsEndpoint(InstanceMetadataServiceCredentialsFetcher.java:58)
at com.amazonaws.auth.InstanceMetadataServiceCredentialsFetcher.getCredentialsResponse(InstanceMetadataServiceCredentialsFetcher.java:46)
at com.amazonaws.auth.BaseCredentialsFetcher.fetchCredentials(BaseCredentialsFetcher.java:112)
at com.amazonaws.auth.BaseCredentialsFetcher.getCredentials(BaseCredentialsFetcher.java:68)
at com.amazonaws.auth.InstanceProfileCredentialsProvider.getCredentials(InstanceProfileCredentialsProvider.java:165)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1266)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:842)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:792)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:779)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:753)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:713)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:695)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:559)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:539)
at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.doInvoke(AWSSecretsManagerClient.java:2454)
at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.invoke(AWSSecretsManagerClient.java:2421)
at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.invoke(AWSSecretsManagerClient.java:2410)
at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.executeGetSecretValue(AWSSecretsManagerClient.java:943)
at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.getSecretValue(AWSSecretsManagerClient.java:912)
at SMtest.getSecret(SMtest.java:52)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at coldfusion.runtime.java.JavaProxy.invoke(JavaProxy.java:106)
at coldfusion.runtime.CfJspPage._invoke(CfJspPage.java:4254)
at coldfusion.runtime.CfJspPage._invoke(CfJspPage.java:4217)
at cfsmtest2ecfm1508275519.runPage(D:\ColdFusion2021\cfusion\wwwroot\smtest.cfm:4)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:257)
at coldfusion.tagext.lang.IncludeTag.handlePageInvoke(IncludeTag.java:749)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:578)
at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:573)
at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:43)
at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
at coldfusion.filter.PathFilter.invoke(PathFilter.java:162)
at coldfusion.filter.IpFilter.invoke(IpFilter.java:45)
at coldfusion.filter.LicenseFilter.invoke(LicenseFilter.java:30)
at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:97)
at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:81)
at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:28)
at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:60)
at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
at coldfusion.filter.CachingFilter.invoke(CachingFilter.java:62)
at coldfusion.CfmServlet.service(CfmServlet.java:231)
at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:311)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:228)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163)
at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:46)
at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:47)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:190)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163)
at coldfusion.inspect.weinre.MobileDeviceDomInspectionFilter.doFilter(MobileDeviceDomInspectionFilter.java:57)
at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:47)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:190)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:190)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:373)
at org.apache.coyote.ajp.AjpProcessor.service(AjpProcessor.java:462)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:893)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1723)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.net.MalformedURLException: Unknown protocol: http
at java.base/java.net.URL.<init>(URL.java:708)
at java.base/java.net.URL.fromURI(URL.java:748)
at java.base/java.net.URI.toURL(URI.java:1139)
at com.amazonaws.internal.ConnectionUtils.connectToEndpoint(ConnectionUtils.java:83)
at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:80)
... 80 more
Caused by: java.lang.IllegalStateException: Unknown protocol: http
at org.apache.felix.framework.URLHandlersStreamHandlerProxy.parseURL(URLHandlersStreamHandlerProxy.java:373)
at java.base/java.net.URL.<init>(URL.java:703)
... 84 more
After noticing that the bottom end of the Stack Trace says "unknown protocol: http" and "Illegal State Exception: Unknown Protocol", I changed the endpoint in my class above to:
String endpoint = "https://secretsmanager.us-east-1.amazonaws.com";
and I am running the web-site via https now. It still works with no problem running it in Eclpse, and I am still getting the same error when using ColdFusion (i.e., when using a browser)
Since the basic credentials are working and since the error is not access denied, it suggests there is no issue with your IAM setup. Instead, likely your process failed to fetch the EC2 instance metadata. For example, timeout when calling the metadata endpoint, hence the Failed to connect to service endpoint error.
One way around this is to retrieve the accessKey and secretKey manually by calling the instance metadata API. For example, using cfhttp to populate variables.
Example curl command from docs: retrieve /api/token and use it to retrieve /meta-data/iam/security-credentials/<rolename>.
[ec2-user ~]$ TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/iam/security-credentials/s3access
Output:
{
"Code" : "Success",
"LastUpdated" : "2012-04-26T16:39:16Z",
"Type" : "AWS-HMAC",
"AccessKeyId" : "ASIAIOSFODNN7EXAMPLE",
"SecretAccessKey" : "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"Token" : "token",
"Expiration" : "2017-05-17T15:09:54Z"
}
From here, if you still face any errors, at least it should be more explicit to you which step has failed and why.
Just a note, AWS recommends caching the credentials until near expiry instead of querying for every transaction to avoid throttling.

StorageException: Anonymous caller does not have storage.objects.get access

When trying to run the below code on CircleCI
fun getJsonFromCloudStorage(): ByteArrayInputStream {
val blobId = BlobId.of("my-company", "creds/my-company-creds.json")
val storage = StorageOptions.getDefaultInstance().service
val get = storage.get(blobId)
return get.getContent().inputStream()
}
it will throw the below error during the integration tests.
> Task :test FAILED
function.GetMetadataFromYouTubeTest > extractIncorrectId FAILED
java.lang.ExceptionInInitializerError
at function.GetMetadataFromYouTube.expand(GetMetadataFromYouTube.kt:17)
at function.GetMetadataFromYouTube.expand(GetMetadataFromYouTube.kt:14)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:491)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:299)
at function.GetMetadataFromYouTubeTest.extractIncorrectId(GetMetadataFromYouTubeTest.kt:71)
Caused by:
com.google.cloud.storage.StorageException: Anonymous caller does not have storage.objects.get access to cni-analytics/creds/cni-awesome.json.
at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:220)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.get(HttpStorageRpc.java:414)
at com.google.cloud.storage.StorageImpl$5.call(StorageImpl.java:198)
at com.google.cloud.storage.StorageImpl$5.call(StorageImpl.java:195)
at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:89)
at com.google.cloud.RetryHelper.run(RetryHelper.java:74)
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:51)
at com.google.cloud.storage.StorageImpl.get(StorageImpl.java:195)
at com.google.cloud.storage.StorageImpl.get(StorageImpl.java:209)
at storage.CredentialHelper$Companion.getJsonFromCloudStorage(CredentialHelper.kt:18)
at service.YoutubeService.initialiseYouTube(YoutubeService.kt:50)
at service.YoutubeService.<init>(YoutubeService.kt:19)
at MainKt.<clinit>(main.kt:15)
... 6 more
Caused by:
com.google.api.client.googleapis.json.GoogleJsonResponseException: 401 Unauthorized
{
"code" : 401,
"errors" : [ {
"domain" : "global",
"location" : "Authorization",
"locationType" : "header",
"message" : "Anonymous caller does not have storage.objects.get access to my-company/creds/my-company-creds.json.",
"reason" : "required"
} ],
"message" : "Anonymous caller does not have storage.objects.get access to my-company/creds/my-company-creds.json."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.get(HttpStorageRpc.java:411)
... 17 more
I followed their documentation.
They said this in their documentation:
Note: To use certain services (like Google Cloud Datastore), you will also need to set the CircleCI $GOOGLE_APPLICATION_CREDENTIALS environment variable to ${HOME}/gcloud-service-key.json.
Instead I set $GOOGLE_APPLICATION_CREDENTIALS in the CircleCI UI to /home/circleci/gcloud-service-key.json and it worked.
I'm assuming this is because I was trying to reference an environment variable from the UI so ${HOME} had not been set when it was setting this environment variable. Perhaps if this environment variable was set in the config.yml ${HOME} would resolve.

Decrypting SAML2 response using pysaml2 Python module

I am integrating my app with okta to have single sign on. Okta will be passing some user information in SAML response which I need to use in my application.
Hence, we decided to encrypt the saml response(xml) at IDP using my server(apache) public key.
Now I am trying to decrypt the saml2 response so that I can get the attributes.
My applications uses
Python 3.5
Django 1.11
pysaml2 python module
I am using below to validate/parse the saml2 response coming from okta
https://github.com/fangli/django-saml2-auth
If the saml response is not encrypted, I am able to process the response and able to get the user identity and user attributes from it.
However once it is encrypted at okta end with my server public key, I am not able to decrypt with my private key.
The saml setting I have at my application is below :
saml_settings = { 'metadata': {
"local": [ metadat_xml
],
},
'service': {
'sp': {
'endpoints': {
'assertion_consumer_service': [
(acs_url, BINDING_HTTP_REDIRECT),
(acs_url, BINDING_HTTP_POST),
(https_acs_url, BINDING_HTTP_REDIRECT),
(https_acs_url, BINDING_HTTP_POST)
],
},
'allow_unsolicited': True,
'authn_requests_signed': False,
'logout_requests_signed': True,
'want_assertions_signed': True,
'want_response_signed': False,
},
},
'key_file': "mykey.key", # private part
'cert_file': "mykey.crt", # public part
'xmlsec_binary': '/usr/bin/xmlsec1',
'encryption_keypairs': [{
'key_file': 'mykey.key',
'cert_file': 'mykey.crt',
}]
}
if 'ENTITY_ID' in settings.SAML2_AUTH:
saml_settings['entityid'] = settings.SAML2_AUTH['ENTITY_ID']
#print('entity id ' , settings.SAML2_AUTH['ENTITY_ID'])
if 'NAME_ID_FORMAT' in settings.SAML2_AUTH:
saml_settings['service']['sp']['name_id_format'] = settings.SAML2_AUTH['NAME_ID_FORMAT']
# NOTE-'NAME_ID_FORMAT is set to None above
spConfig = Saml2Config()
spConfig.load(saml_settings)
spConfig.allow_unknown_attributes = True
saml_client = Saml2Client(config=spConfig)
return saml_client
Then I have
saml_client = _get_saml_client(get_current_domain(r))
resp = r.POST.get('SAMLResponse', None)
authn_response = saml_client.parse_authn_request_response(resp, entity.BINDING_HTTP_POST )
This auth_response object is not returning me anything when the message is encrypted.
In logs I see below error
GbHvkJJM0WIsPYFGtiQ/0n+ux0tV/z/OKpT1AqEE74iRVHEHD7omP41iY/c4=
</ns3:CipherValue></ns3:CipherData><ns3:ReferenceList><ns3:DataReference
URI="#_648cdbd139564492f0bdfe4fbbda92f6" /></ns3:ReferenceList>
</ns3:EncryptedKey></ns1:EncryptedAssertion></ns0:Response>
2018-04-30 18:21:09,232 [DEBUG] sigver saml2.sigver decrypt(): Decrypt input
len: 15187
2018-04-30 18:21:09,233 [DEBUG] sigver saml2.sigver _run_xmlsec(): xmlsec
command: /usr/bin/xmlsec1 --decrypt --privkey-pem
/private.pem --id-attr:ID EncryptedKey --output /tmp/tmp7rt7g95u.xml
/tmp/tmpkhxwo8s4
2018-04-30 18:21:09,247 [DEBUG] sigver saml2.sigver _run_xmlsec(): xmlsec
p_out:
2018-04-30 18:21:09,247 [DEBUG] sigver saml2.sigver _run_xmlsec(): xmlsec
p_erryy:
func=xmlSecXPathDataExecute:file=xpath.c:line=273:obj=unknown:
subj=xmlXPtrEval:error=5:libxml2 library function
failed:expr=xpointer(id('_841612fffac65343e73f8913eeecfb30'))
func=xmlSecXPathDataListExecute:file=xpath.c:line=373:obj=unknown:
subj=xmlSecXPathDataExecute:error=1:xmlsec library function failed:
func=xmlSecTransformXPathExecute:file=xpath.c:line=483:
obj=xpointer:subj=xmlSecXPathDataExecute:error=1:xmlsec library function
failed:
func=xmlSecTransformDefaultPushXml:file=transforms.c:
line=2411:obj=xpointer:subj=xmlSecTransformExecute:error=1:xmlsec library
function failed:
func=xmlSecTransformCtxExecute:file=transforms.c:line=1302:
obj=unknown:subj=xmlSecTransformCtxXmlExecute:error=1:xmlsec library
function failed:
func=xmlSecKeyDataRetrievalMethodXmlRead:file=keyinfo.c:line=1178:
obj=retrieval-method:subj=xmlSecTransformCtxExecute:error=1:xmlsec library
function failed:
func=xmlSecKeyInfoNodeRead:file=keyinfo.c:line=114:obj=retrieval-method:
subj=xmlSecKeyDataXmlRead:error=1:xmlsec library function
failed:node=RetrievalMethod
func=xmlSecKeysMngrGetKey:file=keys.c:line=1349:obj=unknown:
subj=xmlSecKeyInfoNodeRead:error=1:xmlsec library function
failed:node=KeyInfo
func=xmlSecEncCtxEncDataNodeRead:file=xmlenc.c:line=957:
obj=unknown:subj=unknown:error=45:key is not found:
func=xmlSecEncCtxDecryptToBuffer:file=xmlenc.c:line=715:
obj=unknown:subj=xmlSecEncCtxEncDataNodeRead:error=1:xmlsec library function
failed:
func=xmlSecEncCtxDecrypt:file=xmlenc.c:line=623:
obj=unknown:subj=xmlSecEncCtxDecryptToBuffer:error=1:xmlsec library function
failed:
Error: failed to decrypt file
Error: failed to decrypt file "/tmp/tmpkhxwo8s4"
I am not sure why xmlsec1 command is failing and what I am missing here.
I have tried decrypting with my private key(self signed private key) here
https://www.samltool.com/decrypt.php
and it works
Could you please help me here and let me know what I am not doing correctly?
You need to add
saml_settings['id_attr_name'] = 'Id'
The default id attr is ID, but Okta uses Id. See xmlsec FAQ for more details.

Hyperledger chaincode does not get current user metadata

Currently I'm working with Hyperledger chaincode and trying to get at least any info regarding current user who invokes/queries chaincode. For some reason chaincode example asset_management.go results in an error "ERRO 031 Got error: Invalid admin certificate. Empty." I have security.enabled and security.privacy set to true and Membership services running. I've enrolled "admin".
Here are the lines in the code where it happens
// Set the admin
// The metadata will contain the certificate of the administrator
adminCert, err := stub.GetCallerMetadata()
if err != nil {
myLogger.Debug("Failed getting metadata")
return nil, errors.New("Failed getting metadata.")
}
if len(adminCert) == 0 {
myLogger.Debug("Invalid admin certificate. Empty.")
return nil, errors.New("Invalid admin certificate. Empty.")
}
Do you have any ideas how to make the chaincode return any data for stub.GetCallerMetadata() ?
"Metadata" should be provided in your deploy command, an example of "deploy" for asset_management_with_roles:
curl -XPOST -d ‘{“jsonrpc": "2.0", "method": "deploy", "params": {"type": 1,"chaincodeID": {"path": "github.com/hyperledger/fabric/examples/chaincode/go/asset_management_with_roles","language": "GOLANG"}, "ctorMsg": { "args": ["init"] }, "metadata":[97, 115, 115, 105, 103, 110, 101, 114] ,"secureContext": "assigner"} ,"id": 0}' http://localhost:7050/chaincode
In this command "metadata" contains utf-8 encoded string “assigner”. This string will be saved in a ledger and only user with such role will be able to execute “assign” function in smart contract.
"asset_management" example expects that you will provide certificate in metadata field. In order to obtain certificate you can use step 9 described in related question: How is running the asset_management.go different from running a simple chaincode like chaincode_example02.go

Getting a 403 while creating a user, followed every step right so far

Looks like I have followed every step (given that the documentation is extremely lacking, it is sourced from multiple places). This is my code:
def create_user(cred_file_location, user_first_name, user_last_name, user_email):
cred_data = json.loads(open(cred_file_location).read())
access_email = cred_data['client_email']
private_key = cred_data['private_key']
# I have tried with the scope as a single string, and also
# as an array of a single string. Neither worked
credentials = SignedJwtAssertionCredentials(access_email, private_key, ["https://www.googleapis.com/auth/admin.directory.user"])
http = Http()
http = credentials.authorize(http)
service = build('admin', 'directory_v1', http=http)
users = service.users()
userinfo = {
'primaryEmail': user_email,
'name': {
'givenName': user_first_name,
'familyName': user_last_name
},
'password': ''.join(random.SystemRandom().choice(string.ascii_uppercase + string.digits) for _ in range(80))
}
users.insert(body=userinfo).execute()
I downloaded the JSON key right, and it is loading it correctly. This is my JSON key (I am redacting certain parts of identifying information, I have kept some of it there to show that I am loading the correct info):
{
"type": "service_account",
"private_key_id": "c6ae56a9cb267fe<<redacted>>",
"private_key": "<<redacted>>",
"client_email": "account-1#<<redacted>>.iam.gserviceaccount.com",
"client_id": "10931536<<redacted>>",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://accounts.google.com/o/oauth2/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/account-1%40<<redacted>>.iam.gserviceaccount.com"
}
This is how these credentials look in the developer console:
I have also enabled sitewide access for the service account:
I have no clue as to why I am still getting these 403s:
File "/usr/lib/python2.7/site-packages/googleapiclient/http.py", line 729, in execute
raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://www.googleapis.com/admin/directory/v1/users?alt=json returned "Not Authorized to access this resource/api">
Any help is greatly appreciated.
Finally, on some random stackoverflow answer, I found the solutin. I have to sub as a user to execute any request. Esentially:
credentials = SignedJwtAssertionCredentials(
access_email,
private_key,
["https://www.googleapis.com/auth/admin.directory.user"])
changes to:
credentials = SignedJwtAssertionCredentials(
access_email,
private_key,
["https://www.googleapis.com/auth/admin.directory.user"],
sub="user#example.org")
Where all requests will now be made as if they were made on behalf of user#example.org.