What's the best way to store token signing certificate for an AWS web app? - amazon-web-services

I am using IdentityServer4 with .NET Core 2.0 on AWS's ElasticBeanstalk. I have a certificate for signing tokens. What's the best way to store this certificate and retrieve it from the application? Should I just stick it with the application files? Throw it in an environment variable somehow?
Edit: just to be clear, this is a token signing certificate, not an SSL certificate.

I don't really like the term 'token signing certificate' because it sounds so benign. What you have is a private key (as part of the certificate), and everyone knows you should secure your private keys!
I wouldn't store this in your application files. If someone gets your source code, they shouldn't also get the keys to your sensitive data (if someone has your signing cert, they can generate any token they like and pretend to be any of your users).
I would consider storing the certificate in AWS parameter store. You could paste the certificate into a parameter, which can be encrypted at rest. You then lock down the parameter with an AWS policy so only admins and the application can get the cert - your naughty Devs dont need it! Your application would pull the parameter string when needed and turn it into your certificate object.
This is how I store secrets in my application. I can provide more examples/details if required.
Edit -- This was the final result from Stu's guidance
The project needs 2 AWS packages from Nuget to the project
AWSSDK.Extensions.NETCORE.Setup
AWSSDK.SimpleSystemsManagement
Create 2 parameters in the AWS SSM Parameter Store like:
A plain string named /MyApp/Staging/SigningCertificate and the value is a Base64 encoded .pfx file
An encrypted string /MyApp/Staging/SigningCertificateSecret and the value is the password to the above .pfx file
This is the relevant code:
// In Startup class
private X509Certificate2 GetSigningCertificate()
{
// Configuration is the IConfiguration built by the WebHost in my Program.cs and injected into the Startup constructor
var awsOptions = Configuration.GetAWSOptions();
var ssmClient = awsOptions.CreateServiceClient<IAmazonSimpleSystemsManagement>();
// This is blocking because this is called during synchronous startup operations of the WebHost-- Startup.ConfigureServices()
var res = ssmClient.GetParametersByPathAsync(new Amazon.SimpleSystemsManagement.Model.GetParametersByPathRequest()
{
Path = "/MyApp/Staging",
WithDecryption = true
}).GetAwaiter().GetResult();
// Decode the certificate
var base64EncodedCert = res.Parameters.Find(p => p.Name == "/MyApp/Staging/SigningCertificate")?.Value;
var certificatePassword = res.Parameters.Find(p => p.Name == "/MyApp/Staging/SigningCertificateSecret")?.Value;
byte[] decodedPfxBytes = Convert.FromBase64String(base64EncodedCert);
return new X509Certificate2(decodedPfxBytes, certificatePassword);
}
public void ConfigureServices(IServiceCollection servies)
{
// ...
var identityServerBuilder = services.AddIdentityServer();
var signingCertificate = GetSigningCertificate();
identityServerBuilder.AddSigningCredential(signingCertificate);
//...
}
Last, you may need to set an IAM role and/or policy to your EC2 instance(s) that gives access to these SSM parameters.
Edit: I have been moving my web application SSL termination from my load balancer to my elastic beanstalk instance this week. This requires storing my private key in S3. Details from AWS here: Storing Private Keys Securely in Amazon S3

Related

How to resolve Unable to find valid certification path to requested target for AWS Java Application?

My network is behind ZScaler Proxy. I have installed AWS CLI. I have added all the Amazon Root CA Certificates along with ZScaler CA Root Certificate in a pem file. I have setup AWS_CA_Bundle and my aws cli command for fetching secretsmanager worked.
But when on the same machine, I am trying to fetch SecretManagers using AWS SDK, it gives exception - Unable to find valid certification path to requested target.
Can someone guide me what needs to be done?
Below is the source code
public class AwsSecretManager {
public static AWSSecretManagerPojo getRedshiftCredentialsFromSecretManager(String secretName) throws JsonUtilityException, AwsSecretException {
String secret = getSecret(secretName);
// Gaurav added this.
System.out.println("secret \n" + secret);
if (!StringUtility.isNullOrEmpty(secret)) {
AWSSecretManagerPojo AWSSecretManagerPojo = GsonUtility.getInstance().fromJson(secret, AWSSecretManagerPojo.class,
EdelweissConstant.GSON_TAG);
return AWSSecretManagerPojo;
} else {
throw new AwsSecretException("unable to get redshift credentials from aws secret manager");
}
}
private static String getSecret(String secretName) {
// Gaurav commented below and manually supplied the secret as SSL issue is there.
String region = EdelweissConstant.AWS_SECRET_MANAGER_REGION;
AWSSecretsManager client = AWSSecretsManagerClientBuilder.standard()
.withRegion(region)
.build();
String secret = null;
GetSecretValueRequest getSecretValueRequest = new GetSecretValueRequest()
.withSecretId(secretName);
GetSecretValueResult getSecretValueResult = client.getSecretValue(getSecretValueRequest);
if (getSecretValueResult.getSecretString() != null) {
secret = getSecretValueResult.getSecretString();
}
return secret;
}
}
This question is a bit old but other answers to this topic were all programmatic and on a per-application basis which I didn't like, since I wanted this to work for all my applications. To anyone stumbling upon this, these are the steps that I took to fix this issue in my application.
Download the AWS Root CAs from here. CA1 was good enough for me, but you may need others as well. Java requires the .der format one. You can also get the .pem file, but you'll need to convert it to .der. Note that the certificate from Amazon had a .cer extension, but it's still compatible.
Verify that the certificate is legible for the Java keytool:
$ keytool -v -printcert -file AmazonRootCA1.cer
Certificate fingerprints:
SHA1: <...>
SHA256: <...>
Optionally, verify the public key hash if you'd like (outside the scope of this answer, have a look here if you want).
Import the certificate file to your Java keystore. You can do this for your entire JRE by using the keytool command as follows and answering yes when prompted:
keytool -importcert -alias <some_name> -keystore $JAVA_HOME/jre/lib/security/cacerts -storepass changeit -file AmazonRootCA1.cer
If you've changed the default Java keystore pass (changeit), you'll obviously need to use that one instead.
After this your application should be able to connect to AWS without any certificate issues.
I removed all the Amazon Certificates and just added ZScaler Root Certiicate. And it resolved the issue.

Get Azure WebJobs connection strings from KeyVault before host is built

I am following the directions at https://learn.microsoft.com/en-us/azure/app-service/app-service-key-vault-references
Essentially, I am attempting to protect the storage connection string used for AzureWebJobsDashboard and AzureWebJobsStorage behind an Azure Key vault secret. I cannot use my injected KeyVault service to fetch it because my service container has not been built yet. So I found (through the link above) I could express this intent using a "#Microsoft.KeyVault()" expression in configuration. Here is an example where I moved the configuration to inline code to keep it terse:
.ConfigureHostConfiguration(configurationBuilder =>
{
configurationBuilder
.AddConfiguration(configuration)
.AddInMemoryCollection(new Dictionary<string, string>
{
["ConnectionStrings:AzureWebJobsDashboard"] = "#Microsoft.KeyVault(SecretUri=https://host.vault.azure.net/secrets/secret-name/ec545689445a40b199c0e0a956f16fca)",
["ConnectionStrings:AzureWebJobsStorage"] = "#Microsoft.KeyVault(SecretUri=https://host.vault.azure.net/secrets/secret-name/ec545689445a40b199c0e0a956f16fca)",
});
})
If I run this, I get:
FormatException: No valid combination of account information found.
If I change the configuration values from the special annotation to the copied secret value from Key Vault (the blue copy button under the 'Show Secret Value' button), everything just works. This confirms to me the connection string I use is correct.
Also, I manually used KeyVaultClient w/AzureServiceTokenProvider to verify the process should work when running locally in Visual Studio, even before the host has been built. I am able to get the secret just fine. This tells me I have sufficient privileges to get the secret.
So now I am left wondering if this is even supported. There are pages which imply this is possible however, such as https://medium.com/statuscode/getting-key-vault-secrets-in-azure-functions-37620fd20a0b. At least for Azure Functions. I am using Azure Web Jobs which gets deployed as a console application with an ASP.NET Core service, and I cannot find an example with that configuration.
Can anybody clarify if what I am doing is supported? And if not, what is the advisable process for getting connection strings stored in Azure Key Vault before the Azure Web Jobs host has been built?
Thanks
I have gone through a lot of online resources and everything seems to indicate that the special decorated #Microsoft.KeyVault setting only works when the value lives in AppSettings on the Azure Portal, not in local configuration. Somebody please let me know if that is an incorrect assessment.
So to solve this problem, I came up with a solution which in all honesty, feels a little hacky because I am depending on the fact that the connection string is not read/cached from local configuration until the host is ran (not during build). Basically, the idea is to build a configuration provider for which I can set a value after the host has been built. For example:
public class DelayedConfigurationSource : IConfigurationSource
{
private IConfigurationProvider Provider { get; } = new DelayedConfigurationProvider();
public IConfigurationProvider Build(IConfigurationBuilder builder) => Provider;
public void Set(string key, string value) => Provider.Set(key, value);
private class DelayedConfigurationProvider : ConfigurationProvider
{
public override void Set(string key, string value)
{
base.Set(key, value);
OnReload();
}
}
}
A reference to this type gets added during host builder construction:
var delayedConfigurationSource = new DelayedConfigurationSource();
var hostBuilder = new HostBuilder()
.ConfigureHostConfiguration(configurationBuilder =>
{
configurationBuilder
.AddConfiguration(configuration)
.Add(delayedConfigurationSource);
})
...
And just make sure to set the configuration before running the host:
var host = hostBuilder.Build();
using (host)
{
var secretProvider = host.Services.GetRequiredService<ISecretProvider>();
var secret = await secretProvider.YourCodeToGetSecretAsync().ConfigureAwait(false);
delayedConfigurationSource.Set("ConnectionStrings:AzureWebJobsStorage", secret.Value);
await host.RunAsync().ConfigureAwait(false);
}
If there is a more intuitive way to accomplish this, please let me know. If not, the connection string design is plain silly.

AWS SES: Can't send emails from any one of the new regions

I have the problem that i can't send emails from the new aws ses environments, which were introduced a month ago.
All the old ones are working fine (e.g. us-east-1, us-west-2, eu-west-1).
But if I want to send a mail from one of the new environments, e.g. eu-central-1, I just get the error message:
The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
But this can't be the case, because all the old ones are working fine with the same keys.
Therefore I would really appreciate it if sb else could test the sample code with their account to check if they have the same issue.
The new environments are eu-central-1, ap-south-1 and ap-southeast-2. Endpoint Urls
Sample Code:
var ses = require('node-ses');
var client = ses.createClient({ key: '', secret: '', amazon: 'https://email.eu-central-1.amazonaws.com'});
async function sendMessage() {
let options = {};
options.from = "test#aol.com";
options.to = "test2#aol.com";
options.subject = "TestMail";
options.message = "Test";
console.log("Try to sendMessage");
client.sendEmail(options, function (err, data, res) {
console.log("Error: " + JSON.stringify(err));
console.log("Data: " + data);
console.log("res: " + res);
});
}
sendMessage();
The sample code uses the node-ses npm package and you just need to enter aws iam user credentials, which have ses access.
If you want to check different regions, you have to change url in the createClient constructor.
Dont worry, the sample code does not send an email!!!
If the region is working, it should throw an error message similar to this: Email address is not verified. The following identities failed the check in region EU-WEST-1: test#aol.com, test2#aol.com"
Otherwise the error will be the one described above.
I also have to mention that I am currently still in sandbox mode, so maybe the new regions are blocked for sandbox users?
It's because you must be creating the SES credentials from the IAM console . You should instead create the credentials using the SES interface/console.
Follow this article to create smtp credentials using SES interface:
http://docs.amazonwebservices.com/ses/latest/GettingStartedGuide/GetAccessIDs.html.

Get service Name of Task under aws fargate

We need to get the service name under which a fargate task runs so we can perform some per service configuration (we have one service per customer, and use the service name to identify them).
By knowing the service discovery namespace for our cluster and the Task IP address, we are able to do find out the service by doing the following.
Get the task ip address, eaither by calling http://169.254.170.2/v2/metadata endpoint or by using the ECS_ENABLE_CONTAINER_METADATA method in my follow-up answer.
With the cluster namespace we call AWS.ServiceDiscovery.listNamespaces
From there we extract the nameSpace id.
We pass that to AWS.ServiceDiscovery.listServices
We pass the id of each service to AWS.ServiceDiscovery.listInstances
We flat map the results of that and look for an instance that matches our IP address above.
VoilĂ ! that record gives us the service name.
Works fine, it just seems like a super circuitous path! I'm just wondering whether there is some shorter way to get this information.
Here's a working C# example in two steps. It gets the taskARN from the metadata to retrieve the task description, and then reads its Group property, which contains the name of the service. It uses AWSSDK.ECS to get the task description and Newtonsoft.Json to parse the JSON.
private static string getServiceName()
{
// secret keys, should be encoded in license configuration object
var ecsClient = new AmazonECSClient( ACCESS_KEY, SECRET_KEY );
var request = new DescribeTasksRequest();
// need cluster here if not default
request.Cluster = NAME_OF_CLUSTER;
request.Tasks.Add( getTaskArn() );
var asyncResponse = ecsClient.DescribeTasksAsync( request );
// probably need this synchronously for application to proceed
asyncResponse.Wait();
var response = asyncResponse.Result;
string group = response.Tasks.Single().Group;
// group returned in the form "service:[NAME_OF_SERVICE]"
return group.Remove( 0, 8 );
}
private static string getTaskArn()
{
// special URL for fetching internal Amazon information for ECS instances
string url = #"http://169.254.170.2/v2/metadata";
string metadata = getWebRequest( url );
// use JObject to read the JSON return
return JObject.Parse( metadata )[ "TaskARN" ].ToString();
}
private static string getWebRequest( string url )
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create( url );
request.AutomaticDecompression = DecompressionMethods.GZip;
using HttpWebResponse response = (HttpWebResponse)request.GetResponse();
using Stream stream = response.GetResponseStream();
using StreamReader reader = new StreamReader( stream );
return reader.ReadToEnd();
}
You can get the service name from startedBy property of the task definition. Using boto sdk you can call a describe_tasks (or its equivalent in aws-cli: aws ecs describe-tasks) which will provide a
'startedBy': 'string'
The tag specified when a task is started. If the task is started by an Amazon ECS service, then the startedBy parameter contains the deployment ID of the service that starts it.
From:
boto3 ecs client
aws-cli ecs client
Hope it helps.
The answer above requires reading the container metadata that appears if you set the ECS_ENABLE_CONTAINER_METADATA environment variable in the task. The work flow is then:
Read the container metadata file ecs-container-metadata.json to get the taskArn
Call the aws.ecs.describe-tasks function to get the startedBy property
Call aws.servicediscover.get-service.
Two steps instead of three, if you don't count reading the metadata file. Better, to be sure, but I'm probably not going to change the way we do it for now.

Using DeveloperCredentials for AWS S3 Client in C++

I am writing a C++ application in Windows using the AWS C++ SDK and need help with Developer Authenticated Identities in order to upload/download files to/from S3 in my application.
We have a backend application using Cognito to get the temporary credentials for AWS (IdentityID & OpenIDToken). I know the ProviderName and the IdentityPoolID.
Rather than describe what I've attempted, I believe it's easier to show.
Below is the snippet. I am trying to do the equivalent of "Refresh Identity" which is available in C# and Java SDK's. C# and Java both use "RefreshIdentity" which defines an IdentityState(IdentityID, ProviderName, OpenIDToken, fromCacheFlag)
Based on some documentation I found, I created a new class (MyInheritedCognitoIdentityClient) derived from CognitoIdentityClient which overrides the GetId & GetOpenIdToken functions to return the IdentityID and openIdToken received back from the server applicaton. The GetID is called, but GetOpenIdToken is not called - as the documentation implies; instead there is a call to GetCredentialsForIdentity (which fails with NotAuthorizedException).
Below is my code snippet:
`{
std::shared_ptr<MyInheritedPersistentCognitoIdentityProvider> identityProvider = std::make_shared<MyInheritedPersistentCognitoIdentityProvider>();
identityProvider->setIdentityPool(myIdentityPoolID); // us-east-1:c373a2ca-b912-3839-a65c-8d4ce53d512e -> not real
identityProvider->setAccountId(myProviderName); // login.mycompany.net
std::shared_ptr<MyInheritedCognitoIdentityClient> cognitoIdentityClient =
std::make_shared<MyInheritedCognitoIdentityClient>(); // _cognitoID and _openIdToken
std::shared_ptr<Aws::Auth::AWSCredentialsProvider> cognitoCachCredProvider =
std::make_shared<Aws::Auth::CognitoCachingAuthenticatedCredentialsProvider>(identityProvider, cognitoIdentityClient);
Aws::S3::S3Client s3Client(cognitoCachCredProvider);
/* attempt to upload/download here */
}`
you need to call GetCredentialsForIdentity server side, if not anyone can login in your own created provider (I'm assuming login.mycompany.net is custom provider)