System.Security.Cryptography.CryptographicException: 'The key {00000000-0000-0000-0000-000000000000} was not found in the key ring.' - azure-webjobs

I have deployed a webjob (.Net 4.6) which encrypts some url strings and sends it to customers via email. I used .NET Core's IDataProtector for the encrpytion and I had to manually reference the DLL since webjob .net 4.6 was not supporting its library.
For example in webjob:
IDataProtector protector = provider.CreateProtector("myProtector")
http://test.com/Email/Agreement?n=" + protector.Protect(name) + "&a=" + protector.Protect(address)
which becomes a link
http://test.com/Email/Agreement?n=CfDJ8AAAAAAAAAAAAAAAAAAAAAD96g09022UwEG-GyVftzkm-n2JuL1hmvL5HLyghXliIgJ_N014EBqBFfWjmdA&a=CfDJ8AAAAAAAAAAAAAAAAAAAAAALoq9IuMBZKozhzH3jRaHpyFE1wtXzQK3T_0RNuX9LsSVxPITWgU9ZR21jXLk3YGjt
in the email.
when the customers click on the url link in their email, it would go to my client application's (.Net Core 1.1) controller to decrypt the url string in order to pop up an agreement page.
Ex:
public EmailController(IDataProtectionProvider provider)
{
_protector = provider.CreateProtector("myProtector");
}
public IActionResult Agreement(string n, string a)
{
var decryptName = _protector.Unprotect(n);
var decryptAddress = _protector.Unprotect(a);
}
However, I am getting a following error message when I tried to Unprotect them:
System.Security.Cryptography.CryptographicException: 'The key {00000000-0000-0000-0000-000000000000} was not found in the key ring.'
When I searched for an answer, I realized I can configure Data Protection to store the keys to Azure Blob Storage. This link shows me how to persist keys to azure blob storage.
Questions:
What is the best approach other than storing a key to azure blob storage?
If I am on the right track, how do I store it?
How do I configure the same setting for the webjob project as shown in the link which does not have Startup.cs for configuration?
Help is much appreciated.

What is the best approach other than storing a key to azure blob storage?
Based on the document provided by you, we could store the key in file system, Azure Redis, Azure Storage and Registry. Since Registry is not support by Web Job(Azure Web App). If choosing file system, we also need to transfer the keys between the Web Job and your web application. So Azure Redis and Azure Storage would be the good approaches.
If I am on the right track, how do I store it?
Here are the detail steps of how to store keys on Azure Storage.
Step 1, you need to create an Azure Storage account if you doesn't have one.
Step 2, install Microsoft.AspNetCore.DataProtection.AzureStorage package using NuGet.
Step 3, configure DataProtection using following code. We need to invoke SetApplicationName method and use the same application name as your Web Job.
var storageAccount = CloudStorageAccount.Parse("put your azure storage connection string here");
var client = storageAccount.CreateCloudBlobClient();
var container = client.GetContainerReference("key-container");
container.CreateIfNotExistsAsync().GetAwaiter().GetResult();
services.AddDataProtection().SetApplicationName("myapplication")
.PersistKeysToAzureBlobStorage(container, "keys.xml");
Step 4, in your controller, you could use IDataProtectionProvider as following.
public class HomeController : Controller
{
private IDataProtector _protector;
public HomeController(IDataProtectionProvider provider)
{
_protector = provider.CreateProtector("test-purpose");
}
public IActionResult Index()
{
string encryptedTExt = _protector.Protect("abcd");
return Content(encryptedTExt);
}
}
How do I configure the same setting for the webjob project as shown in the link which does not have Startup.cs for configuration?
Step 1, you need to add reference to following DLLs.
Step 2, Add a wrapper class of IDataProtector as following.
public class EncryptService
{
IDataProtector _protector;
// the 'provider' parameter is provided by DI
public EncryptService(IDataProtectionProvider provider)
{
_protector = provider.CreateProtector("test-purpose");
}
public string Protect(string text)
{
return _protector.Protect(text);
}
public string Unprotect(string encryptedText)
{
return _protector.Unprotect(encryptedText);
}
}
Step 3, use ServiceCollection to configure the Data Protection service. Please note that we need to invoke SetApplicationName method and use the same application name as your web application.
static void Main()
{
var storageAccount = CloudStorageAccount.Parse("put your azure storage connection string here");
var client = storageAccount.CreateCloudBlobClient();
var container = client.GetContainerReference("key-container");
container.CreateIfNotExistsAsync().GetAwaiter().GetResult();
var serviceCollection = new ServiceCollection();
serviceCollection.AddDataProtection().SetApplicationName("myapplication")
.PersistKeysToAzureBlobStorage(container, "keys.xml");
var services = serviceCollection.BuildServiceProvider();
}
Step 4, after that, you could use following code to encrypt or decrypt your data.
var encryptService = ActivatorUtilities.CreateInstance<EncryptService>(services);
string text = encryptService.Protect("abcd");

Related

Azure Notification Hubs in C++ apps?

In Microsoft documentation there is the example for C# implementation of Azure Notification Hubs -- https://learn.microsoft.com/en-us/azure/notification-hubs/notification-hubs-windows-store-dotnet-get-started-wns-push-notification#create-a-sample-windows-app
private async void InitNotificationsAsync()
{
var channel = await PushNotificationChannelManager.CreatePushNotificationChannelForApplicationAsync();
var hub = new NotificationHub("<your hub name>", "<Your DefaultListenSharedAccessSignature connection string>");
var result = await hub.RegisterNativeAsync(channel.Uri);
// Displays the registration ID so you know it was successful
if (result.RegistrationId != null)
{
var dialog = new MessageDialog("Registration successful: " + result.RegistrationId);
dialog.Commands.Add(new UICommand("OK"));
await dialog.ShowAsync();
}
}
but our app use C++ and we found part of example code in WinRT/C++ documentation:
https://learn.microsoft.com/en-us/uwp/api/windows.networking.pushnotifications.pushnotificationchannelmanager?view=winrt-22621
...but without part of Azure code:
var hub = new NotificationHub("<your hub name>", "<Your DefaultListenSharedAccessSignature connection string>");
Is it means that for apps with bare C++ Azure Messaging is impossible theme?
At this moment, only a few Azure Services can be used using the c++ SDK. You can check in here:
https://azure.github.io/azure-sdk/cpp_introduction.html
However, you can interact directly to the REST Api:
https://learn.microsoft.com/en-us/rest/api/notificationhubs/create-update-registration

Get Azure WebJobs connection strings from KeyVault before host is built

I am following the directions at https://learn.microsoft.com/en-us/azure/app-service/app-service-key-vault-references
Essentially, I am attempting to protect the storage connection string used for AzureWebJobsDashboard and AzureWebJobsStorage behind an Azure Key vault secret. I cannot use my injected KeyVault service to fetch it because my service container has not been built yet. So I found (through the link above) I could express this intent using a "#Microsoft.KeyVault()" expression in configuration. Here is an example where I moved the configuration to inline code to keep it terse:
.ConfigureHostConfiguration(configurationBuilder =>
{
configurationBuilder
.AddConfiguration(configuration)
.AddInMemoryCollection(new Dictionary<string, string>
{
["ConnectionStrings:AzureWebJobsDashboard"] = "#Microsoft.KeyVault(SecretUri=https://host.vault.azure.net/secrets/secret-name/ec545689445a40b199c0e0a956f16fca)",
["ConnectionStrings:AzureWebJobsStorage"] = "#Microsoft.KeyVault(SecretUri=https://host.vault.azure.net/secrets/secret-name/ec545689445a40b199c0e0a956f16fca)",
});
})
If I run this, I get:
FormatException: No valid combination of account information found.
If I change the configuration values from the special annotation to the copied secret value from Key Vault (the blue copy button under the 'Show Secret Value' button), everything just works. This confirms to me the connection string I use is correct.
Also, I manually used KeyVaultClient w/AzureServiceTokenProvider to verify the process should work when running locally in Visual Studio, even before the host has been built. I am able to get the secret just fine. This tells me I have sufficient privileges to get the secret.
So now I am left wondering if this is even supported. There are pages which imply this is possible however, such as https://medium.com/statuscode/getting-key-vault-secrets-in-azure-functions-37620fd20a0b. At least for Azure Functions. I am using Azure Web Jobs which gets deployed as a console application with an ASP.NET Core service, and I cannot find an example with that configuration.
Can anybody clarify if what I am doing is supported? And if not, what is the advisable process for getting connection strings stored in Azure Key Vault before the Azure Web Jobs host has been built?
Thanks
I have gone through a lot of online resources and everything seems to indicate that the special decorated #Microsoft.KeyVault setting only works when the value lives in AppSettings on the Azure Portal, not in local configuration. Somebody please let me know if that is an incorrect assessment.
So to solve this problem, I came up with a solution which in all honesty, feels a little hacky because I am depending on the fact that the connection string is not read/cached from local configuration until the host is ran (not during build). Basically, the idea is to build a configuration provider for which I can set a value after the host has been built. For example:
public class DelayedConfigurationSource : IConfigurationSource
{
private IConfigurationProvider Provider { get; } = new DelayedConfigurationProvider();
public IConfigurationProvider Build(IConfigurationBuilder builder) => Provider;
public void Set(string key, string value) => Provider.Set(key, value);
private class DelayedConfigurationProvider : ConfigurationProvider
{
public override void Set(string key, string value)
{
base.Set(key, value);
OnReload();
}
}
}
A reference to this type gets added during host builder construction:
var delayedConfigurationSource = new DelayedConfigurationSource();
var hostBuilder = new HostBuilder()
.ConfigureHostConfiguration(configurationBuilder =>
{
configurationBuilder
.AddConfiguration(configuration)
.Add(delayedConfigurationSource);
})
...
And just make sure to set the configuration before running the host:
var host = hostBuilder.Build();
using (host)
{
var secretProvider = host.Services.GetRequiredService<ISecretProvider>();
var secret = await secretProvider.YourCodeToGetSecretAsync().ConfigureAwait(false);
delayedConfigurationSource.Set("ConnectionStrings:AzureWebJobsStorage", secret.Value);
await host.RunAsync().ConfigureAwait(false);
}
If there is a more intuitive way to accomplish this, please let me know. If not, the connection string design is plain silly.

Get service Name of Task under aws fargate

We need to get the service name under which a fargate task runs so we can perform some per service configuration (we have one service per customer, and use the service name to identify them).
By knowing the service discovery namespace for our cluster and the Task IP address, we are able to do find out the service by doing the following.
Get the task ip address, eaither by calling http://169.254.170.2/v2/metadata endpoint or by using the ECS_ENABLE_CONTAINER_METADATA method in my follow-up answer.
With the cluster namespace we call AWS.ServiceDiscovery.listNamespaces
From there we extract the nameSpace id.
We pass that to AWS.ServiceDiscovery.listServices
We pass the id of each service to AWS.ServiceDiscovery.listInstances
We flat map the results of that and look for an instance that matches our IP address above.
VoilĂ ! that record gives us the service name.
Works fine, it just seems like a super circuitous path! I'm just wondering whether there is some shorter way to get this information.
Here's a working C# example in two steps. It gets the taskARN from the metadata to retrieve the task description, and then reads its Group property, which contains the name of the service. It uses AWSSDK.ECS to get the task description and Newtonsoft.Json to parse the JSON.
private static string getServiceName()
{
// secret keys, should be encoded in license configuration object
var ecsClient = new AmazonECSClient( ACCESS_KEY, SECRET_KEY );
var request = new DescribeTasksRequest();
// need cluster here if not default
request.Cluster = NAME_OF_CLUSTER;
request.Tasks.Add( getTaskArn() );
var asyncResponse = ecsClient.DescribeTasksAsync( request );
// probably need this synchronously for application to proceed
asyncResponse.Wait();
var response = asyncResponse.Result;
string group = response.Tasks.Single().Group;
// group returned in the form "service:[NAME_OF_SERVICE]"
return group.Remove( 0, 8 );
}
private static string getTaskArn()
{
// special URL for fetching internal Amazon information for ECS instances
string url = #"http://169.254.170.2/v2/metadata";
string metadata = getWebRequest( url );
// use JObject to read the JSON return
return JObject.Parse( metadata )[ "TaskARN" ].ToString();
}
private static string getWebRequest( string url )
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create( url );
request.AutomaticDecompression = DecompressionMethods.GZip;
using HttpWebResponse response = (HttpWebResponse)request.GetResponse();
using Stream stream = response.GetResponseStream();
using StreamReader reader = new StreamReader( stream );
return reader.ReadToEnd();
}
You can get the service name from startedBy property of the task definition. Using boto sdk you can call a describe_tasks (or its equivalent in aws-cli: aws ecs describe-tasks) which will provide a
'startedBy': 'string'
The tag specified when a task is started. If the task is started by an Amazon ECS service, then the startedBy parameter contains the deployment ID of the service that starts it.
From:
boto3 ecs client
aws-cli ecs client
Hope it helps.
The answer above requires reading the container metadata that appears if you set the ECS_ENABLE_CONTAINER_METADATA environment variable in the task. The work flow is then:
Read the container metadata file ecs-container-metadata.json to get the taskArn
Call the aws.ecs.describe-tasks function to get the startedBy property
Call aws.servicediscover.get-service.
Two steps instead of three, if you don't count reading the metadata file. Better, to be sure, but I'm probably not going to change the way we do it for now.

What's the best way to store token signing certificate for an AWS web app?

I am using IdentityServer4 with .NET Core 2.0 on AWS's ElasticBeanstalk. I have a certificate for signing tokens. What's the best way to store this certificate and retrieve it from the application? Should I just stick it with the application files? Throw it in an environment variable somehow?
Edit: just to be clear, this is a token signing certificate, not an SSL certificate.
I don't really like the term 'token signing certificate' because it sounds so benign. What you have is a private key (as part of the certificate), and everyone knows you should secure your private keys!
I wouldn't store this in your application files. If someone gets your source code, they shouldn't also get the keys to your sensitive data (if someone has your signing cert, they can generate any token they like and pretend to be any of your users).
I would consider storing the certificate in AWS parameter store. You could paste the certificate into a parameter, which can be encrypted at rest. You then lock down the parameter with an AWS policy so only admins and the application can get the cert - your naughty Devs dont need it! Your application would pull the parameter string when needed and turn it into your certificate object.
This is how I store secrets in my application. I can provide more examples/details if required.
Edit -- This was the final result from Stu's guidance
The project needs 2 AWS packages from Nuget to the project
AWSSDK.Extensions.NETCORE.Setup
AWSSDK.SimpleSystemsManagement
Create 2 parameters in the AWS SSM Parameter Store like:
A plain string named /MyApp/Staging/SigningCertificate and the value is a Base64 encoded .pfx file
An encrypted string /MyApp/Staging/SigningCertificateSecret and the value is the password to the above .pfx file
This is the relevant code:
// In Startup class
private X509Certificate2 GetSigningCertificate()
{
// Configuration is the IConfiguration built by the WebHost in my Program.cs and injected into the Startup constructor
var awsOptions = Configuration.GetAWSOptions();
var ssmClient = awsOptions.CreateServiceClient<IAmazonSimpleSystemsManagement>();
// This is blocking because this is called during synchronous startup operations of the WebHost-- Startup.ConfigureServices()
var res = ssmClient.GetParametersByPathAsync(new Amazon.SimpleSystemsManagement.Model.GetParametersByPathRequest()
{
Path = "/MyApp/Staging",
WithDecryption = true
}).GetAwaiter().GetResult();
// Decode the certificate
var base64EncodedCert = res.Parameters.Find(p => p.Name == "/MyApp/Staging/SigningCertificate")?.Value;
var certificatePassword = res.Parameters.Find(p => p.Name == "/MyApp/Staging/SigningCertificateSecret")?.Value;
byte[] decodedPfxBytes = Convert.FromBase64String(base64EncodedCert);
return new X509Certificate2(decodedPfxBytes, certificatePassword);
}
public void ConfigureServices(IServiceCollection servies)
{
// ...
var identityServerBuilder = services.AddIdentityServer();
var signingCertificate = GetSigningCertificate();
identityServerBuilder.AddSigningCredential(signingCertificate);
//...
}
Last, you may need to set an IAM role and/or policy to your EC2 instance(s) that gives access to these SSM parameters.
Edit: I have been moving my web application SSL termination from my load balancer to my elastic beanstalk instance this week. This requires storing my private key in S3. Details from AWS here: Storing Private Keys Securely in Amazon S3

Get AzureWebJobsDashboard through KeyVault

As I have read, Azure Webjobs SDK requires the value of AzureWebJobsDashboard to be defined in the connection string section of an App Service. If I do that manually by going to the portal, and configuring the values, my webjob works fine.
What I want now is to provide the value of the connectionString in KeyVault and provide a secretURI provided by KeyVault in the ApplicationSettings of the App Service. Thereafter, I want the SDK to parse the secretURI and obtain the secret stored underneath in KeyVault.
I have tried to do this but am not able to get a definite solution for this. Is there any way to do this, or is currently not supported ?
Here is working example of what Mike suggested:
var config = new JobHostConfiguration();
if (config.IsDevelopment)
{
config.UseDevelopmentSettings();
}
else
{
var azureServiceTokenProvider = new AzureServiceTokenProvider();
var vaultBaseUrl = ConfigurationManager.AppSettings["KeyVaultUri"];
var client = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(azureServiceTokenProvider.KeyVaultTokenCallback));
config.DashboardConnectionString = client.GetSecretAsync(vaultBaseUrl, "DashboardConnectionString").Result.Value;;
config.StorageConnectionString = client.GetSecretAsync(vaultBaseUrl, "StorageConnectionString").Result.Value;;
}
config.UseTimers();
var host = new JobHost(config);
References:
https://learn.microsoft.com/en-us/azure/key-vault/service-to-service-authentication
WebJobs SDK will not automatically get from KeyVault.
But you could also fetch from keyvault yourself and then programmatically set the result on the JobHostConfiguration.Dashboard property.
You could file a request here to have more direct integration: https://github.com/Azure/azure-webjobs-sdk