We have a WAR file deployed on Tomcat and the database credentials are fetched through JNDI. This WAR now has to be moved to AWS cloud, and the requirement is db credentials has to be stored in AWS Secret Manager. My question is can I continue using JNDI/Tomcat along with Secret Manager ? I understand AWS SM has API and SDKs to access it, can that be integrated with JNDI/Tomcat somehow ? All posts I have seen mentions using the API/SDK directly from code, none i have found say anything about server integration. Is accessing AWS SM from code really the best way to do it ? Thanks.
A side note - for some reason unknown to me, we cannot use BeanStalk, it is just Tomcat on an EC2 instance.
Maybe you could use the JDBC driver wrapper: https://github.com/aws/aws-secretsmanager-jdbc. If you are using a connection pool manager you can follow the example in the README and replace the JDBC library with the wrapper library, specifying the secret in the configuration. The wrapper will then retrieve the secret and pass it to the real JDBC library.
If you are not using a connection pool manager, you could still replace the existing JDBC driver with the wrapper, but this would take some code modifications.
By using the wrapper, you can also turn on auto-rotation on the DB password since the wrapper knows to re-fetch the secret after it changes.
Related
I have googled quite heavily the last couple of hours to see if I could use Google Secret Manager from an external service like AWS Lambda or my local PC. I could not find anything helpful, or something that describes properly the steps to do so.
I do not want to play with the APIs and end up doing the authenticating via OAuth myself, I wish to use the client library. How would I go about doing so?
I have so far referred to the following links:
https://cloud.google.com/secret-manager/docs/configuring-secret-manager - Describes setting up secret manager, and prompts you to set up Google Cloud SDK.
https://cloud.google.com/sdk/docs/initializing - Describes setting up the cloud SDK (doesn't seem like I get some kind of config file that helps me to point my client library to the correct GCP project)
The issue I have is that it doesn't seem like I get access to some form of credential that I can use with the client library that consumes the secret manager service of a particular GCP project. Something like a service account token or a means of authenticating and consuming the service from an external environment.
Any help is appreciated, it just feels like I'm missing something. Or is it simply impossible to do so?
PS: Why am I using GCP secret manager when AWS offers a similar service? The latter is too expensive.
I think that your question applies to all GCP services, there isn't anything that is specific to Secret Manager.
As you mentioned, https://cloud.google.com/docs/authentication/getting-started documents how to create and use a Service Account. But this approach has the downside that now you need to figure out to store the service account key (yet another Secret!)
If you're planning to access GCP Secret Manager from AWS you can consider using: https://cloud.google.com/iam/docs/configuring-workload-identity-federation#aws which uses identity federation to map an AWS service account to a GCP service account, without the need to store an extra Secret somewhere.
How in a work environment with different AWS environments say for example develop, staging and production is it best to store the AWS Access Key and Secret Key other than in the appsettings.json files in .Net Core? I know there is Secret Manager but not sure if that is the best way to store these two values. Looking for someone that may have done this specifically for production and how they handled this within their organization. Thanks for any information.
I believe that your application is running outside of AWS and that it needs to make API calls to AWS services, for example SQS. To make those API calls, your application needs AWS credentials.
Here are approaches for authenticating external applications in a machine-to-machine scenario. In your case, your client seems to need to be able to make arbitrary AWS service requests and that means using AWS signature v4 requests, signed using AWS credentials, which are ideally temporary, rotated credentials from STS rather than persistent credentials (such as IAM user credentials).
Typically, you would configure your application with a base set of IAM credentials that allow the application to assume an IAM role. That role itself, rather than the base credentials, would then give your application the permissions it needs to make SQS API calls etc.
The issue you face is how to securely store the base set of credentials. This is a problem that on-premise applications have had since day one, well before the cloud era, and there are various solutions, depending on the technology you're using.
Typically these credentials would be encrypted, not committed to code repos, and populated on the relevant, locked down application servers in some secure fashion. Some potentially useful resources:
Encrypting sections of a configuration file for an ASP.NET application
Use AWS Secrets Manager to store & read passwords in .Net Core apps
Securely store and retrieve sensitive info in .NET Core apps with Azure Key Vault
AWS Secret Manager securely stores your secrets until you retrieve them at runtime. If your going to be running your ASP.NET Core app in AWS, then AWS Secrets Manager is a great option, as it allows you to finely control the permissions associated with the AWS IAM roles running your apps.
Here are some faqs which were given from the AWS for secrets-manager service and which will clear your doubts also.
Here is the article which you can refer to for implementing secure secrets storage for .net core with AWS Secret Manager
I have a question what is the difference if I just use a Oracle/MySQL service provided by PCF without binding it? What difference will it create. I can anyway access DB using the credentials
There are two differences that come to mind:
When you create a service through the Cloud Foundry marketplace, that will create backing resources for the service but in most cases it does not create credentials. The act of binding a service to your app, in most cases with most service brokers, will actually create service credentials for you. When you unbind, again with most brokers, the service credentials are destroyed. This makes it easy to regenerate your service credentials, just unbind/rebind the service and restart your app. The net result is that if you don't bind, there are no credentials.
Most people do not want to include credentials with the actual application (see https://12factor.net/ for details why). They want to be able to provide configuration external to the app. On Cloud Foundry this commonly amounts to binding a service.
Having said that, how do you want to provide the credentials to your application?
Service bindings are there to try and make life as a developer easier but you don't have to use them. If you want to pass in the configuration some other way, like via environment variables, a config file, or using a config service (Spring Cloud Config Server or Vault) those are fine options too.
If you do not want to bind a service to your app, the only thing you'll need to do is to create a service key instead. A service key is like a binding, but not associated with an application. It will also generate a set of unique credentials. You can then take the credentials from your service key and feed them to your app in the way that works best for you.
Ex:
cf create-service-key service-instance key-name
cf service-key service-instance key-name
The first command creates the service key, the second will display its credentials.
I am trying to write an application in Python.
Through this application I want to create AWS Cognito users and provide services like user Sign-in, Forgot password, etc.
As I understand, boto3, is the standard Python library for accessing AWS APIs, from Python.
https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html
This library needs storing of AWS credentials ( Access key and secure access keys) on the host machine.
Can this be avoided?
I want to distribute this Python application to my users.
I am checking, if I can avoid this configuration of AWS credentials on every user's host.
Is there any alternative option to boto3 library?
If you absolutely need to access internal AWS API's you need to log in to AWS. Access keys is one way, it's also possible to use aws-adfs command line tool to log in though active directory, but that requires your AWS/AD administrators to do some additional setup on their side.
I would suggest looking into writing a client-server / web applications that would be hosted within AWS and only expose relevant functionality to authenticated users.
If costs are an issue for a hosted application, look into lambdas, as there you pay only for cpu/memory time. In case of setting management app it will probably not even exceed free tier.
I was successfully able to connect to RDS like any other database connection.
I use spring jpa data ( repository ) to do CRUD operation on postgres db.
currently I provide the db url and the credential in the properties file
spring:
datasource:
url: jdbc:postgresql://<rds-endpoint>:5432/<dbschema>
username: <dbuser>
password: <dbpassword>
However this is not an option while connecting to production or preproduction.
what is the best practise here.
Does AWS provide any inbuild mechanism to read these details from an endpoint like in the case of accessing S3 ?
My intention is not expose the password.
Several options are available to you:
Use the recently announced IAM access to Postgres RDS
Use Systems Manager Parameter Store to store the password
Use Secrets Manager to store the password and automatically rotate credentials
For 2 and 3, look up the password on application start in Spring using a PropertyPlaceholderConfiguration and the AWSSimpleSystemsManagement client (GetParameter request). SystemsManager can proxy requests to SecretsManager to keep a single interface in your code to access parameters.
IAM credentials is more secure in that:
If using EC2 instance profiles, access to the database uses short lived temporary credentials.
If not on EC2 you can generate short lived authentication tokens.
The password is not stored in your configuration.
If you have a separate database team they can manage access independent of the application user.
Removing access can be done via IAM
another generic option I found was to use AWS Secret Manager
(doc link)
RDS specific solution is to connect to DB Instance Using the AWS SDK using IAMDBAuth