Does anybody know if it is possible to access Amazon Redshift via single sign on or service accounts. Our specific need is to map domain users to Redshift users, and then grant access to specific objects to this mapped users, so if a user wants to query Redhisft via some SQL client or some Excel connector (for example) he can use his domain credentials without having to store or type passwords on every connector. I know the existence of AWS Identity and Access Management (IAM) but from my understanding this works just for SSO to the management console, am i right?
AWS has recently announced the support for Federated authentication with single sign-on support for Redshift.
Using IAM Authentication to Generate Database User Credentials for AWS Redshift
I am currently trying to implement it and will update this answer once I am done with the setup.
Related
I have request from users to be able to connect to my datasets and table in bigquery to fetch the data and manipulate it programmatically outside of GCP
The situation now that i created a service account with credentials to view data and i share the json key of this service account with users in email .
I want to avoid users to use the key inside their code
best way to secure sharing this key with them
The best way to share your application outside Google Cloud is through Workload Identity Federation. Although, creating public/private key pairs is also a secured way to use and share your user-managed service account, it can still impose a threat and security risk if not correctly managed.
Just run through this documentation and use IAM external identities to impersonate a service account to avoid any security issues on your security account keys even without mantaining it.
Currently, we use AWS IAM User permanent credentials to transfer customers' data from our company's internal AWS S3 buckets to customers' Google BigQuery tables following BigQuery Data Transfer Service documentation.
Using permanent credentials possesses security risks related to the data stored in AWS S3.
We would like to use AWS IAM Role temporary credentials, which require the support of a session token on the BiqQuery side to get authorized on the AWS side.
Is there a way that the BigQuery Data Transfer Servce can use AWS IAM roles or temporary credentials to authorise against AWS and transfer data?
We considered Omni framework (https://cloud.google.com/bigquery/docs/omni-aws-cross-cloud-transfer) to transfer data from S3 to BQ, however, we faced several concerns/limitations:
Omni framework targets data analysis use-case rather than data transfer from external services. This concerns us that the design of Omni framework may have drawbacks in relation to data transfer at high scale
Omni framework currently supports only AWS-US-EAST-1 region (we require support at least in AWS-US-WEST-2 and AWS-EU-CENTRAL-1 and corresponding Google regions). This is not backward compatible with current customers' setup to transfer data from internal S3 to customers' BQ.
Our current customers will need to signup for Omni service to properly migrate from the current transfer solution we use
We considered a workaround with exporting data from S3 through staging in GCS (i.e. S3 -> GCS -> BQ), but this will also require a lot of effort from both customers and our company's sides to migrate to the new solution.
Is there a way that the BigQuery Data Transfer Servce can use AWS IAM roles or temporary credentials to authorise against AWS and transfer data?
No unfortunately.
The official Google BigQuery Data Transfer Service only mentions AWS access keys all throughout the documentation:
The access key ID and secret access key are used to access the Amazon S3 data on your behalf. As a best practice, create a unique access key ID and secret access key specifically for Amazon S3 transfers to give minimal access to the BigQuery Data Transfer Service. For information on managing your access keys, see the AWS general reference documentation.
The irony of the Google documentation is that while it refers to best practices and links to the official AWS docs, it actually doesn't endorse best practices and ignores what AWS mention:
We recommend that you use temporary access keys over long term access keys, as mentioned in the previous section.
Important
Unless there is no other option, we strongly recommend that you don't create long-term access keys for your (root) user. If a malicious user gains access to your (root) user access keys, they can completely take over your account.
You have a few options:
hook into both sides manually (i.e. link up various SDKs and/or APIs)
find an alternative BigQuery-compatible service, which does as such
accept the risk of long-term access keys.
In conclusion, Google is at fault here of not following security best practices and you - as a consumer - will have to bear the risk.
In my application I want to users to be able to create an organization i.e. (OrgA) and then have users sign up under said organization using either an invite code or token. Users in OrgA should have access to an Amazon S3 directory (which stores images and files) and access to a database table that has been created for the said organization.
I could not find a solution on how to implement this online and was wondering if using Amazon Cognito groups was a good idea to meet requirements.
This is supported by the Amazon Cognito Service. That is, you can use Amazon Cognito to control permissions for different user groups in your app. This ensures that users have appropriate access to backend resources, determined by the group they belong to. For more information, see Building fine-grained authorization using Amazon Cognito User Pools groups.
From AWS docs:
When to Create an IAM User (Instead of a Role)
...
You want to use the command-line interface (CLI) to work with AWS.
When to Create an IAM Role (Instead of a User)
- You're creating an application that runs on an Amazon Elastic Compute Cloud (Amazon EC2) instance and that application makes requests to AWS.
- You're creating an app that runs on a mobile phone and that makes requests to AWS.
- Users in your company are authenticated in your corporate network and want to be able to use AWS without having to sign in again—that is, you want to allow users to federate into AWS.
But it seems like companies heavily use roles for everything:
Role for groups by creating roles with specific policies and creating custom policies to apply to groups.
Assume role to use the CLI.
Switch role to use different accounts.
Is that excessive or real work based solution?
Is that excessive or real work based solution?
Based on my own experience with AWS, heavily using roles is a real work based solution because, in my company, we use only roles to give access to users (yes, we have 0 users registered in your AWS environments). I'll list the reasons why we chose this way:
We are using AWS Control Tower.
This service enables AWS Organizations with at least 3 AWS accounts to manage your organization. It'd be a mess with we had to create a user for each AWS account. Also, AWS Control Tower enables AWS Single Sign-On.
We're using AWS Single Sign-On.
This service correlates multiples AWS accounts with multiples roles with multiples users. Description:
AWS Single Sign-On (SSO) is a cloud SSO service that makes it easy to centrally manage SSO access to multiple AWS accounts and business applications. With just a few clicks, you can enable a highly available SSO service without the upfront investment and on-going maintenance costs of operating your own SSO infrastructure. With AWS SSO, you can easily manage SSO access and user permissions to all of your accounts in AWS Organizations centrally. AWS SSO also includes built-in SAML integrations to many business applications, such as Salesforce, Box, and Office 365. Further, by using the AWS SSO application configuration wizard, you can create Security Assertion Markup Language (SAML) 2.0 integrations and extend SSO access to any of your SAML-enabled applications. Your users simply sign in to a user portal with credentials they configure in AWS SSO or using their existing corporate credentials to access all their assigned accounts and applications from one place.
Please, check out some features offered by this service. There are a lot of benefits using roles instead of users. In my point of view, with AWS SSO, AWS itself facilitates the use of roles.
The only disadvantage I found is that every time I need to use AWS CLI, I need to access AWS SSO portal, copy the credentials and paste in my terminal because credentials expires after some time. But in the end, this disadvantage is small compared to the security that this process offers - if my computer is stolen, AWS CLI couldn't be accessed because of credentials expiration.
I am setting up an AWS Account, the account will be used by an organization of employees for EC2 use/experimentation. Obviously, I do not want to distribute the root login info or set up one single IAM user for everyone to use. Unfortunately, I do not have the time to manage creating individual IAM users for everyone on a regular basis.
So, is there a way to auto-create IAM users based on a given email's domain on their first login attempt? The users should have read-only roles to begin, then an Administrator could give more roles as needed to each user. I am open to suggestions, perhaps lambda functions or linking to an identity provider?
Keep in mind that these new IAM users need to have access to the AWS Management Console, this is not necessarily intended for login to applications hosted on AWS.
Update:
Moving forward using this AWS Management Console Federation Proxy Sample found in Amazon's code reference, using with Microsoft Exchange hosted email.
If your existing identity provider supports SAML2 Federation, you can set it up to login to the AWS Management Console.
For more details refer Enabling SAML 2.0 Federated Users to Access the AWS Management Console.
Else you can implement a custom Federation Broker to return an URL to the user, after they authenticate with their corporate credentials.
For more details refer Creating a URL that Enables Federated Users to Access the AWS Management Console (Custom Federation Broker).