I currently am running my app on CloudKit by Apple. I would like to start using Amazon Web Services instead but I am not sure which part I should be using.
The app currently allows the user to log in on multiple devices and save a date value.
It then allows all of the user to fetch the same images and strings from CloudKit.
Should I be using AWS Cognito or S3 or something else?
Are there any tutorials for this?
Thank you
Look at Amazon Cognito for authenticating and identifying your users and delivering AWS Credentials to your users' devices. These AWS Credentials enable your users to securely access your AWS Resources. See How Amazon Cognito Keeps Mobile App Users' Data Safe.
Amazon Cognito also provides data synchronization capabilities that enable you to save user-specific information that only the current user can read. Look at Amazon DynamoDB for storing application-wide data that is shared with all users (like CloudKit database/records). Cognito also integrates with DynamoDB to enable fine-grained access control that might be interesting for you to review too. Use Amazon S3 for storing large objects (like CloudKit "Assets") like images or other files.
Related
Currently, we use AWS IAM User permanent credentials to transfer customers' data from our company's internal AWS S3 buckets to customers' Google BigQuery tables following BigQuery Data Transfer Service documentation.
Using permanent credentials possesses security risks related to the data stored in AWS S3.
We would like to use AWS IAM Role temporary credentials, which require the support of a session token on the BiqQuery side to get authorized on the AWS side.
Is there a way that the BigQuery Data Transfer Servce can use AWS IAM roles or temporary credentials to authorise against AWS and transfer data?
We considered Omni framework (https://cloud.google.com/bigquery/docs/omni-aws-cross-cloud-transfer) to transfer data from S3 to BQ, however, we faced several concerns/limitations:
Omni framework targets data analysis use-case rather than data transfer from external services. This concerns us that the design of Omni framework may have drawbacks in relation to data transfer at high scale
Omni framework currently supports only AWS-US-EAST-1 region (we require support at least in AWS-US-WEST-2 and AWS-EU-CENTRAL-1 and corresponding Google regions). This is not backward compatible with current customers' setup to transfer data from internal S3 to customers' BQ.
Our current customers will need to signup for Omni service to properly migrate from the current transfer solution we use
We considered a workaround with exporting data from S3 through staging in GCS (i.e. S3 -> GCS -> BQ), but this will also require a lot of effort from both customers and our company's sides to migrate to the new solution.
Is there a way that the BigQuery Data Transfer Servce can use AWS IAM roles or temporary credentials to authorise against AWS and transfer data?
No unfortunately.
The official Google BigQuery Data Transfer Service only mentions AWS access keys all throughout the documentation:
The access key ID and secret access key are used to access the Amazon S3 data on your behalf. As a best practice, create a unique access key ID and secret access key specifically for Amazon S3 transfers to give minimal access to the BigQuery Data Transfer Service. For information on managing your access keys, see the AWS general reference documentation.
The irony of the Google documentation is that while it refers to best practices and links to the official AWS docs, it actually doesn't endorse best practices and ignores what AWS mention:
We recommend that you use temporary access keys over long term access keys, as mentioned in the previous section.
Important
Unless there is no other option, we strongly recommend that you don't create long-term access keys for your (root) user. If a malicious user gains access to your (root) user access keys, they can completely take over your account.
You have a few options:
hook into both sides manually (i.e. link up various SDKs and/or APIs)
find an alternative BigQuery-compatible service, which does as such
accept the risk of long-term access keys.
In conclusion, Google is at fault here of not following security best practices and you - as a consumer - will have to bear the risk.
In my application I want to users to be able to create an organization i.e. (OrgA) and then have users sign up under said organization using either an invite code or token. Users in OrgA should have access to an Amazon S3 directory (which stores images and files) and access to a database table that has been created for the said organization.
I could not find a solution on how to implement this online and was wondering if using Amazon Cognito groups was a good idea to meet requirements.
This is supported by the Amazon Cognito Service. That is, you can use Amazon Cognito to control permissions for different user groups in your app. This ensures that users have appropriate access to backend resources, determined by the group they belong to. For more information, see Building fine-grained authorization using Amazon Cognito User Pools groups.
I wanted to know if I can accomplish my aim of uploading files or large files from my mobile application to my s3 bucket without login into my amazon console. If there is no way to do this, then. How can I upload files once logged in without the annoying web view? Se what others tried.
Should I upload files to Amazon S3 from mobile devices or from my server? Thanks.
What I need to achieve:
Upload files to S3 with no hassles.
Download files when needed with no hassles.
Able to upload large files on S3.
I really don't want to mess with access key and authentication key. Well if I can at least hide them form onlookers that will be fine.
If you can share all the possible file upload implementation in the mobile application scenario that will be great thanks.
You can use SDK for AWS. It must be an application developed in some programming language available for AWS and be able to access the credentials to connect to S3 and work like an interface between the app and the S3 bucket.
All AWS services can be accessed programmatically via an API. In fact, this is the recommended method for using AWS services.
Mobile applications can certainly communicate directly with Amazon S3.
You will want to secure data stored in S3 so that only authorized and authenticated users can access it. This can be accomplished by having the mobile application authenticate with your own back-end service that handles authentication. Alternatively, you could use Amazon Cognito to handle authentication. This way, your app will handle authentication with AWS and users will never see AWS-specific credentials.
We are building a custom application (using LoopBack) that will need to store many large files coming from multiple users, so naturally we're looking at S3. We've done something similar before, with clients uploading files to the server which then processes and uploads them to S3 under one AWS account, but for this new app, we're looking to allow the clients (using a custom iOS app) to use the iOS S3 SDK to upload directly to their own bucket or folder. User accounts will be created on the server.
Is there any way to handle S3 authentication/authorization using custom code? For example, could the iOS client request a temporary token allowing them to upload to a specific S3 bucket or folder? Or would we need to create unique IAM users for each user in our system?
Is that a terrible idea? It sounds like a terrible idea. :)
I found a similar question here but there was no conclusive answer.
Update: I found this article on Temporary Security Credentials that looks very promising. It also suggests using Cognito, which I've never used, if building a mobile app.
Cognito is the way to go. You should definetly not create IAM users for this. IAM is for managing access to the aws services programatically or from the console. Moreover you would need to hardcode the IAM access keys in the ios app, which is not a best practice.
https://docs.aws.amazon.com/cognito/latest/developerguide/what-is-amazon-cognito.html
https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_oidc_cognito.html
If users of your application are already authenticated, you could generate a pre-signed S3 url on your backend using your credentials. This URL can then be returned to the application and used to upload a file.
It would circumvent having to create individual IAM users/permissions and/or managing bucket policies.
Check out the docs on it here.
Not sure how relevant to your situation.
You can create a role that allows upload to s3 and use SAML web-based identity to authenticate and allow privileges to assume the role and get temp credentials and token.
This will keep very limited time authenticated to S3 upload. ie until the temp credentials expire.
We currently have a webapp running in AWS Region Ireland (service for the UK) and are planing to expand the service into the US.
In order to be sure that the US users get a low latency experience we are considering mirroring the AWS resources used Ireland in the US.
The data for the US users should be stored in the US region, the UK data in Ireland. (There is no need to report across both regions).
We are thinking of building a centralised login services that runs in the Ireland region. After successful login the user will be redirected to the region where his data is stored. (The login service has to know in which region the data is stored)
Has anyone built something similar? Any recommendation how to approach this?
Would Amazon Cognito support such a setup (if we build the login service based on cognito)?
Currently Amazon Cognito does not support this out of the box.
But if you use Cognito User Pools with Federated identities the credentials you get can be used in any AWS region. Along with that you can store a custom attribute for the user defining the region to which that user belongs and then redirect them accordingly.