How do I provide authenticated access to my AWS S3 bucket? - amazon-web-services

I started a bare Expo app with expo init called MyVideoApp. Then I created an AWS account and in the terminal ran:
npm install -g #aws-amplify/cli
amplify configure
This signed me into the console, I went through the default steps and created an account in region:eu-west-2, username:amplify-user, pasted in the accessKeyId & secretAccessKey, profile name:amplify-user-profile.
cd ~/Documents/MyVideoApp/ & amplify init
? Enter a name for the project MyVideoApp
? Enter a name for the environment dev
? Choose your default editor: IntelliJ IDEA
? Choose the type of app that you're building javascript
Please tell us about your project
? What javascript framework are you using react-native
? Source Directory Path: /
? Distribution Directory Path: /
? Build Command: npm run-script build
? Start Command: npm run-script start
Using default provider awscloudformation
? Do you want to use an AWS profile? Yes
? Please choose the profile you want to use amplify-user-profile
Adding backend environment dev to AWS Amplify Console app: d37chh30hholq6
amplify push
At this point I had an amplify folder in my project directory and an S3 bucket called amplify-myvideoapp-dev-50540-deployment. I uploaded an image into the bucket icon_1.png. And tried to download it from the app via a button click.
import React from 'react';
import { StyleSheet, Text, View, SafeAreaView, Button } from 'react-native';
import Amplify, { Storage } from 'aws-amplify';
import awsmobile from "./aws-exports";
Amplify.configure(awsmobile);
async function getImage() {
try {
let data = await Storage.get('icon_1.jpg')
} catch (err) {
console.log(err)
}
}
export default function App() {
return (
<SafeAreaView style={styles.container}>
<Text>Hello, World!</Text>
<Button title={"Click to Download!"} onPress={getImage}/>
</SafeAreaView>
);
}
const styles = StyleSheet.create({
container: {
flex: 1,
justifyContent: 'center',
alignItems: 'center',
},
});
Output:
No credentials
[WARN] 18:54.93 AWSS3Provider - ensure credentials error, No Cognito Identity pool provided for unauthenticated access
...
So I setup (but maybe not correctly?) a user pool (my_first_pool) and an identity pool (myvidapp). This didn't help. Furthermore when I go into my bucket and click Permissions -> Bucket Policy, it's just empty ... not sure if that's okay if only owner is trying to access the bucket & it's contents.
I don't know what's wrong and what else to try. I essentially just want to authenticate my backend so anyone who git clones this code would just be able to run it and access the bucket.
Edit: aws-exports.js
/* eslint-disable */
// WARNING: DO NOT EDIT. This file is automatically generated by AWS Amplify. It will be overwritten.
const awsmobile = {
"aws_project_region": "eu-west-2"
};
export default awsmobile;

Since you've indicated that you're okay with all of the files in the S3 bucket being publicly accessible, I would suggest the following:
Select the bucket from in the AWS console (console.aws.amazon.com)
Under "Permissions" select "Block Public Access" and edit the settings by un-checking all of the options under and including "Block all public access", then save and confirm.
Go to the bucket policy, and paste in the following (Note: replace "YOUR_BUCKET_NAME_HERE" with "amplify-myvideoapp-dev-50540-deployment" first):
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicRead",
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:GetObject",
"s3:GetObjectVersion"
],
"Resource": [
"arn:aws:s3:::[YOUR_BUCKET_NAME_HERE]/*"
]
}
]
}

Related

I wan't to create presigned url post, but always failed

thanks for greate packages!
I have problem when i create development with localstack using S3 service to create presignedurl post.
I have run localstack with SERVICES=s3 DEBUG=1 S3_SKIP_SIGNATURE_VALIDATION=1 localstack start
I have settings AWS_ACCESS_KEY_ID=test AWS_SECRET_ACCESS_KEY=test AWS_DEFAULT_REGION=us-east-1 AWS_ENDPOINT_URL=http://localhost:4566 S3_Bucket=my-bucket
I make sure have the bucket
> awslocal s3api list-buckets
{
"Buckets": [
{
"Name": "my-bucket",
"CreationDate": "2021-11-16T08:43:23+00:00"
}
],
"Owner": {
"DisplayName": "webfile",
"ID": "bcaf1ffd86f41161ca5fb16fd081034f"
}
}
I try create presigned url, and running in console with this
s3_client_sync.create_presigned_post(bucket_name=settings.S3_Bucket, object_name="application/test.png", fields={"Content-Type": "image/png"}, conditions=[["Expires", 3600]])
and have return like this
{'url': 'http://localhost:4566/kredivo-thailand',
'fields': {'Content-Type': 'image/png',
'key': 'application/test.png',
'AWSAccessKeyId': 'test',
'policy': 'eyJleHBpcmF0aW9uIjogIjIwMjEtMTEtMTZUMTE6Mzk6MjNaIiwgImNvbmRpdGlvbnMiOiBbWyJFeHBpcmVzIiwgMzYwMF0sIHsiYnVja2V0IjogImtyZWRpdm8tdGhhaWxhbmQifSwgeyJrZXkiOiAiYXBwbGljYXRpb24vdGVzdC5wbmcifV19',
'signature': 'LfFelidjG+aaTOMxHL3fRPCw/xM='}}
And i test using insomnia
and i have read log in localstack
2021-11-16T10:54:04:DEBUG:localstack.services.s3.s3_utils: Received presign S3 URL: http://localhost:4566/my-bucket/application/test.png?AWSAccessKeyId=test&Policy=eyJleHBpcmF0aW9uIjogIjIwMjEtMTEtMTZUMTE6Mzk6MjNaIiwgImNvbmRpdGlvbnMiOiBbWyJFeHBpcmVzIiwgMzYwMF0sIHsiYnVja2V0IjogImtyZWRpdm8tdGhhaWxhbmQifSwgeyJrZXkiOiAiYXBwbGljYXRpb24vdGVzdC5wbmcifV19&Signature=LfFelidjG%2BaaTOMxHL3fRPCw%2FxM%3D&Expires=3600
2021-11-16T10:54:04:WARNING:localstack.services.s3.s3_utils: Signatures do not match, but not raising an error, as S3_SKIP_SIGNATURE_VALIDATION=1
2021-11-16T10:54:04:INFO:localstack.services.s3.s3_utils: Presign signature calculation failed: <Response [403]>
what i missing, so i cannot create the presignedurl post ?
The problem is with your AWS configuration -
AWS_ACCESS_KEY_ID=test // Should be an Actual access Key for the IAM user
AWS_SECRET_ACCESS_KEY=test // Should be an Actual Secret Key for the IAM user
AWS_DEFAULT_REGION=us-east-1
AWS_ENDPOINT_URL=http://localhost:4566 // Endpoint seems wrong
S3_Bucket=my-bucket // Actual Bucket Name in AWS S3 console
For more information, try to read here and setup your environment with correct AWS credentials - Setup AWS Credentials

Amplify S3 Image - 404 Not Found

I am trying to access an image that I have uploaded to my S3 bucket. I created my bucket using the Amplify CLI (amplify add storage) and granted access to all of my cognito groups. I have also granted my AuthRole AmazonS3FullAccess. My Bucket is set to allow all public access as well.
I have tried all the different ways I can find online to access this image and the only way that works so far is to leave it open to the public and use the image url directly. But even if I use the public method of accessing the image using Amplify's tools, I get the 404 error. Below is my code, am I doing something wrong with the url generation?
resources:
https://docs.amplify.aws/ui/storage/s3-image/q/framework/react
https://docs.amplify.aws/lib/storage/getting-started/q/platform/js#using-amazon-s3
import React, { Component} from 'react'
import Amplify, { Auth, Storage } from 'aws-amplify';
import { AmplifyS3Image} from "#aws-amplify/ui-react";
import { Card } from 'reactstrap';
// FYI, this all matches my aws-exports and matches what I see online in the console
Amplify.configure({
Auth: {
identityPoolId: 'us-east-1:XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX', //REQUIRED - Amazon Cognito Identity Pool ID
region: 'us-east-1', // REQUIRED - Amazon Cognito Region
userPoolId: 'us-east-1_XXXXXXXXX', //OPTIONAL - Amazon Cognito User Pool ID
userPoolWebClientId: 'XXXXXXXXXXXXXXXXX', //OPTIONAL - Amazon Cognito Web Client ID
},
Storage: {
AWSS3: {
bucket: 'xxxxxxxxx-storage123456-prod', //REQUIRED - Amazon S3 bucket name
region: 'us-east-1', //OPTIONAL - Amazon service region
}
}
});
class TestPage extends Component {
constructor(props) {
super(props);
this.state = { }
};
async componentDidMount() {
const user = await Auth.currentAuthenticatedUser();
const deviceKey = user.signInUserSession.accessToken.payload['device_key']
console.log( deviceKey, user );
const storageGetPicUrl = await Storage.get('test_public.png', {
level: 'protected',
bucket: 'xxxxxxxxx-storage123456-prod',
region: 'us-east-1',
});
console.log(storageGetPicUrl);
this.setState({
user,
deviceKey,
profilePicImg: <img height="40px" src={'https://xxxxxxxxx-storage123456-prod.s3.amazonaws.com/test_public.png'} />,
profilePicPrivate: <AmplifyS3Image imgKey={"test_default.png"} />,
profilePicPublic: <AmplifyS3Image imgKey={"test_public.png"} />,
profilePicPrivate2: <AmplifyS3Image imgKey={"test_default.png"} level="protected" identityId={deviceKey} />,
profilePicPublic2: <AmplifyS3Image imgKey={"test_public.png"} level="protected" identityId={deviceKey} />,
profilePicStorage: <img src={storageGetPicUrl} />,
});
};
render() {
return (
<table>
<tbody>
<tr><td><Card>{this.state.profilePicImg}</Card></td></tr>
<tr><td><Card>{this.state.profilePicPrivate}</Card></td></tr>
<tr><td><Card>{this.state.profilePicPublic}</Card></td></tr>
<tr><td><Card>{this.state.profilePicPrivate2}</Card></td></tr>
<tr><td><Card>{this.state.profilePicPublic2}</Card></td></tr>
<tr><td><Card>{this.state.profilePicStorage}</Card></td></tr>
</tbody>
</table>
);
};
};
export default TestPage;
Okay, I've got it figured out! There were 2 problems. One, AWS storage requires you to organize your folder structure in the bucket a certain way for access. Two, I had to update my bucket policy to point at my AuthRole.
When you configure your storage bucket, Amplify CLI will setup your S3 bucket with access permission in such a way that contents in 'public' folder can be accessed by everyone in who's logged into your app. 'private' for user specific contents,' protected' for user specific and can be accessed by other users in the platform. SOURCE
The bucket policy itself needs to be updated to give authentication to your AuthRole which you are using with your webpage login. For me this was the AuthRole that my Cognito users are linked to. This link helped me set the Actions in my policy, but I think it's an old policy format. This link helped me with getting the policy right.
My image is located at: public/test.png within my bucket. The folder name 'public' is necessary to match up with the level specified in the Storage call below. I tested this by setting all permissions Block Public Access. I ran my code without the change to my policy and the images would not load, so they were definitely blocked. After updating the policy, the images loaded perfectly.
Simplified version of the parts of my code that matter:
import { Storage } from 'aws-amplify';
// image name should be relative to the public folder
// example: public/images/test.png => Storage.get('images/test.png' ...
const picUrl= await Storage.get('test.png', {
level: 'public',
bucket: 'bucket-name',
region: 'us-east-1',
});
const img = <img width="40px" name="test" src={picUrl} alt="testImg" />
My bucket polcy:
{
"Version": "2012-10-17",
"Id": "Policy1234",
"Statement": [
{
"Sid": "AllowReadWriteObjects",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::the-rest-of-my-authrole-arn"
},
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::bucket-name/*"
]
}
]
}

gulp-awspublish with AWS profile instead of AWS_ACCESS_KEY and secret

I am trying to deploy my nuxt static website to S3 using this guide.
https://nuxtjs.org/faq/deployment-aws-s3-cloudfront
The deployscript works when using which I tried an a personal AWS account:
AWS_ACCESS_KEY_ID="key"
AWS_SECRET_ACCESS_KEY="secret"
It does not work when unsetting these exports and using the AWS_PROFILE export on a separate AWS account. On this AWS I am not able to get an access key and secret because of company policy.
I also use these AWS profiles for other things so I am sure they are configured properly.
The error I am getting in the console is:
Error: Connect EHOSTUNREACH <EC2 IP address???>
The part in brackets is the IP address I am seeing. Which is weird where it tries to connect to EC2 since the script works on S2 and cloudfront.
The script I am using
#!/bin/bash
export AWS_PROFILE="profile_name"
export AWS_BUCKET_NAME="example.com"
export AWS_CLOUDFRONT="UPPERCASE"
# Load nvm (node version manager), install node (version in .nvmrc), and npm install packages
[ -s "$HOME/.nvm/nvm.sh" ] && source "$HOME/.nvm/nvm.sh" && nvm use
# Npm install if not already.
[ ! -d "node_modules" ] && npm install
npm run generate
gulp deploy
As for the gulpfile:
const gulp = require('gulp')
const awspublish = require('gulp-awspublish')
const cloudfront = require('gulp-cloudfront-invalidate-aws-publish')
const parallelize = require('concurrent-transform')
// https://docs.aws.amazon.com/cli/latest/userguide/cli-environment.html
const config = {
// Required
params: {
Bucket: process.env.AWS_BUCKET_NAME
},
credentials: {
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
signatureVersion: 'v3'
},
// Optional
deleteOldVersions: false, // NOT FOR PRODUCTION
distribution: process.env.AWS_CLOUDFRONT, // CloudFront distribution ID
region: process.env.AWS_DEFAULT_REGION,
headers: {
/* 'Cache-Control': 'max-age=315360000, no-transform, public', */
},
// Sensible Defaults - gitignore these Files and Dirs
distDir: 'dist',
indexRootPath: true,
cacheFileName: '.awspublish',
concurrentUploads: 10,
wait: true // wait for CloudFront invalidation to complete (about 30-60 seconds)
}
gulp.task('deploy', function () {
// create a new publisher using S3 options
// http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#constructor-property
const publisher = awspublish.create(config)
let g = gulp.src('./' + config.distDir + '/**')
// publisher will add Content-Length, Content-Type and headers specified above
// If not specified it will set x-amz-acl to public-read by default
g = g.pipe(
parallelize(publisher.publish(config.headers), config.concurrentUploads)
)
// Invalidate CDN
if (config.distribution) {
console.log('Configured with CloudFront distribution')
g = g.pipe(cloudfront(config))
} else {
console.log(
'No CloudFront distribution configured - skipping CDN invalidation'
)
}
// Delete removed files
if (config.deleteOldVersions) {
g = g.pipe(publisher.sync())
}
// create a cache file to speed up consecutive uploads
g = g.pipe(publisher.cache())
// print upload updates to console
g = g.pipe(awspublish.reporter())
return g
})
The gulp-awspublish docs mention it should be possible to connect with an AWS profile by adding it to the export (which I do in my deploy file).
They also mention using the aws js sdk which I also tried by integrating following snippet.
var AWS = require("aws-sdk");
var publisher = awspublish.create({
region: "your-region-id",
params: {
Bucket: "..."
},
credentials: new AWS.SharedIniFileCredentials({ profile: "myprofile" })
});
When I use the export aws_profile it does at least seam to authenticate. When using the SDK I receive an error mentioning
CredentialsError: Missing Credentials in config, if using
AWS_CONFIG_FILE, set AWS_SDK_LOAD_CONFIG=1
Where adding the latter (AWS_SDK_LOAD_CONFIG=1) to my deployment script does not make any difference.
Any Idea if I a missing something in the script to make it work?
My user policies where set as mentioned in the tutorial. Maybe they forgot something?
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:ListBucket"],
"Resource": ["arn:aws:s3:::example.com"]
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:PutObjectAcl",
"s3:GetObject",
"s3:GetObjectAcl",
"s3:DeleteObject",
"s3:ListMultipartUploadParts",
"s3:AbortMultipartUpload"
],
"Resource": ["arn:aws:s3:::example.com/*"]
},
{
"Effect": "Allow",
"Action": [
"cloudfront:CreateInvalidation",
"cloudfront:GetInvalidation",
"cloudfront:ListInvalidations",
"cloudfront:UnknownOperation"
],
"Resource": "*"
}
]
}
Since awspublish uses the javascript sdk I needed to export AWS_SDK_LOAD_CONFIG=true which solved the issue!

How to get/generate aws quicksight secure dashboard url

I want to embed Quicksight dashboard to an application. I have gone through the AWS quicksight documents, I did not get where I will find secure signed dashboard url.
In order to generate Quicksight secure dashboard url, follow the below steps:
Step 1: Create a new Identity Pool. Go to https://console.aws.amazon.com/cognito/home?region=us-east-1 , click ‘Create new Identity Pool’
Give an appropriate name.
Go to the Authentication Providers section, select Cognito.
Give the User Pool ID(your User pool ID) and App Client ID (go to App Clients in userpool and copy id).
Click ‘Create Pool’. Then click ‘Allow’ to create roles of the identity pool in IAM.
Step 2: Assign Custom policy to the Identity Pool Role
Create a custom policy with the below JSON.
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "quicksight:RegisterUser",
"Resource": "*",
"Effect": "Allow"
},
{
"Action": "quicksight:GetDashboardEmbedUrl",
"Resource": "*",
"Effect": "Allow"
},
{
"Action": "sts:AssumeRole",
"Resource": "*",
"Effect": "Allow"
}
]
}
Note: if you want to restrict the user to only one dashboard, replace the * with the dashboard ARN name in quicksight:GetDashboardEmbedUrl,
then goto the roles in IAM.
select the IAM role of the Identity pool and assign custom policy to the role.
Step 3: Configuration for generating the temporary IAM(STS) user
Login to your application with the user credentials.
For creating temporary IAM user, we use Cognito credentials.
When user logs in, Cognito generates 3 token IDs - IDToken, AccessToken, RefreshToken. These tokens will be sent to your application server.
For creating a temporary IAM user, we use Cognito Access Token and credentials will look like below.
AWS.config.region = 'us-east-1';
AWS.config.credentials = new AWS.CognitoIdentityCredentials({
IdentityPoolId:"Identity pool ID",
Logins: {
'cognito-idp.us-east-1.amazonaws.com/UserPoolID': AccessToken
}
});
For generating temporary IAM credentials, we call sts.assume role method with the below parameters.
var params = {
RoleArn: "Cognito Identity role arn",
RoleSessionName: "Session name"
};
sts.assumeRole(params, function (err, data) {
if (err) console.log( err, err.stack); // an error occurred
else {
console.log(data);
})
You can add additional parameters like duration (in seconds) for the user.
Now, we will get the AccessKeyId, SecretAccessKey and Session Token of the temporary user.
Step 4: Register the User in Quicksight
With the help of same Cognito credentials used in the Step 3, we will register the user in quicksight by using the quicksight.registerUser method with the below parameters
var params = {
AwsAccountId: “account id”,
Email: 'email',
IdentityType: 'IAM' ,
Namespace: 'default',
UserRole: ADMIN | AUTHOR | READER | RESTRICTED_AUTHOR | RESTRICTED_READER,
IamArn: 'Cognito Identity role arn',
SessionName: 'session name given in the assume role creation',
};
quicksight.registerUser(params, function (err, data1) {
if (err) console.log("err register user”); // an error occurred
else {
// console.log("Register User1”);
}
})
Now the user will be registered in quicksight.
Step5: Update AWS configuration with New credentials.
Below code shows how to configure the AWS.config() with new credentials generated Step 3.
AWS.config.update({
accessKeyId: AccessToken,
secretAccessKey: SecretAccessKey ,
sessionToken: SessionToken,
"region": Region
});
Step6: Generate the EmbedURL for Dashboards:
By using the credentials generated in Step 3, we will call the quicksight.getDashboardEmbedUrl with the below parameters
var params = {
AwsAccountId: "account ID",
DashboardId: "dashboard Id",
IdentityType: "IAM",
ResetDisabled: true,
SessionLifetimeInMinutes: between 15 to 600 minutes,
UndoRedoDisabled: True | False
}
quicksight.getDashboardEmbedUrl(params,
function (err, data) {
if (!err) {
console.log(data);
} else {
console.log(err);
}
});
Now, we will get the embed url for the dashboard.
Call the QuickSightEmbedding.embedDashboard from front end with the help of the above generated url.
The result will be the dashboard embedded in your application with filter controls.
this link will give you what you need from aws cli https://aws.amazon.com/blogs/big-data/embed-interactive-dashboards-in-your-application-with-amazon-quicksight/
this is the step 3 aws cli cmd to give you embeded URL ( i was able to excecute)
aws quicksight get-dashboard-embed-url --aws-account-id (your account ID) --dashboard-id (your dashgboard ID) --identity-type IAM
there are many other dependence to enable the embeded dashboard per aws dcouments. i have not able to successfully doen that. GL and let me know if you make it happen!
PHP implementation
(in addition to Siva Sumanth's answer)
https://gist.github.com/evgalak/d0d1adf099e2d7bff741c16a89bf30ba

AWS Amplify AppSync IAM 401

I'm getting GraphQLError: Request failed with status code 401
I followed the automatic configuration instructions from:
https://aws.github.io/aws-amplify/media/api_guide#automated-configuration-with-cli
I tried looking, but there are a lack of resources for IAM. It looks like everything should be setup automatically, and done with the Amplify CLI after I put in the IAM access key and secret.
Is further setup required? Here is my code:
import Amplify, { API, graphqlOperation, Hub } from "aws-amplify";
import aws_config from "../../aws-exports";
Amplify.configure(aws_config);
const ListKeywords = `query ListKeywords {
listKeyword {
keyword {
id
name
}
}
}`;
const loop = async () => {
const allKeywords = await API.graphql(graphqlOperation(ListKeywords));
}
Could it also be because my GraphQL resolvers are not setup yet for ListKeywords?
If you're using IAM as the Authorization type on your AppSync API then the issue is the Cognito Role being used with the Auth category when invoking Amplify.configure() isn't granted permissions for GraphQL operations. It needs something like this attached:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"appsync:GraphQL"
],
"Resource": [
"arn:aws:appsync:us-west-2:123456789012:apis/YourGraphQLApiId/*"
]
}
]
}
More details here: https://docs.aws.amazon.com/appsync/latest/devguide/security.html
Not sure if this helps but I've been struggling with this for a while and found that if I add the API and use IAM as the auth method I need to add 'auth' to the schema too.
See below:
type TimeLapseCamera #model
#auth(rules: [
{ allow: private, provider: iam }
])
{
...
}
I just tested this and my web page is successfully adding a record.
Note to other comment; I do not have AWS at all in this - its a simple VUE app with Amplify.
I just changed ~/.aws/credentials and now it's working.
Looks like even if you have project specific configuration via Amplify's command line tools or ~/.awsmobile/aws-config.js, it still relies on ~/.aws