The best way to send file to GCS wihout user confirmation - google-cloud-platform

I am developing an application that needs to send files to Google Cloud Storage.
The webapp will have a HTML page that the user choose files to do upload.
The user do not have Google Account.
The amount files to send is 5 or less.
I do not want to send files to GAE and GAE send to GCS. I would like that my user to do upload directly to GCS.
I did this code for upload:
function sentStorage() {
var file = document.getElementById("myFile").files[0];
url = 'https://www.googleapis.com/upload/storage/v1/b/XXX/o?uploadType=resumable&name=' + file.name;
xhr = new XMLHttpRequest();
var token = 'ya29.XXXXXXXXXXXXXXX';
xhr.open('POST', url);
xhr.setRequestHeader('Content-Type', file.type);
// resumable
//url = 'https://www.googleapis.com/upload/storage/v1/b/XXXXXX/o?uploadType=resumable&name=' + file.name;
//xhr.setRequestHeader('Content-Type', 'application/json; charset=UTF-8');
//xhr.setRequestHeader('Content-Length', file.size);
xhr.setRequestHeader('x-goog-project-id', 'XXXXXXXXXX');
xhr.setRequestHeader('Authorization', 'Bearer ' + token);
xhr.send(file);
xhr.onreadystatechange = function () {
if (xhr.readyState === 4) {
var response = JSON.parse(xhr.responseText);
if (xhr.status === 200) {
alert('codigo 200');
} else {
var message = 'Error: ' + response.error.message;
console.log(message);
alert(message);
}
}
};
}
I get a serviceaccount information (Google Console) and generate a token Bearer for it. I used a python file that read the "json account information" and generate the token.
My requisit is that user do not need to confirm any Google Account information for send files, this obligation is from my application. (Users do not have Google Account) and the html page send the files directly to GCS without send to GAE or GCE, so, I need to use HTML form or Javascript. I prefer Javascript.
Only users of this application can do upload (the application has an authentication with database), so, anonymous user can not do it.
My questions are:
This token will expire? I used a serviceaccount for generate this token.
There is a better api javascript to do it?
This security solution is better or I should use a different approach?

Sending either a refresh or an access token to an untrusted end user is very dangerous. The bearer of an access token has complete authority to act as the associated account (within the scope used to generate it) until the access token expires a few minutes later. You don't want to do that.
There are a few good alternatives. The easiest way is to create exactly the upload request you want, then sign the URL for that request using the private key of a service account. That signed URL, which will be valid for a few minutes, could then be used to upload a single object. You'll need to sign the URL on the server side before giving it to the customer. Here's the documentation on signed URLs: https://cloud.google.com/storage/docs/access-control/signed-urls

Related

Is it possible to use passport.js auth token in order to limit access to S3 resources only to logged-in users?

This probably makes no sense.
I am using passport.js to authenticate users in the app using a JWT token.
This is how the token is generated:
jwt.sign(
payload,
keys.SecretKey,
{ expiresIn: 3600 * 24 * 356 },
(error, token) => {
res.json({
success: true,
token: "Bearer " + token,
});
}
);
And then in each http request, I add the token in the Authorization header.
I'm trying to limit access to AWS S3 images only to logged-in users.
One way to do this is to generated signed url each time a user tries to access an image like I am trying to do here which is not perfect (at least so far).
So I wonder, if I can do something, maybe use an S3 policy that would allow me to use the JWT token to give access to authenticated users access to S3 resources.

How to efficiently allow for users to view Amazon S3 content?

I am currently creating a basic app with React-Native (frontend) and Flask/MongoDB (backend). I am planning on using AWS S3 as cheap cloud storage for all the images and videos that are going to be uploaded and viewed. My current idea (and this could be totally off), is when a user uploads content, it will go through my Flask API and then to the S3 storage. When a user wants to view content, I am not sure what the plan of attack is here. Should I use my Flask API as a proxy, or is there a way to simply send a link to the content directly on S3 (which would avoid the extra traffic through my API)?
I am quite new to using AWS and if there is already a post discussing this topic, please let me know, and I'd be more than happy to take down this duplicate. I just can't seem to find anything.
Should I use my Flask API as a proxy, or is there a way to simply send a link to the content directly on S3 (which would avoid the extra traffic through my API)?
If the content is public, you just provide an URL which points directly to the file on the S3 bucket.
If the content is private, you generate presigned url on your backend for the file for which you want to give access. This URL should be valid for a short amount of time (for example: 15/30 minutes). You can regenerate it, if it becomes unavailable.
Moreover, you can generate a presigned URL which can be used for uploads directly from the front-end to the S3 bucket. This might be an option if you don't want the upload traffic to go through the backend or you want faster uploads.
There is an API boto3, try to use it.
It is not so difficult, I have done something similar, will post code here.
I have done like #Ervin said.
frontend asks backend to generate credentials
backend sends to frontend the credentials
Frontend upload file to S3
Frontend warns backend it has done.
Backend validate if everything is ok.
Backend will create a link to download, you have a lot of security options.
example of item 6) To generate a presigned url to download content.
bucket = app.config.get('BOTO3_BUCKET', None)
client = boto_flask.clients.get('s3')
params = {}
params['Bucket'] = bucket
params['Key'] = attachment_model.s3_filename
params['ResponseContentDisposition'] = 'attachment; filename={0}'.format(attachment_model.filename)
if attachment_model.mimetype is not None:
params['ResponseContentType'] = attachment_model.mimetype
url = client.generate_presigned_url('get_object', ExpiresIn=3600, Params=params)
example of item 2) Backend will create presigned credentials to post your file on S3, send s3_credentials to frontend
acl_permission = 'private' if private_attachment else 'public-read'
condition = [{'acl': acl_permission},
["starts-with", "$key", '{0}/'.format(folder_name)],
{'Content-Type': mimetype }]
bucket = app.config.get('BOTO3_BUCKET', None)
fields = {"acl": acl_permission, 'Bucket': bucket, 'Content-Type': mimetype}
client = boto_flask.clients.get('s3')
s3_credentials = client.generate_presigned_post(bucket, s3_filename, Fields=fields, Conditions=condition, ExpiresIn=3600)
example of item 5) Here are an example how backend can check if file on S3 are ok.
bucket = app.config.get('BOTO3_BUCKET', None)
client = boto_flask.clients.get('s3')
response = client.head_object(Bucket=bucket, Key=s3_filename)
if response is None:
return None, None
md5 = response.get('ETag').replace('"', '')
size = response.get('ContentLength')
Here are an example how frontend will ask for credentials, upload file to S3 and inform backend it is done.
I tried to remove a lot of particular code.
//frontend asking backend to create credentials, frontend will send some file metadata
AttachmentService.createPostUrl(payload).then((responseCredentials) => {
let form = new FormData();
Object.keys(responseCredentials.s3.fields).forEach(key => {
form.append(key, responseCredentials.s3.fields[key]);
});
form.append("file", file);
let payload = {
data: form,
url: responseCredentials.s3.url
}
//Frontend will send file to S3
axios.post(payload.url, payload.data).then((res) => {
return Promise.resolve(true);
}).then((result) => {
//when it is done, frontend informs backend
AttachmentService.uploadSuccess(...).then((refreshCase) => {
//Success
});
});
});

How does one keep a user logged in with Django Rest Framework?

I'm new to Django coming from the Firebase world, where authentication and keeping a user logged in is super easy.
In learning Django (Rest Framework) I came to find out that you can log in a user, get a token and save the token in Cookies to reuse is next time that same user goes into the website. Is this the best way to keep a user logged in?
So far, I can log a user in, get their token and some additional info, but I'm not sure how to prevent this from happening over and over again. I'd like to know how to keep the user logged in.
Also, whenever the user gets back on the browser, do I place a POST request to get their own information (if needed to display on the screen)? Always?
I'm very confused as to how authentication/logging in works.
An usual way to handle this problem is to use Jwt auth.
You will issue a short lived token alongside a long lived refresh token to your consumer.
https://github.com/jpadilla/django-rest-framework-jwt
On your frontend side you can implement an automatic refresh mechanism when the token expire.
Example with React: https://medium.com/#monkov/react-using-axios-interceptor-for-token-refreshing-1477a4d5fc26
On browser side, it's depend. For example with single page app, you can fetch info only one and store them in a store.
For multi page app, you could still use cookie or local storage to persist data.
Each Request is anonymous/new, even after you login.
Why I keep loggedin?
Once you logged in, server usually reuturn a token and save it in your local browser. Next time you send request, you can add the token in your request. Then server will know it is still the same user.
What is token?
There a many kinds of token: session token, jwt, basic token...
Token is a string of your identity, and jwt(JSON Web Tokens) is one of the most popular authentication(CORS) solution. This is how original jwt looks like:
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c
And this is how actual jwt data looks like:
# HEADER:ALGORITHM
{
"alg": "HS256",
"typ": "JWT"
}
# PAYLOAD:DATA
{
"sub": "1234567890",
"name": "John Doe",
"iat": 1516239022
}
# VERIFY SIGNATURE
HMACSHA256(
base64UrlEncode(header) + "." +
base64UrlEncode(payload),
your-256-bit-secret
)
Where do I keep token
Token is storage in your local cookies. For each request, you can get token from local cookies and add to request head. For example, in React:
import Cookies from "universal-cookie"
localCookies = new Cookies();
...
..
fetch(`/api/logs/:id`, {
headers: {
'Content-Type': 'application/json',
'Authorization': 'JWT ' + localCookies.get('token'),
}
})
...
..

How to get Facebook page feed and Filter its fields as Json using Google App script

I am trying to get a Facebook page feed through Google app script.
As of now I tried different scripts but I am getting only app token with the request and if i change it with a usertoken from graph api I got messages but no images and titles
How to get the user token and get the correct fields for as a json ,
var url = 'https://graph.facebook.com'
+ '/love.to.traavel/feed'
+ '?access_token='+ encodeURIComponent(getToken());
// + '?access_token=' + service.getAccessToken();
var response = UrlFetchApp.fetch(url, {'muteHttpExceptions': true});
var json = response.getContentText();
var jsondata = JSON.parse(json);
Logger.log(jsondata); //check this and adjust following for loop and ht
var posts = {};
for (var i in jsondata) {
posts[i] = {"post":jsondata[i].message};
}
return posts;
You should use a Page Token, not a User Token
You need to ask for fields you want to get, with the fields parameter: https://graph.facebook.com/love.to.traavel/feed?fields=field1,field2,...&access_token=xxx
You get a user token by authorizing your App: https://developers.facebook.com/docs/facebook-login/platforms
Be aware that extended user tokens are valid for 60 days only, so you have to refresh it once in a while. There is no user token that is valid forever. You cannot authorize through code only, it needs user interaction. The easiest way is to just generate a user token by selecting your App in the API Explorer and authorize it, like you did already. Then hardcode it in the script code.
Alternatively, you can try implementing this with the manual login flow, check out the docs for that. You can try adding the functionality using this for a custom interface where you go through the login process: https://developers.google.com/apps-script/guides/html/
Since you donĀ“t own the page, you should read this too: https://developers.facebook.com/docs/apps/review/feature/#reference-PAGES_ACCESS

Identity Server 3 Facebook Login Get Email

Identity server is implemented and working well. Google login is working and is returning several claims including email.
Facebook login is working, and my app is live and requests email permissions when a new user logs in.
The problem is that I can't get the email back from the oauth endpoint and I can't seem to find the access_token to manually request user information. All I have is a "code" returned from the facebook login endpoint.
Here's the IdentityServer setup.
var fb = new FacebookAuthenticationOptions
{
AuthenticationType = "Facebook",
SignInAsAuthenticationType = signInAsType,
AppId = ConfigurationManager.AppSettings["Facebook:AppId"],
AppSecret = ConfigurationManager.AppSettings["Facebook:AppSecret"]
};
fb.Scope.Add("email");
app.UseFacebookAuthentication(fb);
Then of course I've customized the AuthenticateLocalAsync method, but the claims I'm receiving only include name. No email claim.
Digging through the source code for identity server, I realized that there are some claims things happening to transform facebook claims, so I extended that class to debug into it and see if it was stripping out any claims, which it's not.
I also watched the http calls with fiddler, and I only see the following (apologies as code formatting doesn't work very good on urls. I tried to format the querystring params one their own lines but it didn't take)
(facebook.com)
/dialog/oauth
?response_type=code
&client_id=xxx
&redirect_uri=https%3A%2F%2Fidentity.[site].com%2Fid%2Fsignin-facebook
&scope=email
&state=xxx
(facebook.com)
/login.php
?skip_api_login=1
&api_key=xxx
&signed_next=1
&next=https%3A%2F%2Fwww.facebook.com%2Fv2.7%2Fdialog%2Foauth%3Fredirect_uri%3Dhttps%253A%252F%252Fidentity.[site].com%252Fid%252Fsignin-facebook%26state%3Dxxx%26scope%3Demail%26response_type%3Dcode%26client_id%3Dxxx%26ret%3Dlogin%26logger_id%3Dxxx&cancel_url=https%3A%2F%2Fidentity.[site].com%2Fid%2Fsignin-facebook%3Ferror%3Daccess_denied%26error_code%3D200%26error_description%3DPermissions%2Berror%26error_reason%3Duser_denied%26state%3Dxxx%23_%3D_
&display=page
&locale=en_US
&logger_id=xxx
(facebook.com)
POST /cookie/consent/?pv=1&dpr=1 HTTP/1.1
(facebook.com)
/login.php
?login_attempt=1
&next=https%3A%2F%2Fwww.facebook.com%2Fv2.7%2Fdialog%2Foauth%3Fredirect_uri%3Dhttps%253A%252F%252Fidentity.[site].com%252Fid%252Fsignin-facebook%26state%3Dxxx%26scope%3Demail%26response_type%3Dcode%26client_id%3Dxxx%26ret%3Dlogin%26logger_id%3Dxxx
&lwv=100
(facebook.com)
/v2.7/dialog/oauth
?redirect_uri=https%3A%2F%2Fidentity.[site].com%2Fid%2Fsignin-facebook
&state=xxx
&scope=email
&response_type=code
&client_id=xxx
&ret=login
&logger_id=xxx
&hash=xxx
(identity server)
/id/signin-facebook
?code=xxx
&state=xxx
I saw the code parameter on that last call and thought that maybe I could use the code there to get the access_token from the facebook API https://developers.facebook.com/docs/facebook-login/manually-build-a-login-flow
However when I tried that I get a message from the API telling me the code has already been used.
I also tried to change the UserInformationEndpoint to the FacebookAuthenticationOptions to force it to ask for the email by appending ?fields=email to the end of the default endpoint location, but that causes identity server to spit out the error "There was an error logging into the external provider. The error message is: access_denied".
I might be able to fix this all if I can change the middleware to send the request with response_type=id_token but I can't figure out how to do that or how to extract that access token when it gets returned in the first place to be able to use the Facebook C# sdk.
So I guess any help or direction at all would be awesome. I've spent countless hours researching and trying to solve the problem. All I need to do is get the email address of the logged-in user via IdentityServer3. Doesn't sound so hard and yet I'm stuck.
I finally figured this out. The answer has something to do with Mitra's comments although neither of those answers quite seemed to fit the bill, so I'm putting another one here. First, you need to request the access_token, not code (authorization code) from Facebook's Authentication endpoint. To do that, set it up like this
var fb = new FacebookAuthenticationOptions
{
AuthenticationType = "Facebook",
SignInAsAuthenticationType = signInAsType,
AppId = ConfigurationManager.AppSettings["Facebook:AppId"],
AppSecret = ConfigurationManager.AppSettings["Facebook:AppSecret"],
Provider = new FacebookAuthenticationProvider()
{
OnAuthenticated = (context) =>
{
context.Identity.AddClaim(new System.Security.Claims.Claim("urn:facebook:access_token", context.AccessToken, ClaimValueTypes.String, "Facebook"));
return Task.FromResult(0);
}
}
};
fb.Scope.Add("email");
app.UseFacebookAuthentication(fb);
Then, you need to catch the response once it's logged in. I'm using the following file from the IdentityServer3 Samples Repository, which overrides (read, provides functionality) for the methods necessary to log a user in from external sites. From this response, I'm using the C# Facebook SDK with the newly returned access_token claim in the ExternalAuthenticationContext to request the fields I need and add them to the list of claims. Then I can use that information to create/log in the user.
public override async Task AuthenticateExternalAsync(ExternalAuthenticationContext ctx)
{
var externalUser = ctx.ExternalIdentity;
var claimsList = ctx.ExternalIdentity.Claims.ToList();
if (externalUser.Provider == "Facebook")
{
var extraClaims = GetAdditionalFacebookClaims(externalUser.Claims.First(claim => claim.Type == "urn:facebook:access_token"));
claimsList.Add(new Claim("email", extraClaims.First(k => k.Key == "email").Value.ToString()));
claimsList.Add(new Claim("given_name", extraClaims.First(k => k.Key == "first_name").Value.ToString()));
claimsList.Add(new Claim("family_name", extraClaims.First(k => k.Key == "last_name").Value.ToString()));
}
if (externalUser == null)
{
throw new ArgumentNullException("externalUser");
}
var user = await userManager.FindAsync(new Microsoft.AspNet.Identity.UserLoginInfo(externalUser.Provider, externalUser.ProviderId));
if (user == null)
{
ctx.AuthenticateResult = await ProcessNewExternalAccountAsync(externalUser.Provider, externalUser.ProviderId, claimsList);
}
else
{
ctx.AuthenticateResult = await ProcessExistingExternalAccountAsync(user.Id, externalUser.Provider, externalUser.ProviderId, claimsList);
}
}
And that's it! If you have any suggestions for simplifying this process, please let me know. I was going to modify this code to do perform the call to the API from FacebookAuthenticationOptions, but the Events property no longer exists apparently.
Edit: the GetAdditionalFacebookClaims method is simply a method that creates a new FacebookClient given the access token that was pulled out and queries the Facebook API for the other user claims you need. For example, my method looks like this:
protected static JsonObject GetAdditionalFacebookClaims(Claim accessToken)
{
var fb = new FacebookClient(accessToken.Value);
return fb.Get("me", new {fields = new[] {"email", "first_name", "last_name"}}) as JsonObject;
}