Need to list all blobs inside container using sas token .Please guide how to have rest endpoint format.
I read we can get any blob data , blobname?sas token but I want to fetch all blobs under container.
Related
I want to collect all the Facebook link from Facebook search function. What I'm using here is the Facebook graph API tool.
If I want to query the data for a keyword, for example: "limited";
I will use the URL:
https://graph.facebook.com/v11.0/search?q=limited&access_token={my_access_token}
My image
However, I got the result as in the image. When I tried to create a workplace app and tried to get the access token of that app, I got the message as in the second image.
Error message when getting the access token of a workplace app
What I need now is an access token that has the permission to retrieve the data from the Facebook search. After that, I can use python to handle the rest.
Please help me out, thank you very much!
Having issue with http transformation simple post with bearer token.
So, we have a mapping that uses http transformation to SIMPLE POST JSON data into an api. The api uses bearer token authorization. So you send a user/pass and get a token back and then POST your json data with the token in the header.
But when we are trying to POST along with token, we are facing issues. Basically i am not sure how to post json data with token in the header info.
Anyone have any ideas ?
It worked with below approach for Barer token.
Flow:
SQ->exp->HTTP->exp->HTTP->TGT
HTTP connection:
Base URL : Provide the login URL for the API
Authentication : Basic
First HTTP transformation:
Give the login URL in Base URl. Pass a single row as an input for trigger of URL. This should give the generated token as an output (JSON/XML).
Pass the output to expression transf. Add a new output port and extract the token value only.
output port value : 'Bearer '||substr(ouput,2,30)
Methods - POST
Second HTTP transformation:
Add new input port and connect the above expression output as input.
Add new Header port as "Authorization".
Provide the API URL in the base url.
Methods - POST
Connect the output port to Flat file to check the received response.
Note: Do not give/select any connection details to this transformation.
Bearer token is another token-based authorization technique where if you pass user/pass to the token-generating API, it returns a token. You attach that token to API tool (postman, insomnia, or informatica) and POST your data to another API which will process the data.
Now, issue is, token can expire fast or slow. Fast expiring token need to be managed so that API call finishes before expiry. For us luckily its 24hrs.
First, get the token using HTTP trx by passing userid/pass.
Then, I pass the token as part of header column in HTTP transformation.
Methods I used - SIMPLE POST for both trx.
EDIT : Per request from an user, i am adding this edit.
The user and pass in the EXP transformation is hardcode values. And they were added to a json format that auth api can recognize.
The output of http_auth transformation is a token. If hardcode is a problem, you can read user and pass from a file from a secured location.
This token is used in the next json. This next json is created using the token as well as input data to the actual api. The api catches them, validates them against token and then write the data into api DB.
Try postman.. It will give you the code as well. It supports quite a few commonly used development languages
Can anyone help me with the configuration i need to do in API gateway to get a image file and some input param to lambda functions?
E.g. i want to upload image to S3 from lambda and write the metadata in dynamo db.
User can call api with image and metadata (e.g. customer id, mobile number).
For image file, i have added mapping template in integration as
"content" : "$input.body"
and Content-Type as application/png
API gateway settings1
But i can't figure out how to map the query param for the meta data. Can anyone help?
See documentation at https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-mapping-template-reference.html#input-variable-reference
You can map request parameter via
$input.params(x)
Returns the value of a method request parameter from the path, query string, or header value (searched in that order), given a parameter name string x.
I am building an app which lets users upload pictures and share it with his/her friends only.
I am using spring restful services to upload content directly to s3.
The user is authorized using OAuth.
To upload an image an authorized user's js client invokes POST /images/{userid}
To download the image, the client has to invoke GET /images/{userid}/{imageid}
Each user's content is stored in s3 under his own folder name, and the downloading service as explained in point 4 is the one that has to be invoked. Unfortunately, this means I cannot assign this url as source to an image tag <img src="">, because the authorization token should be sent on GET request. I cannot make the contents of user folder public because only user and his friends are allowed to see the images. The current service will soon become a bottleneck and I would like to avoid that.
What is the recommended architecture/design to solve this problem?
Instead of having a service that loads and returns the entire image file from S3, you should have a service that simply generates an S3 presigned URL. Then the URL retrieved from that service can be used in the <img src=""> tags on your site. This will be much more performant since the web browser will ultimately download the image directly from S3, while also still being secure.
The flow for downloading the images would be like this
User invokes GET request to download image
At Server End
Authenticate user
Query DB for metadata
Create a time based auth token.
Create a image URL(S3 based) and append auth token created in previous step
At the client end(User browser) redirect user to new URL(this url is effectively S3 location+auth token )
Now direct request will comes at the server( image URL+ auth token)
authenticate the token and then show image to user
Above URL will not be valid for long time , but your images are secured. As auth token is time based it will cater your case like if some one make the image private/public remove a friend.Deleted images , Copy paste of image url etc ..
When I use the Google Analytics Input Step, all I have to enter is my account username and password for the Authorization. From there, the step looks up the Domain Table ID for me. So by just giving this step my username and password, choosing the id and the metrics, I am able to retrieve all of the information I need--no other authorization required.
However, I am trying to recreate this by using the HTTP Client Lookup step (with a Generate Rows step before it). I gave it the following URL, as described by http://code.google.com/apis/analytics/docs/gdata/v3/reference.html:
https://www.googleapis.com/analytics/v3/data/ga?ids=ga:{*My Domain Table ID*}&start-date=2010-08-01&end-date=2012-04-01&metrics=ga:newVisits
and filled in the Http Login and Http Password fields with my username and password (exactly same as in Google Analytics Input step), respectively. However, when I preview the results of this HTTP Client step, the transformation returns an error that says that Login is required.
I have also tried this with the REST Client Lookup Step (with a Generate Rows step before it). I chose the GET HTTP method, the JSON application type, and filled in my HTTP Login and password for authentication. When I try to run this, it does not return an error, but in the result field of the preview output it says "Invalid Credentials."
What is the Google Analytics Input Step doing differently from the HTTP Client Lookup and REST Client Lookup steps? And how do I access the same information using those lookup steps?
I want to be able to access API's from other web sites as well, not just from Google Analytics, so it is important for me to be able to do this for any API.
Any help appreciated!
I have made a request to google analytics using HTTP client step, and it works perfect.
First, you need a token from Google Analytics:
https://www.google.com/accounts/ClientLogin?accountType=GOOGLE&Email=xxxxxxx#gmail.com&Passwd=xxxxxxx&service=analytics
This token is a long string.
The token is going to step client as HTTP header. The parameter must be called:
Authorization = token
Others parameters:
GData-Version=3.
After you add the request parameters. (ids, start-date, end-date, metrics, filter, segment)
You also have to add the key to your profile id, as the last parameter.
This request returns a XML. Use XML parser step to get metrics value.
which Kettle version are you using? as far as i know there are some changes in google api
read this bug report:
http://jira.pentaho.com/browse/PDI-7942