I'm using the REST API template provided from https://cloud.google.com/dataflow/docs/guides/templates/provided-utilities#api_2 to delete from datastore but I'm getting the following error
"error": {
"code": 403,
"message": "(55ae8f210de971e7): Template file failed to load: gs://dataflow-templates/dataflow-templates/latest/Datastore_to_Datastore_Delete. Permissions denied. Provided scope(s) are not authorized",
"status": "PERMISSION_DENIED"
}
It says the scope isn't authorized but the docs say you only need one of the following: https://www.googleapis.com/auth/compute.readonly, https://www.googleapis.com/auth/compute, https://www.googleapis.com/auth/cloud-platform or https://www.googleapis.com/auth/userinfo.email.
For my oauth2 request I tried adding the devstorage scope as well: https://www.googleapis.com/auth/compute.readonly https://www.googleapis.com/auth/devstorage.read_only
I'm calling POST https://dataflow.googleapis.com/v1b3/projects/{projectid}/templates:launch?gcsPath=gs://dataflow-templates/latest/Datastore_to_Datastore_Delete
With
{
jobName: 'PrunePrintLogs',
environment: { zone: 'europe-west2' },
parameters: {
datastoreReadGqlQuery: 'select * from `00000000test`',
datastoreReadProjectId: '{projectid}',
datastoreDeleteProjectId: '{projectid}'
}
}
My project id is redacted.
I added the Owner role to the account getting the oauth2 token just temporarily for testing so wouldn't all scopes be authorized?
Got it working. Firstly I needed to remove the 'environment' parameter. Secondly I set scope to 'https://www.googleapis.com/auth/cloud-platform'. It also seems the role needs to be Editor, I wasn't able to find a more restrictive role to get it working.
Curiously when I sent the request from postman is shows up in Dataflow jobs as sdk 2.27.0 but I'm sure I'm sending exactly the same from my node app using axios and it shows up as sdk 2.20.0 and warns that it will eventually be no longer supported.
Related
I am getting the following when sending a marketplaceParticipations request to sellers/v1/marketplaceParticipations via Postman after following instructions and examples provided at https://developer-docs.amazon.com/sp-api/docs/connecting-to-the-selling-partner-api
{
"errors": [
{
"message": "Access to requested resource is denied.",
"code": "Unauthorized",
"details": ""
}
]
}
We have registered a self-authorized app client in Draft status which has a user ARN IAM attached as described at https://developer-docs.amazon.com/sp-api/docs/registering-your-application.Ï
I've checked the inline and role policies for the ARN IAM. They are exactly as described at https://developer-docs.amazon.com/sp-api/docs/creating-and-configuring-iam-policies-and-entities#step-4-create-an-iam-role.
We are able to successfully request an LWA access token following the docs at https://developer-docs.amazon.com/sp-api/docs/connecting-to-the-selling-partner-api#step-1-request-a-login-with-amazon-access-token.
Please check that the roles of the user you are using allow to make request to that endpoint in your dev profile at https://sellercentral.amazon.com/
As far as I know, the getMarketplaceParticipations doesn't need a Restricted Data Token (RDT). So you must be able to solve it by giving the user the correct roles.
I was able to get them using Postman. It is a good way to check that the request is correctly built and not a programming issue.
I am trying to invoke GCP functions through my angular app hosted on App Engine. I cannot seem to find any straightforward answer given my users are not signed up through GCP but through Google Workspace. So in short the app I am building is only for internal users. As of right now I am able to log in fine using Google Authentication, the problem is that after I log in, the cloud function is rejecting my request. I have included all the steps that I've taken along with the error I am receiving from the cloud function.
So heres what I've done so far.
I implemented the login button inside of my Angular app using
angularx-social-login.
I then obtained an OAuth 2.0 Client ID from
the GCP project in which the functions are hosted (as this is the
same project anyway).
After this I registered the OAuth consent screen and set it to internal as I don't
want anyone but my internal workspace users to be able to access this
I then went to users identity platform and registered the same OAuth
2.0 client ID that I spoke of in step 2.
I then set up the GCP function to allow allAuthenticatedUsers (I've tried many other
roles but I would accept if I could just get allAuthenticatedUsers to work for now)
Finally back in my angular app I passed into the function call
headers the idToken that I get each time a user logs in using the
Google Login Popup
My code looks like:
DashboardComponent.ts
import {SocialAuthService} from 'angularx-social-login';
...
this.authService.authState.subscribe((user) => {
this.myService.callFunction(user.idToken).subscribe((userRes) => {
...
}
...
FirebaseFunctionService.ts
callFunction(authToken): Observable<any> {
const headers= new HttpHeaders()
.set('content-type', 'application/json')
.set('Access-Control-Allow-Origin', '*')
.set('Authorization', `Bearer ${authToken}`);
return this.http.get('https://my-cloud-function-url/my-function', { headers: headers
});
}
And the response that I get when I call this function is:
error: ProgressEvent {isTrusted: true, lengthComputable: false,
loaded: 0, total: 0, type: 'error', …} headers: HttpHeaders
{normalizedNames: Map(0), lazyUpdate: null, headers: Map(0)} message:
"Http failure response for
https://myFunction.cloudfunctions.net/myFunction: 0 Unknown Error"
name: "HttpErrorResponse" ok: false status: 0 statusText: "Unknown
Error" url: "https://myFunction.cloudfunctions.net/myFunction"
[[Prototype]]: HttpResponseBase
Does anyone know why this might be happening? Any help would be greatly appreciated as I am at my wits end. Thanks in advance :)
Status code 0 in the error message indicates a CORS failure. You can go through this GitHub issue comment, where the stackoverflow thread points to a number of reasons for this error.
Also you need to write this line of code in your initialisations :
const cors = require('cors')({origin: true}) and check out Google’s documentation on how to handle CORS requests. Also you have to provide proper permissions - one of the important ones being giving Cloud Functions invoker role to your cloud functions.
Joe, (author of our question) agreed that it was a CORS error but he solved it by giving allUsers permission (making the function public) and verifying the user in the function itself and the CORS disappeared.
Now the reason behind this :
I think there was some issue with Joe’s authentication mechanism and hence functions were not authenticated. HTTP functions require authentication by default. And as it did not have it, as per documentation the workaround was to make Joe’s function public by setting the --allow-unauthenticated flag, or use the Console to grant the Cloud Functions Invoker role to allUsers. Then handle CORS and authentication in the function code (which he was doing as he mentioned).
So when Joe made the function public by granting allUsers and CORS was handled in code, it started working and the CORS errors disappeared.
I get the following environment-dependent error in terraform.
What action should I take to resolve it?
Error: Post "https://composer.googleapis.com/v1beta1/projects/project/locations/asia-northeast1/environments?alt=json&prettyPrint=false": Post "https://oauth2.googleapis.com/token": dial tcp 172.217.25.202:443: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
The error that was output when opening the above url is as follows.
"error": {
"code": 401,
"message": "Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.",
"status": "UNAUTHENTICATED"
}
}
The terraform code used is as follows.
resource "google_storage_bucket" "auto-expire" {
name = "auto-expiring-bucket"
location = "US"
}
teraform apply without any arguments.
You would need to follow this Terraform doc adding-credentials to setup credentials.
Related access_token client_config.
This could also happen if your config is extremely large as those tokens timeout after an hour.
I am using AWS Amplify in my react app to call my API hosted in API Gateway with AWS_IAM authorization. When I set authorization to NONE, everything works fine in my react app.
However, when I set authorization to AWS_IAM and execute my API using API.get() from Amplify like the code below:
const notes = await API.get('notes', '/notes', init);
I get an error message like:
{
"message": "Missing Authentication Token",
"err": "missing auth"
}
So I tried to use aws-api-gateway-cli-test to test my API gateway. Through the script, I was able to get a valid credential, get authenticated and correct response. I have also tried the POSTMAN with my admin credentials and it worked as well.
After doing some research, I saw people referring this to CORS issue. I have double checked my API gateway settings and confirmed that I have enabled CORS. The same issue persists.
Through the debugging feature of aws-amplify, I was able to see the signing process in my Chrome inspector. The signing process was performed properly with some accessKey and secretKey. I pulled those keys out of the inspector into my POSTMAN and tried to get.
These credentials are not valid and I received the following error message:
{
"message": "The security token included in the request is invalid.",
"err:": "default"
}
Update: I forgot to copy session token into POSTMAN. Now with all credentials generated by my app I am able to get correct result from my API in POSTMAN. Just not in my app.
At this point, it is pretty obvious to me that it is an auth problem. However, I have been using aws-amplify for sign-in and get process. Pretty sure the signing process is done by amplifying behind the scenes.
I have spent 3 days on this problem and basically tried everything... Any ideas on why it doesn't work?
P.S. A lot of people have suggested that typos in the invoke URL could lead to this issue. I have double checked and there is no typo. Below is my amplify configure code:
Amplify.configure({
Auth: {
mandatorySignIn: true,
region: config.cognito.REGION,
userPoolId: config.cognito.USER_POOL_ID,
identityPoolId: config.cognito.IDENTITY_POOL_ID,
userPoolWebClientId: config.cognito.APP_CLIENT_ID
},
Storage: {
region: config.s3.REGION,
bucket: config.s3.BUCKET,
identityPoolId: config.cognito.IDENTITY_POOL_ID
},
API: {
endpoints: [
{
name: "notes",
endpoint: config.apiGateway.URL,
region: config.apiGateway.REGION
}
]
}
});
Just resolved my problem - I have had Authorization settings for OPTIONS method to be AWS_IAM under Resources in API Gateway. However when my browser send a request it will send one to OPTIONS first to check for certain headers without the credentials in this request.
Since I set OPTIONS with IAM authorization, the OPTIONS method then checked against IAM with this request without my credentials. This is why I received "Missing Authentication Token".
The problem was this in my case:
import { Auth } from 'aws-amplify';
import { API } from 'aws-amplify';
I know, I now it's banal. This why I should't code when I am exausted.
Consider I have created an API using with AWS API gateway with the following URL
https://0abcgdefg1.execute-api.ap-northeast-1.amazonaws.com/Employee/.
is it possible to create an error message if the end user tried a nonexisting stage name? For eg
https://0abcgdefg1.execute-api.ap-northeast-1.amazonaws.com/Employee1/
is it possible to give some error information like below?
{
"errors": [
{
"message": "Stage name Employee1doesn't exist",
"type": "InvalidStageError"
}
]
}
No APIGW do not support returning custom error information when stage does not exits.
I am also wondering why you need to return this error message to API user ? can you provide details about your use case ?