I am able to call public endpoints but I need to call this private endpoint. I can put authentication header but trying to find out if I can automate the process through some other google cloud service.
Related
I am setting up a DAG in Cloud Composer that triggers a number of Cloud Run and Cloud Function services. The service account specified in the Cloud Composer Environment (a user created SA) definitely has permissions to invoke both Cloud Run and Cloud Function services, however the Cloud Run functions are giving the following error:
The request was not authenticated. Either allow unauthenticated invocations or set the proper Authorization header. Read more at https://cloud.google.com/run/docs/securing/authenticating
The tasks are like so:
#t1 as request first report
big3_request = SimpleHttpOperator(
task_id= "big3_request",
method='GET',
http_conn_id='trigger_cloud_run_service_conversions_big_3',
endpoint='',
response_check = lambda response: True if response == ("ok", 200) else False
)
I would have thought that the cloud composer environment would be able to use the service accounts IAM roles, but this doesn't seem to be the case. What do I need to do here to enable the services to run? It looks like I can add the keyfile of the service account to the connection, but I don't see why this should be necessary if the same service account is used in the CC environment?
Your service (your SimpleHttpOperator task running in cloud composer) needs to provide authentication credentials in the request. More precisely it needs to
add a Google-signed OpenID Connect ID token as part of the request
You can find here in the official Google doc, different methods to provide such token and a proper request to your Cloud Run service endpoint.
I have a Cloud Run service setup and I have a Cloud Scheduler job that calls an endpoint on that service.
The job sent a GET call (which have an header with a jwt and an api-key, necessary to certify the call to the endpoint) to the Cloud Run endpoint. I have already tried to test the endpoint locally, and it works. Maybe it is necessary a particular configuration for the scheduler to work?
When the job work, I can see from the log console on Cloud Run this:
Cloud Run Console Log.
What can I do to fix this problem?
Thank you
You can follow these docs which walk you through creating or editing a service account for the Scheduler Job to use which is allowed to 'invoke' Cloud Run services.
Thanks,
Josh
According to the "Authenticating service-to-service" documentation for Cloud Run, to use Pub/Sub and Cloud Scheduler on a service, unauthenticated access must be disabled because they rely on HTTP calls because of the zero scaling capability of Cloud Run services.
My services allow internal and Load Balancer traffic and must be publicly available for frontend clients, but they also must be able to communicate with each other privately with Pub/Sub.
Is there a way to achieve this? It feels unnatural to create a separate private service just for using Pub/Sub.
It's a missing piece. You can't plug in your VPC PubSub push subscription and Cloud Scheduler (but also Cloud Task, Cloud Build, Workflows,...). I asked Google Cloud few months ago, and it should be fixed by a new network features, soon. At least in 2021!
So, in your case, if your Cloud Run service is accessible from the public internet through a Load Balancer, you can use this public endpoint to call the path that you want on your service and thus perform the process.
If your Cloud Run in only accessible from ingress=internal, you can't for now.
I have a service listening on 'https://myapp.a.run.app/dosomething', but I want to leverage the scalability features of Cloud Run, so in the controller for 'dosomething', I send off 10 requests to 'https://myapp.a.run.app/smalltask'; with my app configured to allow servicing of only one request per instance, I expect 10 instances to spin up, all do their smalltask, and return (all within the timeout period).
But I don't know how to properly authenticate the request, so those 10 requests all result in 403's. For Cloud Run services, I manually pass in a bearer token with the initial request, though I expect to add some api proxy at some point. But without said API proxy, what's the right way to send the request such that it is accepted? The app is running as a user that does have permissions to access the endpoint.
Authenticating service-to-service
If your architecture is using multiple services, these services will likely need to communicate with each other.
You can use synchronous or asynchronous service-to-service communication:
For asynchronous communication, use
Cloud Tasks for one to one asynchronous communication
Pub/Sub for one to many asynchronous communication
Cloud Scheduler for regularly scheduled asynchronous communication.
Cloud Workflows for orchestration services.
For synchronous communication
One service invokes another one over HTTP using its endpoint URL. In this use case, it's a good idea to ensure that each service is only able to make requests to specific services. For instance, if you have a login service, it should be able to access the user-profiles service, but it probably shouldn't be able to access the search service.
First, you'll need to configure the receiving service to accept requests from the calling service:
Grant the Cloud Run Invoker (roles/run.invoker) role to the calling service identity on the receiving service. By default, this identity is PROJECT_NUMBER-compute#developer.gserviceaccount.com.
In the calling service, you'll need to:
Create a Google-signed OAuth ID token with the audience (aud) set to the URL of the receiving service. This value must contain the schema prefix (http:// or https://) and custom domains are currently not supported for the aud value.
Include the ID token in an Authorization: Bearer ID_TOKEN header. You can get this token from the metadata server, while the container is running on Cloud Run (fully managed). If the application is running outside Google Cloud, you can generate an ID token from a service account key file.
For a full guide and examples in Node/Python/Go/Java and others see: Authenticating service-to-service
I am currently building a rest api, for this I am using Google Cloud API Gateway and Google Cloud Run. I've been looking at all the google cloud documentation and researching elsewhere and I can't find how to add a custom domain to an API gateway instance. The funny thing is that there is more documentation for Google Cloud endpoints, I could find how to do it with endpoints but it does not apply to my use case.
I have 10 instances of google cloud run each one running a microservice respectively and I want to join everything in a single domain and add support with openapi, but I have failed in the attempt.
In any case, if someone has managed to customize the domain of an api gateway instance, I would appreciate if you could guide me, greetings.
For the beta release, custom domain names are not supported on GCP for API Gateway. Since it is still beta as of today, if you want to use a custom domain, you could use Cloud Endpoints in Cloud Run or you could even look into using Microservices in App Engine.