Currently my projects in Cloud Run that make external requests come out with random IP from Google IP's pool.
A new micro-service that I am developing that needs to make an external request on a critical external micro-service that is limited by IP.
Google Cloud Platform has any solution to channel the output from a specific IP to the outside? Some kind of proxy for these kinds of needs?
Thanks
As clarified in this other case here, there is no way to directly setup a static or specific IP for outbound requests for Cloud Run. The only possibility as clarified in this answer from a Google's developer, unless Cloud Run starts supporting Cloud NAT or Serverless VPC Access, you won't be able to achieve such configuration.
There are some workarounds.
One of them would be to create a SOCKS proxy by running a ssh client that routes the traffic through a GCE VM instance that has a static external IP address. More details here.
Another solution is to send your outbound requests through a proxy that has a static IP. You can get details here.
Both these two were provided by developers from Google, so they should be good to go and use it.
Related
The current setup is as follows:
I have a Cloud Run service, which acts as "back-end", which needs to reach external services but wants to be reached ONLY by the second Cloud Run instance. which acts as a "front-end", which needs to reach auth0 and the "back-end" and be reached by any client with a browser.
I recognize that the setup is not optimal, but I've inherited as is and we cannot migrate to another solution (maybe k8n). I'm trying to make this work with the least amount of impact on the infrastructure and, ideally, without having to touch the services themselves.
What I've tried is to restrict the ingress of the back-end service to INTERNAL and place two serverless VPC connectors (one per service), so that the front-end service would be able to reach the back-end but no one else could.
But I've encountered a huge issue: if I set the egress of the front-end all on the VPC it works, but now the front-end cannot reach auth0 and therefore the users cannot authenticate. If I place the egress as "mixed" (only internal ip ranges go through the VPC) the Google Run URL (*.run.app) is resolved not through the VPC and therefore it returns a big bad 403.
What I tried so far:
Placing a load balancer in front of the back-end service. But the serverless NEG only supports the global http load balancer and I'd need an internal one if I wanted an internal ip to resolve against
Trying to see if the VPC accessor itself MAYBE provided an internal (static) ip, but it doesn't seem so
Someone in another question suggested a "MIG as a proxy" but I haven't managed to figure that out (Can I run Cloud Run applications on a private IP (inside dedicated VPC network)?)
Fooled around with the Gateway API, but it seems that I'd have to provide a openAPI specification for the back-end, and I'm still under the delusion that this might be resolved with a cheaper (in terms of effort) approach.
So, I get that the Cloud Run instance cannot possibly have an internal IP by itself, but is there any kind of GCP product that can act as a proxy? Can someone elaborate on the "MIG as a proxy" approach (Managed Instance Group? Of what, though?), which might be the solution I'm looking for? (Sadly, I do not have the reputation needed to comment on that question or I would have).
Any kind of pointer is, as always, deeply appreciated.
You are designing this wrong. Use Cloud Run's identity-based access control instead of trying to route traffic. Google IAP (Identity Aware Proxy) will block all traffic that is not authorized.
Authenticating service-to-service
I would like to develop a Google Cloud Function that will subscribe to file changes in a Google Cloud Storage bucket and upload the file to a third party FTP site. This FTP site requires allow-listed IP addresses of clients.
As such, it is possible to get a static IP address for Google Cloud Functions containers?
Update: This feature is now available in GCP https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
First of all this is not an unreasonable request, don't get gaslighted. AWS Lambdas already support this feature and have for awhile now. If you're interested in this feature please star this feature request: https://issuetracker.google.com/issues/112629904
Secondly, we arrived at a work-around which I also posted to that issue as well, maybe this will work for you too:
Setup a VPC Connector
Create a Cloud NAT on the VPC
Create a Proxy host which does not have a public IP, so the egress traffic is routed through Cloud NAT
Configure a Cloud Function which uses the VPC Connector, and which is configured to use the Proxy server for all outbound traffic
A caveat to this approach:
We wanted to put the proxy in a Managed Instance Group and behind a GCP Internal LB so that it would dynamically scale, but GCP Support has confirmed this is not possible because the GCP ILB basically allow-lists the subnet, and the Cloud Function CIDR is outside that subnet
I hope this is helpful.
Update: Just the other day, they announced an early-access beta for this exact feature!!
"Cloud Functions PM here. We actually have an early-access preview of this feature if you'd like to test it out.
Please complete this form so we can add you..."
The form can be found in the Issue linked above.
See answer below -- it took a number of years, but this is now supported.
https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
For those wanting to associate cloud functions to a static IP address in order to whitelist the IP for an API or something of the sort I recommend checking out this step by step guide which helped me a lot:
https://dev.to/alvardev/gcp-cloud-functions-with-a-static-ip-3fe9 .
I also want to specify that this solution works for Google Cloud Functions and Firebase Functions (as it is based on GCP).
This functionality is now natively part of Google Cloud Functions (see here)
It's a two-step process according to the GCF docs:
Associating function egress with a static IP address In some cases,
you might want traffic originating from your function to be associated
with a static IP address. For example, this is useful if you are
calling an external service that only allows requests from whitelisted
IP addresses.
Route your function's egress through your VPC network. See the
previous section, Routing function egress through your VPC network.
Set up Cloud NAT and specify a static IP address. Follow the guides at
Specify subnet ranges for NAT and Specify IP addresses for NAT to set
up Cloud NAT for the subnet associated with your function's Serverless
VPC Access connector.
Refer to link below:
https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
As per Google, the feature has been released check out the whole thread
https://issuetracker.google.com/issues/112629904
It's not possible to assign a static IP for Google Cloud Functions, as it's pretty much orthogonal to the nature of the architecture being 'serverless' i.e. allocate and deallocate servers on demand.
You can, however, leverage a HTTP proxy to achieve a similar effect. Setup a Google Compute Engine instance, assign it a static IP and install a proxy library such as https://www.npmjs.com/package/http-proxy. You can then route all your external API calls etc through this proxy.
However, this probably reduces scale and flexibility, but it might be a workaround.
I would like to develop a Google Cloud Function that will subscribe to file changes in a Google Cloud Storage bucket and upload the file to a third party FTP site. This FTP site requires allow-listed IP addresses of clients.
As such, it is possible to get a static IP address for Google Cloud Functions containers?
Update: This feature is now available in GCP https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
First of all this is not an unreasonable request, don't get gaslighted. AWS Lambdas already support this feature and have for awhile now. If you're interested in this feature please star this feature request: https://issuetracker.google.com/issues/112629904
Secondly, we arrived at a work-around which I also posted to that issue as well, maybe this will work for you too:
Setup a VPC Connector
Create a Cloud NAT on the VPC
Create a Proxy host which does not have a public IP, so the egress traffic is routed through Cloud NAT
Configure a Cloud Function which uses the VPC Connector, and which is configured to use the Proxy server for all outbound traffic
A caveat to this approach:
We wanted to put the proxy in a Managed Instance Group and behind a GCP Internal LB so that it would dynamically scale, but GCP Support has confirmed this is not possible because the GCP ILB basically allow-lists the subnet, and the Cloud Function CIDR is outside that subnet
I hope this is helpful.
Update: Just the other day, they announced an early-access beta for this exact feature!!
"Cloud Functions PM here. We actually have an early-access preview of this feature if you'd like to test it out.
Please complete this form so we can add you..."
The form can be found in the Issue linked above.
See answer below -- it took a number of years, but this is now supported.
https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
For those wanting to associate cloud functions to a static IP address in order to whitelist the IP for an API or something of the sort I recommend checking out this step by step guide which helped me a lot:
https://dev.to/alvardev/gcp-cloud-functions-with-a-static-ip-3fe9 .
I also want to specify that this solution works for Google Cloud Functions and Firebase Functions (as it is based on GCP).
This functionality is now natively part of Google Cloud Functions (see here)
It's a two-step process according to the GCF docs:
Associating function egress with a static IP address In some cases,
you might want traffic originating from your function to be associated
with a static IP address. For example, this is useful if you are
calling an external service that only allows requests from whitelisted
IP addresses.
Route your function's egress through your VPC network. See the
previous section, Routing function egress through your VPC network.
Set up Cloud NAT and specify a static IP address. Follow the guides at
Specify subnet ranges for NAT and Specify IP addresses for NAT to set
up Cloud NAT for the subnet associated with your function's Serverless
VPC Access connector.
Refer to link below:
https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
As per Google, the feature has been released check out the whole thread
https://issuetracker.google.com/issues/112629904
It's not possible to assign a static IP for Google Cloud Functions, as it's pretty much orthogonal to the nature of the architecture being 'serverless' i.e. allocate and deallocate servers on demand.
You can, however, leverage a HTTP proxy to achieve a similar effect. Setup a Google Compute Engine instance, assign it a static IP and install a proxy library such as https://www.npmjs.com/package/http-proxy. You can then route all your external API calls etc through this proxy.
However, this probably reduces scale and flexibility, but it might be a workaround.
I would like to develop a Google Cloud Function that will subscribe to file changes in a Google Cloud Storage bucket and upload the file to a third party FTP site. This FTP site requires allow-listed IP addresses of clients.
As such, it is possible to get a static IP address for Google Cloud Functions containers?
Update: This feature is now available in GCP https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
First of all this is not an unreasonable request, don't get gaslighted. AWS Lambdas already support this feature and have for awhile now. If you're interested in this feature please star this feature request: https://issuetracker.google.com/issues/112629904
Secondly, we arrived at a work-around which I also posted to that issue as well, maybe this will work for you too:
Setup a VPC Connector
Create a Cloud NAT on the VPC
Create a Proxy host which does not have a public IP, so the egress traffic is routed through Cloud NAT
Configure a Cloud Function which uses the VPC Connector, and which is configured to use the Proxy server for all outbound traffic
A caveat to this approach:
We wanted to put the proxy in a Managed Instance Group and behind a GCP Internal LB so that it would dynamically scale, but GCP Support has confirmed this is not possible because the GCP ILB basically allow-lists the subnet, and the Cloud Function CIDR is outside that subnet
I hope this is helpful.
Update: Just the other day, they announced an early-access beta for this exact feature!!
"Cloud Functions PM here. We actually have an early-access preview of this feature if you'd like to test it out.
Please complete this form so we can add you..."
The form can be found in the Issue linked above.
See answer below -- it took a number of years, but this is now supported.
https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
For those wanting to associate cloud functions to a static IP address in order to whitelist the IP for an API or something of the sort I recommend checking out this step by step guide which helped me a lot:
https://dev.to/alvardev/gcp-cloud-functions-with-a-static-ip-3fe9 .
I also want to specify that this solution works for Google Cloud Functions and Firebase Functions (as it is based on GCP).
This functionality is now natively part of Google Cloud Functions (see here)
It's a two-step process according to the GCF docs:
Associating function egress with a static IP address In some cases,
you might want traffic originating from your function to be associated
with a static IP address. For example, this is useful if you are
calling an external service that only allows requests from whitelisted
IP addresses.
Route your function's egress through your VPC network. See the
previous section, Routing function egress through your VPC network.
Set up Cloud NAT and specify a static IP address. Follow the guides at
Specify subnet ranges for NAT and Specify IP addresses for NAT to set
up Cloud NAT for the subnet associated with your function's Serverless
VPC Access connector.
Refer to link below:
https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
As per Google, the feature has been released check out the whole thread
https://issuetracker.google.com/issues/112629904
It's not possible to assign a static IP for Google Cloud Functions, as it's pretty much orthogonal to the nature of the architecture being 'serverless' i.e. allocate and deallocate servers on demand.
You can, however, leverage a HTTP proxy to achieve a similar effect. Setup a Google Compute Engine instance, assign it a static IP and install a proxy library such as https://www.npmjs.com/package/http-proxy. You can then route all your external API calls etc through this proxy.
However, this probably reduces scale and flexibility, but it might be a workaround.
I am looking to build a scalable REST webservice on the Google Cloud Compute Engine but have a couple of requirements that I am not sure how best to implement.
Structure so far:
2 Instances running a REST webservice connected to a MySQL Cloud database.
(number of instances to scale up in the future)
Load balancer to split request between the two or more Instances.
this part is fine.
What I need next is that the traffic (POST requests from instances to an external webservice) must come from a single IP address. I assume these requests can not route back through the public IP of the load balancer?
I get the impression the solution to this is to route all requests from instances though a 3rd instance running squid. Is this the best way to do this? (side question)
Now to my main question:
I have been reading about ApiAxle which sounds like a nice proxy for Web Services, giving some good access control, throttling and reporting capabilities.
Can I have an instance running ApiAxle followed by a google cloud Load Balancer which shares the request from the proxy to the backend instances that do the leg work and feed the response back through the ApiAxle proxy, thus having everything though a single IP visible to clients using the API? (letting me add new instances to the pool to add capacity.)
and Would the proxy be much of a bottle neck?
Thanks in advance.
/Dave
(new to this, so sorry if its a stupid question because I cant find anything like this on the web)
Sounds like you need to NAT on your outbound traffic so it appears to come from one IP address. You need to do that via a third instance since Google LB stack doesn't provide this. GCLB works only with inbound connections on the load-balanced IP.
You can setup source-NAT using advanced routing, or you can use a proxy as you suggested.