So, let me start in reverse. The issue is that there is no issue. The seemingly relevant Google docs state (https://cloud.google.com/sql/docs/postgres/connect-functions):
If you route all egress traffic through the VPC connector, you
must [my use of bold font] use a Private IP.
Let me now describe my set-up.
1 Google function using a VPC connector. It's invoked via Pub/Sub.
Ingress configuration: internal only;
egress configuration: route all traffic through VPC.
Actions taken when the function is called:
connect to an external API using a whitelisted static IP (provided by virtue of using the aforementioned VPC)
connect to a DB using its internal IP (the same virtual network)
connect to another DB using its connection name (I assume an SQL Proxy tunnel is used implicitly to achieve that)
All work alright. Yet if I understand the docs correctly the third action is bound to fail.
P.S. The context behind this manipulation is: I want to connect to 2 DBs, but as far as I understand that is impossible to achieve with VPC peering (or any current Google solution) if both are in different virtual networks and have only private IPs. I saw multiple years-old feature requests.
Related
I'm running a cloud run service with a working Cloud-SQL connection using the proxy to connect to the Cloud-SQL instance. The Cloud-SQL instance does not have a private IP configured.
Now there is a new requirement that this service needs to connect to a DB outside of GCP, for which it needs a static egress-IP that can be whitelisted. I attempted to achieve this via a serverless-VPC-connector (https://cloud.google.com/run/docs/configuring/static-outbound-ip).
Problem: When I add the VPC-connector to the service, and configure it to route all traffic through the vpc-connector, the service fails to deploy because it cannot connect to Cloud-SQL via the proxy anymore:
CloudSQL connection failed. Please see https://cloud.google.com/sql/docs/mysql/connect-run for additional details: Post "https://sqladmin.googleapis.com/sql/v1beta4/projects/<>/instances/<>/createEphemeral?alt=json&prettyPrint=false": context deadline exceeded
I was able to get this exact setup to work for a cloud function (identical external DB, CloudSQL, and vpc connector), and I'm at a loss as to why this wouldn't work for Cloud Run, and I'm wondering if there is additional configuration required which I'm missing?
Is it possible to connect to Cloud-SQL with the proxy, while at the same time using a VPC-connector to achieve a static egress IP?
If you need a static egress public IP, you only did the half of the path. Now, you need to add a Cloud NAT, that gets the VPC connector range and NAT the traffic to the internet. In this NAT, you can define a public static IP which will be used for all outgoing communications.
In any cases, if you want to reach Cloud SQL, it's better to use Cloud SQL proxy because the connection is secured (encrypted channel). If you use directly the public IP, add a SSL certificat to encrypt the communication over the public internet. (same thing if your instance is located outside GCP)
I would like to develop a Google Cloud Function that will subscribe to file changes in a Google Cloud Storage bucket and upload the file to a third party FTP site. This FTP site requires allow-listed IP addresses of clients.
As such, it is possible to get a static IP address for Google Cloud Functions containers?
Update: This feature is now available in GCP https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
First of all this is not an unreasonable request, don't get gaslighted. AWS Lambdas already support this feature and have for awhile now. If you're interested in this feature please star this feature request: https://issuetracker.google.com/issues/112629904
Secondly, we arrived at a work-around which I also posted to that issue as well, maybe this will work for you too:
Setup a VPC Connector
Create a Cloud NAT on the VPC
Create a Proxy host which does not have a public IP, so the egress traffic is routed through Cloud NAT
Configure a Cloud Function which uses the VPC Connector, and which is configured to use the Proxy server for all outbound traffic
A caveat to this approach:
We wanted to put the proxy in a Managed Instance Group and behind a GCP Internal LB so that it would dynamically scale, but GCP Support has confirmed this is not possible because the GCP ILB basically allow-lists the subnet, and the Cloud Function CIDR is outside that subnet
I hope this is helpful.
Update: Just the other day, they announced an early-access beta for this exact feature!!
"Cloud Functions PM here. We actually have an early-access preview of this feature if you'd like to test it out.
Please complete this form so we can add you..."
The form can be found in the Issue linked above.
See answer below -- it took a number of years, but this is now supported.
https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
For those wanting to associate cloud functions to a static IP address in order to whitelist the IP for an API or something of the sort I recommend checking out this step by step guide which helped me a lot:
https://dev.to/alvardev/gcp-cloud-functions-with-a-static-ip-3fe9 .
I also want to specify that this solution works for Google Cloud Functions and Firebase Functions (as it is based on GCP).
This functionality is now natively part of Google Cloud Functions (see here)
It's a two-step process according to the GCF docs:
Associating function egress with a static IP address In some cases,
you might want traffic originating from your function to be associated
with a static IP address. For example, this is useful if you are
calling an external service that only allows requests from whitelisted
IP addresses.
Route your function's egress through your VPC network. See the
previous section, Routing function egress through your VPC network.
Set up Cloud NAT and specify a static IP address. Follow the guides at
Specify subnet ranges for NAT and Specify IP addresses for NAT to set
up Cloud NAT for the subnet associated with your function's Serverless
VPC Access connector.
Refer to link below:
https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
As per Google, the feature has been released check out the whole thread
https://issuetracker.google.com/issues/112629904
It's not possible to assign a static IP for Google Cloud Functions, as it's pretty much orthogonal to the nature of the architecture being 'serverless' i.e. allocate and deallocate servers on demand.
You can, however, leverage a HTTP proxy to achieve a similar effect. Setup a Google Compute Engine instance, assign it a static IP and install a proxy library such as https://www.npmjs.com/package/http-proxy. You can then route all your external API calls etc through this proxy.
However, this probably reduces scale and flexibility, but it might be a workaround.
I have some Cloud SQL instances that currently have public IP's. It would make certain security-minded people happy if I changed them to have private IP's.
I am following the instructions documented here: https://cloud.google.com/sql/docs/mysql/private-ip
A summary of those instructions:
Ensure your shared VPC host has servicenetworking.googleapis.com enabled
Ensure your project has servicenetworking.googleapis.com enabled
Allocate an IP address range for your new private IP's
Configure VPC network peering (https://cloud.google.com/sql/docs/mysql/configure-private-services-access)
Create cloud sql instance without public IP
Expect new instance's private IP to be in allocated range
I've completed these through step 4, and I'm seeing this:
My interpretation of that page is that I've done my part and now it's google's turn--but that was several days ago. Do I have do do something to prompt google to create the connection?
I think I'm focusing in the right place because if I try to use I private IP, gcloud tells me to go create the network that I'm waiting on:
❯ gcloud --project=my-project-name beta \
sql instances patch foo \
--network=my-network-name --no-assign-ip
The following message will be used for the patch API method.
{"name": "foo", "project": "my-project-name", "settings": {"ipConfiguration": {"ipv4Enabled": false, "privateNetwork": "https://compute.googleapis.com/compute/v1/projects/my-project-name/global/networks/my-network-name"}}}
Patching Cloud SQL instance...failed.
ERROR: (gcloud.beta.sql.instances.patch) [INTERNAL_ERROR] Failed to create subnetwork. Please create Service Networking connection with service 'servicenetworking.googleapis.com' from consumer project '11111111111' network 'my-network-name' again.
In general private services access is implemented as a VPC peering connection between your VPC network and the Google services VPC network where your Cloud SQL instance resides. As #JohnHanley pointed out, the VPC peering should be created within minutes so it’s not expected you have to wait more than that.
To check the peering creation on Stackdriver you can use the following Advanced Filter:
jsonPayload.event_subtype="compute.networks.addPeering"
That said, it makes sense the error you are observing when trying to patch your SQL Instance as the Peering hasn’t been created. Instead of ‘Inactive’ it should be ‘Peer VPC network is connected’
To sum up, in your scenario the Cloud SQL instance cannot get an IP on the aforementioned network as it cannot reach it.
At this specific point I would suggest you focus on the Peering creation. As you mentioned you tried recreating it and the status remains the same, it’s possible that there’s something in your project preventing the peering to be established.
I would also suggest you check the peering limits quota in case it has been reached:
gcloud compute networks peerings list --network='your network'
Also it would be good to review the VPC Peering Restrictions.
All that being said, if you still experience the same issue when creating the VPC Peering, an internal investigation may be required and I would suggest you to you to report this using this link
I hope this helps.
I would like to develop a Google Cloud Function that will subscribe to file changes in a Google Cloud Storage bucket and upload the file to a third party FTP site. This FTP site requires allow-listed IP addresses of clients.
As such, it is possible to get a static IP address for Google Cloud Functions containers?
Update: This feature is now available in GCP https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
First of all this is not an unreasonable request, don't get gaslighted. AWS Lambdas already support this feature and have for awhile now. If you're interested in this feature please star this feature request: https://issuetracker.google.com/issues/112629904
Secondly, we arrived at a work-around which I also posted to that issue as well, maybe this will work for you too:
Setup a VPC Connector
Create a Cloud NAT on the VPC
Create a Proxy host which does not have a public IP, so the egress traffic is routed through Cloud NAT
Configure a Cloud Function which uses the VPC Connector, and which is configured to use the Proxy server for all outbound traffic
A caveat to this approach:
We wanted to put the proxy in a Managed Instance Group and behind a GCP Internal LB so that it would dynamically scale, but GCP Support has confirmed this is not possible because the GCP ILB basically allow-lists the subnet, and the Cloud Function CIDR is outside that subnet
I hope this is helpful.
Update: Just the other day, they announced an early-access beta for this exact feature!!
"Cloud Functions PM here. We actually have an early-access preview of this feature if you'd like to test it out.
Please complete this form so we can add you..."
The form can be found in the Issue linked above.
See answer below -- it took a number of years, but this is now supported.
https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
For those wanting to associate cloud functions to a static IP address in order to whitelist the IP for an API or something of the sort I recommend checking out this step by step guide which helped me a lot:
https://dev.to/alvardev/gcp-cloud-functions-with-a-static-ip-3fe9 .
I also want to specify that this solution works for Google Cloud Functions and Firebase Functions (as it is based on GCP).
This functionality is now natively part of Google Cloud Functions (see here)
It's a two-step process according to the GCF docs:
Associating function egress with a static IP address In some cases,
you might want traffic originating from your function to be associated
with a static IP address. For example, this is useful if you are
calling an external service that only allows requests from whitelisted
IP addresses.
Route your function's egress through your VPC network. See the
previous section, Routing function egress through your VPC network.
Set up Cloud NAT and specify a static IP address. Follow the guides at
Specify subnet ranges for NAT and Specify IP addresses for NAT to set
up Cloud NAT for the subnet associated with your function's Serverless
VPC Access connector.
Refer to link below:
https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
As per Google, the feature has been released check out the whole thread
https://issuetracker.google.com/issues/112629904
It's not possible to assign a static IP for Google Cloud Functions, as it's pretty much orthogonal to the nature of the architecture being 'serverless' i.e. allocate and deallocate servers on demand.
You can, however, leverage a HTTP proxy to achieve a similar effect. Setup a Google Compute Engine instance, assign it a static IP and install a proxy library such as https://www.npmjs.com/package/http-proxy. You can then route all your external API calls etc through this proxy.
However, this probably reduces scale and flexibility, but it might be a workaround.
I would like to develop a Google Cloud Function that will subscribe to file changes in a Google Cloud Storage bucket and upload the file to a third party FTP site. This FTP site requires allow-listed IP addresses of clients.
As such, it is possible to get a static IP address for Google Cloud Functions containers?
Update: This feature is now available in GCP https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
First of all this is not an unreasonable request, don't get gaslighted. AWS Lambdas already support this feature and have for awhile now. If you're interested in this feature please star this feature request: https://issuetracker.google.com/issues/112629904
Secondly, we arrived at a work-around which I also posted to that issue as well, maybe this will work for you too:
Setup a VPC Connector
Create a Cloud NAT on the VPC
Create a Proxy host which does not have a public IP, so the egress traffic is routed through Cloud NAT
Configure a Cloud Function which uses the VPC Connector, and which is configured to use the Proxy server for all outbound traffic
A caveat to this approach:
We wanted to put the proxy in a Managed Instance Group and behind a GCP Internal LB so that it would dynamically scale, but GCP Support has confirmed this is not possible because the GCP ILB basically allow-lists the subnet, and the Cloud Function CIDR is outside that subnet
I hope this is helpful.
Update: Just the other day, they announced an early-access beta for this exact feature!!
"Cloud Functions PM here. We actually have an early-access preview of this feature if you'd like to test it out.
Please complete this form so we can add you..."
The form can be found in the Issue linked above.
See answer below -- it took a number of years, but this is now supported.
https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
For those wanting to associate cloud functions to a static IP address in order to whitelist the IP for an API or something of the sort I recommend checking out this step by step guide which helped me a lot:
https://dev.to/alvardev/gcp-cloud-functions-with-a-static-ip-3fe9 .
I also want to specify that this solution works for Google Cloud Functions and Firebase Functions (as it is based on GCP).
This functionality is now natively part of Google Cloud Functions (see here)
It's a two-step process according to the GCF docs:
Associating function egress with a static IP address In some cases,
you might want traffic originating from your function to be associated
with a static IP address. For example, this is useful if you are
calling an external service that only allows requests from whitelisted
IP addresses.
Route your function's egress through your VPC network. See the
previous section, Routing function egress through your VPC network.
Set up Cloud NAT and specify a static IP address. Follow the guides at
Specify subnet ranges for NAT and Specify IP addresses for NAT to set
up Cloud NAT for the subnet associated with your function's Serverless
VPC Access connector.
Refer to link below:
https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
As per Google, the feature has been released check out the whole thread
https://issuetracker.google.com/issues/112629904
It's not possible to assign a static IP for Google Cloud Functions, as it's pretty much orthogonal to the nature of the architecture being 'serverless' i.e. allocate and deallocate servers on demand.
You can, however, leverage a HTTP proxy to achieve a similar effect. Setup a Google Compute Engine instance, assign it a static IP and install a proxy library such as https://www.npmjs.com/package/http-proxy. You can then route all your external API calls etc through this proxy.
However, this probably reduces scale and flexibility, but it might be a workaround.