I have a Google compute engine vm ubuntu host with stackdriver monitoring agent installed.
The vm host has a VPC firewall rule to deny all communication apart from a proxy server (to get system updates) and it has only an internal IP.
I have configured the stack driver agent according to doc's at https://cloud.google.com/monitoring/agent/install-agent.
The monitoring agent is unable to send monitor data to stackdriver unless i turn off the firewall rule.
What changes should i make to the VPC firewall rule in order for the agent to able to send data to stackdriver?
Stackdriver uses HTTPS to communicate with the Google API endpoints.
However, if your VM only has private IP addresses, you must also configure Private Google Access. I cover the requirements in this article:
https://www.jhanley.com/google-compute-stackdriver-logging-installation-setup-debugging/
These endpoints must be reachable for Stackdriver logging and monitoring to function:
oauth2.googleapis.com
monitoring.googleapis.com
stackdriver.googleapis.com
Related
I have a Google Cloud Run services and i would need to allow traffic from certain machine only.
I use Google cloud armor to allow IPs to access the Cloud Run service.
I have problem in adding dynamic IPs of certain machine as it keeps changing. I also searched on adding mac address to allow, but Cloud armor does not have that feature.
You cannot use MAC addresses for the Internet. The service (Cloud Armor) will never see the client's MAC address, only the MAC address of the last router (which would be a Google router). Google Cloud VPCs do not expose layer 2 information.
Cloud Run is a public service with a public URL. Restricting traffic based upon IP address is not supported by Cloud Run. You can put an HTTP Load Balancer and Cloud Armor in front, but that would not prevent traffic that goes directly to the service.
There are much better techniques to control access to public services. Google Cloud implements authorization using OAuth via Identity Aware Proxy (IAP). That is the correct method to use. Given that your clients have changing IP addresses, that is your best solution.
If I needed access control based upon IP address, I would run my service on Compute Engine using either Container Optimized OS, Docker or just natively using Apache/Nginx. You can dynamically update VPC firewall rules as the client's IP address changes with custom code.
I have 2 google cloud services:
Google Cloud Run Service (Node Js / Strapi)
Google Cloud SQL Service (Mysql)
I have added the Cloud SQL connection to the Google Cloud Run Service from the UI, and have a public IP for the Google Cloud SQL Service. On top of that I have added the Run Service IP to the Authorised networks of SQL Service.
If I try and connect from another server (external from Google cloud) I can easily connect to the Google Cloud SQL Service and execute queries.
But if I try and connect from inside the GCloud Run Service with exactly the same settings (Ip, database_name, etc) my connection hangs and I get a timeout error in the logs...
How to properly allow Gcloud SQL to accept connections from GCloud RUN?
I looked for other answers in here, but they all look very old (around 2015 )
You can use 3 modes to access to your database
Use the built-in feature. In this case, you don't need to specify the IP address, it's a linux socket that is open to communicate with the database as described in the documentation
Use Cloud SQL private IP. This time, no need to configure a connection in the Cloud Run service, you won't use it because you will use the IP, not the linux socket. This solution required 2 things
Firstly attach your database to your VPC and give it a private IP
Then, you need to route the private IP traffic of Cloud Run through your VPC. For this you have to create, and then to attach to the Cloud RUn service, a serverless VPC Connector
Use CLoud SQL public IP. This time again, no need to configure a connection in the Cloud Run service, you won't use it because you will use the IP, not the linux socket. To achieve this, you need more steps (and it's less secure)
You need to route all the egress traffic of Cloud Run through your VPC. For this you have to create, and then to attach to the Cloud RUn service, a serverless VPC Connector
Deploy your Cloud Run service with the Serverless VPC Connector and the egress connectivity param to "all"
Then create a Cloud NAT to route all the VPC Connector ip range traffic to a single IP (or set of IPs) (The link is the Cloud Functions documentation, but it works exactly in the same way)
Finally authorize the Cloud NAT IP(s) on Cloud SQL authorized networks.
In your case, you have whitelisted the Cloud Run IP, but it's a shared IP (other service can use the same!! Be careful) and it's not always the same, there is a pool of IP addresses used by Google cloud.
How do I confirm that my VM connects to my GCP VPN Gateway? The two are already on the same network. I have tried pinging to the VPN Gateway IP from the vm but I cannot.
You would have to review and make sure that:
The VPN is active under Cloud VPN
Ensure that your GCP and on-prem firewall are allowing ingress/egress traffic between them
Depending on the type of VPN you choose, make sure that the IP address of the VM is shared to your on-prem via BPG, Route or Policy
If you see an issue with the VPN, you can review the VPN logs logs via logging (log viewer) and choose GCE Router. https://cloud.google.com/logging/docs/view/overview
If the issue is with the BGP/Route/Policy based, you would need to ensure your VPN IP is part of the shared range on both side (GCP and on-prem). https://cloud.google.com/network-connectivity/docs/vpn/concepts/overview#classic-vpn
If the issue is with Firewall, make sure that nothing is blocking your VM from communicating with your VPN IP range on GCP side and on your on-prem side. https://cloud.google.com/network-connectivity/docs/vpn/how-to/configuring-firewall-rules
Here is more troubleshooting you can review/try: https://cloud.google.com/network-connectivity/docs/vpn/support/troubleshooting
I have connected OpenVPN with Google Cloud VPC by following youtube video
https://www.youtube.com/watch?v=avYcoMKaXRc
Right now I am in Singapore and wants to connect Google Cloud India, setup is perfect but after successful connection if I try to open any website, on-one is opening, like there is no internet connection.
Can any one help me what is the issue? thanks
To monitor and solve common issues with Cloud VPN, follow the GCP Cloud VPN troubleshooting guide and OpenVPN troubleshooting.
Even the VPN gateways are configured correctly, there could be issue in the peer network between the hosts and the gateway, or with the network between the peer gateway and the Cloud VPN gateway.
Check for the Cloud VPN logs in Stackdriver Logging and verify these steps.
Verify connectivity between your host and Google Cloud VMs;
Are you able to reach peer VPN gateway from your host.
Verify that traffic is flowing between the two VPN gateways in both directions. In the VPN logs, check for reported incoming messages from the other VPN gateway.
Make sure to configure your peer VPN gateway to use MTU of no greater than 1460 bytes.
Check network throughput, both within GCP and your host.
Check if supported IKE ciphers are configured.
It's due to the ufw firewall that is active.
You can check the status by running the command
sudo ufw status
In the output you will see
Status: active
Now just execute
sudo ufw disable
This will disable the firewall and now try reconnecting to the server, internet will work.
is there a way to open ICMP on an Azure Pipeline vm? my CI unit tests are expected to send ping requests to the google DNS (8.8.8.8)
According to your description, you are trying to access one external IP through Azure VM endpoint with Ping. This does not allowed.
Please due to this official blog which written by our Azure VM team engineer: HOW TO ALLOW PING FUNCTIONALITY TO WINDOWS AZURE MACHINES?
The Ping functionality on Windows Azure VM is blocked by default for
security reasons.
As we all know, the ICMP protocol which used by Ping can measure the latency of the connection between a local machine and a remote machine. Any connections exceeding a default latency are deemed to be unavailable. See the pic shown below, the only possible connection to that Azure virtual machine is via the Internet. Any internet traffic which trying to enter the virtual network must pass through the load balancer, and this balancer is filtering ICMP traffic, allow UDP and TCP traffic.
By default, Azure denies and blocks all public inbound traffic to an
Azure virtual machine, includes ICMP traffic. This is a good thing
because it can improve security by reducing the attack surface.
Note: This restrict only apply to the network traffic which going through the external IP through configured endpoints. But if the network traffic occurred between internal IPs of VMs which in the same virtual network or in the same cloud service, ICMP would be allowed.
This restrict does not limited permanently. We can set firefall or azure security group to allow this. But, unfortunately, for Azure Devops Pipeline, the hosted agent is using the VM DS2_V2 and DS3_V2, which are all could not be configured\modified with firefall and security group by external users. If build\release with private agent, ICMP will not be limit. You can set a private agent, and execute ping test in it.
(Sometimes, can use VPN or ExpressRoute to skip the load balancer filter and limit. But I don't recommend to use this way)
Since Ping is a very convenient and critical tool for troubleshooting connectivity, we are reviewing and considering to expand this feature in Azure VM. There has a such suggestion ticket raised in our uservoice forum: Enable ICMP traffic to Azure VMs over the Internet. You can vote for it as well to push it faster into the development queue.