I am attempting to take my existing cloud composer environment and connect to a remote SQL database (Azure SQL). I've been banging at my head at this for a few days and I'm hoping someone can point out where my problem lies.
Following the documentation found here I've spun up a GKE Service and SQL Proxy workload. I then created a new airflow connection as show here using the full name of the service azure-sqlproxy-service:
I test run one of my DAG tasks and get the following:
Unable to connect: Adaptive Server is unavailable or does not exist
Not sure on the issue I decide to remote directly into one of the workers, whitelist that IP on the remote DB firewall, and try to connect to the server. With no command line MSSQL client installed I launch python on the worker and attempt to connect to the database with the following:
connection = pymssql.connect(host='database.url.net',user='sa',password='password',database='database')
From which I get the same error above with both the Service and the remote IP entered in as host. Even ignoring the service/proxy shouldn't this airflow worker be able to reach the remote database? I can ping websites but checking the remote logs the DB doesn't show any failed logins. With the generic error and not many ideas on what to do next I'm stuck. A few google results have suggested switching libraries but I'm not quite sure how, or if I even need to, within airflow.
What troubleshooting steps could I take next to get at least a single worker communicating to the DB before moving on the the service/proxy?
After much pain I've found that Cloud composer uses ubuntu 1804 which currently breaks pymssql as per here:
https://github.com/pymssql/pymssql/issues/687
I tried downgrading to 2.1.4 to no success. Needing to get this done I've followed the instructions outlined in this post to use pyodbc.
Google Composer- How do I install Microsoft SQL Server ODBC drivers on environments
Related
I'm trying to run my application on GCE VM. It uses nodeJs as frontend and a Java backend. I use this server to communicate with my local computer using MQTT. This is working but after some time (one hour and a half), the server seems to go to sleep (or the ports close ?).
Both MQTT and ssh terminal interface connections are lost.
When I connect back, the application is not running anymore, it seems like the VM restarted.
Do you have any idea on how to keep my server alive ? I can give further details.
Answering my own question as John Hanley explained the solution in comments:
"By what method are you running your frontend and backend code. VMs do not go to sleep. If you are starting your software via an SSH session, that will be an issue. Your code needs to run as a service. Edit your question with more details."
I was indeed running my application via the ssh terminal which caused the problem. The solution for me was to remotely access the VM via vncserver and to launch the application using the VM's terminal.
I have a Google Cloud compute instance running with Ubuntu 18. We had wireshark running tracking another problem and we noticed that every minute something is accessing the meta data server. Three requests every minute:
GET /computeMetadata/v1/instance/virtual-clock/drift-token?alt=json&last_etag=XXXXXXXXXXXXXXXX&recursive=False&timeout_sec=60&wait_for_change=True
GET /computeMetadata/v1/instance/network-interfaces/?alt=json&last_etag=XXXXXXXXXXXXXXXX&recursive=True&timeout_sec=60&wait_for_change=True
GET /computeMetadata/v1/?alt=json&last_etag=XXXXXXXXXXXXXXXX&recursive=True&timeout_sec=77&wait_for_change=True
In call cases, the wireshark says the source is the IP of my instance, and the destination is the 169.254.169.254 which is the Google metadata server.
I don't have any code we have written that is accessing the server. The first one makes me think that this is some Google specific software that is accessing the meta data? But I haven't been able to prove that. What is worrisome is that the response for the third one contains ssh keys. Also, every minute seem excessive.
I see another post talking about scripts in /usr/share/google, but I don't have that directory. I do see that google-fluent is installed. I also see a installed snap for google-cloud-sdk. Could one of those be it? I don't recall installing them, AFAIK, I am not using it, so if that is it, what is the harm in uninstalling it?
You do not have a problem to worry about. The metadata server is private to your instance. The Google VM guest environment software and Stackdriver (fluentd) are making requests to the metadata server to get credentials, detect changes (new SSH keys), set the clock, etc.
The IP address 169.254.169.254 is an IPv4 Link Local Address. Only your VM has a route to that network.
Compute Engine Guest Environment
Do not attempt to uninstall the Guest Environment. You can remove Stackdriver, but I do not recommend that. Stackdriver provides logging and monitoring features that are very useful.
I created a Redis at GCP. Note it is GCP service, not a Redis created using compute engine. But when I run redis-cli and then run CLIENT command, like "client list", "client kill", etc. it showed:
(error) ERR unknown command 'client', with args beginning with:
Look like it does not support command about CLIENT, and maybe some other commands as well.
If I run "info" in the redis-cli, it shows the version is: redis_version:4.0.14
How to have the GCP managed Redis run command of "client"? I need to disconnect with all clients so I want to run "client kill".
if "client kill" is not available, can I use "shutdown" as a workaround? I am not sure if shutdown will terminate the redis or just stop it. if it just stops the redis and no data is lost, then I can use it as well.
Your issue is with CLIENT LIST command which currently is not available in Memorystore for Redis. Memorystore for Redis is a managed service and comes with some constraints. So some commands that interfere with a managed Redis service are blocked as per documentation.
As a workaround you can use the MONITOR command which is available for instances created after November 4, 2019. From the Redis documentation the MONITOR is a debugging command that streams back every command processed by the Redis server.
MONITOR will show the clients sending traffic to Redis server, as mentioned in this SO answer.
There is a PIT for this issue as Feature request. Please feel free to post there if you have further concern on that issue.
I have HDFS-HA(namenode high availability) setup in my hadoop cluster(using Apache Ambari).
Now, I have one scenario in which my ambari-server machine(which also consist one Namenode i.e. active/Primary) went offline so that my other Namenode(Standby) was active and running but after some time it went offline too for some reason.Services were offline I mean.I was unable to do any operation.What if I have to start the services manually that is used to start using ambari.
I mean using command-line or something
Services can be started from the command line but they should not be in an Ambari environment typically. This is because Ambari does more then just start the service when you issue the start/restart command for any given service. Ambari also makes sure the most up to date configuration is written to each node along with other various house keeping type tasks.
You can look at the logs in Ambari when you start/restart a service to see exactly what Ambari does with respect to writing the configuration, other house keeping, and the exact command to start/restart the given service.
After installing VMware Server I get the following error when I try to access the VMware web-based server manager:
The VMware Infrastructure Web Service
at "http://localhost:8222/sdk" is not
responding
Go into the services manager and check that the 'VMware Host Agent' service is running. If not, then start it and then try browsing to the site again.
Vmware Hostd was not working for me either.
However, in trying to start the service it stopped automatically. Typically when this happens it is because there is an error in your config.xml.
C:\ProgramData\VMware\VMware Server\hostd\config.xml
In my case, checking the logs at:
C:\ProgramData\VMware\VMware Server
showed it erroring out after "Trying hostsvc".
Searching the config.xml for hostsvc showed references to several things, the first thing was the datastore. In checking my datastores.xml file:
C:\ProgramData\VMware\VMware Server\hostd\datastores.xml .
I found it full of all sorts of random characters instead of a properly formed XML document.
Renaming datastores.xml to datastorex.xml.bad allowed me to start the service. At which point I had to add back my datastores through the GUI.
Hopefully this will help someone else out. I did not find any other references in Google to this issue.
Try accessing via "http://localhost:8222" without the /sdk. You can also try the secure site via "https://localhost:8333".