I want to make a connection from Airflow to Google Cloud. I have made a service account in Google Cloud, downloaded the JSON file, and saved the JSON file inside the docker.
I am following this https://cloud.google.com/composer/docs/how-to/managing/connections documentation, but it returns the following error:
HTTPSConnectionPool(host='oauth2.googleapis.com', port=443): Max
retries exceeded with url: /token (Caused by
NewConnectionError('<urllib3.connection.HTTPSConnection object at
0x7f765dfbeb10>: Failed to establish a new connection: [Errno 113] No
route to host'))
Am I doing something wrong or is there an update on the documentation?
Nb:
I use Apache Airflow 2.4.2
I already tested the service account JSON file and it works perfectly to upload data using Python
You can create a connection to Google Cloud from Airflow webserver admin menu. In this menu you can pass the Service Account key file path.
In this picture, the keyfile Path is /usr/local/airflow/dags/gcp.json.
Beforehand you need to mount your key file as a volume in your Docker container with the previous path.
You can also directly copy the key json content in the Airflow connection, in the keyfile Json field :
You can check from these following links :
Airflow-connections
Airflow-with-google-cloud
Airflow-composer-managing-connections
Related
We are facing an issue on our application which uses google big query with OAuth authentication. When we run the application on docker container during runtime we get an authentication URL on the docker console like below :
Please open the following address in your browser:
https://accounts.google.com/o/oauth2/auth?access_type=offline&approval_prompt=auto&client_id=330834658013-dg6niu0iihdcatin3c3mi6haqebrjinm.apps.googleusercontent.com&redirect_uri=http://localhost:61984/Callback&response_type=code&scope=https://www.googleapis.com/auth/bigquery%20https://www.googleapis.com/auth/cloud-platform%20https://www.googleapis.com/auth/drive
If we open the URL outside the container in the browser. We do not get the response and application get failed.
We tried to run the application other than docker container, we found that It automatically redirect to the browser and successfully get the response after that an folder named as 'credentials' containing StoredCredentials file is created locally which get referred next time for authentication.
As described above, due to authentication is not done by URL on docker container. The credential folder is also not created. Could you explain the resolution on priority.
We tried to run the application other than docker, It is running successfully. As it automatically redirect the URL and completes the authentication. Same is expected on docker container as well.
I have uploaded WSO2 IoTS 3.1.0 to my remote virtual machine with static IP address.
I have unzipped zip file then run sudo ./change-ip.sh
In first step I have entered localhost
In second step I have entered my_own_ip_address
Then my hostname is my_own_ip_address
When I launch IoT Server and open device management then I login where it shows me this page:
When I try to open any other page and even try to logout it redirects me to this page.
My logs shows this:
TID[-1234] [IoT] [2017-09-09 08:32:30,079] ERROR {auth-module} -
org.wso2.carbon.apimgt.integration.client.exception.APIMClientOAuthException: failed to retrieve oauth token using jwt
org.mozilla.javascript.Context.throwAsScriptRuntimeEx(Context.java:1754)
org.mozilla.javascript.MemberBox.invoke(MemberBox.java:148)
org.mozilla.javascript.NativeJavaMethod.call(NativeJavaMethod.java:225)
org.mozilla.javascript.optimizer.OptRuntime.callProp0(OptRuntime.java:85)
It shows that there is an error in jaggery JS.
What did I miss?
One reason for the issue could be that you are having a DNS entry against that IP in /etc/hosts file. Please make sure that your /etc/hosts file does not contain any DNS entry pointing your IP to a hostname. When WSO2 IoT Server tries to resolve DNS to call the endpoints, the first thing it looks at is your local DNS setting. If your IP is pointing to a hostname, it takes that host name as the server hostname. And when IoT server tries to verify the common name of the SSL certificate, it gets conflicting information. This leads to hostname verification fail issue. And that could result in JWT token failure as it cannot access the endpoints.
Similar issue was discussed here
I'm trying to deploy a parse application according to this document https://aws.amazon.com/it/blogs/mobile/how-to-set-up-parse-server-on-aws-using-aws-elastic-beanstalk/ .
I deployed the application in aws, created a mongolab cluster, copied the connection string in the environment panel and started the application.
If I try to test using curl according the aws tutorial I receive an interval server error.
How can I understand what's the problem?
I have a web service(asmx) and it has a certificate for SSL. I can access this web service and see whole methods. I want to create a proxy by command prompt and I am using below code.
wsdl MyServiceAdres?wsdl
and this is the error message:
There was an error downloading 'MyServiceAdres?wsdl'.
The request was aborted: Could not create SSL/TLS secure channel.
I solved this problem. You have to save wsdl file to your local and use below code.
wsdl.exe myWebService.wsdl myXsd1.xsd myType1.xsd myXsd2.xsd ...
I'm running Ws02 products in docker containers
I have a strange problem. I publish an API in publisher, subscribe to it using the store and access the API using the curl command. It works fine. The DB is also running in a separate mysql container and Im persisting the DB data to a volume. However, if I shutdown the containers (docker-compose down) , bring it back up and access the API again, I get a 404 error with the following errordetails .Any idea?
errorCode":900000
errorDescription":"Not Found""
errorDetails":"The requested resource is not available
404
In API Manager, some API data is persisted in the File system as well. You can see the artifacts located at "/repository/deployment/server/synapse-configs/default/api. You need to make sure this is not replaced when the docker container is restarted.