Our project is using a Serverless VPC access connector to allow access to DB over private IP from cloud functions and cloud runs. It was working flawlessly for a few months, but today I tried to deploy one of the functions that use such a connector and I got the message:
VPC connector
projects/xxxx/locations/us-central1/connectors/vpc-connector is not
ready yet or does not exist. Please visit
https://cloud.google.com/functions/docs/troubleshooting for in-depth
troubleshooting documentation.
I went to the Serverless VPC access view and found out that indeed the connector has a red marking on it. When I hover on it it says
Connector is in a bad state, manual deletion recommended
but I don't know for what reason, Link to logs doesn't show anything for the past 3 months.
I tried to google about the such error but without success.
I also tried to search through logs but also didn't find anything relevant.
I'm looking for any hints:
Why it happened?
How to fix it? I don't want to recreate the connector, it is related to many functions, and cloud runs
As the issue was blocking us from the deployment of cloud functions I was forced to recreate the connector.
But this time API returned an error:
Error: Error waiting to create Connector: Error waiting for Creating Connector: Error code 7, message: Operation failed: Google APIs Service Agent (<PROJECT_NUMBER>#cloudservices.gserviceaccount.com) needs editor role in the project.
After adding such permission old connector started to work again...
Before there was no such requirement, but it changed in meantime.
Spooky, one time something works other not.
Related
First of all I find google's cloud docs lacking and somewhat incorrect a fair bit of the time.
I am attempting to connect from a cloud function to a cloud sql database and I have having endless issues.
Here is the connection error
"Internal error looking up Cloud SQL instance "project:region:database/.s.PGS""
Error: connect ENOENT /cloudsql/project:region:database/.s.PGSQL.5432
I am able to connect to said database locally with the public ip address and code is all working fine, but when deployed it doesn't work at all.
What I have...
Project A - This has the database in australia-souteast1 region.
Project B - This has all the other logic, also in australia-southeast1
(the database is legacy, hence why its in a different project).
I have a cloud schedule task that triggers a pubsub, which inturn triggers the cloud function. This process works, and is logging what it should, this is also where I am seeing the can't connect error.
Connection host is /cloudsql/projectId:region:database (coppied from the cloud sql connection page, so I know that isn't the issue).
I have also enabled Cloud Sql API and Cloud Sql Admin Api on both Project A and Project B and still no luck.
I have also tried with the default service account by adding the Cloud Sql Client permission in Project B and then adding Project B's default service account into Project A with Cloud Sql Client permissions.
Failing that, I then created a new service account in Project B and gave it Owner permissions and then added that user to Project A with Owner permissions also, I am still getting this error.
I really have no clue now as to what is going on.
We have app engines on Project B connecting to Project A without any issues, I am really confused.
Here is the stack driver error
And my be connection details via an .env file
UPDATE:
Changing the database to a different database instance in Project A seems to connect, so it is looking like it is possibly a problem with the database instance.
Database 1 is working and I can connect to.
Database 2 is the one that I can not get to work.
Database 2 is a clone of Database 1
In this case, the docs are absolutely correct, but you are using the wrong filepath. The unix socket is located at /cloudsql/project:region:database/.s.PGSQL.5432, not /cloudsql/project:region:database/.s.PGS/.s.PGSQL.5432.
I wanted to build a Google assistant with custom actions using actions-sdk. Since I am new to this, I have followed the steps in the tutorial "Build Actions for Google Assistant using Actions SDK (Level 1)" as it is, inorder to build a sample assistant. I followed the tutorial as it is. However, in step 5(Implement fulfillment), when trying to test the the fulfillment by running the command
gactions deploy preview
I am getting the below output in the terminal with error
Sending configuration files...
Sending resources...
Waiting for server to respond. It could take up to 1 minute if your cloud function needs to be redeployed.
[ERROR] Server did not return HTTP 200.
{
"error": {
"code": 400,
"message": "Asset 'webhooks/ActionsOnGoogleFulfillment' cannot be deployed. [An operation on function cf-_CcGD8lKs_F_LHmFYfJZsQ-name in region us-central1 in project <my-project-id> is already in progress. Please try again later.]"
}
}
And when I checked the "Google Cloud Platform -> Cloud Functions Console" for this project, the following is seen.
Image 1(Screenshot)
Cloud Platform Cloud Functions Console
A failed deploy of cloud function with an exclamation mark. And if I delete that functions, then immediately a new function is deployed automatically. But instead of an exclamation mark, a spinning wheel symbol(loading/still deploying) mark is present. I cannot delete that cloud function if it is still loading/deploying. Then after 10-15 min, the spinning symbol changes to exclamation symbol. And then if I delete it, then again a new one automatically appears. And it goes on like this
Image 2 (Screenshot)
Cloud Platform Cloud Function Console
This problem arises only when implementing a webhook/fulfillment(Step 5). For static Actions' response, it successfully deploys for testing on entering the command "gactions deploy preview".(Step 1 to Step 4 are successfully implemented)
I have followed the tutorial as it is, hence the code and directory structure is the same as in tutorial,(only the project-id or actions-console project name will be different).
Github Repository for Code
Since, this is only for the tutorial, at present I am not using a billing account, instead did the following changes in package.json(changed node version from 10 to 8.).
"engines": {
"node": "8"
},
Due to this continuous automatic failed deployment, when I try to explicitly deploy the project, as mentioned above, this error occurs.
"An operation on function cf-_CcGD8lKs_F_LHmFYfJZsQ-name in region us-central1 in project <my-project-id> is already in progress. Please try again later".
Can anyone please suggest how to stop this continuous automatic failed deployment of the cloud functions, so that the function I deploy will be successfully deployed? Would really appreciate your help.
(Note: This is the first time I have posted a question in stack overflow, so please let me know if there are any mistakes or stack overflow question conventions I might not have followed. I will improve it.)
Posting this as Community Wiki as it's based in the comments.
As clarified the issue seems to be the billing account, as the tutorial mentions that it's necessary to have one set for the Cloud Functions to be deployed correctly. Besides that, it's not possible to deploy Cloud Functions (webhooks) without a billing account, so yes, even though that you are not using Node.js 10, you will need to have a billing account configured for your project.
To summarize, billing account will be needed to avoid any possible deployment failure, even if you are not using Node.js 10, as explained in the followed tutorial.
I am trying to use a development Endpoint to interactively run and edit ETL scripts but there seems to some issues in the development endpoint just after creating it as i am getting errors in scala/python REPL and also unable to do SSH tunnel to remote interpreter.
Let me explain what i did exactly - I created a development endpoint in the AWS console with all the default configurations. While creating the development endpoint i only provided three things 'Development endpoint name' and 'IAM Role' and my 'pub ssh key'. This is how it looks after creation
Then Right After creating the endpoint i am connecting to the spark/python REPL, I am able to connect to them successfully but within couple of minutes of connecting, the REPL starts throwing errors without writing a single line of code. This is happening in all the REPL present in the development endpoints.
Also When I am trying to do SSH tunneling to remote interpreter to connect my Local Zeppelin Notebook it is throwing - "bind: Cannot assign requested address".
Couple of things that are working though -
Able to do ssh to the endpoint.
Created a Sagemaker notebook in the AWS glue that is attached to this development endpoint and this notebook seems to be working fine, although surely it is adding an additional cost and i don't want to continue using it.
Can anyone please help what am i doing wrong? Am I missing any important steps that is needed to be done on the machine right after creating the development endpoint?
Thanks in Advance!
Not very sure about this error but if you are using it smaller datasets then probably you would like to use Docker implementation as it will not add any additional cost and you can go on with your developments.
You can refer this blog on how to set it up
https://towardsdatascience.com/develop-glue-jobs-locally-using-docker-containers-bffc9d95bd1
when i'm trying to deploy firebase function from my local machine i'm getting this error.
functions: failed to create function dialogflowFirebaseFulfillment
HTTP Error: 400, Default service account 'project-id#appspot.gserviceaccount.com' doesn't exist. Please recreate this account (for example by disabling and enabling the Cloud Functions API), or specify a different account.
and the project that i'm trying to deploy is, https://github.com/actions-on-google/codelabs-nodejs/tree/master/level1-complete
It seems your service account is removed. You may want to check whether your firebase & actions on google projects are removed or not.
If they are not, check for service accounts on console.cloud.google.com and make sure all your accounts are same as you are trying to deploy. (firebase, dialogflow, app-engine etc.) Also, disabling and enabling the Cloud Functions API may help as mentioned in error.
I notice that your error has 'project-id#appspot.gserviceaccount.com'.
Shouldn't the project-id be your {project-id} from the google action that you created, and not the word project-id.
I have been using the Google Cloud speech recognition service for some time, through a python application.
Due to accidentally copying my Google Cloud json file to a GitHub shared location (I was doing a backup), I suddenly got a warning from Google Cloud that I was violating the rules as json is private. Then, I promptly removed the file, but nevertheless, I got an email saying that my resources for my project "santo1" were suspended, saying some reasons of "cryptocurrency mining" which I have no idea.
I applied to reactivate and my appeal was accepted promptly, saying that my resources about santo1 were reinstated.
Unfortunately, the speech recognition still didn't work.
Launching it from python, it records from the microphone but no answer from the service - and no error messages at all.
Then I attempted the following:
regenerate API
create a new json
create a new project with its own json under my same google account
as suggested by the Google Cloud chat operator, I manually clicked play to the VM resource that appeared stopped
create a new gmail account, with another new project, setup with billing and everything (also reconfigured through "gcloud init")
None of these attempts worked.
I need assistance on this, as the chat operator didn't seem capable of telling me more.
Thank you in advance
Best regards
I would recommend you to contact GCP support for this case as your cloud service could be still in suspended status regardless your access is OK
Apparently, the access key is stolen and used by hackers and they did crypto mining using your GCP account, hence your service account was banned
If it's your testing account/project, you should consider to create a new project rather than continue with it, the hacker could create some other services which you may not realize until too late
Worse case is it's your PROD service, then you'd better review the bill and transaction report thoroughly