Hi i am new to GoogleDataProc and trying to create Custom Image for dataproc. I have followed all the steps as mentioned in documentation but while running it i got SuccessMatch found "BuildSucceeded: Dataproc Initialization Actions Succeeded." and after that i am getting following error:
Command From my local machine with gcloud sdk installed :
python generate_custom_image.py --image-name xyz --dataproc-version 1.2.45 --customization-script customization_script.sh --daisy-path ~/daisy --zone abc-center-b --gcs-bucket gs://xyz_bucket
And got error as :
Error running workflow: step "create-image" run error: googleapi: Error 400: Invalid value for field 'resource.family': 'None'. Must be a match of regex '(?:[a-z] (?:[-a-z0-9]{0,61}[a-z0-9])?)', invalid
Please help me find the cause.
It looks like the cause of this is a recent change.
Just add --family dataproc to your invocation.
I created a pull request to fix this.
Related
I was trying to deploy to CloudRun for the first time by following instruction on a blog (By Tom Ray : Deploying a NestJS app to Cloud Run with Github Actions).
In the section "Use gcloud run deploy", when giving the command gcloud run deploy I got this unclear error message:
Deployment failed
ERROR: (gcloud.run.deploy) Invalid value for [source]: Dockerfile required when specifying --tag
Searching the web I could not find an answer or hint to explain the error message.
After moving on to another tutorial I understood what my mistake was.
It was a simple "case sensitive" syntax error naming of the docker file.
I named it "dockerfile" and it should have been named "Dockerfile".
This is a unique naming convention from all the other filenames in the project, they are either all lowercase or all uppercase.
A better error message would have been "ERROR: Missing Dockerfile", that would have been enough for me to find the cause.
For some reason, I can no longer deploy existing google functions from my local machine or from github actions. Whenever I deploy using the gcloud functions deploy command, I get the following error in the console: ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Build failed: Unknown Syntax Error: Invalid option name ("--production=false"). I am not using a --production=false option in my gcloud deploy command, so I don't really understand where that is coming from.
Build logs always failing on:
Step #1 - "build": Unable to delete previous cache image: DELETE https://us.gcr.io/v2/{{projectId}}/gcf/{{region}}/{{guid}}/cache/manifests/sha256:{{imageId}}: GOOGLE_MANIFEST_DANGLING_TAG: Manifest is still referenced by tag: latest.
Deploy command:
gcloud functions deploy --runtime=nodejs16 --region=us-central1 {{function_name}} --entry-point={{node_function}} --trigger-topic={{topic_name}}
Attempted with the following gcloud versions and got the same result each time:
370, 371, 369, 360
I am not sure where this is coming from. I did not have this problem when I deployed just yesterday and it is not specific to my local machine.
This was due to a regression issue on Google's part. They released a fix for it today and deploys are working again now.
Issue: https://github.com/GoogleCloudPlatform/buildpacks/issues/175#issuecomment-1030519240
Hi I am trying to deploy a node application from cloud 9 to ELB but I keep getting the below error.
Starting environment deployment via CodeCommit
--- Waiting for Application Versions to be pre-processed --- ERROR: Pre-processing of application version app-491a-200623_151654 has
failed. ERROR: Some application versions failed to process. Unable to
continue deployment.
I have attached an image of the IAM roles that I have. Any solutions?
Go to your console and open up your elastic beanstalk console. Go to both applications and environments and delete them. Then in your terminal hit
eb init #Follow instructions
eb create --single ##Follow instructions.
It would fix the error, which is due to some application states which are failed. If you want to check those do
aws elasticbeanstalk describe-application-versions
I was searching for this answer as a result of watching a YouTube tutorial for how to pass the AWS Certified Developer Associate exam. If anyone else gets this error as a result of that tutorial, delete the 002_node_command.config file created in the tutorial and commit that change, as that is causing the error to occur.
A failure within the pre-processing phase, may be caused by an invalid manifest, configuration or .ebextensions file.
If you deploy an (invalid) application version using eb deploy and you enable the preprocess option, The details of the error will not be revealed.
You can remove the --process flag and enable the verbose option to improve error output.
in my case I deploy using this command:
eb deploy -l "XXX" -p
And can return a failure when I mess around with .ebextensions:
ERROR: Pre-processing of application version xxx has failed.
ERROR: Some application versions failed to process. Unable to continue deployment.
With that result I can't figure up what is wrong,
but deploying without -p (or --process)and adding -v (verbose) flag:
eb deploy -l "$deployname" -v
It returns something more useful:
Uploading: [##################################################] 100% Done...
INFO: Creating AppVersion xxx
ERROR: InvalidParameterValueError - The configuration file .ebextensions/16-my_custom_config_file.config in application version xxx contains invalid YAML or JSON.
YAML exception: Invalid Yaml: while scanning a simple key
in 'reader', line 6, column 1:
(... details of the error ...)
, JSON exception: Invalid JSON: Unexpected character (#) at position 0.. Update the configuration file.
Now I can fix the problem.
Have been working on a GCP project involving OCR. Have attempted to follow the tutorial here, but the first function crashes when I try to upload a file to the trigger bucket. Moreover, cloud shell will not allow me to set the env variable GCP_PROJECT; it returns
ERROR: (gcloud.functions.deploy) ResponseError: status=[400], code=[Bad Request], message=[The request has errors
Problems:
environment_variables:
environment variable name GCP_PROJECT is reserved by the system: it cannot be set by users
]
Any suggestions?
I am not 100% sure, but after some work, I am guessing that the answer is that the .json file with the service account being used in the function had to be exported as GOOGLE_APPLICATION_CREDENTIALS in the gcloud SDK when deploying the function.
I got the same error when following this tutorial.
Changing
--set-env-vars "^:^GCP_PROJECT=my_proj:TRANSLATE_TOPIC
to
--project my_proj --set-env-vars "^:^TRANSLATE_TOPIC fixed it.
I have the same error.
I just follow their own tutorial and there is nothing special or customized.
Actually I got errors in every step of the tutorial and search the internet to fix it.
However I stuck on this one.
I'm following this tutorial: https://cloud.google.com/kubernetes-engine/docs/tutorials/hello-app for Google Cloud Platform. I'm using the Google Cloud Shell command line. When I got to the step:
To build the container image of this application and tag it for uploading, run the following command:
docker build -t gcr.io/${PROJECT_ID}/hello-app:v1 .
I get an error:
invalid argument "gcr.io//hello-app:v1" for "-t, --tag" flag: invalid reference format
Bear in mind I already have 3 instances cluster (created from Kubernetes Engine) and one VM instance created on its own, existing in my VM instances, created from previous tutorials. Not sure if this has anything to do with the error.
Thanks in advance.
You missed setting PROJECT_ID. In the the "Before you begin" section of the tutorial you linked to it has you run
gcloud config set project [PROJECT_ID]
and then in Step 1 you run
export PROJECT_ID="$(gcloud config get-value project -q)"
After those two commands you should have the shell variable set correctly.
I also got the same error when running
docker build -t gcr.io/${PROJECT_ID}/hello-app:v1 .
but changing it to (my PROJECT_ID is say deepworld123)
docker build -t gcr.io/deepworld123/hello-app:v1 .
fixed it for me. Even though i did set PROJECT_ID=deepworld123.
Your tutorial link doesn't work (it's a link to a GCP dashboard, not a tutorial), but presumably there was a step where you were supposed to set the PROJECT_ID variable, which you skipped. The error message shows nothing between the two slashes where ${PROJECT_ID} appears in your command.
Had a very similar issue involving PROJECT_ID not being set correctly. The solution has to deal with formating as the error message says.
My PROJECT_ID string has the following format companyname.com:companyname-1 After I followed all the steps in the accepted answer the error message was the same.
It turns out the : needs to be replaced by a /. The final gcr.io string looks like:
gcr.io/companyname.com/companyname-1/hello-app:v1