Cloud SQL creation with Deployment Manager - "Precondition check failed." error - google-cloud-platform

I'm using the gcp-types/sqladmin-v1beta4:instances Resource Type to create a Cloud SQL instance using the Deployment Manager and I'm getting the error below:
{
"ResourceType":"gcp-types/sqladmin-v1beta4:instances",
"ResourceErrorCode":"400",
"ResourceErrorMessage":{
"code":400,
"message":"Precondition check failed.",
"status":"FAILED_PRECONDITION",
"statusMessage":"Bad Request",
"requestPath":"https://www.googleapis.com/sql/v1beta4/projects/[PROJECT_NAME]/instances",
"httpMethod":"POST"
}
}
Here's the configuration inside the JINJA file:
{% set deployment_name = env['deployment'] %}
{% set INSTANCE_NAME = deployment_name + '-instance' %}
resources:
- name: {{ INSTANCE_NAME }}
type: gcp-types/sqladmin-v1beta4:instances
properties:
region: us-central1
rootPassword: root
settings:
tier: db-n1-standard-1
backupConfiguration:
binaryLogEnabled: true
enabled: true
- name: demand_ml_db
type: gcp-types/sqladmin-v1beta4:databases
properties:
name: demand_ml_db
instance: $(ref.{{ INSTANCE_NAME }}.name)
charset: utf8

The FAILED_PRECONDITION error - while not very descriptive, tends to be thrown when you're attempting to deploy over a previous Cloud SQL instance that was recently deleted; as a matter of fact, the instance you selected for deletion is not cleaned up instantly. There's an Issue Tracker thread regarding this here.
I was able to verify this on my end as well. The deployment using the JINJA file you've specified worked fine at first, but when I deleted it, and re-deployed - I received the same error.
The most simple approach is to try using a different deployment (or instance) name.

Related

Unable to delete and upload object containg bucket in GCP using deployment manager

I am using this Deployment Manager jinja template to create a bucket and upload files into the bucket.
resources:
- type: storage.v1.bucket
name: {{ properties['bucket_name'] }}
properties:
location: {{ properties['region'] }}
- name: {{ properties['build_name'] }}
action: gcp-types/cloudbuild-v1:cloudbuild.projects.builds.create
metadata:
runtimePolicy:
- CREATE
properties:
steps:
- name: gcr.io/cloud-builders/git
args: ['clone', 'https://<token>#github.com/{{ properties['username'] }}/{{ properties['repo_name'] }}.git']
- name: gcr.io/cloud-builders/gsutil
args: ['-m', 'cp', '-r', '{{ properties['repo_name'] }}/{{ properties['file_path_name_in_repo'] }}*', 'gs://{{ properties['bucket_name'] }}/']
timeout: 120s
I have taken idea of above code from How to write a file to Google Cloud Storage using Deployment Manager?
While running
gcloud deployment-manager deployments create name --config=file.yaml
then it is working fine, but while executing
gcloud deployment-manager deployments delete name
it is showing that cannot delete a bucket which contains files. Also
gcloud deployment-manager deployments update name --config=file.yaml
command is not working and showing that update option is not supported for Cloud Build.
My goal is to create a bucket and uploads files into that from github. Also if required then I can delete that and update (to put updated file) that using deployment manager template.
I have been told to do that using jinja or yaml template.
It will be good if you give a look into this and clear my problem on that.

GCP Deployment Manager "ResourceErrorCode":"400" while database user creation

I am experimenting with deployment manager and each time I try to deploy an SQL instance with a DB on it and 2 users; some of the tasks are failing. Most of the time they are the users:
conf.yaml:
resources:
- name: mycloudsql
type: gcp-types/sqladmin-v1beta4:instances
properties:
name: mycloudsql-01
backendType: SECOND_GEN
instanceType: CLOUD_SQL_INSTANCE
databaseVersion: MYSQL_5_7
region: europe-west6
settings:
tier: db-f1-micro
locationPreference:
zone: europe-west6-a
activationPolicy: ALWAYS
dataDiskSizeGb: 10
- name: mydjangodb
type: gcp-types/sqladmin-v1beta4:databases
properties:
name: django-db-01
instance: $(ref.mycloudsql.name)
charset: utf8
- name: sqlroot
type: gcp-types/sqladmin-v1beta4:users
properties:
name: root
host: "%"
instance: $(ref.mycloudsql.name)
password: root
- name: sqluser
type: gcp-types/sqladmin-v1beta4:users
properties:
name: user
instance: $(ref.mycloudsql.name)
password: user
Error:
PS C:\Users\user\Desktop\Python\GCP> gcloud --project=sound-catalyst-263911 deployment-manager deployments create dm-sql-test-11 --config conf.yaml
The fingerprint of the deployment is TZ_wYom9Q64Hno6X0bpv9g==
Waiting for create [operation-1589869946223-5a5fa71623bc9-1912fcb9-bc59aafc]...failed.
ERROR: (gcloud.deployment-manager.deployments.create) Error in Operation [operation-1589869946223-5a5fa71623bc9-1912fcb9-bc59aafc]: errors:
- code: RESOURCE_ERROR
location: /deployments/dm-sql-test-11/resources/sqluser
message: '{"ResourceType":"gcp-types/sqladmin-v1beta4:users","ResourceErrorCode":"400","ResourceErrorMessage":{"code":400,"message":"Precondition
check failed.","status":"FAILED_PRECONDITION","statusMessage":"Bad Request","requestPath":"https://www.googleapis.com/sql/v1beta4/projects/sound-catalyst-263911/instances/mycloudsql-01/users","httpMethod":"POST"}}'
- code: RESOURCE_ERROR
location: /deployments/dm-sql-test-11/resources/sqlroot
message: '{"ResourceType":"gcp-types/sqladmin-v1beta4:users","ResourceErrorCode":"400","ResourceErrorMessage":{"code":400,"message":"Precondition
check failed.","status":"FAILED_PRECONDITION","statusMessage":"Bad Request","requestPath":"https://www.googleapis.com/sql/v1beta4/projects/sound-catalyst-263911/instances/mycloudsql-01/users","httpMethod":"POST"}}'
Console View:
It doesn`t say what that precondition failing is or am I missing something?
It seems the installation of database is not completed by the time the Deployment Manager starts to create users despite the reference notation is used in the YAML code to take care of dependencies. That is why you receive the "FAILED_PRECONDITION" error.
As a workaround you can split the deployment into two parts:
Create a CloudSQL instance and a database;
Create users.
This does not look elegant, but it works.
Alternatively, you can consider using Terraform. Fortunately, Cloud Shell instance is provided with Terraform pre-installed. There are sample Terraform code for Cloud SQL out there, for example this one:
CloudSQL deployment with Terraform

Connecting function to empty cloud storage bucket

I am trying to use the following info from Google deployment manager examples on GitHub.
empty_bucket_in_function.yaml
empty_bucket_cf.yaml
imports:
- path: empty_bucket_cf.jinja
resources:
- name: my-function
type: empty_bucket_cf.jinja
properties:
project: <PROJECT_NAME>
region: europe-west1
entryPoint: handler
runtime: nodejs8
bucket: lskflsjfsj
empty_bucket_cf.jinja
{% set BUCKET = properties['bucket'] + '-bucket' %} resources:
#- type: cloudfunctions.v1.function
- type: gcp-types/cloudfunctions-v1:projects.locations.functions name: my-function properties:
parent: projects/{{ properties['project'] }}/locations/{{ properties['region'] }}
location: {{ properties['region'] }}
function: my-{{ properties['bucket'] }}
sourceArchiveUrl: gs://$(ref.{{ BUCKET }}.name)/my-function
entryPoint: {{ properties['entryPoint'] }}
runtime: {{ properties['runtime'] }}
eventTrigger:
resource: $(ref.my-topic.name)
eventType: providers/cloud.pubsub/eventTypes/topic.publish
#- type: pubsub.v1.topic
- type: gcp-types/pubsub-v1:projects.topics name: my-topic properties:
topic: {{ properties['bucket'] }}-topic
#- type: storage.v1.bucket
- type: gcp-types/storage-v1:buckets name: {{ BUCKET }} properties:
predefinedAcl: projectPrivate
projection: full
location: US
storageClass: STANDARD
While deploying using deployment manager I am getting error as
testsetup has resource warnings
my-function: {"ResourceType":"gcp-types/cloudfunctions-v1:projects.locations.functions","ResourceErrorCode":"400","ResourceErrorMessage":"Failed to retrieve function source code"}
Deployment properties
Any idea why this is not a bug in Google Cloud Platform GitHub repository. Isn't it the purpose of empty_bucket config to create CFs with empty bucket.
Note: Sometimes it executes successfully as well.
I don't what Google had in mind when they publish this example, but it can't work. If your bucket is empty, the function has no code. However, when you deploy a function, the code is compiled/parsed, the entry-point checked (exists, correct signature,...), and deployed on the environment.
Here, no entry point, no code to compile/parse thus no deployment -> It's normal, but the example is disturbing. You can open an issue on the repos.

Deployment Manager cannot update instance templates - NO_METHOD_TO_UPDATE_FIELD

I have a deployment comprising a managed instance group and two instance templates (A and B). The deployment was initially created with the instance group referencing instance template A.
I tried updating the sourceImage in instance template B using deployment manager (gcloud beta deployment-manager deployments update my-deployment --template ...), but got the following error:
ERROR: (gcloud.beta.deployment-manager.deployments.update) Error in
Operation [operation-1538798895713-57787898f4ae9-8b478716-0bb72a09]:
errors:
- code: NO_METHOD_TO_UPDATE_FIELD
message: No method found to update field 'properties' on
resource 'fwp-app-preprod-instance-template-a' of type
'compute.v1.instanceTemplate'. The resource may need to be
recreated with the new field.
I should make it clear that the only change I made from the original deployment is the instance template's sourceImage.
Is it possible to perform an update of an instance template via deployment manager so that it references an updated sourceImage?
The error states clearly that the resource (instance template) may need to be recreated, and I'm happy for deployment manager to do that. But I have no idea how to instruct/force deployment manager to take that action.
I don't doubt it can be done outside of deployment manager, but I want to avoid configuration drift.
My app.jinja.schema:
imports:
- path: instance-group.jinja
- path: instance-template.jinja
My app.jinja:
resources:
- name: instance-template-a
type: instance-template.jinja
properties:
name: {{ env["deployment"] }}-instance-template-a
machineType: g1-small
sourceImage: "projects/my-project/global/images/my-image"
diskSizeGb: '30'
- name: instance-template-b
type: instance-template.jinja
properties:
name: {{ env["deployment"] }}-instance-template-b
machineType: g1-small
sourceImage: "projects/my-project/global/images/my-image"
diskSizeGb: '30'
- name: fwp-instance-group
type: instance-group.jinja
My instance-group.jinja:
resources:
- name: 'instance-group-{{ env["deployment"] }}'
type: compute.v1.regionInstanceGroupManager
properties:
baseInstanceName: ig-instance-{{ env["deployment"] }}
instanceTemplate: '$(ref.{{ env["deployment"] }}-instance-template-a.selfLink)'
targetSize: 1
region: australia-southeast1
- name: 'autoscaler-{{ env["deployment"] }}'
type: compute.v1.regionAutoscalers
properties:
autoscalingPolicy:
coolDownPeriodSec: 60
loadBalancingUtilization:
utilizationTarget: 0.9
maxNumReplicas: 10
minNumReplicas: 2
target: $(ref.instance-group-{{ env["deployment"] }}.selfLink)
region: australia-southeast1
And my instance-template.jinja
resources:
- name: {{ properties["name"] }}
type: compute.v1.instanceTemplate
properties:
name: {{ properties["name"] }}
description: ''
properties:
machineType: {{ properties["machineType"] }}
tags:
items:
- no-ip
- web-server
- http-server
- https-server
disks:
- type: 'PERSISTENT'
boot: true
mode: 'READ_WRITE'
autoDelete: true
deviceName: instance-device
initializeParams:
sourceImage: {{ properties["sourceImage"] }}
diskType: 'pd-standard'
diskSizeGb: {{ properties["diskSizeGb"] }}
canIpForward: false
networkInterfaces:
- network: projects/my-project/global/networks/vpc-fwp-nonprod
subnetwork: projects/my-project/regions/australia-southeast1/subnetworks/subnet-private-fwp-nonprod
aliasIpRanges: []
labels: { environment: {{ env["deployment"] }}, tenancy: "fwp-nonprod" }
scheduling:
preemptible: false
onHostMaintenance: MIGRATE
automaticRestart: true
nodeAffinities: []
serviceAccounts:
- email: some-service-account#developer.gserviceaccount.com
scopes:
- https://www.googleapis.com/auth/cloud-platform
To recap the comments:
The DM config includes an instance template for the managed instance group. The change of source image is attempting to change the image used in the template.
Unfortunately, instance templates are immutable once created
"So it is not possible to update an existing instance template or change an instance template after it has been created."
This explains the error message returned. The proper way to change the image you want to use for a Managed Instance Group is to create a new template and perform a rolling update on the group and using the new instance template.

CLOUD DEPLOYMENT MANAGER: Internal Load Balancer create issue

I am using the following to try and create an internal load balancer via Deployment Manager using the following code
- name: {{ env["name"] }}-port389-healthcheck
type: compute.v1.healthChecks
properties:
type: tcp
tcpHealthCheck: {
port: 389
}
- name: {{ env["name"] }}-port389-backend-service
type: compute.beta.backendService
properties:
healthChecks:
- $(ref.{{ env["name"] }}-port389-healthcheck.selfLink)
backends:
- group: $(ref.{{ env['name'] }}-master-instance-groups-managed.instanceGroup)
- group: $(ref.{{ env['name'] }}-slave-instance-groups-managed.instanceGroup)
protocol: TCP
region: {{ properties['region'] }}
loadBalancingScheme: INTERNAL
- name: {{ env["name"] }}-port389-forwarding-rule
type: compute.beta.forwardingRule
properties:
loadBalancingScheme: INTERNAL
ports:
- 389
network: default
region: {{ properties["region"] }}
backendService: $(ref.{{ env["name"] }}-port389-backend-service.selfLink)
It errrors when run with the following
Waiting for create operation-1478651694403-540d36cfdcdb9-cba25532-08697daf...failed.
ERROR: (gcloud.beta.deployment-manager.deployments.create) Error in Operation operation-1478651694403-540d36cfdcdb9-cba25532-08697daf:
errors:
- code: RESOURCE_ERROR
location: /deployments/forgerock/resources/forgerock-frontend-port389-backend-service-us-central1
message: 'Unexpected response from resource of type compute.beta.backendService:
400 {"code":400,"errors":[{"domain":"global","message":"Invalid value for field
''resource.loadBalancingScheme'': ''INTERNAL''. Load balancing scheme must be
external for a global backend service.","reason":"invalid"}],"message":"Invalid
value for field ''resource.loadBalancingScheme'': ''INTERNAL''. Load balancing
scheme must be external for a global backend service.","statusMessage":"Bad Request","requestPath":"https://www.googleapis.com/compute/beta/projects/carbide-tenure-557/global/backendServices"}'
It would appear to be creating using the https://www.googleapis.com/compute/beta/projects/carbide-tenure-557/global/backendServices instead of https://www.googleapis.com/compute/beta/projects/carbide-tenure-557/backendServices
I know this is beta functionality, but trying to develop this solution using GDM instead of a mixture of gcloud commands and GDM
For type instead of type: compute.beta.backendService use:
type: compute.v1.regionBackendService