I've started building an infrastructure using terraform. Within that TF configuration I am calling a module to be used by using relative path. This is successful in a classic release but I have been tasked with converting the pipeline to yaml. When I run terraform init step the agent finds the Tf config files but can't find the modules folder even though the artifact was downloaded on a previous task.
yaml file:
trigger:
- master
resources:
pipelines:
- pipeline: Dashboard-infra
project: Infrastructure
source: IT Dashboard
- pipeline: Infra-modules
project: Infrastructure
source: AWS Modules
trigger: true
stages:
- stage: Test
displayName: Test
variables:
- group: "Non-Prod Keys"
jobs:
- deployment:
displayName: string
variables:
region: us-east-1
app_name: it-dashboard
environment: test
tf.path: 'IT Dashboard'
pool:
vmImage: 'ubuntu-latest'
environment: test
strategy:
runOnce:
deploy:
steps:
- task: DownloadBuildArtifacts#1
inputs:
buildType: 'specific'
project: '23e9505e-a627-4681-9598-2bd8b6c1204c'
pipeline: '547'
buildVersionToDownload: 'latest'
downloadType: 'single'
artifactName: 'drop'
downloadPath: '$(Agent.BuildDirectory)/s'
- task: DownloadBuildArtifacts#1
inputs:
buildType: 'specific'
project: '23e9505e-a627-4681-9598-2bd8b6c1204c'
pipeline: '88'
buildVersionToDownload: 'latest'
downloadType: 'single'
artifactName: 'Modules'
downloadPath: '$(agent.builddirectory)/s'
- task: ExtractFiles#1
inputs:
archiveFilePatterns: 'drop/infrastructure.zip'
destinationFolder: '$(System.DefaultWorkingDirectory)'
cleanDestinationFolder: false
overwriteExistingFiles: false
- task: ExtractFiles#1
inputs:
archiveFilePatterns: 'Modules/drop.zip'
destinationFolder: '$(System.DefaultWorkingDirectory)'
cleanDestinationFolder: false
overwriteExistingFiles: false
- task: TerraformInstaller#0
inputs:
terraformVersion: '0.12.3'
- task: TerraformTaskV2#2
inputs:
provider: 'aws'
command: 'init'
workingDirectory: '$(System.DefaultWorkingDirectory)/$(tf.path)'
commandOptions: '-var "region=$(region)" -var "app_name=$(app.name)" -var "environment=$(environment)"'
backendServiceAWS: 'tf_nonprod'
backendAWSBucketName: 'wdrx-deployments'
backendAWSKey: '$(environment)/$(app.name)/infrastructure/$(region).tfstate'
Raw error log:
2021-10-29T12:30:16.5973748Z ##[section]Starting: TerraformTaskV2
2021-10-29T12:30:16.5981535Z ==============================================================================
2021-10-29T12:30:16.5981842Z Task : Terraform
2021-10-29T12:30:16.5982217Z Description : Execute terraform commands to manage resources on AzureRM, Amazon Web Services(AWS) and Google Cloud Platform(GCP)
2021-10-29T12:30:16.5982555Z Version : 2.188.1
2021-10-29T12:30:16.5982791Z Author : Microsoft Corporation
2021-10-29T12:30:16.5983122Z Help : [Learn more about this task](https://aka.ms/AA5j5pf)
2021-10-29T12:30:16.5983461Z ==============================================================================
2021-10-29T12:30:16.7253372Z [command]/opt/hostedtoolcache/terraform/0.12.3/x64/terraform init -var region=*** -var app_name=$(app.name) -var environment=test -backend-config=bucket=wdrx-deployments -backend-config=key=test/$(app.name)/infrastructure/***.tfstate -backend-config=region=*** -backend-config=access_key=*** -backend-config=secret_key=***
2021-10-29T12:30:16.7532941Z [0m[1mInitializing modules...[0m
2021-10-29T12:30:16.7558115Z - S3-env in ../Modules/S3
2021-10-29T12:30:16.7578267Z - S3-env.Global-Vars in ../Modules/Global-Vars
2021-10-29T12:30:16.7585434Z - global-vars in
2021-10-29T12:30:16.7589958Z [31m
2021-10-29T12:30:16.7597321Z [1m[31mError: [0m[0m[1mUnreadable module directory[0m
2021-10-29T12:30:16.7597847Z
2021-10-29T12:30:16.7599087Z [0mUnable to evaluate directory symlink: lstat ../Modules/global-vars: no such
2021-10-29T12:30:16.7599550Z file or directory
2021-10-29T12:30:16.7599933Z [0m[0m
2021-10-29T12:30:16.7600324Z [31m
2021-10-29T12:30:16.7600779Z [1m[31mError: [0m[0m[1mFailed to read module directory[0m
2021-10-29T12:30:16.7600986Z
2021-10-29T12:30:16.7601405Z [0mModule directory does not exist or cannot be read.
2021-10-29T12:30:16.7601808Z [0m[0m
2021-10-29T12:30:16.7602135Z [31m
2021-10-29T12:30:16.7602573Z [1m[31mError: [0m[0m[1mUnreadable module directory[0m
2021-10-29T12:30:16.7602768Z
2021-10-29T12:30:16.7603271Z [0mUnable to evaluate directory symlink: lstat ../Modules/global-vars: no such
2021-10-29T12:30:16.7603636Z file or directory
2021-10-29T12:30:16.7603964Z [0m[0m
2021-10-29T12:30:16.7604291Z [31m
2021-10-29T12:30:16.7604749Z [1m[31mError: [0m[0m[1mFailed to read module directory[0m
2021-10-29T12:30:16.7604936Z
2021-10-29T12:30:16.7605370Z [0mModule directory does not exist or cannot be read.
2021-10-29T12:30:16.7605770Z [0m[0m
2021-10-29T12:30:16.7743995Z ##[error]Error: The process '/opt/hostedtoolcache/terraform/0.12.3/x64/terraform' failed with exit code 1
2021-10-29T12:30:16.7756780Z ##[section]Finishing: TerraformTaskV2
I have attempted to even move the modules folder inside the tf.path so it is within the same folder as the tf config files and changed the location from "../" to "./". No matter what repo I extract the modules folder to (after downloading as artifact from another build pipeline) it cannot be found when calling it on the tf config files. I am fairly new to DevOps and would appreciate any help or just being pointed in the right direction.
Define system.debug: true variable at global level to enable debug logs - maybe something there will give you a hint:
variables:
system.debug: true
Apart from downloaded artifacts, do you expect to have files checked out from the repo the pipeline is defined in? The deployment job doesn't checkout git files by default, so you may want to add checkout: self to steps there.
Unable to evaluate directory symlink: lstat ../Modules/global-vars - this is suspicious, I wouldn't expect any symlinks in there. But maybe the error message is just misleading.
A useful trick is to log the whole directory structure.
You can do this with a bash script step (might need to apt install tree first):
- script: tree
Or with powershell (will work on MS-hosted linux agent):
- pwsh: Get-ChildItem -Path '$(agent.builddirectory)' -recurse
Related
Introduction
I have a repository with all the infrastructure defined using IaC, separated in folders. For instance, all terraform configuration is in /terraform/. I want to apply all terraform files inside that directory from the CI/CD.
Configuration
The used github action is shown below:
name: 'Terraform'
on: [push]
permissions:
contents: read
jobs:
terraform:
name: 'Terraform'
runs-on: ubuntu-latest
environment: production
# Use the Bash shell regardless whether the GitHub Actions runner is ubuntu-latest, macos-latest, or windows-latest
defaults:
run:
shell: bash
#working-directory: terraform
steps:
# Checkout the repository to the GitHub Actions runner
- name: Checkout
uses: actions/checkout#v3
# Install the latest version of Terraform CLI and configure the Terraform CLI configuration file with a Terraform Cloud user API token
- name: Setup Terraform
uses: hashicorp/setup-terraform#v1
- id: 'auth'
uses: 'google-github-actions/auth#v1'
with:
credentials_json: '${{ secrets.GCP_CREDENTIALS }}'
- name: 'Set up Cloud SDK'
uses: 'google-github-actions/setup-gcloud#v1'
# Initialize a new or existing Terraform working directory by creating initial files, loading any remote state, downloading modules, etc.
- name: Terraform Init
run: terraform init
# Checks that all Terraform configuration files adhere to a canonical format
- name: Terraform Format
run: terraform fmt -check
# On push to "master", build or change infrastructure according to Terraform configuration files
# Note: It is recommended to set up a required "strict" status check in your repository for "Terraform Cloud". See the documentation on "strict" required status checks for more information: https://help.github.com/en/github/administering-a-repository/types-of-required-status-checks
- name: Terraform Apply
run: terraform apply -auto-approve -input=false
Problem
If I log in and then change directory to apply terraform it doesn't find to log in.
storage.NewClient() failed: dialing: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
On the other hand, if I don't change the directory then it doesn't find the configuration files as expected.
Error: No configuration files
Tried to move the terraform configuration files to the root of the repository and works. How could I implement it in a multidirectory repository?
Such feature was requested before. As explained in the issue, auth files is named as follows gha-creds-*.json.
Therefore, added a step just before using terraform to update the variable environment and moving the file itself:
- name: 'Setup google auth in multidirectory repo'
run: |
echo "GOOGLE_APPLICATION_CREDENTIALS=$GITHUB_WORKSPACE/terraform/`ls -1 $GOOGLE_APPLICATION_CREDENTIALS | xargs basename`" >> $GITHUB_ENV
mv $GITHUB_WORKSPACE/gha-creds-*.json $GITHUB_WORKSPACE/terraform/
I am trying to deploy my AWS infrastructure using Terraform from GitLab CI CD Pipeline.
I am using the GitLab managed image and it's default Terraform template.
I have configured S3 backend and it's pointing to the S3 bucket used to store the tf state file.
I had stored CI CD variables in GitLab for: AWS_ACCESS_KEY_ID', 'AWS_SECRET_ACCESS_KEY' and 'S3_BUCKET'.
Everything was working fine, until I changed the 'AWS_ACCESS_KEY_ID', 'AWS_SECRET_ACCESS_KEY' and 'S3_BUCKET' which was pointing to a different AWS account.
Now I am getting the following error:
$ terraform init
Initializing the backend...
Backend configuration changed!
Terraform has detected that the configuration specified for the backend
has changed. Terraform will now check for existing state in the backends.
Error: Error loading state:
AccessDenied: Access Denied
status code: 403, request id: XXXXXXXXXXXX,
host id: XXXXXXXXXXXXXXXXXXXXX
Terraform failed to load the default state from the "s3" backend.
State migration cannot occur unless the state can be loaded. Backend
modification and state migration has been aborted. The state in both the
source and the destination remain unmodified. Please resolve the
above error and try again.
Cleaning up file based variables 00:00
ERROR: Job failed: exit code 1
Since this issue happened because I changed the access_key and secret_key (It was working fine from my local VS Code), I commented out the 'cache:' block in the .gitlab-ci.yml file and it worked!
The following is my .gitlab-ci.yml file:
.gitlab-ci.yml
stages:
- validate
- plan
- apply
- destroy
image:
name: registry.gitlab.com/gitlab-org/gitlab-build-images:terraform
entrypoint:
- '/usr/bin/env'
- 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
# Default output file for Terraform plan
variables:
PLAN: plan.tfplan
JSON_PLAN_FILE: tfplan.json
STATE: dbrest.tfstate
cache:
paths:
- .terraform
before_script:
- alias convert_report="jq -r '([.resource_changes[]?.change.actions?]|flatten)|{\"create \":(map(select(.==\"create\"))|length),\"update\":(map(select(.==\"update\"))|length),\"delete\":(map(select(.==\"delete\"))|length)}'"
- terraform --version
- terraform init
validate:
stage: validate
script:
- terraform validate
only:
- tags
plan:
stage: plan
script:
- terraform plan -out=plan_file
- terraform show --json plan_file > plan.json
artifacts:
paths:
- plan.json
expire_in: 2 weeks
when: on_success
reports:
terraform: plan.json
only:
- tags
allow_failure: true
apply:
stage: apply
extends: plan
environment:
name: production
script:
- terraform apply --auto-approve
dependencies:
- plan
only:
- tags
when: manual
terraform destroy:
extends: apply
stage: destroy
script:
- terraform destroy --auto-approve
needs: ["plan","apply"]
when: manual
only:
- tags
The issue clearly happens if I don't comment out the below block. However it used to work before I made changes to the AWS access_key and secret_key.
#cache:
# paths:
# - .terraform
When the cache was not commented, the following was the result in the CI CI Pipeline:
Is cache being stored anywhere? And How do I clear it?
Think it's related to GitLab.
It seems that runner cache can be cleared off from GitLab from the UI itself.
Go to GitLab -> CI CD -> Pipelines and hit the 'Clear Runner Cache' button to clear the cache.
It actually works!
I want to setup a cloud build trigger so that each time I modify (commit and push) main.py, it execute test_mainpytest.py with pytest
I have a project that look like this :
My_Project\function_one\
main.py
deploy.yaml
requirements.txt
dir_pytest\
test_mainpytest.py
My deploy.yaml contain thoose steps :
steps:
- name: 'python'
args: ['pip3', 'install', '-r', 'My_Project/function_one/requirements.txt', '--user']
- name: 'python'
args: ['python3', 'pytest', 'My_Project/function_one/dir_pytest/']
For the moment I just want to try to execute pytest using the trigger. When I execute the cloud build trigger, I get this error :
ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'My_Project/function_one/requirements.txt'
Also my project is saved in a google cloud repository.
Edit :
I tried to add dir in my step, so it currently look like this :
steps:
- name: 'python'
dir: 'MyProject/function_one/'
args: ['pip3', 'install', '-r', 'My_Project/function_one/requirements.txt', '--user']
- name: 'python'
dir: 'MyProject/function_one/'
args: ['python3', 'pytest', 'My_Project/function_one/dir_pytest/']
Yet I still get the error, (I also tried to put dir after args but it didn't change much
I also noticed; when executing the trigger in Cloud Build; thoose 2 lines :
Initialized empty Git repository in /workspace/.git/
From https://source.developers.google.com/p/my_id_1234/r/My_Project
Should I use https://source.developers.google.com/p/my_id_1234/r/My_Project and add the path to my requirement.txt and my py_test directory ?
Could you show your whole cloudbuild.yaml? If you are using a build trigger, the repository is imported directly in /workspace. If you are doing a git clone, then your repository is inside a directory with the name of the repository. The difference is:
/workspace/my-repository/My_Project/function_one/requirements.txt
versus
/workspace/My_Project/function_one/requirements.txt
If nothing else works, you can do ls -R to show you the directory structure within the build. Add this as a first build step:
- name: 'list recursively'
args: ['ls', '-R']
Notice that Cloud Build uses a directory called /workspace as a working directory in order to persist the contents. You can add the dir field within your cloudbuild.yaml file in order for Cloud Build to find the requirements.txt file and then run the tests.
Does anyone have any experience of using Azure DevOps to deploy React build package to AWS using their extension?
I'm stuck on uploading only the build package of npm build.
Here is my scripts so far:
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
steps:
- task: NodeTool#0
inputs:
versionSpec: '10.x'
displayName: 'Install Node.js'
- script: |
npm install
npm test
npm run build
- task: S3Upload#1
inputs:
awsCredentials: 'AWS Deploy User'
regionName: 'us-east-1'
bucketName: 'test'
globExpressions: '**'
createBucket: true
displayName: 'npm install and build'
The only options on the task for S3Upload that stands out is sourceFolder. They use something like "$(Build.ArtifactStagingDirectory)" but since I've never used that before that doesn't make a lot of sense to me. Would it just be as simple as like $(Build.ArtifactStagingDirectory)/build
The predefined variable $(Build.ArtifactStagingDirectory) is mapped to c:\agent_work\1\a, which is the local path on the agent where any artifacts are copied to before being pushed to their destination.
In your yaml pipeline, your source code is downloaded in folder $(Build.SourcesDirectory)(ie. c:\agent_work\1\s). And the npm commands in the script task all runs in this folder. So the npm build result is this folder $(Build.SourcesDirectory)\build (ie.c:\agent_work\1\s\build).
S3Upload task will upload file from $(Build.ArtifactStagingDirectory) by default. You can specifically point the sourceFolder attribute (default is $(Build.ArtifactStagingDirectory)) of S3Upload task to folder $(Build.SourcesDirectory)\build. See below:
- task: S3Upload#1
inputs:
awsCredentials: 'AWS Deploy User'
regionName: 'us-east-1'
bucketName: 'test'
globExpressions: '**'
createBucket: true
sourceFolder: '$(Build.SourcesDirectory)/build'
Another workaround is to use copy file task to copy the build results from $(Build.SourcesDirectory)\build to folder $(Build.ArtifactStagingDirectory). See example here.
- task: CopyFiles#2
inputs:
Contents: 'build/**' # Pull the build directory (React)
TargetFolder: '$(Build.ArtifactStagingDirectory)'
I currently have an Azure Devops pipeline to build and deploy a next.js application via the serverless framework.
Upon reaching the AWSPowerShellModuleScript#1 task I get these errors:
[warning]MSG:UnableToDownload «https:...» «»
[warning]Unable to download the list of available providers. Check
your internet connection.
[warning]Unable to download from URI 'https:...' to ''.
[error]No match was found for the specified search criteria for the
provider 'NuGet'. The package provider requires 'PackageManagement'
and 'Provider' tags. Please check if the specified package has the
tags.
[error]No match was found for the specified search criteria and
module name 'AWSPowerShell'. Try Get-PSRepository to see all
available registered module repositories.
[error]The specified module 'AWSPowerShell' was not loaded because no
valid module file was found in any module directory.
I do have the AWS.ToolKit installed and it's visible when I go to manage extensions within Azure Devops.
My pipeline:
trigger: none
stages:
- stage: develop_build_deploy_stage
pool:
name: Default
demands:
- msbuild
- visualstudio
jobs:
- job: develop_build_deploy_job
steps:
- checkout: self
clean: true
- task: NodeTool#0
displayName: Install Node
inputs:
versionSpec: '12.x'
- script: |
npm install
npx next build
displayName: Install Dependencies and Build
- task: CopyFiles#2
inputs:
Contents: 'build/**'
TargetFolder: '$(Build.ArtifactStagingDirectory)'
- task: PublishBuildArtifacts#1
displayName: Publish Artifact
inputs:
pathtoPublish: $(Build.ArtifactStagingDirectory)
artifactName: dev_artifacts
- task: AWSPowerShellModuleScript#1
displayName: Deploy to Lambda#Edge
inputs:
awsCredentials: '###'
regionName: '###'
scriptType: 'inline'
inlineScript: 'npx serverless --package dev_artifacts'
I know I can use the ubuntu vmImage and then make use of the awsShellScript but the build agent I have available to me doesn't support bash.