I making the codebuild with cdk
It can accept the buildspec as yaml, however ,how can I do the same thing in cdk?
What I want to do is like this, of course it doesn't work though,
I forcely put the yaml code in commands.
const buildProject = new codebuild.PipelineProject(this, 'project', {
environment: {// I guess I need to select ubuntu and image 4.0},
buildSpec: codebuild.BuildSpec.fromObject({
version: '0.2',
phases: {
build: {
commands:['
version: 0.2
phases:
install:
runtime-versions:
docker: 18
build:
commands:
- apt-get install jq -y
- ContainerName="tnkDjangoContainer"
- ImageURI=$(cat imageDetail.json | jq -r '.ImageURI')
- printf '[{"name":"CONTAINER_NAME","imageUri":"IMAGE_URI"}]' > imagedefinitions.json
- sed -i -e "s|CONTAINER_NAME|$ContainerName|g" imagedefinitions.json
- sed -i -e "s|IMAGE_URI|$ImageURI|g" imagedefinitions.json
- cat imagedefinitions.json
artifacts:
files:
- imagedefinitions.json
',
],
},
}
})
});
And also I guess I need to choose the image to do the buildspec such as Ubuntu
Where can I set these?
CDK does not expose a method to inline a YAML buildspec at synth-time. You could do this yourself by parsing existing YAML into a JS object and passing the result to BuildSpec.fromObject.
CDK's codebuild.Project gives you several other ways to provide a buildSpec:
BuildSpec.fromObject inlines a buildspec from key-value pairs at synth-time. It should follow the CodeBuild buildspec format. CDK will output a stringified JSON buildspec in the CloudFormation template. If you want CDK to output YAML instead, use fromObjectToYaml. Both methods take key-value pairs (type: [key: string]: any;) as input, so TS can't offer much typechecking help.
BuildSpec.fromSourceFilename tells CodeBuild to use a buildspec file in your source at run-time. The filename is passed in the CloudFormation template.
Here's an example of parsing a YAML string into inlined YAML output, using the yaml package. Note that the environment is defined outside the buildspec:
import * as yaml from 'yaml';
const fromYaml = yaml.parse(`
version: '0.2'
phases:
build:
commands:
- npm run build
`);
new codebuild.Project(this, 'YamlInYamlOutProject', {
environment: {
buildImage: codebuild.LinuxBuildImage.STANDARD_5_0, // Ubuntu Standard 5
},
buildSpec: codebuild.BuildSpec.fromObjectToYaml(fromYaml),
});
Related
I have a JSON body that looks something like this.
This was retrieved through AWS cli secrets manager command.
{
"user1": {
"username": "user",
"password": "123",
},
"user2": {
"username": "user2",
"password": "1234"
}
}
I need a command that would put this JSON into a file called something like user.json.
Is this possible through a command?
## buildspec.yml
version: 0.2
phases:
install:
runtime-versions:
nodejs: latest
commands:
- npm install
pre_build:
commands:
- CREDS="$(aws secretsmanager get-secret-value --secret-id arn:aws:secretsmanager:{secret} | jq '.SecretString | fromjson')"
build:
commands:
- echo $CREDS (prints the correct credentials in json format)
- Need to send $CREDS > user.json or something similar. Is this possible?
- npm start test:run
I am deploying a build spec for AWS codebuild using Serverless Framework. When I deploy, the new line after the first line is absent in the build spec. This resource previously deployed without a problem and I cannot see anything I have done to break it. Is this a problem on my end or a bug with Serverless/CloudFormation?
Below is the CloudFormation template and the resulting build spec copied from the AWS console.
Resources:
CodeBuild:
Type: 'AWS::CodeBuild::Project'
Properties:
Name: sls-retrobase-frontend-CodeBuild-${opt:stage}
ServiceRole: !GetAtt CodeBuildRole.Arn
Artifacts:
Type: CODEPIPELINE
Name: sls-retrobase
Environment:
Type: LINUX_CONTAINER
ComputeType: BUILD_GENERAL1_SMALL
Image: "aws/codebuild/amazonlinux2-x86_64-standard:3.0"
Source:
Type: CODEPIPELINE
BuildSpec: !Sub
- >
version: 0.2
phases:
pre_build:
commands:
- echo List directory files...
- ls
- echo Installing source NPM dependencies...
- npm install
build:
commands:
- echo List active directory...
- ls
- echo Inserting Api Url into config.json from environment
- node makeConfig.js ${apiUrl} ${auth0Audience} ${auth0Domain} ${auth0ClientId}
- echo Build started on `date`
- npm run build
post_build:
commands:
- echo List build directory...
- ls ./build
- aws s3 cp --recursive --acl public-read ./build s3://${Website}
artifacts:
files:
- '**/*'
- apiUrl: !Join ['', [ "https://", !Ref QueryRestApi, ".execute-api.us-east-1.amazonaws.com/${opt:stage}/sls-retrobase-${opt:stage}"]]
websiteUrl: !GetAtt Website.WebsiteURL
auth0Audience: '***'
auth0Domain: '***'
auth0ClientId: '***'
build spec:
version: 0.2 phases:
pre_build:
commands:
- echo List directory files...
- ls
- echo Installing source NPM dependencies...
- npm install
build:
commands:
- echo List active directory...
- ls
- echo Inserting Api Url into config.json from environment
- node makeConfig.js https://h0suk54yw0.execute-api.us-east-1.amazonaws.com/test/sls-retrobase-test https://sls-retrobase https://dev-y33gimcf.eu.auth0.com/ Xn6SDc43vE8P0sQHkVLtiBSBVFT5rJMU
- echo Build started on `date`
- npm run build
post_build:
commands:
- echo List build directory...
- ls ./build
- aws s3 cp --recursive --acl public-read ./build s3://serverless-retrobase-resources-test-website-7cvhqlgbkfj7
artifacts:
files:
- '**/*'
This is probably because of your use of >. Please change it to |:
BuildSpec: !Sub
- |
version: 0.2
phases:
Alternatively, fix your spaces when using >. > and your code are not aligned.
I'm using cloudbuild to deploy new version of my app when a new commit appears in github.
Everything is working good.
Now I'm trying to setup a variable substitution in the trigger configuration, because I want to put my version number in the trigger once, so that I can find the deployed correct version without modifying cloudbuild configuration file.
Variabile substitution works great in my cloudbuild file, for example:
(cloudbuild.yaml)
# TEST: PRINT VARIABLE IN LOG
- name: 'gcr.io/cloud-builders/gcloud'
entrypoint: 'bash'
args: ['-c', 'echo', '${_VERSION}']
# DEPLOY APP
- name: "gcr.io/cloud-builders/gcloud"
args: ["app", "deploy", "-v", "${_VERSION}", "app.yaml"]
dir: 'frontend'
timeout: "20m"
${_VERSION} is correctly replaced with the string I put into my trigger.
Now I want to obtain the same result in app.yaml file, substituting an env variabile, something like:
(app.yaml)
runtime: nodejs
env: flex
service: backend
env_variables:
VERSION: "${_VERSION}"
TEST_ENV: "read from google"
When I read TEST_ENV from my app, it works, but _VERSION is not replaced.
Any suggestion?
When you perform this step
# DEPLOY APP
- name: "gcr.io/cloud-builders/gcloud"
args: ["app", "deploy", "-v", "${_VERSION}", "app.yaml"]
dir: 'frontend'
timeout: "20m"
The app.yaml is provided as-is to the gcloud command, and it's not evaluated. You have to update it manually. Something like this
# REPLACE: PUT THE CORRECT VALUE IN APP.YAML FILE
- name: 'gcr.io/cloud-builders/gcloud'
entrypoint: 'bash'
args: ['-c', 'sed', "-i", "sed -i "s/\$${_VERSION}/${_VERSION}/g", 'app.yaml']
Of course if you let the
env_variables:
VERSION: "${_VERSION}"
as-is in your app.yaml file. You can change this replacement string
I want to add this solution in case someone has problems with the one proposed by giullade (in my case, cloudbuild gave me an error in executing the sed command).
I also changed my replacement string to one more readable and to avoid escaping the $ sign.
# Step 0: REPLACE variables in app.yaml file
- name: 'gcr.io/cloud-builders/gcloud'
entrypoint: 'bash'
dir: 'backend'
args:
- '-c'
- |
sed -i "s/__VERSION/${_VERSION}/g" app-staging.yaml
and in my app.yaml:
env_variables:
VERSION_ENV: "__VERSION"
I want to deploy aws lamda .net core project using bit bucket pipeline
I have created bitbucket-pipelines.yml like below but after build run getting error -
MSBUILD : error MSB1003: Specify a project or solution file. The current working directory does not contain a project or solution file.
file code -
image: microsoft/dotnet:sdk
pipelines:
default:
- step:
caches:
- dotnetcore
script: # Modify the commands below to build your repository.
- export PROJECT_NAME=TestAWS/AWSLambda1/AWSLambda1.sln
- dotnet restore
- dotnet build $PROJECT_NAME
- pipe: atlassian/aws-lambda-deploy:0.2.1
variables:
AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
AWS_DEFAULT_REGION: 'us-east-1'
FUNCTION_NAME: 'my-lambda-function'
COMMAND: 'update'
ZIP_FILE: 'code.zip'
project structure is like this -
The problem is here:
PROJECT_NAME=TestAWS/AWSLambda1/AWSLambda1.sln
This is the incorrect path. Bitbucket Pipelines will use a special path in the Docker image, something like /opt/atlassian/pipelines/agent/build/YOUR_PROJECT , to do a Git clone of your project.
You can see this when you click on the "Build Setup" step in the Pipelines web console:
Cloning into '/opt/atlassian/pipelines/agent/build'...
You can use a pre-defined environment variable to retrieve this path: $BITBUCKET_CLONE_DIR , as described here: https://support.atlassian.com/bitbucket-cloud/docs/variables-in-pipelines/
Consider something like this in your yml build script:
script:
- echo $BITBUCKET_CLONE_DIR # Debug: Print the $BITBUCKET_CLONE_DIR
- pwd # Debug: Print the current working directory
- find "$(pwd -P)" -name AWSLambda1.sln # Debug: Show the full file path of AWSLambda1.sln
- export PROJECT_NAME="$BITBUCKET_CLONE_DIR/AWSLambda1.sln"
- echo $PROJECT_NAME
- if [ -f "$PROJECT_NAME" ]; then echo "File exists" ; fi
# Try this if the file path is not as expected
- export PROJECT_NAME="$BITBUCKET_CLONE_DIR/AWSLambda1/AWSLambda1.sln"
- echo $PROJECT_NAME
- if [ -f "$PROJECT_NAME" ]; then echo "File exists" ; fi
I can trigger my AWS pipeline from jenkins but I don't want to create buildspec.yaml and instead use the pipeline script which already works for jenkins.
In order to user Codebuild you need to provide the Codebuild project with a buildspec.yaml file along with your source code or incorporate the commands into the actual project.
However, I think you are interested in having the creation of the buildspec.yaml file done within the Jenkins pipeline.
Below is a snippet of a stage within a Jenkinsfile, it creates a build spec file for building docker images and then sends the contents of the workspace to a codebuild project. This uses the plugin for Codebuild.
stage('Build - Non Prod'){
String nonProductionBuildSpec = """
version: 0.1
phases:
pre_build:
commands:
- \$(aws ecr get-login --registry-ids <number> --region us-east-1)
build:
commands:
- docker build -t ces-sample-docker .
- docker tag $NAME:$TAG <account-number>.dkr.ecr.us-east-1.amazonaws.com/$NAME:$TAG
post_build:
commands:
- docker push <account-number>.dkr.ecr.us-east-1.amazonaws.com/$NAME:$TAG
""".replace("\t"," ")
writeFile file: 'buildspec.yml', text: nonProductionBuildSpec
//Send checked out files to AWS
awsCodeBuild projectName: "<codebuild-projectname>",region: "us-east-1", sourceControlType: "jenkins"
}
I hope this gives you an idea of whats possible.
Good luck!
Patrick
You will need to write a buildspec for the commands that you want AWS CodeBuild to run. If you use the CodeBuild plugin for Jenkins, you can add that to your Jenkins pipeline and use CodeBuild as a Jenkins build slave to execute the commands in your buildspec.
See more details here: https://docs.aws.amazon.com/codebuild/latest/userguide/jenkins-plugin.html
#hynespm - excellent example mate.
Here is another one based off yours but with stripIndent() and "withAWS" to switch roles:
#!/usr/bin/env groovy
def cbResult = null
pipeline {
.
.
.
script {
echo ("app_version TestwithAWS value : " + "${app_version}")
String buildspec = """\
version: 0.2
env:
parameter-store:
TOKEN: /some/token
phases:
pre_build:
commands:
- echo "List files...."
- ls -l
- echo "TOKEN is ':' \${TOKEN}"
build:
commands:
- echo "build':' Do something here..."
- echo "\${CODEBUILD_SRC_DIR}"
- ls -l "\${CODEBUILD_SRC_DIR}"
post_build:
commands:
- pwd
- echo "postbuild':' Done..."
""".stripIndent()
withAWS(region: 'ap-southeast-2', role: 'CodeBuildWithJenkinsRole', roleAccount: '123456789123', externalId: '123456-2c1a-4367-aa09-7654321') {
sh 'aws ssm get-parameter --name "/some/token"'
try {
cbResult = awsCodeBuild projectName: 'project-lambda',
sourceControlType: 'project',
credentialsType: 'keys',
awsAccessKey: env.AWS_ACCESS_KEY_ID,
awsSecretKey: env.AWS_SECRET_ACCESS_KEY,
awsSessionToken: env.AWS_SESSION_TOKEN,
region: 'ap-southeast-2',
envVariables: '[ { GITHUB_OWNER, special }, { GITHUB_REPO, project-lambda } ]',
artifactTypeOverride: 'S3',
artifactLocationOverride: 'special-artifacts',
overrideArtifactName: 'True',
buildSpecFile: buildspec
} catch (Exception cbEx) {
cbResult = cbEx.getCodeBuildResult()
}
}
} //script
.
.
.
}