Running VirtualBox in Concourse Task - virtualbox

I'm trying to build vagrant boxes with concourse. I'm using the concourse/buildbox-ci image which is used in concourse's own build pipeline to build the concourse-lite vagrant box.
Before running packer I'm creating the virtualbox devices so they match the hosts devices. Nevertheless the packer build fails with:
==> virtualbox-iso: Error starting VM: VBoxManage error: VBoxManage: error: The virtual machine 'packer-virtualbox-iso-1488205144' has terminated unexpectedly during startup with exit code 1 (0x1)
==> virtualbox-iso: VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component MachineWrap, interface IMachine
Has somebody got this working?
Is the concourse hetzner worker configuration accessible anywhere?
Additional configuration info:
in the concourse job container:
# ls -al /dev/vboxdrv /dev/vboxdrvu /dev/vboxnetctl
crw------- 1 root root 10, 53 Feb 27 14:19 /dev/vboxdrv
crw------- 1 root root 10, 52 Feb 27 14:19 /dev/vboxdrvu
crw------- 1 root root 10, 51 Feb 27 14:19 /dev/vboxnetctl
on the worker host:
# ls -al /dev/vbox*
crw------- 1 root root 10, 53 Feb 24 09:40 /dev/vboxdrv
crw------- 1 root root 10, 52 Feb 24 09:40 /dev/vboxdrvu
crw------- 1 root root 10, 51 Feb 24 09:40 /dev/vboxnetctl
concourse job:
jobs:
- name: mpf
serial_groups: [build]
plan:
- get: vagrant
trigger: true
- get: version
resource: version-mpf
- task: build
privileged: true
file: vagrant/ci/tasks/build.yml
tags: [vm-builder]
params:
TEMPLATE_FILE: virtualbox-mpf.json
vagrant/ci/scripts/build.sh:
#!/bin/bash -ex
mknod -m 0600 /dev/vboxdrv c 10 53
mknod -m 0600 /dev/vboxdrvu c 10 52
mknod -m 0600 /dev/vboxnetctl c 10 51
for name in $(VBoxManage list hostonlyifs | grep '^Name:' | awk '{print $NF}'); do
VBoxManage hostonlyif remove $name
done
VERSION=$(cat version/version)
packer build -var 'version=${VERSION}' vagrant/packer/${TEMPLATE_FILE}
vagrant/ci/tasks/build.yml:
---
platform: linux
image_resource:
type: docker-image
source: {repository: concourse/buildbox-ci}
inputs:
- name: vagrant
- name: version
outputs:
- name: build
run:
path: vagrant/ci/scripts/build.sh

Unfortunately the Hetzner worker configuration is basically just us periodically upgrading VirtualBox and fixing things when it falls over. (edit: we also make sure to use the same OS distro in the host and in the container - in our case, Arch Linux).
Make sure your VirtualBox version matches the version in the container - down to the patch version.
The device IDs (10,53 and 10,52 and 10,51) also must match those found on the host - these vary from version to version of VirtualBox.
We also make sure to use a special backend that does not perform any network namespacing, which is important if you're spinning up VMs that need a host-only network.
This whole thing's tricky. :/

Related

AWS CodeBuild not pausing on breakpoint

Using steps provided here, I kicked off a CodeBuild with the following advanced options checked:
Enable session connection
Allow AWS CodeBuild to modify this service role so it can be used with this build project
The buildspec included a codebuild-breakpoint:
version: 0.2
phases:
pre_build:
commands:
- ls -al
- codebuild-breakpoint
- cd "${SERVICE_NAME}"
- ls -al
- $(aws ecr get-login)
- TAG="$SERVICE_NAME"
build:
commands:
- docker build --tag "${REPOSITORY_URI}:${TAG}" .
post_build:
commands:
- docker push "${REPOSITORY_URI}:${TAG}"
- printf '{"tag":"%s"}' $TAG > ../build.json
artifacts:
files: build.json
The build started and produced the following logs without pausing:
[Container] 2022/02/28 13:49:03 Entering phase PRE_BUILD
[Container] 2022/02/28 13:49:03 Running command ls -al
total 148
drwxr-xr-x 2 root root 4096 Feb 28 13:49 .
drwxr-xr-x 3 root root 4096 Feb 28 13:49 ..
-rw-rw-rw- 1 root root 1818 Feb 28 10:54 user-manager\Dockerfile
-rw-rw-rw- 1 root root 140 Feb 28 10:34 user-manager\body.json
-rw-rw-rw- 1 root root 0 Feb 28 10:54 user-manager\shared-modules\
-rw-rw-rw- 1 root root 4822 Feb 21 14:52 user-manager\shared-modules\config-helper\config.js
-rw-rw-rw- 1 root root 2125 Feb 21 14:52 user-manager\shared-modules\config-helper\config\default.json
-rw-rw-rw- 1 root root 366 Feb 21 14:52 user-manager\shared-modules\config-helper\package.json
-rw-rw-rw- 1 root root 9713 Feb 21 14:52 user-manager\shared-modules\dynamodb-helper\dynamodb-helper.js
-rw-rw-rw- 1 root root 399 Feb 21 14:52 user-manager\shared-modules\dynamodb-helper\package.json
-rw-rw-rw- 1 root root 451 Feb 21 14:52 user-manager\shared-modules\token-manager\package.json
-rw-rw-rw- 1 root root 13885 Feb 21 14:52 user-manager\shared-modules\token-manager\token-manager.js
-rw-rw-rw- 1 root root 44372 Feb 28 10:34 user-manager\src\cognito-user.js
-rw-rw-rw- 1 root root 706 Feb 28 10:34 user-manager\src\package.json
-rw-rw-rw- 1 root root 32734 Feb 28 10:34 user-manager\src\server.js
[Container] 2022/02/28 13:49:03 Running command codebuild-breakpoint
2022/02/28 13:49:03 Build is paused temporarily and you can use codebuild-resume command in the session to resume this build
[Container] 2022/02/28 13:49:03 Running command cd "${SERVICE_NAME}"
/codebuild/output/tmp/script.sh: 4: cd: can't cd to user-manager
My primary question is: Why didn't the build pause and session manager link become available?
Side-quest: The reason I'm trying to debug the session is to try to determine why the process can't CD to the user-manager folder (which clearly exists). Any ideas why?
TLDR: The image on the build machine was too old.
Main quest
The template specified aws/codebuild/ubuntu-base:14.04 as the CodeBuild image. Presumably that image pre-dated the Session Manager functionality (which requires a specific version of the SSM agent to be installed).
I update the agent to aws/codebuild/standard:5.0 and was able to successfully pause on the breakpoint and connect to the session.
Side quest
Once I connected I was able to investigate the cause of the inability to CD to the folder. I can confirm that Tim's shot in the dark was correct! All the entries were in fact files - no folders.
This QuickStart is the gift that keeps on giving! When/if I get all the issues resolved I'll submit a PR to update the project. Those interested in the cause of the file/folder issue can follow up there.
Side quest update
The strange flattening behaviour was due to creating the zip file on a Windows machine and unzipping it on a unix machine (the build agent uses an Ubuntu image). Just zipped it using 7-Zip and that did the job.

Define specific docker-compose file to use for AWS Elastic Beanstalk Deployment

Before I run eb create command, how can I tell Elastic Beanstalk to use a DIFFERENT docker-compose file?
For example, my project directory:
HelloWorldDocker
├──.elasticbeanstalk
│ └──config.yml
├──app/
├──proxy/
└──docker-compose.prod.yml
└──docker-compose.yml
My docker-compose.yml is what I use for local development
My docker-compose.prod.yml is what I want to use for production
Is there a way to define this configuration before running the eb create command from the EB CLI?
Stating the obvious: I realize I could use docker-compose.yml for my production file and a docker-compose.dev.yml for my local development but then running the docker-compose up command becomes more tedious locally (ie: docker-compose -f docker-compose.dev.yml up --build...). Further, I'm mainly interested if this is even possible as I'm learning Elastic Beanstalk, and how I could do it if I wanted to.
EDIT / UPDATE: June 11, 2021
I attempted to rename docker-compose.prod.yml to docker-compose.yml in .ebextensions/docker-settings.config with this:
container_commands:
rename_docker_compose:
command: mv docker-compose.prod.yml docker-compose.yml
>eb deploy:
2021-06-11 16:44:45 ERROR Instance deployment failed.
For details, see 'eb-engine.log'.
2021-06-11 16:44:45 ERROR Instance deployment: Both
'Dockerfile' and 'Dockerrun.aws.json' are missing in your
source bundle. Include at least one of them. The deployment
failed.
In eb-engine.log, I see:
2021/06/11 16:44:45.818876 [ERROR] An error occurred during
execution of command [app-deploy] - [Docker Specific Build
Application]. Stop running the command. Error: Dockerfile and
Dockerrun.aws.json are both missing, abort deployment
Based on my testing, this is due to AWS needing to call /bin/sh -c docker-compose config before getting to the later steps of container_commands.
Edit / Update #2
If I use commands instead of container_commands:
commands:
rename_docker_compose:
command: mv docker-compose.prod.yml docker-compose.yml
cwd: /var/app/staging
it does seem to do the replacement successfully:
2021-06-11 21:40:44,809 P1957 [INFO] Command find_docker_compose_file
2021-06-11 21:40:45,086 P1957 [INFO] -----------------------Command Output-----------------------
2021-06-11 21:40:45,086 P1957 [INFO] ./var/app/staging/docker-compose.prod.yml
2021-06-11 21:40:45,086 P1957 [INFO] ------------------------------------------------------------
2021-06-11 21:40:45,086 P1957 [INFO] Completed successfully.
but I still am hit with:
2021/06/11 21:40:45.192780 [ERROR] An error occurred during
execution of command [app-deploy] - [Docker Specific Build
Application]. Stop running the command. Error: Dockerfile and
Dockerrun.aws.json are both missing, abort deployment
EDIT / UPDATE: June 12, 2021
I'm on a Windows 10 machine. Before running eb deploy command locally, I opened up Git Bash which uses MINGW64 terminal. I cdd to the prebuild directory where build.sh exists. I ran:
chmod +x build.sh
If I do ls -l, it returns:
-rwxr-xr-x 1 Jarad 197121 58 Jun 12 12:31 build.sh*
I think this means the file is executable.
I then committed to git.
I then ran eb deploy.
I am seeing a build.sh: permission denied error in eb-engine.log. Below is an excerpt of the relevant portion.
...
2021/06/12 19:41:38.108528 [INFO] application/zip
2021/06/12 19:41:38.108541 [INFO] app source bundle is zip file ...
2021/06/12 19:41:38.108547 [INFO] extracting /opt/elasticbeanstalk/deployment/app_source_bundle to /var/app/staging/
2021/06/12 19:41:38.108556 [INFO] Running command /bin/sh -c /usr/bin/unzip -q -o /opt/elasticbeanstalk/deployment/app_source_bundle -d /var/app/staging/
2021/06/12 19:41:38.149125 [INFO] finished extracting /opt/elasticbeanstalk/deployment/app_source_bundle to /var/app/staging/ successfully
2021/06/12 19:41:38.149142 [INFO] Executing instruction: RunAppDeployPreBuildHooks
2021/06/12 19:41:38.149190 [INFO] Executing platform hooks in .platform/hooks/prebuild/
2021/06/12 19:41:38.149249 [INFO] Following platform hooks will be executed in order: [build.sh]
2021/06/12 19:41:38.149255 [INFO] Running platform hook: .platform/hooks/prebuild/build.sh
2021/06/12 19:41:38.149457 [ERROR] An error occurred during execution of command [app-deploy] - [RunAppDeployPreBuildHooks]. Stop running the command. Error: Command .platform/hooks/prebuild/build.sh failed with error fork/exec .platform/hooks/prebuild/build.sh: permission denied
2021/06/12 19:41:38.149464 [INFO] Executing cleanup logic
2021/06/12 19:41:38.149572 [INFO] CommandService Response: {"status":"FAILURE","api_version":"1.0","results":[{"status":"FAILURE","msg":"Engine execution has encountered an error.","returncode":1,"events":[{"msg":"Instance deployment failed. For details, see 'eb-engine.log'.","timestamp":1623526898,"severity":"ERROR"}]}]}
2021/06/12 19:41:38.149706 [INFO] Platform Engine finished execution on command: app-deploy
...
Any idea why I am getting a permission denied error?
My Conclusion From This Madness
Elastic Beanstalk's EB CLI eb deploy command does not zip files (the app_source_bundle it creates) correctly on Windows machines.
Proof
I was able to recreate Marcin's example by zipping it locally and manually uploading it through the Elastic Beanstalk online interface. When I do that and check the source bundle, it shows that build.sh does have executable permissions (-rwxr-xr-x).
[root#ip-172-31-11-170 deployment]# zipinfo app_source_bundle
Archive: app_source_bundle
Zip file size: 993 bytes, number of entries: 5
drwxr-xr-x 3.0 unx 0 bx stor 21-Jun-13 03:08 .platform/
drwxr-xr-x 3.0 unx 0 bx stor 21-Jun-13 03:08 .platform/hooks/
drwxr-xr-x 3.0 unx 0 bx stor 21-Jun-13 03:08 .platform/hooks/prebuild/
-rwxr-xr-x 3.0 unx 58 tx defN 21-Jun-13 03:09 .platform/hooks/prebuild/build.sh
-rw-r--r-- 3.0 unx 98 tx defN 21-Jun-13 03:08 docker-compose.prod.yml
When I initialize and create using the EB CLI and the exact same files, build.sh does NOT have executable permissions (-rw-rw-rw-).
[ec2-user#ip-172-31-5-39 deployment]$ zipinfo app_source_bundle
Archive: app_source_bundle
Zip file size: 1092 bytes, number of entries: 5
drwxrwxrwx 2.0 fat 0 b- stor 21-Jun-12 20:32 ./
-rw-rw-rw- 2.0 fat 98 b- defN 21-Jun-12 20:08 docker-compose.prod.yml
-rw-rw-rw- 2.0 fat 993 b- defN 21-Jun-12 20:15 myzip.zip
drwxrwxrwx 2.0 fat 0 b- stor 21-Jun-12 20:08 .platform/hooks/prebuild/
-rw-rw-rw- 2.0 fat 58 b- defN 21-Jun-12 20:09 .platform/hooks/prebuild/build.sh
Therefore, I think this is a bug with AWS EB CLI deploy command in regards to how it zips files for Windows users.
You can't do this from command level. But I guess you could write container_commands script to rename your docker-compose file from docker-compose.dev.yml to docker-compose.yml:
You can use the container_commands key to execute commands that affect your application source code. Container commands run after the application and web server have been set up and the application version archive has been extracted, but before the application version is deployed.
UPDATE 12 Jun 2021
I tried to replicate the issue using simplified setup with just docker-compose.prod.yml and Docker running on 64bit Amazon Linux 2 3.4.1 EB platform.
docker-compose.prod.yml
version: "3"
services:
client:
image: nginx
ports:
- 80:80
I can confirm and reproduce the issue with container_commands. So in my tests, the solution was to setup prebuild deployment hook.
So my deployment zip had the structure:
├── docker-compose.prod.yml
└── .platform
└── hooks
└── prebuild
└── build.sh
where
build.sh
#!/bin/bash
mv docker-compose.prod.yml docker-compose.yml
I also made the build.sh executable before creating deployment zip.
app_source_bundle permissions (zipinfo -l)
Zip file size: 1008 bytes, number of entries: 5
drwxr-xr-x 3.0 unx 0 bx 0 stor 21-Jun-12 07:37 .platform/
drwxr-xr-x 3.0 unx 0 bx 0 stor 21-Jun-12 07:37 .platform/hooks/
drwxr-xr-x 3.0 unx 0 bx 0 stor 21-Jun-12 07:38 .platform/hooks/prebuild/
-rwxr-xr-x 3.0 unx 77 tx 64 defN 21-Jun-12 07:24 .platform/hooks/prebuild/build.sh
-rw-r--r-- 3.0 unx 92 tx 68 defN 21-Jun-12 07:01 docker-compose.prod.ym
I was able to circumvent this annoying bug by:
Using git and AWS CodeCommit
Running git add --chmod=+x .platform/hooks/prebuild/build.sh
This circumvents the Windows-related issue because:
When you configure CodeCommit with your EB CLI repository, the EB CLI
uses the contents of the repository to create source bundles. When you
run eb deploy or eb create, the EB CLI pushes new commits and uses the
HEAD revision of your branch to create the archive that it deploys to
the EC2 instances in your environment.
Source: Deploying from your CodeCommit repository

docker socker at /var/run/docker.sock with AWS

I have an issue where a few tools, Portainer for example, can't find the docker socket on AWS.
I have some setup scripts that were run to set various containers.
On MacOS, it works without problems.
On a CentOS box, no problem as well.
On CentOS / AWS, containers cannot connect to the docker socket.
I am talking about a local unsecured connection to /var/run/docker.sock
What could be different on AWS?
I can see the socket:
➜ run ls -ld /var/run/docker*
drwxr-xr-x 8 root root 200 Nov 27 14:04 /var/run/docker
-rw-r--r-- 1 root root 4 Nov 27 14:03 /var/run/docker.pid
srw-rw-r-- 1 root docker 0 Nov 27 14:03 /var/run/docker.sock

can not start greengrassd (AWS IOT greengrass) on raspberrypi

I have register AWS IoT Greengrass group.
I also download Greengrass certificate from console and AmazonRoot-CA1
here is list my certificate files(store in /greengrass/certs/):
-rw-r--r-- 1 pi pi 1220 Jan 15 10:07 82ab16xxxx.cert.pem
-rw-r--r-- 1 pi pi 1679 Jan 15 10:07 82ab16xxxx.private.key
-rw-r--r-- 1 pi pi 451 Jan 15 10:07 82ab16xxxx.public.key
-rw-r--r-- 1 pi pi 1188 Jan 15 10:07 root.ca.pem
When I start greengrassd by command:
sudo ./greengrassd start
I have error:
Setting up greengrass daemon
Validating hardlink/softlink protection
Waiting for up to 40s for Daemon to start
Error occured while generating TLS config: ErrUnknownURIScheme: no handlers matched for path: .../greengrass/certs/root.ca.pem
The Greengrass daemon process with [pid = 18029] died
I have try to re-install OS but still error.
I also install mosquitto-clients and mosquitto on raspberrypi
Thanks.
I'm guessing your issue is that you haven't activated your root CA from the console.
try this instead:
sudo wget -O root.ca.pem https://www.amazontrust.com/repository/AmazonRootCA1.pem
try doing this directly in your certs directory then restarting your deamon.

AWS Block Devices name doesn't match with CentOS SoftLink

On AWS EC2 Block Device is identified as /dev/sda, /dev/sdf and /dev/sdg, but inside EC2 CentOS instance when I do ll /dev/sd* it gives following:
lrwxrwxrwx. 1 root root 4 Feb 17 03:10 /dev/sda -> xvde
lrwxrwxrwx. 1 root root 4 Feb 17 03:10 /dev/sdj -> xvdj
lrwxrwxrwx. 1 root root 4 Feb 17 03:10 /dev/sdk -> xvdk
lrwxrwxrwx. 1 root root 5 Feb 17 03:10 /dev/sdk1 -> xvdk1
When I run ec2-describe-instances --aws-access-key xxxxxx<MyKey>xxx --aws-secret-key xxxxxx<MyKey>xxx --region us-east-1 ``curl -s http://169.254.169.254/latest/meta-data/instance-id`` | grep -i BLOCKDEVICE output is as follow:
/dev/sda
/dev/sdf
/dev/sdg
I am wondering how to link these two: AWS GUI Console's Block Devices and within EC2 instance Block Devices?
Thanks,
This is a device mapping alias problem. You can see more details with a solution here:
https://forums.aws.amazon.com/message.jspa?messageID=255240
Make sure you take backups of everything before making any changes!