I started to experiment with Gitlab-CI but i have some problems while running my builds. At this point i started to run PHPMD from the building using the following .gitlab-ci.yml
phpmd:
script:
- /tmp/phpmd/src/bin/phpmd app/code/local text cleancode,codesize,design
All this is ok and the build is fired. The problem is that the build keeps failing:
gitlab-ci-multi-runner 0.5.2 (adb8153)
Using Docker executor with image ruby:2.1 ...
Starting service mysql:latest ...
Starting service redis:latest ...
Waiting for services to be up and running...
Running on runner-42afea22-project-4-concurrent-0 via LINXAACC101...
Cloning repository...
Cloning into '/builds/MyProject/Foo'...
Checking out a3e2b48f as development...
$ /tmp/phpmd/src/bin/phpmd app/code/local text cleancode,codesize,design
/builds/Dynalean/Tekenzeker.sh: line 22: /tmp/phpmd/src/bin/phpmd: No such file or directory
ERROR: Build failed with: exit code 1
This is strange since when i run the command on the commandline it runs like a charm:
/tmp/phpmd/src/bin/phpmd /tmp/phpmd/src/main text
cleancode,codesize,design
For further debugging i set the PHPMD executable to 777 (the full directory) using sudo chmod 777 -R /tmp/phpmd. This also reflects on the executable:
-rwxrwxrwx 1 root root 7350 Oct 16 12:03 phpmd*
Thanks for the help
Related
First time trying out cifuzz/jazzer but I am unable to get the source code from github without circumventing my organisation's restrictions on downloading external code (and risking their wrath). Fortunately, I can download and use the cifuzz/jazzer docker image. However all roads lead this error:
ERROR: Could not find jazzer_standalone.jar. Please provide the pathname via the --agent_path flag.
Obviously, I'm no jazzer expert, nor am I too seasoned with docker beyond the (very) basics, however by overriding the entrypoint of the image with:
docker run -it --entrypoint /bin/sh cifuzz/jazzer
and navigating to the /app directory where these files exist:
/fuzzing # cd /app/
/app # ls -alrt
total 10192
-r-xr-xr-x 1 root root 9764956 Oct 24 21:09 jazzer.jar
-r-xr-xr-x 1 root root 658288 Oct 24 21:09 jazzer
drwxr-xr-x 2 root root 4096 Oct 24 21:09 .
drwxr-xr-x 1 root root 4096 Nov 6 16:54 ..
Running ./jazzer results in the same error seen when trying to start the app through the instructions on the github page.
ERROR: Could not find jazzer_standalone.jar. Please provide the pathname via the --agent_path flag.
Looking in the github repo online in the search facility for "jazzer_standalone.jar", it finds this code in the BUILD.bazel file on line 34:
remap_paths = {
"driver/src/main/java/com/code_intelligence/jazzer/jazzer_standalone_deploy.jar": "jazzer_standalone.jar",
"launcher/jazzer": "jazzer",
},
Seems that jazzer_standalone_deploy.jar isn't remapped and/or included in the image?
From the github instructions at https://github.com/CodeIntelligenceTesting/jazzer it says:
The "distroless" Docker image cifuzz/jazzer includes Jazzer together with OpenJDK 11. Just mount a directory containing your compiled fuzz target into the container under /fuzzing by running:
docker run -v path/containing/the/application:/fuzzing cifuzz/jazzer <arguments>
I tried:
docker run -v path-to-my-applicatiuon-jar:/fuzzing cifuzz/jazzer
So I missed out the arguments, just to get some error output and see if I had got the volume path correct, etc.
The result is:
ERROR: Could not find jazzer_standalone.jar. Please provide the pathname via the --agent_path flag.
This has been fixed in Jazzer and new images have been pushed: https://github.com/CodeIntelligenceTesting/jazzer/issues/524
I updated my docker file to upgrade ubuntu but it started failing and I'm unsure why...
dockerfile:
# using digest for version 20.04 as there is multiple digest that used this tag#
FROM ubuntu#sha256:82becede498899ec668628e7cb0ad87b6e1c371cb8a1e597d83a47fac21d6af3
ENV DEBIAN_FRONTEND=noninteractive
RUN echo "APT::Get::Assume-Yes \"true\";" > /etc/apt/apt.conf.d/90assumeyes
#install tools
#removed for clarity
WORKDIR /azp
COPY ./start.sh .
RUN chmod +x start.sh
CMD ["./start.sh"]
my evens from the pod
Successfully assigned se-agents/agent-se-linux-5c9f647768-25p7v to aks-linag-56790600-vmss000002
Pulling image "compregistrynp.azurecr.io/agent-se-linux:25319"
Successfully pulled image "comregistrynp.azurecr.io/agent-se-linux:25319"
Created container agent-se-linux
Started container agent-se-linux
Back-off restarting failed container
When I check the error in the pod, I see the following message:
standard_init_linux.go:228: exec user process caused: no such file or directory
Not even sure where to look anymore. The only difference in the dockerfile was the ubuntu tag and I added 1 tool to install. I tried to deploy what was in Prod to dev and it's failing with the same error. I'm convinced there's something in my AKS...
So the issue was that someone on my team had modified the shell script and didn't set the proper End of Line characters to Lf.
I will be running a script to convert the file to Linux to ensure this doesn't happen again in my pipeline!
I was trying to setup a build trigger for an kotlin app that is build using gradle. For that I put together the following Dockerfile:
FROM gradle:jdk8 as builder
WORKDIR /home/gradle/project
COPY . .
WORKDIR ./Kuroji-Eventrouter-Server
RUN gradle shadowJar
FROM openjdk:8-jre-alpine
WORKDIR /app
COPY --from=builder /home/gradle/project/Kuroji-Eventrouter-Server/build/libs/kuroji-eventrouter-server-*-all.jar kuroji-eventrouter-server.jar
ENTRYPOINT ["java", "-jar", "kuroji-eventrouter-server.jar"]
And that file works on my machine with docker build and it starts normally on google container registry however during the RUN gradle shadowJar task it crashes with some gradle error:
Step 5/9 : RUN gradle shadowJar
---> Running in ddd190fc2323
Starting a Gradle Daemon (subsequent builds will be faster)
[91m
[0m[91mFAILURE: [0m[91mBuild failed with an exception.[0m[91m
[0m[91m
[0m[91m* What went wrong:
[0m[91mCould not create service of type ScriptPluginFactory using BuildScopeServices.createScriptPluginFactory().
[0m[91m> [0m[91mCould not create service of type CrossBuildFileHashCache using BuildSessionScopeServices.createCrossBuildFileHashCache().
[0m[91m
[0m[91m* Try:
[0m[91mRun with [0m[91m--stacktrace[0m[91m option to get the stack trace. Run with --info[0m[91m or --debug[0m[91m option to get more log output. Run with [0m[91m--scan[0m[91m to get full insights.[0m[91m
[0m[91m
[0m[91m* Get more help at https://help.gradle.org
[0m[91m
[0m[91mBUILD FAILED in 3s
The command '/bin/sh -c gradle shadowJar' returned a non-zero code: 1
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/docker" failed: exit status 1
[0m
I tried building the Image on docker HUB and the same thing happend: https://hub.docker.com/r/usbpc/kuroji-eventrouter-server/builds/bnknnpqowwabdy82ydxiypc/
This is very confusing to me as I thought containers should be able to run anywhere and not depend on the enviroment. What can I do to make google build my container?
The problem was a file permission problem. Using the --stacktrace option I found that the gradle process didn't have permissions to create a folder inside the sources.
The solution I would like to do is use the --chown=gradle:gradle option on the COPY instruction, unfortunatly this it not supported in the google cloud yet.
So the solution is to add USER root before executing the gradle build.
I have a pre-existing golang project with the a following folder structure (minimized the folder for readability).
- postgre
- service.go
- cmd
- vano
- main.go
- vanoctl
- main.go
vano.go
Now since my project web server is in ./cmd/vano I need to create a custom Buildfile and Procfile. So I did that
Here is my Buildfile
make: ./build.sh
build.sh file:
#!/usr/bin/env bash
# Install dependencies.
go get ./...
# Build app
go build ./cmd/vano -o bin/application
and finally my Procfile:
web: bin/application
So now my folder structure looks like this:
- postgre
- service.go
- cmd
- vano
- main.go
- vanoctl
- main.go
vano.go
Buildfile
build.sh
Procfile
I zip up the source using git:
git archive --format=zip HEAD > vano.zip
And upload it to AWS Beanstalk. How ever I keep getting errors and AWS errors don't seem to be the most read. Here is my error
Command execution completed on all instances. Summary: [Successful: 0, Failed: 1].
Error Message
[Instance: i-0d8f642474e3b2c68] Command failed on instance. Return code: 1 Output: (TRUNCATED)...' Failed to execute 'HOME=/tmp /opt/elasticbeanstalk/lib/ruby/bin/ruby /opt/elasticbeanstalk/lib/ruby/bin/foreman start --procfile /tmp/d20170213-1941-1baz0rh/eb-buildtask-0 --root /var/app/staging --env /var/elasticbeanstalk/staging/elasticbeanstalk.env'. Hook /opt/elasticbeanstalk/hooks/appdeploy/pre/01_configure_application.sh failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
Extra Error info:
Failed to execute 'HOME=/tmp /opt/elasticbeanstalk/lib/ruby/bin/ruby /opt/elasticbeanstalk/lib/ruby/bin/foreman start --procfile /tmp/d20170213-1941-1baz0rh/eb-buildtask-0 --root /var/app/staging --env /var/elasticbeanstalk/staging/elasticbeanstalk.env'
Another approach here instead of using a procfile etc would be to cross-compile your binary (usually pretty painless in go) and upload it that way, as per the simple instructions in the guide:
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/go-environment.html
You can just compile it locally with:
GOARCH=amd64 GOOS=linux go build -o bin/application ./cmd/vano
Then upload zip of the application file and it should work, assuming your setup only requires this one binary to run.
I try to support the use of the webp format with EB, however it's not working as expected...
I created a .config file in .ebextensions with this:
commands:
01-command:
command: wget https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-0.5.0.tar.gz
02-command:
command: tar xvzf libwebp-0.5.0.tar.gz
03-command:
command: cd libwebp-0.5.0
04-command:
command: ./configure
05-command:
command: make
06-command:
command: sudo make install
But when deploying I got this error:
ERROR: Command failed on instance. Return code: 127 Output: /bin/sh: ./configure: No such file or directory.
Am I doing something wrong?
(environment: 64bit Amazon Linux 2015.09 v2.0.6 running PHP 5.6)
You need to execute the install post deployment. AWS hasn't really documented how to execute commands post deployment, so I'll do so here.
commands:
create_post_dir:
command: "mkdir /opt/elasticbeanstalk/hooks/appdeploy/post"
ignoreErrors: true
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/99_install_libwebp.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
. /opt/elasticbeanstalk/support/envvars
cd $EB_CONFIG_APP_CURRENT
wget https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-0.5.0.tar.gz
tar xvzf libwebp-0.5.0.tar.gz
cd libwebp-0.5.0
sudo ./configure
sudo make
sudo make install
As I mentioned, AWS has not really documented that you can actually execute scripts on ElasticBeanstalk post deployment. If you talk a look in the eb-commandprocessor.log file you will see that eb looks for AppDeployPreHook (4 of 6) and AppDeployPostHook (1 of 2). It will look something like this:
[2016-04-13T14:15:22.955Z] DEBUG [8851] : Loaded 6 actions for stage 0.<br>
[2016-04-13T14:15:22.955Z] INFO [8851] : Running 1 of 6 actions: InfraWriteConfig...<br>
[2016-04-13T14:15:22.962Z] INFO [8851] : Running 2 of 6 actions: DownloadSourceBundle...<br>
[2016-04-13T14:15:23.606Z] INFO [8851] : Running 3 of 6 actions: EbExtensionPreBuild...<br>
[2016-04-13T14:15:24.229Z] INFO [8851] : Running 4 of 6 actions: AppDeployPreHook...<br>
[2016-04-13T14:15:28.469Z] INFO [8851] : Running 5 of 6 actions: EbExtensionPostBuild...<br>
[2016-04-13T14:15:28.970Z] INFO [8851] : Running 6 of 6 actions: InfraCleanEbextension...<br>
[2016-04-13T14:15:28.974Z] INFO [8851] : Running stage 1 of command CMD-AppDeploy...<br>
[2016-04-13T14:15:28.974Z] DEBUG [8851] : Loaded 2 actions for stage 1.<br>
[2016-04-13T14:15:28.974Z] INFO [8851] : Running 1 of 2 actions: AppDeployEnactHook...<br>
[2016-04-13T14:15:29.600Z] INFO [8851] : Running 2 of 2 actions: AppDeployPostHook...<br>
[2016-04-13T14:16:42.048Z] INFO [8851] : Running AddonsAfter for command CMD-AppDeploy... <br>
That little "AppDeployPostHook" tells us that it is searching for scripts to run post deployment. You can find the eb deployment scripts in the /opt/elasticbeanstalk directory on the server, and if you ssh in and ls on that directory you'll find hooks, which is what we're looking for, and if you cd hooks you'll find the appdeploy directory, cd appdeploy and then ls and you'll get two directories pre and enact. This seems mundane but is really great, because now we know where eb is looking for scripts it's running. Since the AppDeployPreHook scripts are executing from the "pre" directory we know that we'll need a "post" directory to execute a command post deployment with that AppDeployPostHook that eb is running. Now that we know what to do, we can start writing our commands.
create_post_dir First step is to actually going to create the "post" directory on the server using the mkdir command. mkdir "/opt/elasticbeanstalk/hooks/appdeploy/post" will do that for us, so we'll create that as the command.
files The files config allows us to create a file in a directory via ElasticBeanstalk. Pretty convenient for our purposes! The first line of the files action gives us the name of the file to create. We'll create a shell script to execute out commands, and you can call it whatever you want, but I'd start with 99 and go onwards. We'll call this shell script that we're creating "99_install_libwebp.sh".
File settings The next three lines set the file settings. Make sure root owns them and that there 000755'd.
File Contents This is the content of the file we're creating. Straight forward. Put your shell script in there and you're good to go.
Load environment vars We opted to load the eb environment variables so our script can know where the current version of the app is. It's usually in /var/app/current but it could be elsewhere depending on a variety of factors. We'll use the environment variables to make life a bit easier for us.
Change to our current app directory We're going to cd to our current app directory so we can do what we we're here to do.
Get the package we want use wget to get the libwebp we want
Unpack the package self explanatory
Change to the package directory Now that we've unpacked the package we can change to the package directory.
Do what we need to do We can now run our ./configure, make, and make install.
That's it. You can use the stealthy AppDeployPostHook to run pretty much any post deployment command that you need. Super useful if you need to install packages, restart services, or do anything else post deployment.
I added the code I deployed to Github, for easy reference too. https://github.com/hephalump/testphp
Note: I did this successfully running a slightly different environment. I used ElasticBeanstalk to deploy a new PHP application using the latest environment version which is PHP 5.6 on 64bit Amazon Linux 2016.03 v2.1.0; the environment type that you are using was not available as an option to me... Actually, this was the only version with PHP 5.6 that was available to me so I went with that.