Running gcloud run deploy from inside Cloud Build results in error - google-cloud-platform

I have a custom build step in Google Cloud Build, which first builds a docker image and then deploys it as a cloud run service.
This last step fails, with the following log output;
Step #2: Deploying... Step #2: Setting IAM Policy.........done Step
2: Creating Revision............................................................................................................................failed
Step #2: Deployment failed Step #2: ERROR: (gcloud.run.deploy) Cloud
Run error: Invalid argument error. Invalid ENTRYPOINT. [name:
"gcr.io/opencobalt/silo#sha256:fb860e758eb1957b90ff3761fcdf68dedb9d10f832f2bb21375915d3de2aaed5"
Step #2: error: "Invalid command \"/bin/sh\": file not found" Step #2:
]. Finished Step #2 ERROR ERROR: build step 2
"gcr.io/cloud-builders/gcloud" failed: step exited with non-zero
status: 1
The build steps look like this;
["run","deploy","silo","--image","gcr.io/opencobalt/silo","--region","us-central1","--platform","managed","--allow-unauthenticated"]}
The image is built an exists in the registry, and if I change the last build step to deploy a compute engine VM instead, it works. Those build steps looks like this;
{"name":"gcr.io/cloud-builders/gcloud","args":["compute","instances",
"create-with-container","silo","--container-image","gcr.io/opencobalt/silo","--zone","us-central1-a","--tags","silo,pharo"]}
I can also build the image locally but run into the same error when running gcloud run deploy locally.
I am trying to figure out how to solve this problem. The image works, since it runs fine locally and runs fine when deployed as a Compute Engine VM, the error only show up when I'm trying to deploy the image as a Cloud Run service.
(added) The Dockerfile looks like this;
######################################
# Based on Ubuntu image
######################################
FROM ubuntu
######################################
# Basic project infos
######################################
LABEL maintainer="PeterSvensson"
######################################
# Update Ubuntu apt and install some tools
######################################
RUN apt-get update \
&& apt-get install -y wget \
&& apt-get install -y git \
&& apt-get install -y unzip \
&& rm -rf /var/lib/apt/lists/*
######################################
# Have an own directory for the tool
######################################
RUN mkdir webapp
WORKDIR webapp
######################################
# Download Pharo using Zeroconf & start script
######################################
RUN wget -O- https://get.pharo.org/64/80+vm | bash
COPY service_account.json service_account.json
RUN export certificate="$(cat service_account.json)"
COPY load.st load.st
COPY setup.sh setup.sh
RUN chmod +x setup.sh
RUN ./setup.sh; echo 0
RUN ./pharo Pharo.image load.st; echo 0
######################################
# Expose port 8080 of Zinc outside the container
######################################
EXPOSE 8080
######################################
# Finally run headless as server
######################################
CMD ./pharo --headless Pharo.image --no-quit
Any advice warmly welcome.
Thank you.

After a lot of testing, I managed to come further. It seems that the /bin/sh missing file thing is a red herring.
I tried to change the startup command from CMD to ENTRYPOINT, since that was mentioned in the error, but it did not work. However, when I copied the startup instruction into a new file 'startup.sh' and changed the last line of the Dockerfile to;
ENTRYPOINT ./startup.sh
It did work. I needed to chmod +x the new file of course, but the strange thing is that ENTRYPOINT ./pharo --headless Pharo.image --no-quit gave the same error, and even ENTRYPOINT ["./pharo", "--headless", "Pharo.image", "--no-quit"] also gave the same error.
But having just one argument to ENTRYPOINT made cloud run work. Go figure.

It appears that Google Cloud Run has a dislike for the ubuntu:20.04 image. I have the exact same problem with a Play framework application.
The command
ENTRYPOINT /opt/play-codecheck/bin/play-codecheck -Dconfig.file=/opt/codecheck/production.conf
failed with
error: "Invalid command \"/bin/sh\": file not found"
I also tried
ENTRYPOINT ["/bin/bash", "/opt/play-codecheck/bin/play-codecheck", "-Dconfig.file=/opt/codecheck/production.conf"]
and was rewarded with
error: "Invalid command \"/bin/bash\": file not found"
The trick of putting the command in a shell script didn't work for me either. However, when I changed
FROM ubuntu:20.04
to
FROM ubuntu:18.04
the image deployed. At this point, that's an acceptable fix for me, but it seems like something that Google needs to address.

See also:
Unable to deploy Ubuntu 20.04 Docker container on Google Cloud Run
My workaround was to use a CMD directive that calls Python directly rather than a shell (either /bin/sh or /bin/bash). It's working well so far.

Related

"RUN true" in dockerfile

When I took over a project, I found a command "RUN true" in the Dockerfile.
FROM xxx
RUN xxx
RUN true
RUN xxx
I don't know what this command does, can anyone help explain. In my opinion, this command makes no sense, but i'm not sure if there is any other use.
There is doc about Creating Images, you can see it:
RUN true \
&& dnf install -y --setopt=tsflags=nodocs \
httpd vim \
&& systemctl enable httpd \
&& dnf clean all \
&& true
#David Maze
test for it. docker file:
FROM centos:7.9.2009
RUN yum install tmux -y
RUN yum install not_exists -y
build log:
Sending build context to Docker daemon 2.048kB
Step 1/3 : FROM centos:7.9.2009
---> eeb6ee3f44bd
Step 2/3 : RUN yum install tmux -y
---> Running in 6c6e29ea9f2c
...omit...
Complete!
Removing intermediate container 6c6e29ea9f2c
---> 7c796c2b5260
Step 3/3 : RUN yum install not_exists -y
---> Running in e4b7096cc42b
...omit...
No package not_exists available.
Error: Nothing to do
The command '/bin/sh -c yum install not_exists -y' returned a non-zero code: 1
modify dockefile:
FROM centos:7.9.2009
RUN yum install tmux -y
RUN yum install tree -y
build log:
Sending build context to Docker daemon 2.048kB
Step 1/3 : FROM centos:7.9.2009
---> eeb6ee3f44bd
Step 2/3 : RUN yum install tmux -y
---> Using cache
---> 7c796c2b5260
Step 3/3 : RUN yum install tree -y
---> Running in 180b32cb44f3
...omit...
Installed:
tree.x86_64 0:1.6.0-10.el7
Complete!
Removing intermediate container 180b32cb44f3
---> 4e905ed25cc2
Successfully built 4e905ed25cc2
Successfully tagged test:v0
you can see Using cache 7c796c2b5260. without a command "RUN true", but the first "RUN" cache is reusged.
RUN true as a standalone command does absolutely nothing and it's safe to delete it.
/bin/true is a standard shell command. It reads no input, produces no output, and neither reads nor writes files; it just exits with a status code of 0 ("success"). Running it as a Docker step will have no effect on the final image other than inserting an additional layer into the docker history.
The one clever use I can think of for this is to cause a later part of a Dockerfile to re-run. Imagine a Dockerfile like
RUN some_expensive_command http://server-a.example.com/input1
RUN another_expensive_command http://server-b.example.com/input2
If the second input changes, you could want to rebuild this image. docker build --no-cache will re-run the first step too, though, and this could take longer than you want. Inserting a RUN true line between the two lines would break Docker's layer caching, but only after the first command has run.
# identical RUN line as before, from cache
RUN some_expensive_command http://server-a.example.com/input1
# not the same RUN line, so "executes" (but does nothing)
RUN true
# not running commands from cache any more
RUN another_expensive_command http://server-b.example.com/input2
I found an already existing answer which explains it quite well.
And if I quote the answer here:
Running and thus creating a new container even if it terminates still keeps the resulting container image and metadata lying around which can still be linked to.
So when you run docker run ... /bin/true you are essentially creating a new container for storage purposes and running the simplest thing you can.
In Docker 1.5 was introduced the docker create command so I believe you can now "create" containers without confusingly running something like /bin/true
And I found an quick explanation from the best practices github page under section '#chaining-commands' saying:
The first and last commands of the block are special.
If you would like to prepend or append a one-line command to the block, you will have to edit two lines - one that you are adding and the first or last commands. The first command is on the same line as the RUN directive, whereas the last command lacks the trailing backslash.
Editing a line with a command that that you don’t want to change presents a risk of introducing a bug, and you also obscure the line’s history. This can be mitigated by having both the first and last commands true - they don’t do anything.

docker image runs ok locally but in ECS I get a message: executable file not found in $PATH

I've a weird error, I'm trying to run a python script in ECS, the dockerfile is pretty basic:
FROM python:3.8
COPY . /
RUN pip install -r requirements.txt
CMD ["python", "./get_historical_data.py"]
building this in my local machine works perfect,
docker run --network=host historical-price
I uploaded this image to ECR and run on ECS, a basic config, just set container name, pointing the Image to my ECR repo and set some environment variables...when I run this I get
Status reason CannotStartContainerError: Error response from daemon: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "python": executable file not found in $PATH: unknown
but (really weird) if I enter in the EC2 server and run the container manually
docker run -it -e TICKER='SOL/USDT' -e EXCHANGE='BINANCE' -e DB_HOST='xxx' -e DB_NAME='xxx' -e DB_PASSWORD='xxx' -e DB_PORT='xxx' -e DB_USER='xxx' xxx.dkr.ecr.ap-southeast-2.amazonaws.com/xxx:latest /bin/bash
I can see this running ok...
I've tried several dockerfiles, using
CMD python ./get_historical_data.py
or using python3 command instead of python
also I tried to skip the CMD command in the Dockerfile and add this in the ECS task definition
nothing work...
I really don't know what can be happen here because the last week I ran a similar task and this worked perfectly, hope you can help me
thank you, please let me know if you need more details

Can't launch Chromium as executable doesn't exists if customized image is build from Dockerfile

FROM alpine:latest
# Copy source to container
RUN mkdir -p /usr/app
# Copy source code
COPY package.json /usr/app
COPY package-lock.json /usr/app
COPY . /usr/app
# Set working directory
WORKDIR /usr/app
# Environment variables
ENV BASE_URL="Local https url"
ENV PARALLEL_RUN=false
ENV TAG=int
ENV PLAYWRIGHT_BROWSERS_PATH=/usr/lib
# npm install
RUN apk add --update npm
RUN apk add chromium
# Run tests
RUN npm run codeceptjs
Above is the Dockerfile. When tried to Build the image from docker file then I am getting below error:
13 8.596 Error: browserType.launch: Failed to launch chromium because executable doesn't exist at /usr/lib/chromium-888113/chrome-linux/chrome
#13 8.596 Try re-installing playwright with "npm install playwright"**
Although, I can see chromium is getting installed at the mentioned path but still it saying "executable not present".
I believe your problem lies with using alpine.
According to the playwright developers, there are no plans to support playwright on alpine. This makes your whole undertaking more complex. It's correct that you need to provide your own chromium and cannot use the browsers that come with playwright. Therefore, you should set PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD to prevent any (incompatible) browsers from being loaded.
The chromium executable should be in your Docker image under /usr/bin/chromium-browser. You need to use playwright's browserType.launch to set the path to the executable:
const { chromium } = require("playwright-chromium");
// ...
const browser = await chromium.launch({
executablePath: process.env.PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH,
});
If you want a simpler solution, I would suggest using the Docker image of Zanika, containing chromium and playwright already. Here the link to the tag on DockerHub. At the very least you can see it as a reference implementation if you still want to use your own image.

why is the serverless framework rust plugin failling in dockerless and dockerised build mode

I am currently building my rust lambda using the dockerless method, it builds and deploys fine but when I attempt to test it throws the error:
Error: fork/exec /var/task/bootstrap: no such file or directory Runtime.InvalidEntrypoint
I then tried building it using a builder image following this method but that when it ran it returned the error
/var/task/bootstrap: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by /var/task/bootstrap)
The image used for the build is
FROM lambci/lambda:20200812-build-provided.al2
ARG RUST_VERSION=1.51.0
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \
| CARGO_HOME=/cargo RUSTUP_HOME=/rustup sh -s -- -y --profile minimal --default-toolchain $RUST_VERSION
ADD build.sh /usr/local/bin/
RUN chmod 777 -R /usr/local
VOLUME ["/code"]
WORKDIR /code
ENTRYPOINT ["/usr/local/bin/build.sh"]
Which does have an version of glibc higher that 2.18
At this point I am out of ideas as to what to try next and would really appreciate it if some one could give me any ideas that allow me to deploy my rust lambda functions using the serverless framework
P.s I did originally try using the plugin with it's defaults but its to far out of date to be usefull

How do I make bower install work with aws.push?

As a starting point to making my own app that uses meanjs, I went to the meanjs website and used their yeomen generator to create the template/sample app. Following the instructions getting the sample application running out of the box on my local desktop machine worked within minutes. To complete the exercise I tried to deploy the sample app to an AWS/EC2 instance before making any changes to it. I have used the command line deployment tools in the past and liked it. Also it is nice how now you can just select an EC2 Linux instance with node and npm already installed and ready.
After checking the sample into git, I run "git aws.push" to deploy the app.
The problem is in the package.json the line:
"postinstall": "bower install --config.interactive=false"
In the eb-activity.log:
npm WARN cannot run in wd meansample#0.0.1 bower install --config.interactive=false (wd=/tmp/deployment/application)
The result is that AngularJS ends up not getting installed in /public/lib.
First thing I tried was giving the full path in the package.json file: node_modules/bower/bin/bower. This didn't help and results in the same error. Also noting that other commands like "grunt" don't need the full path specified in the package.json and they work.
I don't understand enough of the black box magic that aws.push does to understand why this error is happening. For example what user does it run as? What permissions does that user have? what options if any does it use when it runs npm install?
I did figure out a work-around, but it adds a lot of extra steps that shouldn't be required if aws.push was able to run bower install directly. Basically I can manually run the bower install in the ssh client connected to my EC2 instance, set the owner/group on the installed files, and restart the server.
Work-around steps:
1) On local command prompt run git aws.push. Wait for unsuccessfully deployment to finish.
2) Connect ssh client to EC2 instance. From the command prompt:
cd /var/app/current
/* NOTE: if I don't use sudo the ec2user I am logged in as does not have permission to create /public/lib needed to install AngularJS into*/
sudo node_modules/bower/bin/bower install --config.interactive=false --allow-root
/* NOTE: just changing the owner and group to match the same as the other files that aws.push deployed */
sudo chown -R nodejs public/lib
sudo chgrp -R nodejs public/lib
From AWS dashboard, select the correct EC2 instance, Action = Restart App Server(s)
Now AngularJS is install and the sample app works.
How do I eliminate the extra steps and make it so aws.push can do the bower install successfully?
I have experienced the same problem when trying to publish my nodejs app in a private server running CentOs using root user. The same error is fired by "postinstall": "./node_modules/bower/bin/bower install" in my package.json file so the only solution that was working for me is to use both options to avoid the error:
1: use --allow-root option for bower install command
"postinstall": "./node_modules/bower/bin/bower --allow-root install"
2: use --unsafe-perm option for npm install command
npm install --unsafe-perm