How do I deploy a Go app with modules to AWS EB - amazon-web-services

When I deploy the app, it runs fine on first install. But any following eb deploy procedures fail with an error that: go.mod was found, but not expected.
Is there a specific configuration I have to set for deploying with Go modules?
I switched to Dockerizing the app and deploying that way, which works fine. But it sounds a bit cumbersome to me as AWS Elastic Beanstalk provided specific Go environments.

You can work with go modules.
build.sh
#!/usr/bin/env bash
set -xe
# get all of the dependencies needed
go get
# create the application binary that EB uses
go build -o bin/application application.go
and override GOPATH to point to $HOME which defaults to /var/app/current as given in the EB configuration management dashboard.
.ebextensions/go.config
option_settings:
aws:elasticbeanstalk:application:environment:
GOPATH: /home/ec2-user

I had the same problem, I was finally able to fix it adding this line in my build.sh script file:
sudo rm /var/app/current/go.*
So it is like this, in my case:
#!/usr/bin/env bash
# Stops the process if something fails
set -xe
sudo rm /var/app/current/go.*
# get all of the dependencies needed
go get "github.com/gin-gonic/gin"
go get "github.com/jinzhu/gorm"
go get "github.com/jinzhu/gorm/dialects/postgres"
go get "github.com/appleboy/gin-jwt"
# create the application binary that eb uses
GOOS=linux GOARCH=amd64 go build -o bin/application -ldflags="-s -w"

Related

Elastic BeanStalk app deploy post hook not executing my command

I recently was able to get my Laravel app deployed using codepipeline on Elastic Beanstalk but ran into a problem. I noticed that my routes where failing because of php.conf Nginx configuration. I had to add a few lines of code to EB's nginx php.conf file to get it to work.
My problem now was that after every deployment, the instance of the application I modified the php.conf file was destroyed and recreated fresh. I wanted a way to dynamically update the file after every successful deployment. I had a version of the file I wanted versioned with my application and so wanted to create a symlink to that file after deployment.
After loads of research, I stumbled on appDeploy Hooks on Elastic Beanstalk that runs post scripts after deployment so did this
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/91_post_deploy_script.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
sudo mkdir /var/testing1
sudo ln -sfn /var/www/html/php.conf.example /etc/nginx/conf.d/elasticbeanstalk/php.conf
sudo mkdir /var/testing
sudo nginx -s reload
And this for some reason does not work. The symlink is not created so my routes are still not working..
I even added some mkdir so am sure the commands in that script runs, none of those commands ran because none of those directories where created.
Please note that if I ssh into the ec2 instance and run the commands there it works. That bash script also exists in the post directory and if I manually run in on the server it works too.
Any pointers to how I could fix this would be helpful. Maybe I am doing something wrong too.
Now I have gotten my scripts to run by following this. However, the script is not running. I am getting an error
2020/06/28 08:22:13.653339 [INFO] Following platform hooks will be executed in order: [01_myconf.config]
2020/06/28 08:22:13.653344 [INFO] Running platform hook: .platform/hooks/postdeploy/01_myconf.config
2020/06/28 08:22:13.653516 [ERROR] An error occurred during execution of command [app-deploy] - [RunPostDeployHooks]. Stop running the command. Error: Command .platform/hooks/postdeploy/01_myconf.config failed with error fork/exec .platform/hooks/postdeploy/01_myconf.config: permission denied
I tried to follow this forum post here to make my file executable by adding to my container command a new command like so:
01_chmod1:
command: "chmod +x .platform/hooks/postdeploy/91_post_deploy_script.sh"
I am still running into the same issue. Permission denied
Sadly, the hooks you are describing (i.e. /opt/elasticbeanstalk/hooks/appdeploy) are for Amazon Linux 1.
Since you are using Amazon Linux 2, as clarified in the comments, the hooks you are trying to use do not apply. Thus they are not being executed.
In Amazon Linux 2, there are new hooks as described here and they are:
prebuild – Files here run after the Elastic Beanstalk platform engine downloads and extracts the application source bundle, and before it sets up and configures the application and web server.
predeploy – Files here run after the Elastic Beanstalk platform engine sets up and configures the application and web server, and before it deploys them to their final runtime location.
postdeploy – Files here run after the Elastic Beanstalk platform engine deploys the application and proxy server.
The use of these new hooks is different than in Amazon Linux 1. Thus you have to either move back to Amazon Linux 1 or migrate your application to Amazon Linux 2.
General migration steps from Amazon Linux 1 to Amazon Linux 2 in EB are described here
Create a folder called .platform in your project root folder and create a file with name 00_myconf.config inside the .platform folder.
.platform/
00_myconf.config
Open 00_myconf.config and add the scripts
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/91_post_deploy_script.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
sudo mkdir /var/testing1
sudo ln -sfn /var/www/html/php.conf.example /etc/nginx/conf.d/elasticbeanstalk/php.conf
sudo mkdir /var/testing
sudo nginx -s reload
Commit your changes or reupload the project. This .platform folder will be considered in each new instance creation and your application will deploy properly in all the new instances Amazon Elastic beanstalk creates.
If you access the documentation here and scroll to the section with the title "Application example with extensions" you can see an example of the folder structure of your .platform folder so it adds your custom configuration to NGINX conf on every deploy.
You can either replace the entire nginx.conf file with your file or add additional configuration files to the conf.d directory
Replace conf file with your file on app deploy:
.platform/nginx/nginx.conf
Add configuration files to nginx.conf:
.platform/nginx/conf.d/custom.conf

Elastic Beanstalk - .ebextensions not running on deploy

I have a dotnet core application. And I'm packed it for docker.
My aim is deploying this application to EB but I need to run some commands after deploy.
Thats why I have created a Dockerfile
# https://hub.docker.com/_/microsoft-dotnet-core
FROM mcr.microsoft.com/dotnet/core/sdk:2.2
WORKDIR /
# copy csproj and restore as distinct layers
COPY . ./App/
WORKDIR /App/WebApi
RUN dotnet restore
RUN dotnet publish -c release -o /build --no-restore
WORKDIR /build
ENV ASPNETCORE_URLS=http://+:8080
EXPOSE 8080
ENTRYPOINT ["dotnet", "WebApi.dll"]
And I have a Dockerrun.aws.json
{
"AWSEBDockerrunVersion": "1"
}
And finally this is my .ebextensions/01_nginx.conf
commands:
test_command:
command: "touch /tmp/x.f"
Then I'm creating an EB application
$ eb init
and creating an enviorment
$eb create
It is deploying my application successfully.
What is expected?
When I login to my EC2 container with ssh I want to see the /tmp/x.f file.
What is the problem?
I have tried several ways, I'm sure that .ebextensions/01_nginx.conf not running any way, because /tmp/x.f file not exists.
Notes :
I'm sure that the zip file which is deployed has .ebextensions/01_nginx.conf file
I'm sure that it is not about git. Because I'm including .ebignore in my root directory.
I can react the end point without any problem, my application is deploying successfully.
What is my mistake?
A probable reason is wrong extension of your files in .ebextensions. It should be .config, not .conf:
Configuration files are YAML- or JSON-formatted documents with a .config file extension that you place in a folder named .ebextensions and deploy in your application source bundle.

Elastic Beanstalk Procfile for go

I'm trying to deploy my go restful server program to EC2 Linux using Elastic Beanstalk. The document says that I need to create a Procfile at the root. So I did. Here are the steps:
Build my go program myapp.go to using
$ go build -o myapp -i myapp.go
Create a Procfile with exact name at the root with
web: myapp
Zip up the Procfile and the myapp image to a myapp.zip file.
Upload to the server via Elastic Beanstalk console. But I keep getting Degraded health and warning with
WARN Process termination taking longer than 10 seconds.
Any suggestions. By the way, I tried to use the same procfile procedure on the simple application.go zip file came from the Elastic Beanstalk example library. It didn't work either.
I was finally able to get a Go application to deploy with Elastic Beanstalk using the eb client. There are a few things that EB requires:
The name of your main file should be application.go.
Make sure your app is listening on port 5000.
You'll need a Procfile in the main root with
web: bin/application
You'll need a Buildfile with
make: ./build.sh
And finally you'll need a build.sh file with
#!/usr/bin/env bash
# Stops the process if something fails
set -xe
# All of the dependencies needed/fetched for your project.
# FOR EXAMPLE:
go get "github.com/gin-gonic/gin"
# create the application binary that eb uses
GOOS=linux GOARCH=amd64 go build -o bin/application -ldflags="-s -w"
Then if you run eb deploy (after creating your initial eb repository), it should work. I wrote a whole tutorial for deploying a Gin application on EB here. The section specifically on deploying with Elastic Beanstalk is here.

WebP support with AWS ElasticBeanstalk

I try to support the use of the webp format with EB, however it's not working as expected...
I created a .config file in .ebextensions with this:
commands:
01-command:
command: wget https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-0.5.0.tar.gz
02-command:
command: tar xvzf libwebp-0.5.0.tar.gz
03-command:
command: cd libwebp-0.5.0
04-command:
command: ./configure
05-command:
command: make
06-command:
command: sudo make install
But when deploying I got this error:
ERROR: Command failed on instance. Return code: 127 Output: /bin/sh: ./configure: No such file or directory.
Am I doing something wrong?
(environment: 64bit Amazon Linux 2015.09 v2.0.6 running PHP 5.6)
You need to execute the install post deployment. AWS hasn't really documented how to execute commands post deployment, so I'll do so here.
commands:
create_post_dir:
command: "mkdir /opt/elasticbeanstalk/hooks/appdeploy/post"
ignoreErrors: true
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/99_install_libwebp.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
. /opt/elasticbeanstalk/support/envvars
cd $EB_CONFIG_APP_CURRENT
wget https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-0.5.0.tar.gz
tar xvzf libwebp-0.5.0.tar.gz
cd libwebp-0.5.0
sudo ./configure
sudo make
sudo make install
As I mentioned, AWS has not really documented that you can actually execute scripts on ElasticBeanstalk post deployment. If you talk a look in the eb-commandprocessor.log file you will see that eb looks for AppDeployPreHook (4 of 6) and AppDeployPostHook (1 of 2). It will look something like this:
[2016-04-13T14:15:22.955Z] DEBUG [8851] : Loaded 6 actions for stage 0.<br>
[2016-04-13T14:15:22.955Z] INFO [8851] : Running 1 of 6 actions: InfraWriteConfig...<br>
[2016-04-13T14:15:22.962Z] INFO [8851] : Running 2 of 6 actions: DownloadSourceBundle...<br>
[2016-04-13T14:15:23.606Z] INFO [8851] : Running 3 of 6 actions: EbExtensionPreBuild...<br>
[2016-04-13T14:15:24.229Z] INFO [8851] : Running 4 of 6 actions: AppDeployPreHook...<br>
[2016-04-13T14:15:28.469Z] INFO [8851] : Running 5 of 6 actions: EbExtensionPostBuild...<br>
[2016-04-13T14:15:28.970Z] INFO [8851] : Running 6 of 6 actions: InfraCleanEbextension...<br>
[2016-04-13T14:15:28.974Z] INFO [8851] : Running stage 1 of command CMD-AppDeploy...<br>
[2016-04-13T14:15:28.974Z] DEBUG [8851] : Loaded 2 actions for stage 1.<br>
[2016-04-13T14:15:28.974Z] INFO [8851] : Running 1 of 2 actions: AppDeployEnactHook...<br>
[2016-04-13T14:15:29.600Z] INFO [8851] : Running 2 of 2 actions: AppDeployPostHook...<br>
[2016-04-13T14:16:42.048Z] INFO [8851] : Running AddonsAfter for command CMD-AppDeploy... <br>
That little "AppDeployPostHook" tells us that it is searching for scripts to run post deployment. You can find the eb deployment scripts in the /opt/elasticbeanstalk directory on the server, and if you ssh in and ls on that directory you'll find hooks, which is what we're looking for, and if you cd hooks you'll find the appdeploy directory, cd appdeploy and then ls and you'll get two directories pre and enact. This seems mundane but is really great, because now we know where eb is looking for scripts it's running. Since the AppDeployPreHook scripts are executing from the "pre" directory we know that we'll need a "post" directory to execute a command post deployment with that AppDeployPostHook that eb is running. Now that we know what to do, we can start writing our commands.
create_post_dir First step is to actually going to create the "post" directory on the server using the mkdir command. mkdir "/opt/elasticbeanstalk/hooks/appdeploy/post" will do that for us, so we'll create that as the command.
files The files config allows us to create a file in a directory via ElasticBeanstalk. Pretty convenient for our purposes! The first line of the files action gives us the name of the file to create. We'll create a shell script to execute out commands, and you can call it whatever you want, but I'd start with 99 and go onwards. We'll call this shell script that we're creating "99_install_libwebp.sh".
File settings The next three lines set the file settings. Make sure root owns them and that there 000755'd.
File Contents This is the content of the file we're creating. Straight forward. Put your shell script in there and you're good to go.
Load environment vars We opted to load the eb environment variables so our script can know where the current version of the app is. It's usually in /var/app/current but it could be elsewhere depending on a variety of factors. We'll use the environment variables to make life a bit easier for us.
Change to our current app directory We're going to cd to our current app directory so we can do what we we're here to do.
Get the package we want use wget to get the libwebp we want
Unpack the package self explanatory
Change to the package directory Now that we've unpacked the package we can change to the package directory.
Do what we need to do We can now run our ./configure, make, and make install.
That's it. You can use the stealthy AppDeployPostHook to run pretty much any post deployment command that you need. Super useful if you need to install packages, restart services, or do anything else post deployment.
I added the code I deployed to Github, for easy reference too. https://github.com/hephalump/testphp
Note: I did this successfully running a slightly different environment. I used ElasticBeanstalk to deploy a new PHP application using the latest environment version which is PHP 5.6 on 64bit Amazon Linux 2016.03 v2.1.0; the environment type that you are using was not available as an option to me... Actually, this was the only version with PHP 5.6 that was available to me so I went with that.

Elastic Beanstalk .ebextensions config file not getting deployed with git aws.push

I've linked a git branch to my Elastic Beanstalk environment and using git aws.push it deploys correctly.
I've now added a .extensions directory which contains a config script which should be creating a couple of directories. However, nothing appears to be happening.
I understand that the .extensions directory should be copied across to the ec2 instance as well but I'm not seeing it.
I've checked eb-tools.log and it's not mentioned in the upload.
Is there something additional that's required?
The script contains:
commands:
cache:
command: mkdir /tmp/cache
items:
command: mkdir /tmp/cache/items
chmod:
command: chmod -R 644 /tmp
You can find the run logs for this at /var/log/cfn-init.log.
In here I could see that the mkdir commands had worked initially but subsequently failed as the directory already existed.
Turns out that eb extensions run commands in alphabetical order so I had to change the commands to:
01command1:
02command2:
etc.
From this point on it worked fine.
Something else that was confusing me is that the .ebextensions directory in my local git repo was not appearing on the target instance directory. this is because once it's been run it will delete the directory.
Double check that your local script file has a .config extension. I was having a similar problem because my local file was called .ebextensions/01_stuff.yaml and it was fixed once I renamed it to .ebextensions/01_stuff.config.