Is this the right Make file code if I want to deploy my Golang server on Amazon AWS EC2 with this configuration:
Im not allowed to embed image so click here to see the image
Here is the Make File Code:
Shipping package
Assumes user is on MacOS, if other OS, please change PROTO_ROOT_DIR to the path of protobuf installation PROTO_ROOT_DIR = $(shell brew
--prefix)/Cellar/protobuf/3.6.0/include PROJECT_NAME = hello-grpc
Dart requires you to manually ship all google provided proto files too.
_gendart: #mkdir -p model/gen/ship/dart #protoc -I=model/protodefs --dart_out=grpc:model/gen/ship/dart model/protodefs/.proto #protoc -I$(PROTO_ROOT_DIR) --dart_out=model/gen/ship/dart $(PROTO_ROOT_DIR)/google/protobuf/.proto
_gengo: #mkdir -p model/gen #protoc -I=model/protodefs --go_out=plugins=grpc:model/gen model/protodefs/*.proto
gen: _gengo _gendart
build: get gen #env CGO_ENABLED=0 GOOS=linux GOARCH=386 go build
-ldflags '-w -extldflags "-static"' -o build/${PROJECT_NAME}_linux_amd64 . #env GOARCH=amd64 go build
-ldflags '-w -extldflags "-static"' -o build/${PROJECT_NAME}_macosx_amd64 .
get: #go get -u github.com/golang/dep/cmd/dep #dep ensure
install: get gen #cp config_template.json config.json
My confusion is that for building files for Amazon AWS EC2 with the above configuration I need to use (assuming my server go file name is main):
GOOS=linux GOARCH=amd64 go build -o main
But in the Make file this "GOARCH=386" is mentioned in this line:
build: get gen #env CGO_ENABLED=0 GOOS=linux GOARCH=386 go build
What could be the right make file code when Im building Go gRPC Server on Mac OS for the above mentioned Amazon AWS EC2 instance
Please help me :(
Related
Im trying to download a file using wget using Cloudformation. However, I dont see the file in the directory I downloaded it to ( /home/ubuntu/). Ive also tried different things like creating directories in /home/ubuntu but I also dont see the created directory
The following is the portition of the cloudformation code I am having trouble with. Lets assume that I am calling these commands correctly with cfn-init. I see the user data execute the cfn-init command but I dont see the files.
I wget the file and put it into /home/ubuntu/odbc_connector.tar.gz. When I go onto the server I do not find the file.
Instance:
Type: AWS::EC2::Instance
Metadata:
AWS::CloudFormation::Init:
configSets:
Instance_install:
# Install ODBC connector / pyodbc
- setup_pyodbc
setup_pyodbc:
commands:
# Install ODBC connector
download_connector:
command: wget https://dev.mysql.com/get/Downloads/Connector-ODBC/8.0/mysql-connector-odbc-8.0.19-linux-ubuntu18.04-x86-64bit.tar.gz -O /home/ubuntu/odbc_connector.tar.gz
Your wget command should be:
wget https://dev.mysql.com/get/Downloads/Connector-ODBC/8.0/mysql-connector-odbc-8.0.19-linux-ubuntu18.04-x86-64bit.tar.gz -O /home/ubuntu/odbc_connector.tar.gz
There may be other issues, that fail in your template, but you should still correct your wget command.
You can use the below command to debug the issue. You'll have to go into the instance =>
cat /var/log/cloud-init-output.log - to check cf error
cat /var/lib/cloud/instance/scripts/part-001 - to view script
The cloud-init-output.log should tell you why the command didn't run
When I deploy the app, it runs fine on first install. But any following eb deploy procedures fail with an error that: go.mod was found, but not expected.
Is there a specific configuration I have to set for deploying with Go modules?
I switched to Dockerizing the app and deploying that way, which works fine. But it sounds a bit cumbersome to me as AWS Elastic Beanstalk provided specific Go environments.
You can work with go modules.
build.sh
#!/usr/bin/env bash
set -xe
# get all of the dependencies needed
go get
# create the application binary that EB uses
go build -o bin/application application.go
and override GOPATH to point to $HOME which defaults to /var/app/current as given in the EB configuration management dashboard.
.ebextensions/go.config
option_settings:
aws:elasticbeanstalk:application:environment:
GOPATH: /home/ec2-user
I had the same problem, I was finally able to fix it adding this line in my build.sh script file:
sudo rm /var/app/current/go.*
So it is like this, in my case:
#!/usr/bin/env bash
# Stops the process if something fails
set -xe
sudo rm /var/app/current/go.*
# get all of the dependencies needed
go get "github.com/gin-gonic/gin"
go get "github.com/jinzhu/gorm"
go get "github.com/jinzhu/gorm/dialects/postgres"
go get "github.com/appleboy/gin-jwt"
# create the application binary that eb uses
GOOS=linux GOARCH=amd64 go build -o bin/application -ldflags="-s -w"
I have a pre-existing golang project with the a following folder structure (minimized the folder for readability).
- postgre
- service.go
- cmd
- vano
- main.go
- vanoctl
- main.go
vano.go
Now since my project web server is in ./cmd/vano I need to create a custom Buildfile and Procfile. So I did that
Here is my Buildfile
make: ./build.sh
build.sh file:
#!/usr/bin/env bash
# Install dependencies.
go get ./...
# Build app
go build ./cmd/vano -o bin/application
and finally my Procfile:
web: bin/application
So now my folder structure looks like this:
- postgre
- service.go
- cmd
- vano
- main.go
- vanoctl
- main.go
vano.go
Buildfile
build.sh
Procfile
I zip up the source using git:
git archive --format=zip HEAD > vano.zip
And upload it to AWS Beanstalk. How ever I keep getting errors and AWS errors don't seem to be the most read. Here is my error
Command execution completed on all instances. Summary: [Successful: 0, Failed: 1].
Error Message
[Instance: i-0d8f642474e3b2c68] Command failed on instance. Return code: 1 Output: (TRUNCATED)...' Failed to execute 'HOME=/tmp /opt/elasticbeanstalk/lib/ruby/bin/ruby /opt/elasticbeanstalk/lib/ruby/bin/foreman start --procfile /tmp/d20170213-1941-1baz0rh/eb-buildtask-0 --root /var/app/staging --env /var/elasticbeanstalk/staging/elasticbeanstalk.env'. Hook /opt/elasticbeanstalk/hooks/appdeploy/pre/01_configure_application.sh failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
Extra Error info:
Failed to execute 'HOME=/tmp /opt/elasticbeanstalk/lib/ruby/bin/ruby /opt/elasticbeanstalk/lib/ruby/bin/foreman start --procfile /tmp/d20170213-1941-1baz0rh/eb-buildtask-0 --root /var/app/staging --env /var/elasticbeanstalk/staging/elasticbeanstalk.env'
Another approach here instead of using a procfile etc would be to cross-compile your binary (usually pretty painless in go) and upload it that way, as per the simple instructions in the guide:
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/go-environment.html
You can just compile it locally with:
GOARCH=amd64 GOOS=linux go build -o bin/application ./cmd/vano
Then upload zip of the application file and it should work, assuming your setup only requires this one binary to run.
I try to support the use of the webp format with EB, however it's not working as expected...
I created a .config file in .ebextensions with this:
commands:
01-command:
command: wget https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-0.5.0.tar.gz
02-command:
command: tar xvzf libwebp-0.5.0.tar.gz
03-command:
command: cd libwebp-0.5.0
04-command:
command: ./configure
05-command:
command: make
06-command:
command: sudo make install
But when deploying I got this error:
ERROR: Command failed on instance. Return code: 127 Output: /bin/sh: ./configure: No such file or directory.
Am I doing something wrong?
(environment: 64bit Amazon Linux 2015.09 v2.0.6 running PHP 5.6)
You need to execute the install post deployment. AWS hasn't really documented how to execute commands post deployment, so I'll do so here.
commands:
create_post_dir:
command: "mkdir /opt/elasticbeanstalk/hooks/appdeploy/post"
ignoreErrors: true
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/99_install_libwebp.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
. /opt/elasticbeanstalk/support/envvars
cd $EB_CONFIG_APP_CURRENT
wget https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-0.5.0.tar.gz
tar xvzf libwebp-0.5.0.tar.gz
cd libwebp-0.5.0
sudo ./configure
sudo make
sudo make install
As I mentioned, AWS has not really documented that you can actually execute scripts on ElasticBeanstalk post deployment. If you talk a look in the eb-commandprocessor.log file you will see that eb looks for AppDeployPreHook (4 of 6) and AppDeployPostHook (1 of 2). It will look something like this:
[2016-04-13T14:15:22.955Z] DEBUG [8851] : Loaded 6 actions for stage 0.<br>
[2016-04-13T14:15:22.955Z] INFO [8851] : Running 1 of 6 actions: InfraWriteConfig...<br>
[2016-04-13T14:15:22.962Z] INFO [8851] : Running 2 of 6 actions: DownloadSourceBundle...<br>
[2016-04-13T14:15:23.606Z] INFO [8851] : Running 3 of 6 actions: EbExtensionPreBuild...<br>
[2016-04-13T14:15:24.229Z] INFO [8851] : Running 4 of 6 actions: AppDeployPreHook...<br>
[2016-04-13T14:15:28.469Z] INFO [8851] : Running 5 of 6 actions: EbExtensionPostBuild...<br>
[2016-04-13T14:15:28.970Z] INFO [8851] : Running 6 of 6 actions: InfraCleanEbextension...<br>
[2016-04-13T14:15:28.974Z] INFO [8851] : Running stage 1 of command CMD-AppDeploy...<br>
[2016-04-13T14:15:28.974Z] DEBUG [8851] : Loaded 2 actions for stage 1.<br>
[2016-04-13T14:15:28.974Z] INFO [8851] : Running 1 of 2 actions: AppDeployEnactHook...<br>
[2016-04-13T14:15:29.600Z] INFO [8851] : Running 2 of 2 actions: AppDeployPostHook...<br>
[2016-04-13T14:16:42.048Z] INFO [8851] : Running AddonsAfter for command CMD-AppDeploy... <br>
That little "AppDeployPostHook" tells us that it is searching for scripts to run post deployment. You can find the eb deployment scripts in the /opt/elasticbeanstalk directory on the server, and if you ssh in and ls on that directory you'll find hooks, which is what we're looking for, and if you cd hooks you'll find the appdeploy directory, cd appdeploy and then ls and you'll get two directories pre and enact. This seems mundane but is really great, because now we know where eb is looking for scripts it's running. Since the AppDeployPreHook scripts are executing from the "pre" directory we know that we'll need a "post" directory to execute a command post deployment with that AppDeployPostHook that eb is running. Now that we know what to do, we can start writing our commands.
create_post_dir First step is to actually going to create the "post" directory on the server using the mkdir command. mkdir "/opt/elasticbeanstalk/hooks/appdeploy/post" will do that for us, so we'll create that as the command.
files The files config allows us to create a file in a directory via ElasticBeanstalk. Pretty convenient for our purposes! The first line of the files action gives us the name of the file to create. We'll create a shell script to execute out commands, and you can call it whatever you want, but I'd start with 99 and go onwards. We'll call this shell script that we're creating "99_install_libwebp.sh".
File settings The next three lines set the file settings. Make sure root owns them and that there 000755'd.
File Contents This is the content of the file we're creating. Straight forward. Put your shell script in there and you're good to go.
Load environment vars We opted to load the eb environment variables so our script can know where the current version of the app is. It's usually in /var/app/current but it could be elsewhere depending on a variety of factors. We'll use the environment variables to make life a bit easier for us.
Change to our current app directory We're going to cd to our current app directory so we can do what we we're here to do.
Get the package we want use wget to get the libwebp we want
Unpack the package self explanatory
Change to the package directory Now that we've unpacked the package we can change to the package directory.
Do what we need to do We can now run our ./configure, make, and make install.
That's it. You can use the stealthy AppDeployPostHook to run pretty much any post deployment command that you need. Super useful if you need to install packages, restart services, or do anything else post deployment.
I added the code I deployed to Github, for easy reference too. https://github.com/hephalump/testphp
Note: I did this successfully running a slightly different environment. I used ElasticBeanstalk to deploy a new PHP application using the latest environment version which is PHP 5.6 on 64bit Amazon Linux 2016.03 v2.1.0; the environment type that you are using was not available as an option to me... Actually, this was the only version with PHP 5.6 that was available to me so I went with that.
I am trying to copy files out to a network directory during a build, and I keep getting a " No such file or directory" error message.
Copying to local drive works fine:
cp -Rf c:/Jenkins/deployments/TW_ISSUE_A/src c:/Jenkins/deployments/TW_ISSUE_A/target
The following all throw the same message:
cp -Rf c:/Jenkins/deployments/TW_ISSUE_A/src H:/some_dir
cp -Rf c:/Jenkins/deployments/TW_ISSUE_A/src H:\some_dir
cp -Rf c:/Jenkins/deployments/TW_ISSUE_A/src //Hubbell/MISGenl/some_dir
cd c:/Jenkins/deployments/TW_ISSUE_A/src
rsync -avuzb //Hubbell/MISGenl/Projects/Tronweb/TronwebBuilds/test/ora/sql/
cp -Rf c:/Jenkins/deployments/TW_ISSUE_A/src /cygdrive/h/some_dir
I've even created a shell script to call from Jenkins, but I continue to receive that message.
#!/bin/bash
url="http://as-test02:8080/job/TW_ISSUE_A_BUILD/lastSuccessfulBuild/artifact/bui
ld-TW_ISSUE_A_BUILD.tar";
remote_stage_dir="/cygdrive/h/some_dir"
#fetch the artifacts
(cd "$remote_stage_dir" && wget "$url" && tar xvf build-TW_ISSUE_A_BUILD.tar dat
java ora && rm -rf *.tar && cp -r ./ora/* ../INTEGRATION)
Is there any way to copy files out to a mapped drive on the build machine?
Thank you!!
I would guess the mapped drive isn't available in the services context, or that the user executing Jenkins doesn't have access to it. What user is Jenkins running as?
Edit:
I think your problem has two aspects:
The user running the Jenkins service isn't allowed to connect to the network.
h: isn't known to the user.
If you haven't modified it, the service is most likely running under the LocalSystem account. You can modify this by running services.msc (or navigate to services via the Windows control panel) and locating the jenkins service. This should resolve the first problem.
The second problem can be resolved by using UNC paths (as you tried above) instead of network drives.
The Jenkins wiki has an article about problems like this: My software builds on my computer but not on Jenkins