Go & Docker: I'm able to run a go web server when using stdlib, when I use custom packages errors occur - amazon-web-services

Note the code works perfectly fine when I'm running the code on my laptop.
The following two groups of code will run on my laptop. However the second group (which uses my custom package) doesn't work on Elastic Beanstalk running docker.
Standard Lib only
import (
"net/http"
"os"
)
func main() {
port := os.Getenv("PORT")
if port == "" {
port = "3000"
}
http.ListenAndServe(":"+port, nil)
}
Uses Custom Package
import (
"os"
"github.com/sim/handlers"
)
func main() {
port := os.Getenv("PORT")
if port == "" {
port = "3000"
}
handlers.ServeAndHandle(port) // wrapper of ListenAndServe
}
Error Messages:
Failed to build Docker image aws_beanstalk/staging-app: andlers: exit status 128 [0mtime="2015-08-14T05:08:17Z" level="info" msg="The command [/bin/sh -c go-wrapper download] returned a non-zero code: 1" . Check snapshot logs for details.
2015-08-14 01:08:15 UTC-0400 WARN Failed to build Docker image aws_beanstalk/staging-app, retrying...
cron.yaml
version: 1
cron:
- name: "task1"
url: "/scheduled"
schedule: "* * * * *"

You need a Dockerfile or/and Dockerrun.aws.json for your environment as per the documentation
Dockerfile
FROM FROM golang:1.3-onbuild
EXPOSE 3000
CMD ["go run <your.go.file>"]
Dockerrun.aws.json
{
"AWSEBDockerrunVersion": "1",
"Image": {
"Name": "golang:1.3-onbuild", # <-- don't need this if you are using a Dockerfile
"Update": "true"
},
"Ports": [
{
"ContainerPort": "3000"
}
],
"Logging": "/var/log/go"
}
Using the eb command line to deploy ?

Related

Could not launch AWS SAM application when using VS Code Debugger "Function not implemented"

I have a generic SAM application generated by AWS Toolkit.
My end goal is to debug my go application line by line.
Sorry if this question gets too long, but I have tried my best to add as few relevant details as possible, while trying to make sure I don't miss anything important.
I am using the default launch.json configurations generated by the Toolkit:
{
"configurations": [
{
"type": "aws-sam",
"request": "direct-invoke",
"name": "sunflowers:HelloWorldFunction (go1.x)",
"invokeTarget": {
"target": "template",
"templatePath": "${workspaceFolder}/template.yaml",
"logicalId": "HelloWorldFunction"
},
"lambda": {
"payload": {},
"environmentVariables": {},
"runtime": "go1.x"
}
},
{
"type": "aws-sam",
"request": "direct-invoke",
"name": "API sunflowers:HelloWorldFunction (go1.x)",
"invokeTarget": {
"target": "api",
"templatePath": "${workspaceFolder}/template.yaml",
"logicalId": "HelloWorldFunction"
},
"api": {
"path": "/hello",
"httpMethod": "get",
"payload": {
"json": {}
}
},
"lambda": {
"runtime": "go1.x"
}
}
]
}
But when I run sunflowers:HelloWorldFunction (go1.x), I get the following error in the AWS Toolkit output:
could not launch process: fork/exec /var/task/hello-world: function not implemented
Full error Log:
2022-08-04 18:41:12 [INFO]: Preparing to debug locally: Lambda "hello-world"
2022-08-04 18:41:12 [INFO]: Building SAM application...
2022-08-04 18:41:12 [INFO]: Command: (not started) [/opt/homebrew/bin/sam build --build-dir /tmp/aws-toolkit-vscode/vsctky6JmVn/output --template /Users/varungawande/playground/goLambdaDebug/sunflowers/template.yaml]
2022-08-04 18:41:13 [INFO]: Your template contains a resource with logical ID "ServerlessRestApi", which is a reserved logical ID in AWS SAM. It could result in unexpected behaviors and is not recommended.
2022-08-04 18:41:13 [INFO]: Building codeuri: /Users/varungawande/playground/goLambdaDebug/sunflowers/hello-world runtime: go1.x metadata: {} architecture: x86_64 functions: HelloWorldFunction
2022-08-04 18:41:13 [INFO]: Running GoModulesBuilder:Build
2022-08-04 18:41:13 [INFO]:
Build Succeeded
2022-08-04 18:41:13 [INFO]:
Built Artifacts : ../../../../../private/tmp/aws-toolkit-vscode/vsctky6JmVn/output
Built Template : ../../../../../private/tmp/aws-toolkit-vscode/vsctky6JmVn/output/template.yaml
Commands you can use next
=========================
[*] Validate SAM template: sam validate
[*] Invoke Function: sam local invoke -t ../../../../../private/tmp/aws-toolkit-vscode/vsctky6JmVn/output/template.yaml
[*] Test Function in the Cloud: sam sync --stack-name {stack-name} --watch
[*] Deploy: sam deploy --guided --template-file ../../../../../private/tmp/aws-toolkit-vscode/vsctky6JmVn/output/template.yaml
2022-08-04 18:41:14 [INFO]: Build complete.
2022-08-04 18:41:14 [INFO]: Starting SAM application locally
2022-08-04 18:41:14 [INFO]: AWS.running.command
Invoking hello-world (go1.x)
Skip pulling image and use local one: public.ecr.aws/sam/emulation-go1.x:rapid-1.53.0-x86_64.
Mounting /tmp/aws-toolkit-vscode/vsctky6JmVn/output/HelloWorldFunction as /var/task:ro,delegated inside runtime container
START RequestId: 8cc3eb7a-20f2-4599-81bc-2f29f8d02102 Version: $LATEST
API server listening at: [::]:5858
2022-08-04T13:11:16Z warning layer=rpc Listening for remote connections (connections are not authenticated nor encrypted)
2022-08-04T13:11:16Z info layer=debugger launching process with args: [/var/task/hello-world]
2022-08-04 18:41:16 [INFO]: Waiting for SAM application to start...
could not launch process: fork/exec /var/task/hello-world: function not implemented
2022/08/04 13:11:16 exit status 1
04 Aug 2022 13:11:16,067 [ERROR] (rapid) Init failed error=Runtime exited with error: exit status 1 InvokeID=
API server listening at: [::]:5858
2022-08-04T13:11:16Z warning layer=rpc Listening for remote connections (connections are not authenticated nor encrypted)
2022-08-04T13:11:16Z info layer=debugger launching process with args: [/var/task/hello-world]
could not launch process: fork/exec /var/task/hello-world: function not implemented
2022/08/04 13:11:16 exit status 1
END RequestId: 0dcb2785-a232-41aa-8f78-f61c82add96b
REPORT RequestId: 0dcb2785-a232-41aa-8f78-f61c82add96b Init Duration: 1.26 ms Duration: 838.60 ms Billed Duration: 839 ms Memory Size: 128 MB Max Memory Used: 128 MB
2022-08-04 18:41:17 [INFO]: Attaching debugger to SAM application...
Command stopped: "sam local invoke"
2022-08-04 18:41:23 [ERROR]: Retry limit reached while trying to attach the debugger.
2022-08-04 18:41:23 [ERROR]: Unable to attach Debugger. Check AWS Toolkit logs. If it took longer than expected to start, you can still attach.
sam local start-api and sam local invoke still work.
The path they mounted at /tmp/aws-toolkit-vscode/vsctky6JmVn/output/HelloWorldFunction does have the file they're looking for.
❯ tree /tmp/aws-toolkit-vscode/vsctky6JmVn/output/HelloWorldFunction
/tmp/aws-toolkit-vscode/vsctky6JmVn/output/HelloWorldFunction
└── hello-world
The hello-world program does have a main function:
package main
import (
"errors"
"fmt"
"io/ioutil"
"net/http"
"github.com/aws/aws-lambda-go/events"
"github.com/aws/aws-lambda-go/lambda"
)
var (
// DefaultHTTPGetAddress Default Address
DefaultHTTPGetAddress = "https://checkip.amazonaws.com"
// ErrNoIP No IP found in response
ErrNoIP = errors.New("No IP in HTTP response")
// ErrNon200Response non 200 status code in response
ErrNon200Response = errors.New("Non 200 Response found")
)
func handler(request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
str := "Hello World"
str = exclaim(str)
for i := 3; i < 30; i++ {
str = exclaim(str)
fmt.Println(str, "at", i)
}
resp, err := http.Get(DefaultHTTPGetAddress)
if err != nil {
return events.APIGatewayProxyResponse{}, err
}
if resp.StatusCode != 200 {
return events.APIGatewayProxyResponse{}, ErrNon200Response
}
ip, err := ioutil.ReadAll(resp.Body)
if err != nil {
return events.APIGatewayProxyResponse{}, err
}
if len(ip) == 0 {
return events.APIGatewayProxyResponse{}, ErrNoIP
}
return events.APIGatewayProxyResponse{
Body: fmt.Sprintf("Hello, %v", string(ip)),
StatusCode: 200,
}, nil
}
func main() {
lambda.Start(handler)
}
func exclaim(str string) string {
return str + "!"
}
Note: I'm on a Mac, using the M1 chip(Arch: arm64) but the build-file is being executed for x86-64.
hello-world: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=O4tsTAbXATrthCbExZYS/IcUitw8QuK2KVYZ9ln5Y/QZl7mLeZv6TDrrYMwafh/81ITMlD5ZHrz3gae1BfY, with debug_info, not stripped```
Since it can find the file, Does this mean that the executor can't find the main?

PM2 doesn't launch Loopback 4 app on DigitalOcean Ubuntu server

I am trying to launch Loopback on Digital Ocean following this tutorial:
https://loopback.io/doc/en/lb4/deploying-with-pm2-and-nginx.html
The problem is that when I launch it with "pm2 start" command, it seems that pm2 starts, but it doesn't start the Loopback app. The logs don't show anything (screenshot - https://i.stack.imgur.com/MvaV2.png).
I double checked - the Loopback app is fine and it starts with "npm run start:local" successfully.
Here's my package.json commands:
"start:local": "node -r source-map-support/register .",
"start": "pm2 start ecosystem.config.js --env production",
"stop": "pm2 stop ecosystem.config.js --env production",
ecosystem.config.js:
module.exports = {
apps: [
{
name: 'BFF',
script: './dist/index.js',
instances: 1,
interpreter : 'node#10.20.1',
autorestart: true,
watch: true,
max_memory_restart: '1G',
env: {
NODE_ENV: 'development',
},
env_production: {
NODE_ENV: 'production',
},
},
],
};

Meteor deploy error (mup): pushing meteor app bundle to server failed

I am trying to deploy a meteor app to an AWS server, but am getting this message:
Started TaskList: Configuring App
[52.41.84.125] - Pushing the Startup Script
nodemiral:sess:52.41.84.125 copy file - src: /
Users/Olivia/.nvm/versions/node/v7.8.0/lib/node_modules/mup/lib/modules/meteor/assets/templates/start.sh, dest: /opt/CanDu/config/start.sh, vars: {"appName":"CanDu","useLocalMongo":0,"port":80,"bind":"0.0.0.0","logConfig":{"opts":{"max-size":"100m","max-file":10}},"docker":{"image":"abernix/meteord:base","imageFrontendServer":"meteorhacks/mup-frontend-server","imagePort":80},"nginxClientUploadLimit":"10M"} +0ms
[52.41.84.125] x Pushing the Startup Script: FAILED Failure
Previously I had been able to deploy using mup, but now I am getting this message. The only major thing I've changed is the Python path in my .noderc. I am also able to SSH into my amazon server directly from the terminal. My mup file is:
module.exports = {
servers: {
one: {
host: '##.##.##.###',
username: 'ec2-user',
pem: '/Users/Olivia/.ssh/oz-pair.pem'
// password:
// or leave blank for authenticate from ssh-agent
}}meteor: {
name: 'CanDu',
path: '/Users/Olivia/repos/bene_candu_v2',
servers: {
one: {}
},
buildOptions: {
serverOnly: true,
mobileSettings: {
public: {
"astronomer": {
"appId": "<key>",
"disableUserTracking": false,
"disableRouteTracking": false,
"disableMethodTracking": false
},
"googleMaps": "<key>",
"facebook":{
"permissions":["email","public_profile","user_friends"]
}
},
},
},
env: {
ROOT_URL: 'http://ec2-##-##-##-###.us-west-2.compute.amazonaws.com',
MONGO_URL: 'mongodb://. . . "
},
/*ssl: {
crt: '/opt/keys/server.crt', // this is a bundle of certificates
key: '/opt/keys/server.key', // this is the private key of the certificate
port: 443, // 443 is the default value and it's the standard HTTPS port
upload: false
},*/
docker: {
image: 'abernix/meteord:base'
},
deployCheckWaitTime: 60
}
};
And I have checked to make sure there are no trailing commas, and have tried increasing the wait time. etc. The error message I'm getting is pretty unhelpful. Does anyone have any insight? Thank you so much!

Rails app deployment with AZK fail on Digital Ocean

I'm trying to push a very simple rails app on a DigitalOcean droplet. Unfortunatly, i'm enable to continue the deployment : i get stuck with a very elusive error message :
info: [agent] get agent status
info: [agent] agent is running: true
info: [agent] get agent status
info: [agent] agent is running: true
info: [agent] get agent status
info: [agent] agent is running: true
info: [agent] get agent status
info: [agent] agent is running: true
info: Connecting to http://192.168.50.4:2375
PLAY [all] *********************************************************************
TASK [setup] *******************************************************************
fatal: [default]: FAILED! => {"changed": false, "failed": true, "module_stderr": "", "module_stdout": "/bin/sh: 1: /usr/bin/python: not found\r\n", "msg": "MODULE FAILURE", "parsed": false}
NO MORE HOSTS LEFT *************************************************************
to retry, use: --limit #playbooks/setup.retry
PLAY RECAP *********************************************************************
default : ok=0 changed=0 unreachable=0 failed=1
Am i the only one who ever encounter this problem ?
Here is my Azkfile too :
/**
* Documentation: http://docs.azk.io/Azkfile.js
*/
// Adds the systems that shape your system
systems({
'apptelier-website': {
// Dependent systems
depends: [],
// More images: http://images.azk.io
image: {docker: 'azukiapp/ruby:2.3.0'},
// Steps to execute before running instances
provision: [
"bundle install --path /azk/bundler"
],
workdir: "/azk/#{manifest.dir}",
shell: "/bin/bash",
command: ["bundle", "exec", "rackup", "config.ru", "--pid", "/tmp/ruby.pid", "--port", "$HTTP_PORT", "--host", "0.0.0.0"],
wait: 20,
mounts: {
'/azk/#{manifest.dir}': sync("."),
'/azk/bundler': persistent("./bundler"),
'/azk/#{manifest.dir}/tmp': persistent("./tmp"),
'/azk/#{manifest.dir}/log': path("./log"),
'/azk/#{manifest.dir}/.bundle': path("./.bundle")
},
scalable: {"default": 1},
http: {
domains: [
'#{env.HOST_DOMAIN}', // used if deployed
'#{env.HOST_IP}', // used if deployed
'#{system.name}.#{azk.default_domain}' // default azk domain
]
},
ports: {
// exports global variables
http: "3000/tcp"
},
envs: {
// Make sure that the PORT value is the same as the one
// in ports/http below, and that it's also the same
// if you're setting it in a .env file
RUBY_ENV: "production",
RAILS_ENV: "production",
RACK_ENV: 'production',
WORKER_RETRY: 1,
BUNDLE_APP_CONFIG: '/azk/bundler',
APP_URL: '#{system.name}.#{azk.default_domain}'
}
},
deploy: {
image: {docker: 'azukiapp/deploy-digitalocean'},
mounts: {
'/azk/deploy/src': path('.'),
'/azk/deploy/.ssh': path('#{env.HOME}/.ssh'), // Required to connect with the remote server
'/azk/deploy/.config': persistent('deploy-config')
},
// This is not a server. Just run it with `azk deploy`
scalable: {default: 0, limit: 0},
envs: {
GIT_REF: 'master',
AZK_RESTART_COMMAND: 'azk restart -Rvv',
BOX_SIZE: '512mb'
}
}
});
Thanks for the help !
Edouard, nice to meet you. I'm from azk core team and I'm not sure if you're using the latest deployment image.
Please follow these steps to update it:
adocker pull azukiapp/deploy-digitalocean:0.0.7;
Edit the deploy system in your Azkfile and add the tag 0.0.7 to the used deployment image to ensure we're using the latest one. It should be like: image: {docker: 'azukiapp/deploy-digitalocean:0.0.7'};
Next, be sure you have the env DEPLOY_API_TOKEN set in your .env file. If you don't have it set yet, take a look on the Step 7 of the article we've published on DigitalOcean Community Tutorials: https://www.digitalocean.com/community/tutorials/how-to-deploy-a-rails-app-with-azk#step-7-%E2%80%94-obtaining-a-digitalocean-api-token
Finally, re-run the deploy command:
azk deploy clear-cache;
azk deploy
Please let me know if this is enough to solve your problem.

How do you mock OpsWorks specific services/dependencies when developing locally with Kitchen and Chef?

I'm writing Chef wrappers around some of the built in OpsWorks cookbooks. I'm using Berkshelf to clone the OpsWorks cookbooks from their github repo.
This is my Berksfile:
source 'https://supermarket.getchef.com'
metadata
def opsworks_cookbook(name)
cookbook name, github: 'aws/opsworks-cookbooks', branch: 'release-chef-11.10', rel: name
end
%w(dependencies scm_helper mod_php5_apache2 ssh_users opsworks_agent_monit
opsworks_java gem_support opsworks_commons opsworks_initial_setup
opsworks_nodejs opsworks_aws_flow_ruby
deploy mysql memcached).each do |cb|
opsworks_cookbook cb
end
My metadata.rb:
depends 'deploy'
depends 'mysql'
depends 'memcached'
The problem is, when I try to override attributes that depend on the opsworks key in the node hash, I get a:
NoMethodError
-------------
undefined method `[]=' for nil:NilClass
OpsWorks has a whole bunch of pre-recipe dependencies that create these keys and do a lot of their setup. I'd like to find a way to either pull in those services and run them against my Kitchen instances or mock them in a way that I can actually test my recipes.
Is there a way to do this?
I would HIGHLY recommend you check out Mike Greiling's blog post Simplify OpsWorks Development With Packer and his github repo opsworks-vm which help you to mock the entire opsworks stack including the install of the opsworks agent so you can also test app deploy recipes, multiple layers, multiple instances at the same time, etc . It is extremely impressive.
Quick Start on Ubuntu 14.04
NOTE: This can NOT be done from an ubuntu virtual machine because virtualbox does not support nested virtualization of 64-bit machines.
Install ChefDK
mkdir /tmp/packages && cd /tmp/packages
wget https://opscode-omnibus-packages.s3.amazonaws.com/ubuntu/12.04/x86_64/chefdk_0.8.1-1_amd64.deb
sudo dpkg -i chefdk_0.8.0-1_amd64.deb
cd /opt/chefdk/
chef verify
which ruby
echo 'eval "$(chef shell-init bash)"' >> ~/.bash_profile && source ~/.bash_profile
Install VirtualBox
echo 'deb http://download.virtualbox.org/virtualbox/debian vivid contrib' > /etc/apt/sources.list.d/virtualbox.list
wget -q https://www.virtualbox.org/download/oracle_vbox.asc -O- | sudo apt-key add -
sudo apt-get update -qqy
sudo apt-get install virtualbox-5.0 dkms
Install Vagrant
cd /tmp/packages
wget https://dl.bintray.com/mitchellh/vagrant/vagrant_1.7.4_x86_64.deb
sudo dpkg -i vagrant_1.7.4_x86_64.deb
vagrant plugin install vagrant-berkshelf
vagrant plugin install vagrant-omnibus
vagrant plugin list
Install Packer
mkdir /opt/packer && cd /opt/packer
wget https://dl.bintray.com/mitchellh/packer/packer_0.8.6_linux_amd64.zip
unzip packer_0.8.6_linux_amd64.zip
echo 'PATH=$PATH:/opt/packer' >> ~/.bash_profile && source ~/.bash_profile
Build Mike Greiling's opsworks-vm virtualbox image using Packer
mkdir ~/packer && cd ~/packer
git clone https://github.com/pixelcog/opsworks-vm.git
cd opsworks-vm
rake build install
This will install a new virtualbox vm to ~/.vagrant.d/boxes/ubuntu1404-opsworks/
To mock a single opsworks instance, create a new Vagrantfile like so:
Vagrant.configure("2") do |config|
config.vm.box = "ubuntu1404-opsworks"
config.vm.provision :opsworks, type: 'shell', args: 'path/to/dna.json'
end
The dna.json file path is set relative to the Vagrantfile and should contain any JSON data you wish to send to OpsWorks Chef.
For example:
{
"deploy": {
"my-app": {
"application_type": "php",
"scm": {
"scm_type": "git",
"repository": "path/to/my-app"
}
}
},
"opsworks_custom_cookbooks": {
"enabled": true,
"scm": {
"repository": "path/to/my-cookbooks"
},
"recipes": [
"recipe[opsworks_initial_setup]",
"recipe[dependencies]",
"recipe[mod_php5_apache2]",
"recipe[deploy::default]",
"recipe[deploy::php]",
"recipe[my_custom_cookbook::configure]"
]
}
}
To mock multiple opsworks instances and include layers see his AWS OpsWorks "Getting Started" Example which includes the stack.json below.
Vagrantfile (for multiple instances)
Vagrant.configure("2") do |config|
config.vm.box = "ubuntu1404-opsworks"
# Create the php-app layer
config.vm.define "app" do |layer|
layer.vm.provision "opsworks", type:"shell", args:[
'ops/dna/stack.json',
'ops/dna/php-app.json'
]
# Forward port 80 so we can see our work
layer.vm.network "forwarded_port", guest: 80, host: 8080
layer.vm.network "private_network", ip: "10.10.10.10"
end
# Create the db-master layer
config.vm.define "db" do |layer|
layer.vm.provision "opsworks", type:"shell", args:[
'ops/dna/stack.json',
'ops/dna/db-master.json'
]
layer.vm.network "private_network", ip: "10.10.10.20"
end
end
stack.json
{
"opsworks": {
"layers": {
"php-app": {
"instances": {
"php-app1": {"private-ip": "10.10.10.10"}
}
},
"db-master": {
"instances": {
"db-master1": {"private-ip": "10.10.10.20"}
}
}
}
},
"deploy": {
"simple-php": {
"application_type": "php",
"document_root": "web",
"scm": {
"scm_type": "git",
"repository": "dev/simple-php"
},
"memcached": {},
"database": {
"host": "10.10.10.20",
"database": "simple-php",
"username": "root",
"password": "correcthorsebatterystaple",
"reconnect": true
}
}
},
"mysql": {
"server_root_password": "correcthorsebatterystaple",
"tunable": {"innodb_buffer_pool_size": "256M"}
},
"opsworks_custom_cookbooks": {
"enabled": true,
"scm": {
"repository": "ops/cookbooks"
}
}
}
For those not familiar with vagrant you just do a vagrant up to start the instance(s). Then you can modify your cookbook locally and any changes can be applied by re-running chef against the existing instance(s) with vagrant provision. You can do a vagrant destroy and vagrant up to start from scratch.
You'll have to do it manually. You can add arbitrary attributes to your .kitchen.yml, just go on an OpsWorks machine and log the values you need and either use them directly or adapt them to workable test data.