I'm managing a repo which maintains common utilities, this repo has my common utils, jenkins file and all the dependencies are managed by Poetry so I've some TOML files to manage. We keep adding new utils as we go on.
.
├── Jenkinsfile
├── README.md
├── common_utils
│ ├── __init__.py
│ ├── aws.py
│ ├── sftp.py
├── pre-commit.yaml
├── poetry.lock
└── pyproject.toml
When I push my code to git, my Jenkinsfile packages it and publishes it to AWS code artifact. But everything time I push it, I've to manually update the TOML version in my local first before I push it to dev branch and then pull a new branch from dev to update the version again before pushing to master branch to update the version again otherwise I get a conflict on TOML version in code artifact.
I can't modify the Jenkins because even though it solves the conflict issue by updating the version, the version in the source code isn't modified, which means I still need to manually update all the versions manually. I'm not sure if pre-commit will help because I dont want to update the version every time I push to feature branch.
Is there a known way to handle this or is going with a release branch the only option?
by default ebs uses Dockerrun.aws.json file to deploy. we have a requirement to have multiple dockerrun files, like Dockerrun.aws.json and Dockerrun-staging.aws.json.
Is it possible to make sure deployment uses Dockerrun-Staging.aws.json file instead of Dockerrun.aws.json?
I had the same problem and could not find any documentation on how to provide target file on deploy. I ended up creating separate folders for staging and production then execute the eb deploy on the target environment folder
:docker-run$ tree
.
├── production
│ └── Dockerrun.aws.json
└── staging
└── Dockerrun.aws.json
2 directories, 2 files
Tesseract OCR build
I want to deploy tesseract-OCR on lambda. Scroll to the section where it says adaptions for tesseract 4. I have built tesseract following this tutorial. But I am facing an issue with this as the build is not portable.
When ever I try to use the built on a new Linux instance, I have to set the environmental variable PATH to /tmp or else this wont work.
Structure
├── cv2
├── lib
├── numpy
├── PIL
├── pytesseract
├── tessdata
├── tesseract
├── test.png
└── zzz.py
https://s3.amazonaws.com/tesseractstandalone/complete-package.zip
This is the link for the standalone tesseract. There is a sample program zzz.py which has the script for running the tesseract. When I download the zip and extract to /tmp/ folder in an ec2 instance, the program works fine. But I am having an issue when working with lambda. When I try to download the same thing to lambda /tmp/ folder, I am getting an error that says tesseract is not installed or it's not in your path. Don't know where things are going wrong. Not sure whether its a PATH issue or lambda issue.
Finally got the support from AWS help. It seems that the executable doesn't have the permissions to execute when it is getting downloaded to lambda. Solve my doing chmod 755 to the executable.
I'm new to Go.
I am trying to deploy a simple web project to EB without success.
I would like to deploy a project with the following local structure to Amazon EB:
$GOPATH
├── bin
├── pkg
└── src
├── github.com
│ ├── AstralinkIO
│ │ └── api-server <-- project/repository root
│ │ ├── bin
│ │ ├── cmd <-- main package
│ │ ├── pkg
│ │ ├── static
│ │ └── vendor
But I'm not sure how to do that, when building the command, Amazon is treating api-server as the $GOPATH, and of course import paths are broken.
I read that most of the time it's best to keep all repos under the same workspace, but it makes deployment harder..
I'm using Procfile and Buildfile to customize output path, but I can't find a solution to dependencies.
What is the best way to deploy such project to EB?
Long time has past since I used Beanstalk, so I'm a bit rusty on the details. But basic idea is as follows. AWS Beanstalk support for go is a bit odd by design. It basically extracts your source files into a folder on the server, declares that folder as GOPATH and tries to build your application assuming that your main package is at the root of your GOPATH. Which is not a standard layout for go projects. So your options are:
1) Package your whole GOPATH as "source bundle" for Beanstalk. Then you should be able to write build.sh script to change GOPATH and build it your way. Then call build.sh from your Buildfile.
2) Change your main package to be a regular package (e.g. github.com/AstralinkIO/api-server/cmd). Then create an application.go file at the root of your GOPATH (yes, outside of src, while all actual packages are in src as they should be). Your application.go will become your "package main" and will only contain a main function (which will call your current Main function from github.com/AstralinkIO/api-server/cmd). Should do the trick. Though your mileage might vary.
3) A bit easier option is to use Docker-based Go Platform instead. It still builds your go application on the server with mostly same issues as above, but it's better documented and possibility to test it locally helps a lot with getting configuration and build right. It will also give you some insights into how Beanstalk builds go applications thus helping with options 1 and 2. I used this option myself until I moved to plain EC2 instances. And I still use skills gained as a result of it to build my current app releases using docker.
4) Your best option though (in my humble opinion) is to build your app yourselves and package it as a ready to run binary file. See second bullet point paragraph here
Well, which ever option you choose - good luck!
I'm trying to upload my Flask application to AWS however I receive an error on doing so:
Your WSGIPath refers to a file that does not exist.
After doing some digging online I found that in the .ebextensions folder, I should specify the path. There was not a .ebextensions folder so I created one and added the following code to a file named settings.config:
option_settings:
"aws:elasticbeanstalk:container:python":
WSGIPath: project/application.py
the WSGIPath is the correct path to the application.py file so I'm not sure what raises this error. Am I changing the WSGIPath right, is there a better way or is there an issue with something else which causes this to happen? Thanks.
There's a lot of configuration issues that can arise with Flask deployed on AWS. I was running into a similar issue as you, so I can at least show you what I did to resolve the WSGI error.
First, apparently you can do this without the .ebextensions folder (see this post here. and look at davetw12's answer. However, be aware that while this works, I'm not entirely sure that davetw12's conclusion about .ebextensions is correct, based on some of the comments below). Instead, (in the Terminal), I navigated to my project at the same level as my .elasticbeanstalk directory and used the command eb config. This will open up a list of options you can set to configure your beanstalk application. Go down through the options until you find the WSGI path. I notice you have yours set to project/application.py, however, this should not include the folder reference, just application.py. Here is how it looks on my Mac in the terminal (WSGI path is near the bottom).
Note that once you get that set, EB will probably redeploy. That's fine. Let it.
Once you get that set, go into your application.py file and make sure you call your app application. For example, mine looks like this:
from flask import Flask
from flask import render_template
application = Flask(__name__)
#application.route('/')
#application.route('/index')
def index():
return render_template('index.html',
title='Home')
This took away the WSGI path error - although I still had to fix some other issues following this :-) But that is a different set of questions.
If anyone here is doing via AWS Console (GUI), modify the configuration and put your script name in WSGIPath as below.
I'm giving bonus hints if you are newer.
You should match the script name and the Flask object too.
Common mistake: When you're compressing the source code, you need to
select the files and compress, not the folder. (make sure you have the
.py in the root of the zip)
from flask import Flask
application = Flask(__name__)
#application.route("/")
def hello():
return "Hello"
if __name__ == "__main__":
application.run()
I had the same message, but for a very stupid reason.
Apparently, when I cloned the repo to my Windows PC and then pushed back the changes, somewhere along the way Windows changed ".ebextensions" folder to "ebextensions" (dropping the leading ".").
So when I renamed back the folder to ".ebextensions" in the master repo, everything started working again perfectly.
For me the problem was I had misspelled a filename:
I wrote: ..ebextensions/django.conf
When I needed: ..ebextensions/django.config
This cost me about 3 hours of my life today. The trouble was that the AWS error is misleading, because the "WSGIPath" it refers to is not the file above, but some invisible default.
In my case trying many solutions didn't solve the issue but changing WSGIPath from
option_settings:
"aws:elasticbeanstalk:container:python":
WSGIPath: project_name/application_name.py
to
option_settings:
"aws:elasticbeanstalk:container:python":
WSGIPath: project_name.wsgi
worked like a charm. Here is the file structure:
├── manage.py
├── mysite ***
│ ├── __init__.py
│ ├── settings.py
│ ├── urls.py
│ └── wsgi.py ***
├── myvenv
│ └── ...
└── requirements.txt