Tesseract OCR build
I want to deploy tesseract-OCR on lambda. Scroll to the section where it says adaptions for tesseract 4. I have built tesseract following this tutorial. But I am facing an issue with this as the build is not portable.
When ever I try to use the built on a new Linux instance, I have to set the environmental variable PATH to /tmp or else this wont work.
Structure
├── cv2
├── lib
├── numpy
├── PIL
├── pytesseract
├── tessdata
├── tesseract
├── test.png
└── zzz.py
https://s3.amazonaws.com/tesseractstandalone/complete-package.zip
This is the link for the standalone tesseract. There is a sample program zzz.py which has the script for running the tesseract. When I download the zip and extract to /tmp/ folder in an ec2 instance, the program works fine. But I am having an issue when working with lambda. When I try to download the same thing to lambda /tmp/ folder, I am getting an error that says tesseract is not installed or it's not in your path. Don't know where things are going wrong. Not sure whether its a PATH issue or lambda issue.
Finally got the support from AWS help. It seems that the executable doesn't have the permissions to execute when it is getting downloaded to lambda. Solve my doing chmod 755 to the executable.
Related
I'm managing a repo which maintains common utilities, this repo has my common utils, jenkins file and all the dependencies are managed by Poetry so I've some TOML files to manage. We keep adding new utils as we go on.
.
├── Jenkinsfile
├── README.md
├── common_utils
│ ├── __init__.py
│ ├── aws.py
│ ├── sftp.py
├── pre-commit.yaml
├── poetry.lock
└── pyproject.toml
When I push my code to git, my Jenkinsfile packages it and publishes it to AWS code artifact. But everything time I push it, I've to manually update the TOML version in my local first before I push it to dev branch and then pull a new branch from dev to update the version again before pushing to master branch to update the version again otherwise I get a conflict on TOML version in code artifact.
I can't modify the Jenkins because even though it solves the conflict issue by updating the version, the version in the source code isn't modified, which means I still need to manually update all the versions manually. I'm not sure if pre-commit will help because I dont want to update the version every time I push to feature branch.
Is there a known way to handle this or is going with a release branch the only option?
I'm quite new to the Amplify function world. I've been struggling to deploy my Golang function, connected to a DynamoDB stream. Even though I am able to run my lambda successfully by manually uploading a .zip I created myself after I built the binary using GOARCH=amd64 GOOS=linux go build src/index.go (I develop on a mac), when I use the Amplify CLI tools I am not able to deploy my function.
This is the folder structure of my function myfunction
+ myfunction
├── amplify.state
├── custom-policies.json
├── dist
│ └── latest-build.zip
├── function-parameters.json
├── go.mod
├── go.sum
├── parameters.json
├── src
│ ├── event.json
│ └── index.go
└── tinkusercreate-cloudformation-template.json
The problem is I can't use the amplify function build command, since looks like it creates a .zip file with my source file index.go in there (not the binary), so the lambda, regardless of the handler I set, it seems not able to run it from that source. I record errors like
fork/exec /var/task/index.go: exec format error: PathError null or
fork/exec /var/task/index: no such file or directory: PathError null
depending on the handler I set.
Is there a way to make Amplify build function work for a Golang lambda? I would like to successfully execute the command amplify function build myfunction so that I will be able to deliver the working deployment by amplify push to my target environment.
We are trying to install python package that can be shipped in plugins.zip for MWAA environment and added the reference in requirements.txt as described in the docs but we are still seeing the error file not found.
directory structure
├── README.md
├── dags
│ └── dummy.py
├── plugins
│ └── dummy-wheel-0.1.0-py3-none-any.whl
└── requirements.txt
The contents of requirements.txt
/usr/local/airflow/plugins/dummy-wheel-0.1.0-py3-none-any.whl
From here I have just created a zip file containing dummy-wheel-0.1.0-py3-none-any.whl and push it to S3://<bucket_name>/plugins.zip
Followed instructions here https://aws.amazon.com/premiumsupport/knowledge-center/mwaa-install-custom-packages/
What am I missing?
You may utilize BashOperator to list the directory /usr/local/airflow/plugins/ to verify whether the file is present there. While zipping, the whl file is supposed to be in the root path for it to be available in /usr/local/airflow/plugins/. Whereas if zipping a folder (eg. plugins), the extracted path will vary accordingly (eg. /usr/local/airflow/plugins/plugins/).
#mad_
Yeah I would try listing the contents of /usr/local/airflow/plugins first:
ls_airflow_plugins = BashOperator(
task_id="ls_airflow_plugins",
bash_command="ls -laR /usr/local/airflow/plugins",
dag=dag,
priority_weight=300,
)
Also, some things to try are:
put the actual plugins.zip file in a plugins/ folder that is at the same level as dag/ folder.
Before zipping the whl file, make sure you have given 755 file permissions (chmod 755 command)
at the top of your requirements.txt file, include:
--find-links /usr/local/airflow/plugins
--no-index
Hope one of these fixes it, lmk of questions
#mad_ Can you explain where you are seeing the file not found error?
If you zip packages and then edit/save your environment to point to the plugins.zip file, enable logging in the monitoring section before you re-deploy. The scheduler CloudWatch log will list packages that were installed (or errors that occurred). That may point you in the right direction.
The other thing to check is dependencies. If your package depends on a certain version of a package that MWAA already installs, you may need to include that dependency too.
I'm new to Go.
I am trying to deploy a simple web project to EB without success.
I would like to deploy a project with the following local structure to Amazon EB:
$GOPATH
├── bin
├── pkg
└── src
├── github.com
│ ├── AstralinkIO
│ │ └── api-server <-- project/repository root
│ │ ├── bin
│ │ ├── cmd <-- main package
│ │ ├── pkg
│ │ ├── static
│ │ └── vendor
But I'm not sure how to do that, when building the command, Amazon is treating api-server as the $GOPATH, and of course import paths are broken.
I read that most of the time it's best to keep all repos under the same workspace, but it makes deployment harder..
I'm using Procfile and Buildfile to customize output path, but I can't find a solution to dependencies.
What is the best way to deploy such project to EB?
Long time has past since I used Beanstalk, so I'm a bit rusty on the details. But basic idea is as follows. AWS Beanstalk support for go is a bit odd by design. It basically extracts your source files into a folder on the server, declares that folder as GOPATH and tries to build your application assuming that your main package is at the root of your GOPATH. Which is not a standard layout for go projects. So your options are:
1) Package your whole GOPATH as "source bundle" for Beanstalk. Then you should be able to write build.sh script to change GOPATH and build it your way. Then call build.sh from your Buildfile.
2) Change your main package to be a regular package (e.g. github.com/AstralinkIO/api-server/cmd). Then create an application.go file at the root of your GOPATH (yes, outside of src, while all actual packages are in src as they should be). Your application.go will become your "package main" and will only contain a main function (which will call your current Main function from github.com/AstralinkIO/api-server/cmd). Should do the trick. Though your mileage might vary.
3) A bit easier option is to use Docker-based Go Platform instead. It still builds your go application on the server with mostly same issues as above, but it's better documented and possibility to test it locally helps a lot with getting configuration and build right. It will also give you some insights into how Beanstalk builds go applications thus helping with options 1 and 2. I used this option myself until I moved to plain EC2 instances. And I still use skills gained as a result of it to build my current app releases using docker.
4) Your best option though (in my humble opinion) is to build your app yourselves and package it as a ready to run binary file. See second bullet point paragraph here
Well, which ever option you choose - good luck!
I'm trying to upload my Flask application to AWS however I receive an error on doing so:
Your WSGIPath refers to a file that does not exist.
After doing some digging online I found that in the .ebextensions folder, I should specify the path. There was not a .ebextensions folder so I created one and added the following code to a file named settings.config:
option_settings:
"aws:elasticbeanstalk:container:python":
WSGIPath: project/application.py
the WSGIPath is the correct path to the application.py file so I'm not sure what raises this error. Am I changing the WSGIPath right, is there a better way or is there an issue with something else which causes this to happen? Thanks.
There's a lot of configuration issues that can arise with Flask deployed on AWS. I was running into a similar issue as you, so I can at least show you what I did to resolve the WSGI error.
First, apparently you can do this without the .ebextensions folder (see this post here. and look at davetw12's answer. However, be aware that while this works, I'm not entirely sure that davetw12's conclusion about .ebextensions is correct, based on some of the comments below). Instead, (in the Terminal), I navigated to my project at the same level as my .elasticbeanstalk directory and used the command eb config. This will open up a list of options you can set to configure your beanstalk application. Go down through the options until you find the WSGI path. I notice you have yours set to project/application.py, however, this should not include the folder reference, just application.py. Here is how it looks on my Mac in the terminal (WSGI path is near the bottom).
Note that once you get that set, EB will probably redeploy. That's fine. Let it.
Once you get that set, go into your application.py file and make sure you call your app application. For example, mine looks like this:
from flask import Flask
from flask import render_template
application = Flask(__name__)
#application.route('/')
#application.route('/index')
def index():
return render_template('index.html',
title='Home')
This took away the WSGI path error - although I still had to fix some other issues following this :-) But that is a different set of questions.
If anyone here is doing via AWS Console (GUI), modify the configuration and put your script name in WSGIPath as below.
I'm giving bonus hints if you are newer.
You should match the script name and the Flask object too.
Common mistake: When you're compressing the source code, you need to
select the files and compress, not the folder. (make sure you have the
.py in the root of the zip)
from flask import Flask
application = Flask(__name__)
#application.route("/")
def hello():
return "Hello"
if __name__ == "__main__":
application.run()
I had the same message, but for a very stupid reason.
Apparently, when I cloned the repo to my Windows PC and then pushed back the changes, somewhere along the way Windows changed ".ebextensions" folder to "ebextensions" (dropping the leading ".").
So when I renamed back the folder to ".ebextensions" in the master repo, everything started working again perfectly.
For me the problem was I had misspelled a filename:
I wrote: ..ebextensions/django.conf
When I needed: ..ebextensions/django.config
This cost me about 3 hours of my life today. The trouble was that the AWS error is misleading, because the "WSGIPath" it refers to is not the file above, but some invisible default.
In my case trying many solutions didn't solve the issue but changing WSGIPath from
option_settings:
"aws:elasticbeanstalk:container:python":
WSGIPath: project_name/application_name.py
to
option_settings:
"aws:elasticbeanstalk:container:python":
WSGIPath: project_name.wsgi
worked like a charm. Here is the file structure:
├── manage.py
├── mysite ***
│ ├── __init__.py
│ ├── settings.py
│ ├── urls.py
│ └── wsgi.py ***
├── myvenv
│ └── ...
└── requirements.txt