Missing temp folder in elastic beanstack with newspaper library - flask

Every once in a while, the temp folder on my deployment server seems to go missing. I am using Flask and Newspaper on AWS elastic beanstalk. I am using the Newspaper library to scrape meta tags from external urls.
Error on the server:
{'code_content': 500, 'error': "[Errno 2] No such file or directory: '/tmp/.newspaper_scraper/article_resources'"}
Everything works fine after the server is restared. Is there a config setting that might fix this?

We've had a similar problem using AWS Elastic Beanstalk platform "Java 8 running on 64bit Amazon Linux". There was a daily cron-job (/etc/cron.daily/tmpwatch) running that deletes all files in the /tmp folder that are older than 10 days.
Try placing the .newspaper_scraper folder somewhere else.

Related

Elastic BeanStalk post deploy hook not executed

I'm facing a similar issue to this question while trying to implement the new hooks logic on a AWS Linux 2 managed platform running Docker.
I have created my file inside this .platform/hooks/postdeploy/configure_nginx.sh
which is living in the src/ folder of my app.
I can see the file in my host after the deploy but it's never executed /var/app/current/src/.platform/hooks/postdeploy/configure_nginx.sh
It has the right privileges and I can run it if I ssh into my instance.
The EBS environment is initialized through a docker-compose file where I start 2 public containers and my custom app (in which I have put the hook folders)
The script is not being executed so I'm a bit lost on where I need to put it. FYI, my eb deploy is simply copying a Dockerrun file which is grabbing an image from one of my ECR so basically nothing is done in there.
Thanks for your help!
[SOLUTION]
I've found how it should work.
You have to create the .platform folder at the same level as the .ebextensions one.
In my case I'm deploying a zip archive simply containing my Dockerrun.aws.json and both .ebextensions and .platform folders.
So remember to zip it with the whole package before deploying it to your EBS environment.

Spring Boot on AWS Elastic Beanstalk and logging to a file

I have been looking to find an easy way to view debug statements on Beanstalk as I develop. I thought I could simply log to a file on Beanstalk.
In my application.properties file I set
logging.file.path=/var/log
And that did not produce any results even on my local machine. I am not sure if it's a permission issue or what, but locally I set the path to my home directory and then I saw the file, spring.log, appear.
With Beanstalk I tried /var/log, var/log/tomcat, /home/webapp/, ./, ~, and various other values. Nothing worked.
I even tried what was suggested here with no luck: https://medium.com/vividcode/logging-practice-for-elastic-beanstalk-java-apps-308ed7c4d63f
If logging to file is not a good idea, what are the alternatives? I have Googled a lot about this issue and all answers are not very clear.
Yes, this is permission issues. Your app runs under webapp user, while /var/log is own by root. Thus you can't write to it.
The proper way of adding your log files to be recognized by EB is through config files.
Specifically, assuming Amazon Linux 2, you can create .ebextensions/mylogfiles.config with the content of:
files:
"/opt/elasticbeanstalk/config/private/logtasks/bundle/myapplogs.conf":
mode: "000644"
owner: root
group: root
content: |
/var/app/current/log/*.log
Obviously, /var/app/current/log/*.log would point to location where your app stores its log files. /var/app/current is the home folder of your app.

AWS Elastic Beanstalk deploy not working

I'm new to AWS Eleastic Beanstalk. I'm trying to deploy a new application through awsebcli and I'm getting the following error:
"Error: OSError :: [WinError 145] The directory is not empty '.elasticbeanstalk\app_versions'
I was able to init the eb application. I am running the command line under administrator privileges.
Please Help.
I've just ran into the same issue.
"eb deploy" temporarily creates a subfolder "app_versions" in the ".elasticbeanstalk" folder at the root of the project that contains the zip file to be uploaded to S3. Once done, the folder gets deleted. Check whether any software on your computer might be responsible for preventing this.
The cause for me was a files-syncing software (Dropbox-like) that was watching the entire project for file/folder changes.
I'm developing a Django Application and I get this message -
Uploading app to S3. This may take a while. Upload Complete.
How to fix every time it happens
Disable/Pause file syncing applications, such as: Google Drive Sync/OneDrive/DropBox
Delete the (If exists) mysite.elasticbeanstalk\app_versions , don't worry, it's created each time you type "eb deploy"
Open Command prompt in the folder mysite\ and run the command
pip freeze > requirements.txt
Navigate mysite\ and run again eb deploy should work
The message I get when it's not working
The message I get when it's working

Elastic Beanstalk ebextension config files using VS toolkit

Im trying to set Folder Permissions for creating additional folders/files inside Temp directory in my .Net Project.
I know there a lot of references for a similar question like this (as given)
Running a .config file on Elastic Beanstalk?
How To Set Folder Permissions in Elastic Beanstalk Using YAML File?
Im having trouble verifying if its given the permission, im not able to create any folders/files in the Temp folder; i cant find any errors either during deployment in the Elastic Beanstalk (last 100 lines) related to permission setting.
Im using the following code in my config file
command: icacls \"C:/inetpub/wwwroot/myapp_deploy/Temp\" /grant DefaultAppPool:(OI)(CI)
(i have replaced myapp with the EB application name)
Please help anyone.
i was having trouble posting the code and a general unknown exception error was being generated with no additional details.
Figured out the mistake i was making:
ICACLS command requires the directory path to have double slash '\'.
Verified command for above would be:
command: "icacls \"C:\inetpub\wwwroot\Temp\" /grant DefaultAppPool:(OI)(CI)F"

Best way to deploy play2 app using Amazon Beanstalk

I found fragmented instructions here and some other places about deploying Play2 app on amazon ec2. But did not find any neat way to deploy using Beanstalk.
Play is a nice framework and AWS beanstalk is one of the most popular services then why is there no official instruction to do this?
Has anyone found any better solution?
Deploying a Play2 app on elastic beanstalk is now easy with Docker Containers in combination with sbt's experimental docker feature.
In build.sbt specify the exposed docker ports:
dockerExposedPorts in Docker := Seq(9000)
You should automate the following steps, but you can try this out manually to test that it works:
Generate a Dockerfile for the project by running the command: sbt docker:stage.
Go to the ./target/docker/ directory.
Create an elastic beanstalk Dockerrun.aws.json file with the following contents:
{
"AWSEBDockerrunVersion": "1",
"Ports": [
{
"ContainerPort": "9000"
}
]
}
Zip up everything in that directory, let's say into a file called play2-test-docker.zip. The zip file should contain the files: Dockerfile, Dockerrun.aws.json, and files/* directory.
Go to aws beanstalk console and create a new application using the m3.medium or any instance type with enough memory for the jvm to run. Any instance with too little memory will result in a JVM error.
Select "Docker Container" in the Predefined Configuration dropdown.
In the application selection screen, select "Upload" and select the zip file you created earlier. Launch the app and then go brew some tea. This can take a very long time. Minutes. Subsequent deployments of the same app version should be slightly quicker.
Once the app is running and green in the aws console, click on the app's url and you should see the welcome screen of the application (or whatever your index file is).
Here's my solution that doesn't require any additional services/containers like Docker or Jenkins.
Create a dist folder in the root of your Play application's directory. Create a Procfile file containing the following contents and put it in the dist folder (EB requires port 5000):
web: ./bin/YOUR_APP_FILE_NAME -Dhttp.port=5000 -Dconfig.file=conf/application.conf
The YOUR_APP_FILE_NAME is the name of the executable in the bin directory, which is inside the .zip created by activator dist.
After running activator dist, you can just upload the created zip file into Elastic Beanstalk and it will automatically deploy the app. You also put whatever .ebextension folders and configuration files into the dist folder that you require for Elastic Beanstalk configuration. Ex. I have dist/.ebextensions/nginx/conf.d/proxy.conf for NGINX reverse proxy settings or dist/.ebextensions/env.config for environment variables.
Edit 2016: There's now a much better way to deploy your Playframework apps onto ElasticBeanstalk using the new Java SE containers.
Here's an article that walks you through deploying step by step using Jenkins to build and deploy your project:
https://www.davemaple.com/articles/deploy-playframework-elastic-beanstalk-jenkins/
You can use custom AMIs that I keep updated here:
https://github.com/davemaple/playframework-nginx-elastic-beanstalk
These run Nginx + Playframework and support standard zip files created using "activator dist".
We also saw this as being too much of a pain and have added native Play 2 support to Boxfuse to address this.
You can now simply do boxfuse run my-play-app-1.0.zip -env=prod and this will automatically:
create a minimal AMI tailor-made for your Play 2 app
create an elastic IP
create a security group with the correct permissions
launch an instance of your app
All future updates are performed as blue/green deployments with zero downtime.
This also works with Elastic Load Balancers and Auto-Scaling Groups and the Boxfuse free tier is designed to fit the AWS free tier.
You can read more about it here: https://boxfuse.com/blog/playframework-aws
Disclaimer: I'm the founder and CEO of Boxfuse
I had some problems with other solutions found here and there. I guess that the problem is that I'm developing on Play 2.4.
Anyway, I could deploy the app to Beanstalk using Typesafe Activator and Docker:
In build.sbt I added this lines:
import com.typesafe.sbt.packager.docker.{ExecCmd, Cmd}
// [...]
dockerCommands := Seq(
Cmd("FROM","java:openjdk-8-jre"),
Cmd("MAINTAINER","myname"),
Cmd("EXPOSE","9000"),
Cmd("ADD","stage /"),
Cmd("WORKDIR","/opt/docker"),
Cmd("RUN","[\"chown\", \"-R\", \"daemon\", \".\"]"),
Cmd("RUN","[\"chmod\", \"+x\", \"bin/myapp\"]"),
Cmd("USER","daemon"),
Cmd("ENTRYPOINT","[\"bin/myapp\", \"-J-Xms128m\", \"-J-Xmx512m\", \"-J-server\"]"),
ExecCmd("CMD")
)
I went to the project's directory and ran this command in the terminal
$ ./activator clean docker:stage
I opened the [project]/target/dockerdirectory and created the file Dockerrun.aws.json. This was its content:
{
"AWSEBDockerrunVersion": "1",
"Ports": [
{
"ContainerPort": "9000"
}
]
}
In the same target/docker directory, I tested the result, built, checked and ran the image:
$ docker build -t myapp .
$ docker images
$ docker run -p 9000:9000 myapp
As everything was ok, I zipped the content:
$ zip -r myapp.zip *
My zip file had Dockerfile, Dockerrun.aws.json and stage/* files
Finally, I created a new Beanstalk app and uploaded the zip created on the last step. I took care of select "Generic Docker" on "Predefined configuration", when I was creating the app.
Beanstalk only supports WAR deployment and Play doesn't officially support WAR deployment. If you want to use EC2 then you should instead just create an EC2 instance and follow the deployment instructions: http://www.playframework.com/documentation/2.2.x/ProductionDist
Deploying play 2.* apps in aws ec2 is very diffrent until you have found this much better way to do it. I mean ansible is promising a great solution to that. though it is still needed to work with new setup of ansible, and its playbook but that must be worthy.
I have found these reads very recently and yet to apply them in my project. I hope following reads will help you to learn more:
Ansible + play + aws ec2
Read it to know more about Ansible to deply play in aws
Thanks!
Hope this will help you to kick your start. Please do share more knowledge you gain during the procedure or if there is any simple way to solve this complicated deployment problem.