Good day,
I am in the process of deploying some of my applications to elasticbeanstalk on AWS, now from reading the documentation and tutorials i get it all deployed and working, but there is a big thing missing in the AWS documentation that i need to know. i can not find the information i am seeking for anywhere, can someone please give me a link to the documentation explaining this or just explain it to me please.
Who and what and from where does is the EbExtensionPreBuild and EbExtensionPostBuild actions executed, who calls them, what do they run and where do they get the commands from?
There are in total 6 actions being performed and nowhere on the internet does AWS explain what happends in these actions.
InfraWriteConfig...
DownloadSourceBundle...
EbExtensionPreBuild...
AppDeployPreHook...
EbExtensionPostBuild...
InfraCleanEbextension...
can someone please explain these actions and link them to the bits of code they execute from the .ebextensions folder .config files.
Thank you
The environment that used to answer you question is a PHP 7.3 running on 64bit Amazon Linux/2.9.2, but, maybe to others platforms, like docker the answer is the same, or at least in how to find the answer.
You can find in log file /var/log/eb-commandprocessor.log the log of all tasks that was executed in your server, the most common task is the deployment task CMD-AppDeploy.
This task is responsible to execute the following scripts:
CMD-AppDeploy
First stage : AppDeployStage0
DownloadSourceBundle:
- /opt/elasticbeanstalk/bin/download-source-bundle
EbExtensionPreBuild
- /opt/elasticbeanstalk/eb_infra/infra-embedded_prebuild.rb
AppDeployPreHook
- /opt/elasticbeanstalk/hooks/appdeploy/pre
EbExtensionPostBuild
/opt/elasticbeanstalk/eb_infra/infra-embedded_postbuild.rb
InfraCleanEbextension
/opt/elasticbeanstalk/eb_infra/infra-clean_ebextensions_dir.rb
Second stage : AppDeployStage1
AppDeployEnactHook
- /opt/elasticbeanstalk/hooks/appdeploy/enact
AppDeployPostHook
- /opt/elasticbeanstalk/hooks/appdeploy/post
You have more than one task available in Beanstalk, you can find the full config in file: /opt/elasticbeanstalk/deploy/configuration/containerconfiguration
Each script is a small part in the deployment process, if you need more details in how the deployment is done, I suggest you check each script individually.
Related
For a project, I have to create a Dataproc cluster that has one of the outdated versions (for example, 1.3.94-debian10) that contain the vulnerabilities in Apache Log4j 2 utility. The goal is to get the alert related (DATAPROC_IMAGE_OUTDATED), in order to check how SCC works (it is just for a test environment).
I tried to run the command gcloud dataproc clusters create dataproc-cluster --region=us-east1 --image-version=1.3.94-debian10 but got the following message ERROR: (gcloud.dataproc.clusters.create) INVALID_ARGUMENT: Selected software image version 1.3.94-debian10 is vulnerable to remote code execution due to a log4j vulnerability (CVE-2021-44228) and cannot be used to create new clusters. Please upgrade to image versions >=1.3.95, >=1.4.77, >=1.5.53, or >=2.0.27. For more information, see https://cloud.google.com/dataproc/docs/guides/recreate-cluster, which makes sense, in order to protect the cluster.
I did some research and discovered that I will have to create a custom image with said version and generate the cluster from that. The thing is, I have tried to read the documentation or find some tutorial, but I still can't understand how to start or to run the file generate_custom_image.py, for example, since I am not confortable with cloud shell (I prefer the console).
Can someone help? Thank you
I'm currently trying to set up an Airflow environment via MWAA. I've gone through the create environment steps twice with both ending at the page listing Airflow environments with a banner saying I was successful. However, for the past 2 days, this environments page has just shown Loading Environments, as shown below. I also see a (0) for the environment number.
So far, I've added 2 interfaces for ECR and VPC for the API and the environment but no luck. Has anyone else run into this issue or have any clue what might be happening? Thanks!
Were you able to find the solution to this issue? I had similar issues when I tried to set up the first-time MWAA on AWS Account.
https://github.com/awslabs/aws-support-tools/tree/master/MWAA
Here's a link to how to verify if all the resources are set up correctly for MWAA. If you run the script mentioned on the repo you should be able to see where the issue lies.
I've been searching for an answer to this question for quite some time now. I've read almost every official aws written tutorial on their services and watched several youtube videos, read some third party tutorials on the subject but there doesn't seem to be a simple easy-to-follow solution to this. Then I tried to search on stackoverflow and although there are 2-3 similiar questions they are either not an exact problem or their solutions are not applicable/understandable and require further explanations.
Problem:
On my PC I do docker-compose build and then docker-compose up and voila, I have my instances up and running without problems.
I want to do the exact same thing but so that every service in docker-compose.yml file starts on its own EC2 Instance (or whichever service that AWS offers). Then I want to be able to change my code and push it to github/gitlab/bitbucket (in my case gitlab) and for it to be deployed to my instances.(but this goes into topic about CI/CD so not important for this exact question)
I think that most of people who are used to docker and docker compose and want to start using AWS will or already have encountered this problem.
A step by step solution( or at least a link or pointing me in the right direction) would be really useful because at the moment there is just too much information on EC2, ECR, ECS, IAM and bunch of other stuff and for a beginner in the AWS world it is really really hard to understand and follow.
EDIT:
I know that it probably isn't that simple but a solution must exists even if it is something as cumbersome as creating every single service by itself on ECS (as mentioned in comments). If it is the only way then sure, I all for it, and I did try several of those tutorials but still didn't succeed in my goal.
Actually I want to integrate AWS CodeCommit with AWS Elastic Beanstalk. But I am stuck in code upload on AWS CodeCommit. I have code of size 900 MB around. I have no idea so much about it so I am attaching image containing my problem related to hang process after completing the code upload process successfully. Please see the image for that.
Actually, I have setup this parameter to increase the buffer size with following command:
git config --global http.postBuffer 157286400
Main Issue is how we upload code of bigger size approx size in GB on aws codeCommit successfully.
But, after that I am facing this issue so please if you have any idea about that, please help me. Thanks in advance.
This is the image containing my problem definition
Based on your information, it seems that you are under the limits of CodeCommit, which are listed here: https://docs.aws.amazon.com/codecommit/latest/userguide/limits.html. I strongly suggest you to review them, just in case that you are falling in one of them.
Could you please provide more details about your git client, AWS region and try to run git push again with the GIT_CURL_VERBOSE=1 GIT_TRACE=1 options?
I think I'm on the right path. I can use .ebextensions to change some of the conf files for the instance I'm running. Since I'm using Elastic Beanstalk, and that a lot of the software is shrinkwrapped (which I'm fine with), I should be using .ebextensions as a means of modifying the environment.
I want to employ some form of mod_rewrite config, but I know nothing of this Amazon Linux. I don't even know what the web server is. I've been through the console for the past few hours and see no trace of the things I want to override.
Apparently I can setup a shell to take a look around, but modifying things that way will cause things to be overridden since Beanstalk is handling config. I'm not entirely sure on that last point.
Should I just ssh and play in userland like a typical unix host?
You can definitely ssh to the instance, and see around. But remember, that your changes are not persistent. You should look at .ebextensions config files as the way to re-run your commands on the host, plus more.
It might take some time to see where ElasticBeanstalk stores configuration files and all other interesting things.
To get you started, your app files are located at: /opt/python/current/app and if you are using Python, it is located in virtual environment at: /opt/python/run/venv/bin/python27
Customizing the Software on EC2 Instances Running Linux guide contains detailed information on what you can do:
Packages - install packages
Sources - retrieve archives
Files - operations with files
Users - anything with users
Groups - anything with groups
Commands - execute instance commands
Container_commands - execute commands after the container is
extracted
Services - launch services
Option_settings - configure
container settings
See if that satisfies your requirements, if not, come back to StackOverflow and ask more questions.