I've been searching for an answer to this question for quite some time now. I've read almost every official aws written tutorial on their services and watched several youtube videos, read some third party tutorials on the subject but there doesn't seem to be a simple easy-to-follow solution to this. Then I tried to search on stackoverflow and although there are 2-3 similiar questions they are either not an exact problem or their solutions are not applicable/understandable and require further explanations.
Problem:
On my PC I do docker-compose build and then docker-compose up and voila, I have my instances up and running without problems.
I want to do the exact same thing but so that every service in docker-compose.yml file starts on its own EC2 Instance (or whichever service that AWS offers). Then I want to be able to change my code and push it to github/gitlab/bitbucket (in my case gitlab) and for it to be deployed to my instances.(but this goes into topic about CI/CD so not important for this exact question)
I think that most of people who are used to docker and docker compose and want to start using AWS will or already have encountered this problem.
A step by step solution( or at least a link or pointing me in the right direction) would be really useful because at the moment there is just too much information on EC2, ECR, ECS, IAM and bunch of other stuff and for a beginner in the AWS world it is really really hard to understand and follow.
EDIT:
I know that it probably isn't that simple but a solution must exists even if it is something as cumbersome as creating every single service by itself on ECS (as mentioned in comments). If it is the only way then sure, I all for it, and I did try several of those tutorials but still didn't succeed in my goal.
Related
I've made a pretty simple web application which has a REST API backend service written in Python/Django and a FE service written in JS/React. Both parts are containerized and can be launched locally via docker-compose. They are situated in separate github repositories, and every time a new tag is pushed to the corresponding repo, an image is built and pushed to the corresponding ecr repo via github actions. Until that point everything works smoothly, but the problem is that, I don't know how to properly automate the deployment process to the test and production environments. The goal is to have those environments as similar as possible.
My current solution for test env is to simply upload docker-compose file to the ec2 instance via github actions, then manually run the docker-compose command, which pulls images from ecr.
Even though it's a simple solution, it's not very scalable or automated and requires some work to be done in order to update an application. The desired flow is to have a manual GitHub action in each repository, which would deploy either BE or FE to the test or prod environment without any need to ssh into the server and do any other manipulations with docker.
I was looking at ECS, but it seems that it's a solution for bigger apps, which need several or more instances to run. I want my app to be used by many users, but I'm not sure, when it will happen. So maybe i should stick to something less complicated than ECS. Are there any simpler solutions, which i am missing? Like Elastic beanstack or something from any other provider?
I will be happy to receive a feedback on anything I wrote in this post, thanks!
So I've recently joined in a project. They asked me to develop a particular module so I did. Now I need to integrate it with the system. They gave me AWS login credentials for the integration purpose. I'm new to AWS and I don't wanna sound dumb to them by asking where's the code. I saw that there's an EC2 instance running but I see no option to see code there. So can you please let me know where can I see the code of running EC2 instance?
Never feel dumb about asking questions on your team. It's much better to ask questions and seek clarification, rather than assume and waste your time and theirs.
So if your team is tasking you with integrating a module you've built with something running on EC2, they probably have an API of some sorta to integrate. They likely aren't expecting you to go to EC2 and view code or decompile DLLs to view source code.
However, to potentially answer your question if your EC2 instance is running some sort of application that has DLLs, you can download those and decompile them using various tools to view the actual source code. YOu would of course need the keypair to access the EC2 instances so you'd have to get that first.
I would just ask someone on your team how to integrate with the system running on EC2. They likely have the source code stored somewhere in a repository.
I'm presently looking into GCP's Deployment Manager to deploy new projects, VMs and Cloud Storage buckets.
We need a web front end that authenticated users can connect to in order to deploy the required infrastructure, though I'm not sure what Dev Ops tools are recommended to work with this system. We have an instance of Jenkins and Octopus Deploy, though I see on Google's Configuration Management page (https://cloud.google.com/solutions/configuration-management) they suggest other tools like Ansible, Chef, Puppet and Saltstack.
I'm supposing that through one of these I can update something simple like a name variable in the config.yaml file and deploy a project.
Could I also ensure a chosen name for a project, VM or Cloud Storage bucket fits with a specific naming convention with one of these systems?
Which system do others use and why?
I use Deployment Manager, as all 3rd party tools are reliant upon the presence of GCP APIs, as well as trusting that those APIs are in line with the actual functionality of the underlying GCP tech.
GCP is decidedly behind the curve on API development, which means that even if you wanted to use TF or whatever, at some point you're going to be stuck inside the SDK, anyway. So that's why I went with Deployment Manager, as much as I wanted to have my whole infra/app deployment use other tools that I was more comfortable with.
To specifically answer your question about validating naming schema, what you would probably want to do is write a wrapper script that uses the gcloud deployment-manager subcommand. Do your validation in the wrapper script, then run the gcloud deployment-manager stuff.
Word of warning about Deployment Manager: it makes troubleshooting very difficult. Very often it will obscure the error that can help you actually establish the root cause of a problem. I can't tell you how many times somebody in my office has shouted "UGGH! Shut UP with your Error 400!" I hope that Google takes note from my pointed survey feedback and refactors DM to pass the original error through.
Anyway, hope this helps. GCP has come a long way, but they've still got work to do.
I have successfully deployed my website on AWS Elastic Beanstalk. Now I want to change the code in one of my file.
If I do eb deploy, it will completely deploy a new version of my code which I don't want. I already have an updated DB on Elastic Beanstalk. If I deploy the whole code again, it will overwrite my DB file.
How can I deploy the changed file only successfully?
This may not be the answer you're looking for, but I would highly recommend deleting this file from your code repository. Hopefully you're using a version control system like Git; if you want to keep the original file for historical purposes, I would create an entirely different repository and put it in there.
Why? Even if you did come up with a solution to only deploy changed files...would you really want to trust it? If there's any problem with the solution you came up with, you would entirely erase/overwrite your production database. Not good.
In addition, if you want to build a really robust system to entirely create your app from scratch in AWS, take a look at Cloud Formation. It takes some learning and work, but you can build a script -- and maintain it in version control -- that will scaffold your entire cloud infrastructure.
Good day,
I am in the process of deploying some of my applications to elasticbeanstalk on AWS, now from reading the documentation and tutorials i get it all deployed and working, but there is a big thing missing in the AWS documentation that i need to know. i can not find the information i am seeking for anywhere, can someone please give me a link to the documentation explaining this or just explain it to me please.
Who and what and from where does is the EbExtensionPreBuild and EbExtensionPostBuild actions executed, who calls them, what do they run and where do they get the commands from?
There are in total 6 actions being performed and nowhere on the internet does AWS explain what happends in these actions.
InfraWriteConfig...
DownloadSourceBundle...
EbExtensionPreBuild...
AppDeployPreHook...
EbExtensionPostBuild...
InfraCleanEbextension...
can someone please explain these actions and link them to the bits of code they execute from the .ebextensions folder .config files.
Thank you
The environment that used to answer you question is a PHP 7.3 running on 64bit Amazon Linux/2.9.2, but, maybe to others platforms, like docker the answer is the same, or at least in how to find the answer.
You can find in log file /var/log/eb-commandprocessor.log the log of all tasks that was executed in your server, the most common task is the deployment task CMD-AppDeploy.
This task is responsible to execute the following scripts:
CMD-AppDeploy
First stage : AppDeployStage0
DownloadSourceBundle:
- /opt/elasticbeanstalk/bin/download-source-bundle
EbExtensionPreBuild
- /opt/elasticbeanstalk/eb_infra/infra-embedded_prebuild.rb
AppDeployPreHook
- /opt/elasticbeanstalk/hooks/appdeploy/pre
EbExtensionPostBuild
/opt/elasticbeanstalk/eb_infra/infra-embedded_postbuild.rb
InfraCleanEbextension
/opt/elasticbeanstalk/eb_infra/infra-clean_ebextensions_dir.rb
Second stage : AppDeployStage1
AppDeployEnactHook
- /opt/elasticbeanstalk/hooks/appdeploy/enact
AppDeployPostHook
- /opt/elasticbeanstalk/hooks/appdeploy/post
You have more than one task available in Beanstalk, you can find the full config in file: /opt/elasticbeanstalk/deploy/configuration/containerconfiguration
Each script is a small part in the deployment process, if you need more details in how the deployment is done, I suggest you check each script individually.