I would like to develop a web application for AWS practice labs where I can create
users and allow them to practice their AWS knowledge with live labs just like https://qwiklabs.com/
In this application, according to labs I need to provision AWS resources automatically as per the lab requirements and allow the user to complete the lab
with limited access.
So, Please share your suggestion, how can I do this whole setup with my AWS account.
Thanks in advance.
I took a look at how QwikLabs provided their service early on when they first started offering services. Back then their IAM policies were not well written and I could see the resources that they were provisioning. I studied many of their CloudFormation templates. This is not true today as they have tightened everything very well.
1) They developed their own management system for managing users, payments, account access, etc.
2) They use CloudFormation with very tight IAM permissions to create the labs that you access. Via IAM policies they control what you can do and access. They also time you out and drop everything after a period of time. This is one of the better/best executions of CloudFormation that I have experienced.
Their setup is actually very well thought-out and executed.
Related
We're adding a GCP project to be used for greenfield development, e.g. sort of a developer sandbox. My inclination is to give application/service developers full permissions in that project, to reduce friction and let them get stuff done as quickly and easily as possible.
We then have a separate beta project which we use where we prepare work for production, where application/service developers would have limited-to-no access, but the devops team could productionize things. And then, of course, we have the production project, where everything is locked down tight.
Is a sandbox like this a good idea? What permission(s) would I grant? Owner? GCP recommends not using those legacy roles...
List all of what each team is allowed to do on each env.
Translate this to a list of IAM permissions per team per env.
If there is some predefined role/s that matches exactly these permissions then use that role/s
If not, then create your own custom role/s for each team per each env.
For example, in the sandbox env:
if developers team is only allowed to create GKE clusters and deploy workloads to these GKEs then list all required permissions for such operation and find a predefined role that have permissions that only allows this operation. See here.
Or, if this is too wide and does not apply the least privilege concept for you then create your own custom role.
I personally don't recommend to restrict the IAM permission. Indeed, in a sandbox project, you want to try things, and maybe thing totally outside of the box and unexpected as usual way of working/processing. Using IAM to limit the set of allowed product restrict the creativity and protect you against (almost) nothing.
Indeed, if you want to perform security restriction it's for what? Limit the access to the service in Beta environment? Not sure... Prevent the overuse of resources in a non-production (and no profitable) environment? I think yes!
That's why, I recommend to use the Quotas to restrict the number of resources available for a project (i.e. only 10 CPUs in 1 region and not 3600 in 20 regions as by default). Like that, the app team will be able to try and experiment safely, without any restriction, but without killing your budget.
I’m my scenario wants to separate out the production environment from our development environments.
We'd like to only have our production systems on one AWS account and all other systems and services on another.
I'd like to split/separate for billing purposes. If I do add more monitoring services many charge by the number of running instances. I have considerably more running instances than I need to monitor though so I'd like the separation. This also would make managing permissions in the future a lot easier I believe (e.g. security hub scores wouldn't be affected by LMS instances).
I'd like to split out all public facing assets to a separate AWS account. So RDS, all EC2 instances relating to prod-webserver (instances, target group, AMI, scaling, VPC, etc.), S3 cloudfront.abc.com bucket, jenkins, OpenVPN, all Seoul assets.
Perhaps I could achieve the goal with 'Organizations' or the 'Control Tower' as well. Could anyone please advise what would be best in my scenario? Is there Better alternative for this ?
The fact that you was to split for billing purposes means you should use separate AWS Accounts. While you could split some billing by tags within a single account, it's much easier to use multiple accounts to split the billing.
The typical split is Production / Testing / Development.
You can join the accounts together by using AWS Organizations, which gives some overall security controls.
Separating workloads and environments is considered a best practice in AWS according to the AWS Well-Architected Framework. Nowadays Control Tower (which builds upon AWS Organizations) is the standard for building multi-account setups in AWS.
Regarding multi-account setups I recommend reading the Organizing Your AWS Environment Using Multiple Accounts.
Also have a look at the open-source AWS Quickstart superwerker which sets up a well-architected AWS landing zone using AWS Control Tower, Security Hub, GuardDuty, and more.
AWS provides a lot information about this topic. E.g. a very detailed Whitepaper about Organizing Your AWS Environment in which they say
Using multiple AWS accounts to help isolate and manage your business applications and data can
help you optimize across most of the AWS Well-Architected Framework pillars, including operational
excellence, security, reliability, and cost optimization.
With accounts, you logically separate all resources (unless you allow something else) and therefore ensure independence between e.g. the development environment and the production environment.
You should also take a look at Organizational Units (OUs)
The following benefits of using OUs helped shape the Recommended OUs and accounts and Patterns for organizing your AWS accounts.
Group similar accounts based on function
Apply common policies
Share common resources
Provision and manage common resources
Control Tower is a tool which allows you to manage all your AWS accounts in one place. You can apply policies for every account, OU, or prohibit regions. You can use the Account Factory to create new accounts based on blueprints.
But still you need to collect a lot of knowledge about these tools and best practices because they're just that. Best practices and recommendations you can use to get started and build a good foundation, but they're nothing you can fully rely on because you may have individual factors.
So understanding these factor and consequences is very important.
Please how can get the list of all services I am using.
I have gone to Service Quotas at
https://ap-east-1.console.aws.amazon.com/servicequotas/home?region=ap-east-1
on the dashboard. I could see a list of Items e.g. EC2, VPC, RDS, Dynamo etc but I did not understand what is there.
As I did not request for some of the services I am seeing I even went into budget at
https://console.aws.amazon.com/billing/home?region=ap-east-1#/budgets
and also credits. Maybe I can get the services I have been given credits to use
https://console.aws.amazon.com/billing/home?region=ap-east-1#/budgets?
Also, how can I stop any service which I do not want?
The Billing service is not giving me tangible information also. I do not want the bill to pile up before I start taking needed steps.
Is there a location where I can see all services I am using or maybe there is a code I can enter somewhere which would produce such result?
You can use AWS Config Resource Inventory feature.
AWS Config will discover resources that exist in your account, record their current configuration, and capture any changes to these configurations. Config will also retain configuration details for resources that have been deleted. A comprehensive snapshot of all resources and their configuration attributes provides a complete inventory of resources in your account.
https://aws.amazon.com/config/
There is not an easy answer on this one, as there is not an AWS service that you can use to do this out of the box (yet).
There are some AWS services that you can use to get you close, like:
AWS Config (as suggested by #kepils)
Another option is to use Resource Groups and Tagging to list all resources within a region within account (as described in this answer).
In both cases however, the issue is that both Config and Resource Groups come with the same limitation - they can't see all AWS services on their own.
Another option would be to use a third party tool to do this, if your end goal is to find what do you currently have running in your account like aws-inventory or cloudmapper
On the second part of your question on how to stop any services which you don't want you can do the following:
Don't grant excessive permissions to your users. If someone needs to work on EC2 instances, then their IAM role and respective policy should allow only that instead of for example full access.
You can limit the scope and services permitted for use within account by creating Service Control Policies which are allowing only the specific resources you plan to use.
Set-up an AWS Budget Notifications and potentially AWS Budget Actions.
[Not sure this is the correct forum for this question, but I'll give it a shot.]
I'm looking at duplicating an existing solution built on AWS into an AWS China account. From what I've read in AWS' getting started blog post and AWS China's list of services per region, it seems to me that deploying a solution in Beijing or Ningxia using the AWS services we're used to and dependent on would be feasible. But since you cannot create an AWS China account without having a business license (which seems to be a topic in itself, hmm), it seems impossible to actually try things out to get a feel for if there are any differences. I also cannot seem to find any blog posts with testimonies, experiences from developers or architects who've done this, which is surprising.
Basically I want to understand if taking an existing solution built on AWS and setting it up on Chinese infrastructure is straightforward or if I should expect some differences in how things work etc. I know that AWS does not operate these two regions themselves, but through Chinese partner companies. But I'm not sure if the service capabilities, APIs etc are identical (even including the timing of releases of new versions etc).
The only real limitations I can find on the AWS blog is that the free tier is not available, and that EC2 classic instances is not supported. But let's say I have a solution using very stadnard AWS services like Cloudfront, S3, DynamoDB, Lambda, ECS, Elastic Beanstalk, Cognito, KMS etc. Will it be fairly simple to migrate it to an AWS China account or should I expect a struggle?
Regarding the difference, basically AWS China and AWS Global are two seperate cloud and they are not connected to earch other, thus they will have separate Marketplace,Endpoints and ARNs, different service capablities etc. However those differences are not capatured in such details in official AWS documentation.
For example most security related features, landing zone related feature are not available in AWS China. I have tried to customize some AWS global solutions to China, and met lot of issues and challenges, so plug and play won't work here. The best way is to have some parnters or local presence to overcome those challenges especially the team with similar capabilities.
I'm a noobie to CloudFormation. But reading the documentation for CloudFormation, Amazon seems to think it is the method we should use to consistently, repeatedly deploy a given topology of AWS service instances. However AWS has been around for over a decade, and the AWS push for CF seems to be only within the last 5 years.
I stumbled across a great post, AWS OpsWorks vs AWS Beanstalk vs AWS CloudFormation?, which explores the strengths of different AWS deployment offerings. And given the needs of my organization for flexible and repeatable IaaS/PaaS deployments, CF seems to fit the bill.
What I want to know is: How prevalent is the use of CF, vs other "template" deployment technologies? What is YOUR team using for deploying repeated configurations of AWS services?
How usable/learnable is it? If I adopt CF, how likely is it that existing developers on AWS will already be familiar with it, and be able to use it straight off the bat? CF seems to support many or most AWS services already, but are people actually using it to repeatedly stamp out identically-configured topologies of services?
Or do most teams favor a simpler, less endlessly-configurable option? And if so, why?
What pitfalls do I need to watch out for when using CloudFormation Templates? What doesn't CF handle, which it really should?
I'll try to answer most of your questions based on my personal experience:
What I want to know is: How prevalent is the use of CF, vs other "template" deployment technologies?
I can't assert to specific usage distribution, but I know people who use Terraform. Although Terraform supports CF, my team decided not to use it simply because CF already satisfies our needs.
What is YOUR team using for deploying repeated configurations of AWS
services?
My team uses CloudFormation (without Terraform) to deploy our whole infrastructure to AWS
How usable/learnable is it?
Pretty easy. Start with a small template (ideally YAML), then build up from there. The aws cloudformation deploy will speed up your feedback loop.
If I adopt CF, how likely is it that existing developers on AWS will already be familiar with it, and be able to use it straight off the bat?
I think developers who are familiar to AWS can easily pick up CF. If you can find your way around AWS documentation, CF is just another service to learn. I can't assert to the likeliness that existing AWS devs are familiar with CF.
CF seems to support many or most AWS services already, but are people actually using it to repeatedly stamp out identically-configured topologies of services?
My team uses it to provision testing and production environments that have the same topology. Some parts of our infrastructure is duplicated for redundancy using shared CF templates.
What pitfalls do I need to watch out for when using CloudFormation Templates?
You have to watch out for some CF limits, namely the template body's maximum size, which is capped at 46KB. We have hit this limit a few times, especially when provisioning EC2 instances with larger user data scripts. That being said, you should not hit that limit early on, and there are many workarounds
What doesn't CF handle, which it really should?
From the top of my head: Elastic Transcoder, EC2 AMIs, API Gateway VPC Links. My team has circumvented these limitations using Lambda-backed custom resources, which allow you to extend CF to your needs.
Overall, my team is very satisfied with CloudFormation. It definitely helps us maintain our AWS accounts in order.
Hope this helps!