I have 2 VPCs in my account:
aws ec2 describe-vpcs --query 'Vpcs[*].VpcId'
[
"vpc-654bf20c",
"vpc-184bf271"
]
But when I check my account-attributes, I don't see any default VPC tagged:
aws ec2 describe-account-attributes --query 'AccountAttributes[5]'
{
"AttributeName": "default-vpc",
"AttributeValues": [
{
"AttributeValue": "none"
}
]
}
How do I make any of the VPCs as a default VPC so that when I push stacks using CloudFormation, I don't have to mention VPCs?
You wont be able to convert a non-default VPC into default on your own, you need to get in touch with Amazon support for that.
If you have UI access to amazon console can you cross check if "Default VPC" for either of your VPC is Yes. See reference image.
If there is none, this means that you have intentionally/mitakenly deleted this VPC in the past. You need to raise a support ticket for getting a new default VPC.
Other work around could be that you move all configurations to some other region, if that is possible, you will get a default VPC there.
Thanks
Hope it helps!
Related
I accidentally deleted all the default subnets in aws，I want to recreate default subnets。I make CLI command: "aws ec2 create-default-subnet --availability-zone us-west-2a"
，but always get the error message
"An error occurred (DefaultSubnetAlreadyExistsInAvailabilityZone) when calling the CreateDefaultSubnet operation: 'subnet-015c449cab525d947' is already the default subnet in us-west-2d."
how to solve this problem?
There is only one default subnet can exist in each availability zone, seems you already have yours on us-west-2a, login to you AWS account search for VPC > Subnets and delete what you have there, then you can re-create it with this command:
ws ec2 create-default-subnet --availability-zone us-west-2a
check AWS document fore more info:
https://aws.amazon.com/premiumsupport/knowledge-center/recreate-default-vpc/
NOTE: This is about ELBv2 ALBs, not legacy load balancers, ELBv1, but my humble rep won't let me improve the tagging.
I'm attempting to create an AWS ECS Fargate service. I have created Application Load Balancers one for each container in the Task, and since create-service now supports multiple loadbalancers according to the docs - we should be all good, right? Since it's an ALB, I specify targetGroupArn rather than loadBalancerName, and since ECS has a service linked role created by default, AWSServiceRoleForECS I ought to be able to map target groups and go ahead and create the service, right? Not like it used to be when you had to create ecsServiceRole manually from what the internet tells me.
my command is
aws ecs create-service --region $REGION --cluster $CLUSTER --service-name production-svc --task-definition $TASK_ARN --desired-count 2 --launch-type "FARGATE" --network-configuration "awsvpcConfiguration={subnets=[${SUBNET_1}, ${SUBNET_2}],securityGroups=[${SECURITYGROUP_ID}]}" --load-balancers=tgt-a,containerName=a,containerPort=5000,targetGroupArn=tgt-b,containerName=b,containerPort=6000
And the error I get is
An error occurred (InvalidParameterException) when calling the CreateService operation: Unable to assume role and validate the specified targetGroupArn. Please verify that the ECS service role being passed has the proper permissions.
Now, looking though the internet it could either be that my load balancers don't exist (you'll have to trust me, they do - I supply target group ARNs, and those target groups exist as confirmed with aws elbv2 cli and are mapped to valid active application load balancers that all live in the same region as the cluster), or that the service mapped role (the one AWS created for me earlier) won't have enough rights to assume to verify the targetGroupArn.
Do I really have to add rights to the automatic ECS service mapped role? If so - which rights?
"elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
"elasticloadbalancing:DeregisterTargets",
"elasticloadbalancing:Describe*",
"elasticloadbalancing:RegisterInstancesWithLoadBalancer",
"elasticloadbalancing:RegisterTargets",
My Automatic role (that has AmazonECSServiceRolePolicy) already has all of those? Which ones are missing?
I was having the same issue, and can confirm what Ali says:
Turns out that if you are creating the service via CLI with aws ecs create-service you need the full arn for the loadbalancer.
The documentation confirms that:
https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ecs/create-service.html
It's a bit confusing because in the CloudFormation template you can just use the "short" name. Also the error message is a bit misleading.
Hope this helps.
I have a Lambda function in my VPC, and I want to access S3 bucket.
I have setup the S3 VPC endpoint correctly, I think, because I created an EC2 instance in the same subnet and security group as the Lambda function. When I ran a copy of Lambda function code on the EC2 instance, it can correctly showed the S3 file content.
But when I run the code in Lambda, it failed. So, I want to know what is the difference between "run in EC2" and "run in Lambda"? Why did it fail when I ran it in Lambda?
Here is my Lambda function code:
import boto3
s3 = boto3.client('s3', region_name='ap-northeast-1')
def lambda_handler(event, context):
bucket = '*xxxxxx*'
key = 's3-upload.json'
try:
response = s3.get_object(Bucket=bucket, Key=key)
print('--------------------------------------')
print(response)
print('--------------------------------------')
body = response['Body'].read()
print(body)
print('--------------------------------------')
print("CONTENT TYPE: " + response['ContentType'])
except Exception as e:
print('Error getting object.')
print(e)
raise e
If you want to allow an AWS Lambda to access Amazon S3, use one of these methods:
Do not associate the function to a VPC. Access is then automatic.
If the function is attached to a public subnet in the VPC, associate an Elastic IP to the Lambda function's ENI that appears in the VPC (Not recommended)
If the function is attached to a private subnet in the VPC, launch a NAT Gateway in the public subnet and update Route Tables. Traffic will flow to the Internet via the NAT Gateway.
Add an Amazon S3 VPC Endpoint in the VPC and update Route Tables. Traffic will flow through that instead of the Internet Gateway.
Even though they're in the same VPC, EC2 and Lambda are still different environments within AWS. Being able to run your code in one and not the other implies that your code is fine and works, so it's likely to be a configuration issue with AWS.
Have you checked the service/execution role that the lambda is using?
You need to ensure that the IAM role that it's using is allowed the correct level of S3 access.
This documentation on execution roles for lambda might provide a useful jumping off point: https://docs.aws.amazon.com/lambda/latest/dg/intro-permission-model.html#lambda-intro-execution-role
An IAM policy like this would give whatever execution role you use read-only access to all your S3 buckets, and happens to be one of the AWS managed policies.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:Get*",
"s3:List*"
],
"Resource": "*"
}
]
}
Thanks everyone! I found the reason.
My Lambda have two subnets, private_sn_1 and private_sn_2,
private_sn_1 have correctly set the vpc endpoint route table,
but the private_sn_2 set a wrong route table,
and my ec2 created in private_sn_1 so it can access the vpc endpoint.
In normal, Lambda will run randomly in private_sn_1 or private_sn_2,
but in my case it always run in private_sn_2(I don't know why),
so when I fixed the private_sn_2 route table,
everything is right.
In addition to all said above, it is also possible that VPC Endpoint policy can be prohibitive and not allowing traffic to/from S3 through. Make sure you allow traffic through endpoint by using "Full access" policy.
Edit: here's related bit of documentation:
Your policy must contain a Principal element. For gateway endpoints
only, you cannot limit the principal to a specific IAM role or user.
Specify "*" to grant access to all IAM roles and users. Additionally,
for gateway endpoints only, if you specify the principal in the format
"AWS":"AWS-account-ID" or "AWS":"arn:aws:iam::AWS-account-ID:root",
access is granted to the AWS account root user only, and not all IAM
users and roles for the account.
So for S3 endpoints to work you need to specify '*' as a principal in general case
I'm quite bad at using AWS but I'm trying to automate the set up of an ECS cluster with private DNS names in route53, using the new service discovery mechanism. I am able to click my way through the AWS UI to accomplish a DNS entry showing up in a private hosted zone but I cannot figure out the JSON parameters to add to the json for the command below to accomplish the same thing.
aws ecs create-service --cli-input-json file://aws/createService.json
and below is the approximate contents of the createService.json
referenced above
"cluster": "clustername",
"serviceName": "servicename",
"taskDefinition": "taskname",
"desiredCount": 1,
// here is where I'm guessing there should be some DNS config referencing some
// namespace or similar that I cannot figure out...
"networkConfiguration": {
"awsvpcConfiguration": {
"subnets": [
"subnet-11111111"
],
"securityGroups": [
"sg-111111111"
],
"assignPublicIp": "DISABLED"
}
}
I'd be grateful for any ideas since my googling skills apparently aren't good enough for this problem as it seems. Many thanks!
To automatically have an ECS service register instances into a servicediscovery service you can use the serviceRegistries attribute. Add the following to the ECS service definition json:
{
...
"serviceRegistries": [
{
"registryArn": "arn:aws:servicediscovery:region:aws_account_id:service/srv-utcrh6wavdkggqtk"
}
]
}
The attribute contains a list of autodiscovery services that should be updated by ECS when it creates or destroys a task as part of the service. Each registry is referenced using the ARN of the autodiscovery service.
To get the Arn use the AWS cli command aws servicediscovery list-services
Strangely the documentation of the ECS service definition does not contain information about this attribute. However this tutorial about service discovery does.
As it turns out there is no support in ecs create service for adding it to the service registry, i.e. the route53 private hosted zone. Instead I had to use aws servicediscovery create-service and then servicediscovery register-instance to finally get an entry in my private hosted zone.
This became a quite complicated solution so I'll instead give Terraform a shot at it since I found they recently added support for ECS service discovery and see where that takes me...
Looking for a quick way to pull my account number, I had originally thought of using aws iam get-account-authorization-details --max-items 1 but there are several issues with doing it this way. Is there a way to do this that might not cross account origins?
You can get the account number from the Secure Token Service subcommand get-caller-identity using the following:
aws sts get-caller-identity --query Account --output text
From my related answer for the AWS PowerShell CLI, your Account ID is a part of the Arn of resources that you create... and those that are automatically created for you. Some resources will also list you as an OwnerId.
The Default Security Group is automatically created for you in each region's default VPC as a reserved security group. From the documentation:
You can't delete a default security group. If you try to delete the EC2-Classic default security group, you'll get the following error: Client.InvalidGroup.Reserved: The security group 'default' is reserved. If you try to delete a VPC default security group, you'll get the following error: Client.CannotDelete: the specified group: "sg-51530134" name: "default" cannot be deleted by a user.
This makes it a reliable candidate for retrieving our account Id, as long as you are in EC2 classic or have a default VPC (*see edge cases if you don't).
Example:
aws ec2 describe-security-groups \
--group-names 'Default' \
--query 'SecurityGroups[0].OwnerId' \
--output text
This uses --query to filter the output down to the "owner ID" for the first result from this request, and then uses --output to output your account ID as plaintext:
123456781234
Edge cases:
(Thanks #kenchew) Note that if you've deleted your default VPC in a given region, this security group no longer exists and you should use one of these alternative solutions:
query STS get-caller-identity, per #Taras
use the first security group returned, per #Phillip
Further reading:
AWS EC2 Documentation: Default Security Groups
AWS CLI Documentation: aws ec2 describe-security-groups
Controlling Command Output from the AWS Command Line Interface
If you are running on a server that is running with an assumed role you can't call aws sts get-caller-identity. Also, with describe-security-groups you can't always use the --group-names filter (it doesn't work if you don't have a default VPC), so just pick the first security group. I've found this to be the most reliable regardless of what sort of authentication you use or what sort of VPC you have.
aws ec2 describe-security-groups --query 'SecurityGroups[0].OwnerId' --output text
My favorite method is to use aws iam get-user [--profile <profile>] since you only need IAM self service role for this to work.