I am trying to run the following command by reading cidr ips out of a txt file.
aws ec2 authorize-security-group-ingress --group-id $sgid --ip-permissions '[{"IpProtocol": "tcp", "FromPort": 9092, "ToPort": 9092, "IpRanges": [{"CidrIp": '${cidreip}', "Description": "$train us-east-2"}]}]'
When I try this with a single quote around my variable for cidr ip, I get the below error.
Error parsing parameter '--ip-permissions': Invalid JSON: Expecting ',' delimiter: line 1 column 87 (char 86)
Trying it with a double quote around the variable, the command takes it as a literal value. Not a variable.
How do I work around it? I tried escaping. I tried using the following
aws ec2 authorize-security-group-ingress --group-id $sgid --ip-permissions "[{"IpProtocol": "tcp", "FromPort": 9092, "ToPort": 9092, "IpRanges": [{"CidrIp": "$cidreip", "Description": "$train us-east-2"}]}]"
and I get a Error parsing parameter '--ip-permissions': Invalid JSON: error.
Please help. Thanks in advance.
First JSON does not use ', but you need ". Also you have to build your string in chunks:
aws ec2 authorize-security-group-ingress --group-id 33333 --ip-permissions '[{"IpProtocol": "tcp", "FromPort": 9092, "ToPort": 9092, "IpRanges": [{"CidrIp": "'${cidreip}'", "Description": "'${train}' us-east-2"}]}]'
Related
I'm running into quoting issues with AWS CLI when trying to update description of security group rules.
Can anyone advise how I can rewrite this piece to accommodate a list of IPs?
while read -r line; do
aws ec2 update-security-group-rule-descriptions-ingress \
--group-id sg-123456 \
--region us-east-2 \
--ip-permissions "[{'IpProtocol': 'tcp', 'FromPort': 443, 'ToPort': 443, 'IpRanges': [{'CidrIp': ${line}, 'Description': 'Meaningful description'}]}]"
done < ip_list
Move the single quotes to encompass the whole JSON, then use double quotes for the JSON content, which typically expects double quotes.
--ip-permissions '[{"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": ${line}, "Description": "Meaningful description"}]}]'
I ended up having to use this format:
while IFS=, read -r IP CLIENT_DESC
do
aws ec2 authorize-security-group-ingress --region us-east-2 --group-id sg-123456 --ip-permissions IpProtocol=tcp,FromPort=443,ToPort=443,IpRanges="[{CidrIp=${IP},Description=${CLIENT_DESC}}]"
done < server_rules
At my AWS account, I have few VPC. I'm trying to find a way to list all resources that located under a certain VPC.
Thanks!!
You can do it in three ways: AWS CLI, AWS console or code.
AWS CLI
You can use AWS CLI to list all ENIs associated with the VPC and prettify the output using the --query parameter to get a resource list with the desired fields (AZ, instance-id, etc.).
aws ec2 describe-network-interfaces --filters Name=vpc-id,Values=<vpc-id> --query 'NetworkInterfaces[*].[AvailabilityZone, OwnerId, Attachment.InstanceId, PrivateIpAddresses[*].Association.PublicIp]'
aws ec2 describe-network-interfaces --filters Name=vpc-id,Values=<vpc-id> --query 'NetworkInterfaces[*].[RequesterId,Description]'
A sample of the raw output (only one instance on the VPC):
"NetworkInterfaces": [
{
"Association": {
"IpOwnerId": "amazon",
"PublicDnsName": "ec2-54-196-57-169.compute-1.amazonaws.com",
"PublicIp": "54.196.57.169"
},
"Attachment": {
"AttachTime": "2020-08-24T10:59:16+00:00",
"AttachmentId": "eni-attach-047e562690aabbffd",
"DeleteOnTermination": true,
"DeviceIndex": 0,
"InstanceId": "i-0fe495a6c17bd0f82",
"InstanceOwnerId": "570398916848",
"Status": "attached"
},
"AvailabilityZone": "us-east-1d",
"Description": "",
"Groups": [
{
"GroupName": "launch-wizard-1",
"GroupId": "sg-0aa7d8257bb487e1b"
}
],
"InterfaceType": "interface",
"Ipv6Addresses": [],
"MacAddress": "0e:58:38:33:9a:31",
"NetworkInterfaceId": "eni-0b20855178d276783",
"OwnerId": "570398916848",
"PrivateDnsName": "ip-172-31-34-30.ec2.internal",
"PrivateIpAddress": "172.31.34.30",
"PrivateIpAddresses": [
{
"Association": {
"IpOwnerId": "amazon",
"PublicDnsName": "ec2-54-196-57-169.compute-1.amazonaws.com",
"PublicIp": "54.196.57.169"
},
"Primary": true,
"PrivateDnsName": "ip-172-31-34-30.ec2.internal",
"PrivateIpAddress": "172.31.34.30"
}
],
"RequesterManaged": false,
"SourceDestCheck": true,
"Status": "in-use",
"SubnetId": "subnet-e2bc5fbd",
"TagSet": [],
"VpcId": "vpc-6ad2e110"
}
]
And now filtered:
For the first --query
[
"us-east-1d",
"57039816848",
"i-0fe495a6c17bd0f82",
[
"44.196.57.169"
]
]
And for the second --query (another VPC):
[
"amazon-elasticache",
"ElastiCache alon-001"
],
[
"amazon-elasticache",
"ElastiCache alon-002"
],
[
"975289786086",
"arn:aws:ecs:us-east-2:57039916848:attachment/22a90802-fae7-4afb-9a7e-43e6f4be8ca4"
],
[
"074689309192",
"Interface for NAT Gateway nat-069344579d8bda20"
],
[
"amazon-elb",
"ELB app/EC2Co-EcsEl-YX74WCWEGOK/0b6d7bc60b540b1"
],
[
"amazon-elb",
"ELB app/EC2Co-EcsEl-YX74WCWGGOK/0b6bd7c60b540b1"
],
[
"amazon-elasticache",
"ElastiCache alon-003"
]
AWS Console
You can do the same using the AWS console.
Under EC2->Network Interfaces, search for the desired vpc-id in the search bar.
Code
Using a python script called vpc-inside.py you can describe all of your VPC resources.
usage: vpc-inside.py [-h] -v VPC [-r REGION] [-p PROFILE]
optional arguments:
-h, --help show this help message and exit
-v VPC, --vpc VPC The VPC to annihilate
-r REGION, --region REGION AWS region that the VPC resides in
-p PROFILE, --profile PROFILE AWS profile
And the output will look like this:
EKSs in VPC vpc-07ef7f777429cfd82:
Omikron
--------------------------------------------
ASGs in VPC vpc-07ef7f777429cfd82:
eks-pooks-9ebf225b-70a9-a026-034f-c7431df9b7ba resides in vpc-07ef7f777429cfd82
eks-pooks-9ebf225b-70a9-a026-034f-c7431df9b7ba
--------------------------------------------
RDSs in VPC vpc-07ef7f777429cfd82:
--------------------------------------------
EC2s in VPC vpc-07ef7f777429cfd82:
i-0c63874d77ea2ba78
i-043740f224015e69e
--------------------------------------------
Lambdas in VPC vpc-07ef7f777429cfd82:
--------------------------------------------
Classic ELBs in VPC vpc-07ef7f777429cfd82:
--------------------------------------------
ELBs V2 in VPC vpc-07ef7f777429cfd82:
--------------------------------------------
NAT GWs in VPC vpc-07ef7f777429cfd82:
--------------------------------------------
VPC EndPoints in VPC vpc-07ef7f777429cfd82:
--------------------------------------------
IGWs in VPC vpc-07ef7f777429cfd82:
--------------------------------------------
ENIs in VPC vpc-07ef7f777429cfd82:
eni-079231232dc136305
eni-05ff227eca8341a08
eni-0c01b2871887ac3f7
eni-00e11d4f9590161b4
--------------------------------------------
Security Groups in VPC vpc-07ef7f777429cfd82:
sg-0b4554a65e1560745
sg-0f93574d6b180b263
--------------------------------------------
Routing tables in VPC vpc-07ef7f777429cfd82:
rtb-0694bdbdd696b2bed
rtb-072ec82a18d8a04ba
--------------------------------------------
ACLs in VPC vpc-07ef7f777429cfd82:
acl-0c0087eabf9335940
--------------------------------------------
Subnets in VPC vpc-07ef7f777429cfd82:
subnet-0b8cc1132727e5b5d
subnet-0e47ee92a9ca80280
subnet-0c25990d9a138616b
--------------------------------------------
You can try in AWS Config > Advanced queries and run below query :
All resources:
SELECT
resourceId,
resourceName,
resourceType
Resources directly associated to VPC:
SELECT
resourceId,
resourceName,
resourceType
WHERE
relationships.resourceId = 'vpc-02368dae78f1387e5'
Query can be further enhanced, see some example of preconfigured query.
VPCs mostly contain EC2 instances, RDS instances, Load Balancers and Lambda functions. Plus, things that use EC2 underneath, like Elasticache. These are the types of resources that connect into a VPC.
Some people suggest using the Tag Editor to find resources: Is there a way to list all resources in AWS.
I also like aws inventory, which simply runs in your browser and does a great job of showing resources. Just give it an Access Key and Secret Key to run.
There's no built in service to easily do this.
The best hope you'd have of find all resources is programatically looping over resources that support:
SubnetId
VpcId
My exposed service on nodePort seems to not allow traffic through it.
So how to add rules to allow traffic for that port range on CLI not on the console?
EC2 Security groups
There is a security group on your screen.
See more about security groups:
EC2 Security Groups
Creating a Security Group
CLI for AWS Security groups
As for CLI for working with AWS Security groups, see this article: Creating, Configuring, and Deleting Security Groups for Amazon EC2 - AWS Command Line Interface
$ aws ec2 create-security-group --group-name my-sg --description "My security group" --vpc-id vpc-1a2b3c4d
{
"GroupId": "sg-903004f8"
}
$ aws ec2 authorize-security-group-ingress --group-id sg-903004f8 --protocol tcp --port 3389 --cidr 203.0.113.0/24
The following command adds another rule to enable SSH to instances in the same security group.
$ aws ec2 authorize-security-group-ingress --group-id sg-903004f8 --protocol tcp --port 22 --cidr 203.0.113.0/24
To view the changes to the security group, run the describe-security-groups command.
$ aws ec2 describe-security-groups --group-ids `sg-903004f8`
O/P is:
{
"SecurityGroups": [
{
"IpPermissionsEgress": [
{
"IpProtocol": "-1",
"IpRanges": [
{
"CidrIp": "0.0.0.0/0"
}
],
"UserIdGroupPairs": []
}
],
"Description": "My security group"
"IpPermissions": [
{
"ToPort": 22,
"IpProtocol": "tcp",
"IpRanges": [
{
"CidrIp": "203.0.113.0/24"
}
]
"UserIdGroupPairs": [],
"FromPort": 22
}
],
"GroupName": "my-sg",
"OwnerId": "123456789012",
"GroupId": "sg-903004f8"
}
]
}
P.S. awless.io - A Mighty CLI for AWS
There is also a bit outdated but still convenient CLI tool:
wallix/awless: A Mighty CLI for AWS
A Mighty CLI for AWS http://awless.io/
Here the Medium post about it
after the VPC is created, how I can grep only 1 VPC-ID from specific VPC by aws ec2 describe-vpcs, so that VPC ID can be passed inside the script for the next step, I know I can see it manually from that command or from AWS console,
for example:
aws ec2 describe-vpcs --vpc-ids |grep VpcId
"VpcId": "vpc-00a0338c2f671a77c",
"VpcId": "vpc-0b3697513d5987516",
"VpcId": "vpc-061e25f5f78877798",
it gives me all of them, or:
aws ec2 describe-vpcs --vpc-ids |grep -i ansible
"Value": "ANSIBLE_VPC",
but I need only to get the VPC-ID for that specific VPC from command.
If you just issued a create-vpc command, then the VPC ID of that VPC would have been returned in response to that command:
Output:
{
"Vpc": {
"CidrBlock": "10.0.0.0/16",
"DhcpOptionsId": "dopt-5EXAMPLE",
"State": "pending",
"VpcId": "vpc-0a60eb65b4EXAMPLE", <-- This is the VPC ID
"OwnerId": "123456789012",
"InstanceTenancy": "default",
"Ipv6CidrBlockAssociationSet": [],
"CidrBlockAssociationSet": [
{
"AssociationId": "vpc-cidr-assoc-07501b79ecEXAMPLE",
"CidrBlock": "10.0.0.0/16",
"CidrBlockState": {
"State": "associated"
}
}
],
"IsDefault": false,
"Tags": []
}
}
Thus, you could create the VPC and store its ID like this:
$ ID=`aws ec2 create-vpc --cidr-block 10.0.0.0/16 --query Vpc.VpcId --output text`
$ echo $ID
vpc-0fb4d08f9d6501e94
If, instead, you are seeking the VPC ID for a VPC given its Name tag, you could use:
$ ID=`aws ec2 describe-vpcs --filter Name=tag:Name,Values=ANSIBLE_VPC --query Vpcs[].VpcId --output text`
$ echo $ID
vpc-0fb4d08f9d6501e94
You can use just the aws cli for this, with filters and query:
aws ec2 describe-vpcs --filters Name=tag:Name,Values=ANSIBLE_VPC --query "Vpcs[].VpcId" --output text"
Or you can use a mix of the --filters command with grep to accomplish your task:
aws ec2 describe-vpcs --filters Name=tag:Name,Values=ANSIBLE_VPC | grep VpcId | grep -oh "vpc-\w*"
I have my CloudFormation script like this now:
"SecurityGroupIngress" : [{
"IpProtocol" : "tcp",
"FromPort" : "0",
"ToPort" : "65535",
"CidrIp" : "0.0.0.0/0"
}]
and it looks like this, which is fine:
But I am wondering how to I update the template to get this:
Notice the Ports say All. I also wonder if they are different?
The original solution I posted (and accepted by the original poster) stopped working as AWS no longer supports it. To avoid the barrage of downvotes, I deleted the answer. The alternatives are:
Specify the ports 0 and 65535
or
Open all ports for all protocols not just TCP (as suggested by thewire247 below)
"SecurityGroupIngress" : [{
"IpProtocol" : "-1",
"CidrIp" : "0.0.0.0/0"
}]
If you are looking to allow all protocols and all ports, then you can do the following
{
"IpProtocol" : "-1"
"CidrIp" : "0.0.0.0/0"
}
FromPort
Start of port range for the TCP and UDP protocols, or an ICMP type number. If you specify icmp for the IpProtocol property, you can specify -1 as a wildcard (i.e., any ICMP type number).
ToPort
End of port range for the TCP and UDP protocols, or an ICMP code. If you specify icmp for the IpProtocol property, you can specify -1 as a wildcard (i.e., any ICMP code).
ex.
{ "IpProtocol" : "icmp", "FromPort" : "8", "ToPort" : "-1", "CidrIp" : "10.0.0.0/24" }
ref:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-security-group-ingress.html