I need to find what IP address was used in each individual execution of a AWS CodeBuild.
My first approach was to print out the value of the environment variable $HOSTNAME into a file then export it as an artifact. But this solution is not ideal since I need to "inject" these commands into the buildspec.yml (which is not on my control).
So my second approach was to find out what ENI the execution used through the batchGetBuilds method from the SDK. Link to the docs.
One of the properties returned from that method is exactly what I need, the ENI ID: networkInterface.networkInterfaceId
The problem is, the ENI gets deleted after the execution of the build, it is not listed in the EC2 Network Interfaces console anymore.
I was able to find the excluded ENI in AWS Config when listing excluded resources. But since it is now excluded, there is no IP address attached to it anymore.
So, what can I do now? Is it possible to "restore" this ENI so I can find out what IP address it used? Or is there a better way of finding out the IP address used by CodeBuild in each execution?
Related
I would like to try to setup AWS Launch Template, or just Spot request (persistance) and I need automatically attach my specific volume.
The main idea - spot instance will be process data and store it in a separate volume. When Spot will die, another Spot should be requested automatically (which will be built from an image with predefined software) and data should continue processing automatically (and again, storing in my second volume).
But, I can`t setup it in AWS console, so, looks like it is not possible. Am I wrong? Is it possible in some another way?
The same according IP address - I would like to have the same IP address for any of "versions" of Spot (after recreating for example)
It is possible to attach and detach Elastic IP using AWS CLI to achieve what you want.
However there are other possible workarounds in case you do not want to script AWS CLI :
Using Route53 you can define an A record with TTL of 60 seconds if you can accept few minutes of transition. That way you get to use a domain to access your underlying instance instead.
Setup a ALB and forward the request to the EC2 fleet
I've been trying to set up Google Cloud SQL with a private IP connection, where
the IP range it's bound to is manually allocated, and have not succeeded. I
don't know if this is a bug in the implementation because it's still in beta, if
there's something missing from the docs, or if I'm just doing something wrong.
(A command-line session is at the bottom, for a quick summary of what I'm
seeing.)
Initially, I set it up to automatically allocate the IP range. It all worked
just fine, except that it chose 172.17.0.0/24, which is one of the networks
managed by docker on my GCE instance, so I couldn't connect from there (but
could on another machine without docker). So then I tried going down the manual
allocation route.
First, I tore down all the associated network objects that had been created on
my behalf. There were two VPC Peerings, cloudsql-postgres-googleapis-com and
servicenetworking-googleapis-com, which I deleted, and then I confirmed that
the routing entry associated with them disappeared as well.
Then, I followed the directions at https://cloud.google.com/vpc/docs/configure-private-services-access#allocating-range, creating 10.10.0.0/16, because I wanted it in my default network, which is
auto mode, so I'm limited to the low half (which is currently clear).
At that point, I went back to the Cloud SQL instance creation page, since it
should be doing the rest for me. I checked the "Private IP" box, and chose the
default network.
I wasn't taking notes at the time, so my recollection may be flawed,
particularly since my experience in later attempts was consistently different,
but what I remember seeing was that below the network choice dropdown, it said
"This instance will use the existing managed service connection". I assumed
that meant it would use the address range I'd created, and went forward with the
instance creation, but the instance landed on the 172.17.0.0/24 network again.
Back around the third time, where that message was before, it had a choice box
listing my address range. Again, my recollection was poor, so I don't know if I
either saw or clicked on the "Connect" button, but the end result was the same.
On the fourth attempt, I did notice the "Connect" button, and made sure to click
it, and wait for it to say it succeeded. Which it did, sort of: it replaced the
dropdown and buttons with the same message I'd seen before about using the
existing connection. And again, the instance was created on the wrong network.
I tried a fifth time, this time having created a new address range with a new
name -- google-managed-services-default -- which was the name that the
automatic allocation had given it back when I first started (and what the
private services access docs suggest). But even with that name, and explicitly
choosing it, I still ended up with the instance on the wrong network.
Indeed, I now see that after I click "Connect", I can go check the routes and
see that the route that was created is to 172.17.0.0/24.
The same thing seems to happen if I do everything from the command-line:
$ gcloud beta compute addresses list
NAME ADDRESS/RANGE TYPE PURPOSE NETWORK REGION SUBNET STATUS
google-managed-services-default 10.11.0.0/16 INTERNAL VPC_PEERING default RESERVED
$ gcloud beta services vpc-peerings connect \
--service=servicenetworking.googleapis.com \
--ranges=google-managed-services-default \
--network=default \
--project=...
$ gcloud beta services vpc-peerings list --network=default
---
network: projects/.../global/networks/default
peering: servicenetworking-googleapis-com
reservedPeeringRanges:
- google-managed-services-default
---
network: projects/.../global/networks/default
peering: cloudsql-postgres-googleapis-com
reservedPeeringRanges:
- google-managed-services-default
$ gcloud beta compute routes list
NAME NETWORK DEST_RANGE NEXT_HOP PRIORITY
peering-route-ad7b64a0841426ea default 172.17.0.0/24 cloudsql-postgres-googleapis-com 1000
So now I'm not sure what else to try. Is there some state I didn't think to clear? How is the route supposed to be connected to the address range? Why is it creating two peerings when I only asked for one? If I were to create a route manually to the right address range, I presume that wouldn't work, because the Postgres endpoint would still be at the wrong address.
(Yes, I could reconfigure docker, but I'd rather not.)
I found here https://cloud.google.com/sql/docs/mysql/private-ip that this seems to be the correct behaviour:
After you have established a private services access connection, and created a Cloud SQL instance with private IP configured for that connection, the corresponding (internal) subnet and range used by the Cloud SQL service cannot be modified or deleted. This is true even if you delete the peering and your IP range. After the internal configuration is established, any Cloud SQL instance created in that same region and configured for private IP uses the original internal configuration.
There turned out to be a bug somewhere in the service machinery of Google Cloud, which is now fixed. For reference, see the conversation at https://issuetracker.google.com/issues/118849070.
The Cloudwatch docs show its possible, based on the outcome of an alarm, to (amongst others) either reboot, or recover an instance. What is the difference between a reboot and a recovery?
The docs say:
A recovered instance is identical to the original instance, including the instance ID, private IP addresses, Elastic IP addresses, and all instance metadata.
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-recover.html
But from the tests I have done, that is the case for rebooted instances too.
Just found what I needed, But Ill leave this here for others searching for the same thing.
Recovery will move the instance to a new hypervisor, rebooting keeps the instance on the same hypervisor.
Is it possible to do AutoScaling with Static IPs in AWS ? The newly created instances should either have a pre-defined IP or pick from a pool of pre-defined IPs.
We are trying to setup ZooKeeper in production, with 5 zooKeeper instances. Each one should have a static-IP which are to hard-coded in the Kafka's AMI/Databag that we use. It should also support AutoScaling, so that if one of the zooKeeper node goes down, a new one is spawned with the same IP or from a pool of IPs. For this we have decided to go with 1 zoo-keeper instance per AutoScaling group, but the problem is with the IP.
If this is the wrong way, please suggest the right way. Thanks in advance !
One method would be to maintain a user data script on each instance, and have each instance assign itself an elastic IPs from a set of EIPs assigned for this purpose. This user data script would be referenced in the ASGs Launch Configuration, and would run on launch.
Say the user script is called "/scripts/assignEIP.sh", using the AWS CLI you would have it consult the pool to see which ones are available and which ones are not (already in use). Then it would assign itself one of the available EIPS.
For ease of IP management, you could keep the pool of IPs in a simple text properties file on S3, and have the instance download and consult that list when the instance starts.
Keep in mind that each instance will need an to be assigned IAM instance profile that will allow each instance to consult and assign EIPs to itself.
I'm puzzled my EC2's use of the bare IP address 169.254.169.254 for the URI's to retrieve user and instance metadata. Wouldn't it be a better design decision for both Amazon and users if a hostname that was easier to remember was used, say metadata.ec2.amazonaws.com? If Amazon decides to change the bare IP address in the future, all the associated scripts that fetch user or instance metadata stop working.
You might say that I should use the Amazon supplied tool EC2 Metadata, but it hasn't been updated in close to two years. Besides, the script itself would need to be updated should Amazon decide to change the IP address from the random 169.254.169.254 to something equally random, say 170.11.19.142.
Is there something I'm missing here?
Is there something I'm missing here?
Yes - the 169.254.0.0/16 block is specified as a private block - see 169.254.0.0/16 addresses explained. Therefore, it's accessibly on that IP from machine within the private network - like your instance. Amazon isn't going to change this address to a whole other block, like your 170.11.19.142, because it wouldn't be a private internal block.
The last two numbers, 169.254 are likely random, as you say. They were chosen by Amazon at some point in time, and will likely stay that way for quite a long time, seeing as Amazon has full control over that IP space.
You might say that I should use the Amazon supplied tool EC2 Metadata
You should.
, but it hasn't been updated in close to two years. Besides, the script itself would need to be updated should Amazon decide to change the IP address
Not necessarily. I haven't seen the script source code, but it's likely that, if the address is expected to change any time soon, it would check somehow with a root EC2 controller what IP the metadata server is at.