Why don't I create Amazon lightsailclient and set up UserData?
var shuju = new CreateInstancesRequest()
{
BlueprintId = "centos_7_1901_01",
BundleId = "micro_2_0",
AvailabilityZone = "ap-northeast-1d",
InstanceNames = new System.Collections.Generic.List<string>() { "test" },
UserData = "echo root:test123456- |sudo chpasswd root\r\nsudo sed -i 's/^#\\?PermitRootLogin.*/PermitRootLogin yes/g' /etc/ssh/sshd_config;\r\nsudo sed -i 's/^#\\?PasswordAuthentication.*/PasswordAuthentication yes/g' /etc/ssh/sshd_config;\r\nsudo reboot\r\n"
};
If you wish to run a User Data script on a Linux instance, then the first line must begin with #!.
It uses the same technique as an Amazon EC2 instance, so see: Running Commands on Your Linux Instance at Launch - Amazon Elastic Compute Cloud
Related
I'm wondering if it's possible to know when the script in user data executes completely?
data "template_file" "script" {
template = file("${path.module}/installing.sh")
}
data "template_cloudinit_config" "config" {
gzip = false
base64_encode = false
# Main cloud-config configuration file.
part {
filename = "install.sh"
content = "${data.template_file.script.rendered}"
}
}
resource "aws_instance" "web" {
ami = "ami-04e7b4117bb0488e4"
instance_type = "t2.micro"
key_name = "KEY"
vpc_security_group_ids = [aws_default_security_group.default.id]
subnet_id = aws_default_subnet.default_az1.id
associate_public_ip_address = true
iam_instance_profile = "Role_S3"
user_data = data.template_cloudinit_config.config.rendered
tags = {
Name = "Terraform-Ansible"
}
}
And in the content of the script I have this.
It tells me Terraform successfully apply the changes, but the script is still running, is there a way I can monitor that?
#!/usr/bin/env bash
exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1
echo BEGIN
sudo apt update
sudo apt upgrade -y
sudo apt install -y unzip
echo END
No, You can not confirm the user data status from the terraform, as it posts launching script that executes once EC2 instance launched. But you will need some extra effort on init script that one way to check.
How to check User Data status while launching the instance in aws
If you do something that is mentioned above to make some marker file once user data completed, then you can try this to check.
resource "null_resource" "user_data_status_check" {
provisioner "local-exec" {
on_failure = "fail"
interpreter = ["/bin/bash", "-c"]
command = <<EOT
echo -e "\x1B[31m wait for few minute for instance warm up, adjust accordingly \x1B[0m"
# wait 30 sec
sleep 30
ssh -i yourkey.pem instance_ip ConnectTimeout=30 -o 'ConnectionAttempts 5' test -f "/home/user/markerfile.txt" && echo found || echo not found
if [ $? -eq 0 ]; then
echo "user data sucessfully executed"
else
echo "Failed to execute user data"
fi
EOT
}
triggers = {
#remove this once you test it out as it should run only once
always_run ="${timestamp()}"
}
depends_on = ["aws_instance.my_instance"]
}
so this script will check marker file on the newly launch server by doing ssh with timeout 30 seconds with max attempts 5.
Here are some pointers to remember:
User data shell scripts must start with the Shebang #! characters and the path to the interpreter you want to read the script (commonly /bin/bash).
Scripts entered as user data are run as the root user, so no need to use the sudo command in the init script.
When a user data script is processed, it is copied to and run from /var/lib/cloud/instances/instance-id/. The script is not deleted after it is run and can be found in this directory with the name user-data.txt So to check if your shell script made to the server refer this directory and the file.
The cloud-init output log file (/var/log/cloud-init-output.log) captures console output of your user_data shell script. to know how your user_data shell script was executed and its output check this file.
Source: https://www.middlewareinventory.com/blog/terraform-aws-ec2-user_data-example/
Well I use these two ways to confirm.
At the end of cloudinit config file this line sends me a notification through whatsapp (using callmebot). Thus no matter how much does it take to setup, I always get notified when it's ready to use. I watch some series or read something in that time. no time wasted.
curl -X POST "https://api.callmebot.com/whatsapp.php?phone=12345678910&text=Ec2+transcoder+setup+complete&apikey=12345"
At the end of cloudinit config this line runs -
echo "for faster/visual confirmation of above execution.."
wget https://www.sample-videos.com/video123/mp4/720/big_buck_bunny_720p_1mb.mp4 -O /home/ubuntu/dpnd_comp.mp4
When I sign in to the instance I can see directly the file.
And I'm loving it. Hope this helps someone. Also, don't forget to tell me your method too.
Im a novice to google cloud compute api in node
im using this library
https://googleapis.dev/nodejs/compute/latest/index.html
im authenticated and can make API requests that is all set up
all im trying to do is make a start up script that will download from this URL
http://eve-robotics.com/release/EveAIO_setup.exe and places the folder on the desktop
i have this but im 100% sure this is way off based on some articles and docs i am seeing but i know nothing ab bash, start up scripts
this is what i have
const Compute = require('#google-cloud/compute');
const compute = new Compute();
const zone = compute.zone('us-central1-c')
async function createVM(){
vmName = 'start-script-trial3'
// const [vm, operation] = await zone.createVM(vmName, {
// })
const config = {
os: 'windows',
http: true,
metadata: {
items: [
{
key: 'startup-script',
value: `curl http://eve-robotics.com/release/EveAIO_setup.exe --output Eve`,
},
]}
}
const vm = zone.vm(vmName)
const [gas, operation] = await vm.create(config)
console.log(operation.id)
}
createVM()
I was able to do it in bash:
I made a 'bat' script for windows:
#ECHO OFF
curl http://eve-robotics.com/release/EveAIO_setup.exe --output C:\Users\Eve
I copied the script to GCS:
gsutil cp file.bat gs://my-bucket/
Then I run the gcloud command:
gcloud compute instances create example-windows-instance --scopes storage-ro --image-family=windows-1803-core --image-project=windows-cloud --metadata windows-startup-script-url=gs://marian-b/file.bat --zone=europe-west1-c
My problem
I have successfully deployed a nomad job with a few dozen Redis Docker containers on AWS, using the default Redis image from Dockerhub.
I've slightly altered the default config file created by nomad init to change the number of running containers, and everything works as expected
The problem is that the actual image I would like to run is in ECR, which requires AWS permissions (access and secret key), and I don't know how to send these.
Code
job "example" {
datacenters = ["dc1"]
type = "service"
update {
max_parallel = 1
min_healthy_time = "10s"
healthy_deadline = "3m"
auto_revert = false
canary = 0
}
group "cache" {
count = 30
restart {
attempts = 10
interval = "5m"
delay = "25s"
mode = "delay"
}
ephemeral_disk {
size = 300
}
task "redis" {
driver = "docker"
config {
# My problem here
image = "https://-whatever-.dkr.ecr.us-east-1.amazonaws.com/-whatever-"
port_map {
db = 6379
}
}
resources {
network {
mbits = 10
port "db" {}
}
}
service {
name = "global-redis-check"
tags = ["global", "cache"]
port = "db"
check {
name = "alive"
type = "tcp"
interval = "10s"
timeout = "2s"
}
}
}
}
}
What have I tried
Extensive Google Search
Reading the manual
Placing the aws credentials in the machine which runs the nomad file (using aws configure)
My question
How can nomad be configured to pull Docker containers from AWS ECR using the AWS credentials?
Pretty late for you, but aws ecr does not handle authentication in the way that docker expects. There you need to run sudo $(aws ecr get-login --no-include-email --region ${your region}) Running the returned command actually authenticates in a docker compliant way
Note that region is optional if aws cli is configured. Personally, I allocate an IAM role the box (allowing ecr pull/list/etc), so that I don't have to manually deal with credentials.
I don't use ECR, but if it acts like a normal docker registry, this is what I do for my registry, and it works. Assuming the previous sentence, it should work fine for you as well:
config {
image = "registry.service.consul:5000/MYDOCKERIMAGENAME:latest"
auth {
username = "MYMAGICUSER"
password = "MYMAGICPASSWORD"
}
}
According to documentation, using terraform, I'm able to create a droplet on digital ocean:
resource "digitalocean_volume" "foobar" {
region = "nyc1"
name = "baz"
size = 100
description = "an example volume"
}
So, I'm also able to add a volume to it:
resource "digitalocean_droplet" "foobar" {
name = "baz"
size = "1gb"
image = "coreos-stable"
region = "nyc1"
volume_ids = ["${digitalocean_volume.foobar.id}"]
}
I'd like to know how to mount this on a desired location.
I need to mount it automatically. I mean, when droplet is up I need to the volume is mounted. I was thinking about using chef...
Any ideas?
To mount the volume automatically, you can use user_data via cloud init to run a script as follow:
This is how your digitalocean_droplet resources should reflect:
resource "digitalocean_droplet" "foobar" {
name = "baz"
size = "1gb"
image = "coreos-stable"
region = "nyc1"
volume_ids = ["${digitalocean_volume.foobar.id}"]
# user data
user_data = "${data.template_cloudinit_config.cloudinit-example.rendered}"
}
Then your cloud.init file that contains the cloudinit_config should be as bellow. It will reference the shell script in ${TERRAFORM_HOME}/scripts/disk.sh that would mount your volume automatically:
provider "cloudinit" {}
data "template_file" "shell-script" {
template = "${file("scripts/disk.sh")}"
}
data "template_cloudinit_config" "cloudinit-example" {
gzip = false
base64_encode = false
part {
content_type = "text/x-shellscript"
content = "${data.template_file.shell-script.rendered}"
}
}
The shell script to mount the volume automatically on startup is in ${TERRAFORM_HOME}/scripts/disk.sh
It will first check if a file system exist. If true it wouldn't format the disk if not it will
#!/bin/bash
DEVICE_FS=`blkid -o value -s TYPE ${DEVICE}`
if [ "`echo -n $DEVICE_FS`" == "" ] ; then
mkfs.ext4 ${DEVICE}
fi
mkdir -p /data
echo '${DEVICE} /data ext4 defaults 0 0' >> /etc/fstab
mount /data
I hope this helps
Mounting the volume needs to be done from the guest OS itself using mount, fstab, etc.
The digital ocean docs cover this here.
Using Chef you could use resource_mount to mount it in an automated fashion.
The device name will be /dev/disk/by-id/scsi-0DO_Volume_YOUR_VOLUME_NAME. So, using the example from the Terraform docs, it would be /dev/disk/by-id/scsi-0DO_Volume_baz.
I am trying to attach the ephemeral partitions on my ec2 instances using boto.
I have tried that with AWS tools and worked:
./ec2-run-instances ami-018c9568
--instance-type m2.4xlarge
--availability-zone us-east-1a
-n 1 -k my_keypair -g sg-11111111
-b "/dev/xvdb=ephemeral0"
-b "/dev/xvdc=ephemeral1"
-b "/dev/xvdd=ephemeral2"
-b "/dev/xvde=ephemeral3"
When I try it with boto it doesn't work:
mapping = BlockDeviceMapping()
eph0 = BlockDeviceType()
eph1 = BlockDeviceType()
eph2 = BlockDeviceType()
eph3 = BlockDeviceType()
eph0.ephemeral_name = 'ephemeral0'
eph1.ephemeral_name = 'ephemeral1'
eph2.ephemeral_name = 'ephemeral2'
eph3.ephemeral_name = 'ephemeral3'
mapping['/dev/xvdb'] = eph0
mapping['/dev/xvdc'] = eph1
mapping['/dev/xvdd'] = eph2
mapping['/dev/xvde'] = eph3
It is correct. I was experimenting with instance type with a single ephemeral (I thought it had two). Hope it helps other people.