I POST the following data to https://api.digitalocean.com/v2/droplets
{
"name":"1470293222",
"image":"ubuntu-16-04-x64",
"size":"512mb",
"user_data":"#!/bin/bash\ncurl http://www.myserver.com",
"region":"nyc1"
}
This should create a new droplet and run the script in user_data, but not matter what I do, I can't seem to get the script to run.
Strangely if I launch a Droplet from the DigitalOcean console, which appears NOT to use the REST API, then the userdata script appears to work OK
Does anyone have any ideas on how to make a DigitalOcean boot script work?
What response are you receiving when you send that request? Based on the DigitalOcean API documentation for creating a droplet, it appears that your size field is incorrect. Instead of using 512mb, try something like s-1vcpu-1gb.
Related
I am fairly new to AWS and would like your suggestions. The problem I would like to solve is that I want to automate the process. I have this ec2 image running ubuntu and I want to call this executable "executable_hello_world_repeat" inside the image which prints "Hello World" every second. and when calling the executable I want to add input parameters such as "executable_hello_world_repeat -n10" this would print "hello world" 10 times.
Manually I can do the following:
go to AWS management console and choose the ec2 image to start
check if the instance is running successfully
from the terminal call "executable_hello_world_repeat -n10"
it prints the "Hello World"
I want to write a program to do them all programatically. Eventually I will have a web page in React/JS and automate this process.
Thanks for reading.
When an Amazon EC2 instance is first launched, a User Data script can be provided, which is automatically executed as the root user towards the end of the boot process. You can use this script to install software, configure settings, start process, etc.
Please note that this script only runs on the first boot, because the software does not need to be installed on subsequent boots.
If you want a script to run on every boot, put it in the /var/lib/cloud/scripts/per-boot/ directory.
If you later want to trigger a script to run, then you will need some mechanism that receives this request and runs the script. A few ways you could do this are:
Run a web server on the instance and the request comes via an HTTP / REST request, or
Trigger the AWS Systems Manager Run Command that will cause a script to be run on the instance, or even multiple instances, or
Have a program or script running on the instance that is continuously polling an Amazon SQS queue. When a message is received from the queue, trigger a program/script to process the message. This is known as a "Worker" that pulls from the Queue
The EC2 instance is basically just a normal Linux instance, so you'll need to somehow get something to trigger on the instance when desired.
I'm trying to deploy apps using with cf cli commands with jenkins, and have some weird issue now.
It works fine with 1 or 2 concurrent deployments, but if there are more than 3-4 jobs running, any cf cli command returns strange errors randomly like;
No space targeted, use 'cf target -s' to target a space.
or
Server error, status code: 404, error code: 100004, message: The app
could not be found: 0da4xxxx-9476-473a-b77d-f02xxxxxx
However, there is no issue cf cli command itself if I run each cf command one by one.
(I'm only assigned to 1 org, 1 space, so no issue to choose space/target, and app is there if I do 'cf a' later.)
I fixed the config.json issue by this comment, but still blocked by strange behavior of cf cli. Any idea?
https://stackoverflow.com/a/35247160/5862540
The cf CLI stores your configured API endpoint and access & refresh tokens in a local file, $CF_HOME/config.json.
Most cf CLI commands read this file when you invoke them, and many commands write to the file when they finish. Writing is performed for two reasons: when your access token expires, the cf CLI automatically requests a new token from UAA and updates the one in config.json. Also, we simply don't have any logic to check if any updates were made that need persisting, so the file gets written out again just in case.
So it's important to configure a different CF_HOME for any parallel executions of cf CLI commands to avoid random errors. And when your config.json is corrupted, just delete the file and configure your API endpoint & login again.
I have two python projects running locally:
A cloud endpoints python project using the latest App Engine version.
A client project which consumes the endpoint functions using the latest google-api-python-client (v 1.5.1).
Everything was fine until I renamed one endpoint's function from:
#endpoints.method(MyRequest, MyResponse, path = "save_ocupation", http_method='POST', name = "save_ocupation")
def save_ocupation(self, request):
[code here]
To:
#endpoints.method(MyRequest, MyResponse, path = "save_occupation", http_method='POST', name = "save_occupation")
def save_occupation(self, request):
[code here]
Looking at the local console (http://localhost:8080/_ah/api/explorer) I see the correct function name.
However, by executing the client project that invokes the endpoint, it keeps saying that the new endpoint function does not exist. I verified this using the ipython shell: The dynamically-generated python code for invoking the Resource has the old function name despite restarting both the server and client dozens of times.
How can I force the api client to get always the latest endpoint api document?
Help is appreciated.
Just after posting the question, I resumed my Ubuntu PC and started Eclipse and the python projects from scratch and now everything works as expected. This sounds like a kind of a http client cache, or a stale python process, which prevented from getting the latest discovery document and generating the corresponding resource code.
This is odd as I have tested running these projects outside and inside Eclipse without success. But I prefer documenting this just in case someone else has this issue.
Scenario: I'm uploading an image file to the server, when trying to access it, using curl, I'm getting it on every 2nd request.
e.g curl http://staging.muserver.com/system/assets/images/000/000/test.png
When i am getting 200ok, i can see that i am having ETag, when it fails i can see X-Request-Id and X-Runtime.
this error happen only on the amazon pre production, on local machine i cannot reproduce it.
Frederick was right...i didn't setup LB...but engineyard supplied by default, when disabled, everything works fine.
I have encountered an issue when creating custom AMIs (images) on EC2 instances. If I start up a Windows default 2012 server instance with a custom bootstrap/user-data script such as;
<powershell>
PowerShell "(New-Object System.Net.WebClient).DownloadFile('http://download.microsoft.com/download/3/2/2/3224B87F-CFA0-4E70-BDA3-3DE650EFEBA5/vcredist_x64.exe','C:\vcredist_x64.exe')"
</powershell>
It will work as intended and go to the URL and download the file, and store it on the C: Drive.
But if I setup a Windows Server Instance, then create a image from it, and store it as a Custom AMI, then deploy it with the exact same custom user-data script it will not work. But if I go to the instance url (http://169.254.169.254/latest/user-data) it will show the script has imported successfully but has not been executed.
After checking the error logs I have noticed this on a regular occasion:
Failed to fetch instance metadata http://169.254.169.254/latest/user-data with exception The remote server returned an error: (404) Not Found.
Update 4/15/2017: For EC2Launch and Windows Server 2016 AMIs
Per AWS documentation for EC2Launch, Windows Server 2016 users can continue using the persist tags introduced in EC2Config 2.1.10:
For EC2Config version 2.1.10 and later, or for EC2Launch, you can use
true in the user data to enable the plug-in after
user data execution.
User data example:
<powershell>
insert script here
</powershell>
<persist>true</persist>
For subsequent boots:
Windows Server 2016 users must additionally enable configure and enable EC2Launch instead of EC2Config. EC2Config was deprecated on Windows Server 2016 AMIs in favor of EC2Launch.
Run the following powershell to schedule a Windows Task that will run the user data on next boot:
C:\ProgramData\Amazon\EC2-Windows\Launch\Scripts\InitializeInstance.ps1 –Schedule
By design, this task is disabled after it is run for the first time. However, using the persist tag causes Invoke-UserData to schedule a separate task via Register-FunctionScheduler, to persist your user data on subsequent boots. You can see this for yourself at C:\ProgramData\Amazon\EC2-Windows\Launch\Module\Scripts\Invoke-Userdata.ps1.
Further troubleshooting:
If you're having additional issues with your user data scripts, you can find the user data execution logs at C:\ProgramData\Amazon\EC2-Windows\Launch\Log\UserdataExecution.log for instances sourced from the WS 2016 base AMI.
Original Answer: For EC2Config and older versions of Windows Server
User data execution is automatically disabled after the initial boot. When you created your image, it is probable that execution had already been disabled. This is configurable manually within C:\Program Files\Amazon\Ec2ConfigService\Settings\Config.xml.
The documentation for "Configuring a Windows Instance Using the EC2Config Service" suggests several options:
Programmatically create a scheduled task to run at system start using schtasks.exe /Create, and point the scheduled task to the user data script (or another script) at C:\Program Files\Amazon\Ec2ConfigServer\Scripts\UserScript.ps1.
Programmatically enable the user data plug-in in Config.xml.
Example, from the documentation:
<powershell>
$EC2SettingsFile="C:\Program Files\Amazon\Ec2ConfigService\Settings\Config.xml"
$xml = [xml](get-content $EC2SettingsFile)
$xmlElement = $xml.get_DocumentElement()
$xmlElementToModify = $xmlElement.Plugins
foreach ($element in $xmlElementToModify.Plugin)
{
if ($element.name -eq "Ec2SetPassword")
{
$element.State="Enabled"
}
elseif ($element.name -eq "Ec2HandleUserData")
{
$element.State="Enabled"
}
}
$xml.Save($EC2SettingsFile)
</powershell>
Starting with EC2Config version 2.1.10, you can use <persist>true</persist> to enable the plug-in after user data execution.
Example, from the documentation:
<powershell>
insert script here
</powershell>
<persist>true</persist>
Another solution that worked for me is to run Sysprep with EC2Launch.
The issue is that AWS doesn't reestablish the route to the profile service (169.254.169.254) in your custom AMI. See response by SanjitPatel in this post. So when I tried to use my custom AMI to create spot requests, my new instances were failing to find user data.
Shutting down with Sysprep, essentially forces AWS re-do all setup work on the instance, as if it were run for the first time. So when you create your instance, shut it down with Sysprep and then create your custom AMI, AWS will setup the profile service route correctly for the new instances and execute your user data. This also avoids manually changing Windows Tasks and executing user data on subsequent boots, as persist tag does.
Here is a quick step-by-step:
Create an instance using one of the AWS Windows AMIs (Windows Server 2016 Nano Server doesn't support Sysprep) and passing your desired user data (this may be optional, but good to make sure AWS wires setup scripts correctly to handle user data).
Customize your instance as needed.
Shut down your instance with Sysprep. Just open EC2LaunchSettings application and click "Shutdown with Sysprep". Full instructions here.
Create your custom AMI from the instance you just shut down.
Use your custom AMI to create other instances, passing user data on instance creation. User data will be executed on instance launch. In my case, I used Spot Request screen, which had a User Data text box.
Hope this helps!
At the end of initial bootstrap (UserData) script, just append persist tag as shown below.
Works perfectly.
<powershell>
insert script here
</powershell>
<persist>true</persist>
For those people that got here from Google and are running a Server 2016 instance, it seems that this is no longer possible.
Server2016 doesn't have ec2config service and so you can't use the persist flag.
<persist>true</persist>
Described in Anthony Neace's post.
Server 2016 uses EC2Launch and I haven't yet seen how it's possible to run a script at every boot. You can run a script on the first boot, but subsequent boots will not run it.
I added below powershell script to run during the AMI bake process which helped me fix this issue. This was Windows server 2019.
$EC2LaunchInitInstance = "C:\ProgramData\Amazon\EC2-Windows\Launch\Scripts\InitializeInstance.ps1"
$EC2LaunchSysprep = "C:\ProgramData\Amazon\EC2-Windows\Launch\Scripts\SysprepInstance.ps1"
Invoke-Expression -Command "$EC2LaunchInitInstance -Schedule"
Invoke-Expression -Command "$EC2LaunchSysprep -NoShutdown"