I'm trying to follow the instructions on https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/SSLNET.SingleInstance.html to install a self-signed certificate on an Elastic Beanstalk dev instance. That is, I have (in addition to other prerequisites listed there), a .ebextensions/https-instance-dotnet.config file of the form:
files:
"C:\\certs\\install-cert.ps1":
content:
Echo "I want some SSL"
import-module webadministration
## Settings - replace the following values with your own
$bucket = "my-bucket"
$certkey = "example.com.pfx"
$pwdkey = "password.txt"
[...]
commands:
00_install_ssl:
command: powershell -NoProfile -ExecutionPolicy Bypass -file C:\\certs\\install-cert.ps1 > c:\\certs\\log.txt
I'm publishing from Visual Studio to AWS EB using AWS Toolkit and not seeing errors. The file seems to be formatted correctly according to a YAML validator. But I can't tell if the script is working or not, or even being run, or if C:\certs\install-cert.ps1 and log.txt are being created in my environment.
My question is: How do I debug or trace this so I can see (for example) the results of the Echo commands?
Related
I am trying to create a couple of os policy assignments to configure - run some scripts with PowerShell - and install some security agents on a Windows VM (Windows Server 2022), by using the VM Manager. I am following the official Google documentation to setup the os policies. The VM Manager is already enabled, nevertheless I have difficulties creating the appropriate .yaml file which is required for the policy assignment since I haven't found any detailed examples.
Related topics I have found:
Google documentation offers a very simple example of installing an .msi file - Example OS policies.
An example of a fixed policy assignment in Terraform registry - google_os_config_os_policy_assignment, from where I managed to better comprehend the required structure for the .yaml file even though it is in a .json format.
Few examples provided at GCP GitHub repository (OSPolicyAssignments).
OS Policy resources in JSON representation - REST Resource, from where you can navigate to sample cases based on the selected resource.
But, it is still not very clear how to create the desired .yaml file. (ie. Copy some files, run a PowerShell script to perform an installation or an authentication). According to the Google documentation pkg, repository, exec, and file are the supported resource types.
Are there any more detailed examples I could use to understand what is needed? Have you already tried something similar?
Update: Adding an additional source.
You need to follow these steps:
Ensure that the OS Config agent is installed in your VM by running the below command in PowerShell:
PowerShell Get-Service google_osconfig_agent
you should see an output like this:
Status Name DisplayName
------ ---- -----------
Running google_osconfig... Google OSConfig Agent
if the agent is not installed, refer to this tutorial.
Set the metadata values to enable OSConfig agent with Cloud Shell command:
gcloud compute instances add-metadata $YOUR_VM_NAME \
--metadata=enable-osconfig=TRUE
Generate an OS policy and OS policy assignment yaml file. As an example, I am generating an OS policy that installs a msi file retrieved from a GCS bucket, and an OS policy assignment to run it in all Windows VMs:
# An OS policy assignment to install a Windows MSI downloaded from a Google Cloud Storage bucket
# on all VMs running Windows Server OS.
osPolicies:
- id: install-msi-policy
mode: ENFORCEMENT
resourceGroups:
- resources:
- id: install-msi
pkg:
desiredState: INSTALLED
msi:
source:
gcs:
bucket: <your_bucket_name>
object: chrome.msi
generation: 1656698823636455
instanceFilter:
inventories:
- osShortName: windows
rollout:
disruptionBudget:
fixed: 10
minWaitDuration: 300s
Note: Every file has its own generation number, you can get it with the command gsutil stat gs://<your_bucket_name>/<your_file_name>.
Apply the policies created in the previous step using Cloud Shell command:
gcloud compute os-config os-policy-assignments create $POLICY_NAME --location=$YOUR_ZONE --file=/<your-file-path>/<your_file_name.yaml> --async
Refer to the Examples of OS policy assignments for more scenarios, and check out this example of a PowerShell script.
Down below you can find the the .yaml file that worked, in my case. It copies a file, and executes a PowerShell command, so as to configure and deploy a sample agent (TrendMicro) - again this is specifically for a Windows VM.
.yaml file:
id: trendmicro-windows-policy
mode: ENFORCEMENT
resourceGroups:
- resources:
- id: copy-exe-file
file:
path: C:/Program Files/TrendMicro_Windows.ps1
state: CONTENTS_MATCH
permissions: '755'
file:
gcs:
bucket: [your_bucket_name]
generation: [your_generation_number]
object: Windows/TrendMicro/TrendMicro_Windows.ps1
- id: validate-running
exec:
validate:
interpreter: POWERSHELL
script: |
$service = Get-Service -Name 'ds_agent'
if ($service.Status -eq 'Running') {exit 100} else {exit 101}
enforce:
interpreter: POWERSHELL
script: |
Start-Process PowerShell -ArgumentList '-ExecutionPolicy Unrestricted','-File "C:\Program Files\TrendMicro_Windows.ps1"' -Verb RunAs
To elaborate a bit more, this .yaml file:
copy-exe-file: It copies the necessary installation script from GCS to a specified location on the VM. Generation number can be easily found on "VERSION HISTORY" when you select the object on GCS.
validate-running: This stage contains two different steps. On the validate it checks if the specific agent is up and running on the VM. If not, then it proceeds with the enforce step, where it executes the "TrendMicro_Windows.ps1" file with PowerShell. This .ps1 file downloads, configures and installs the agent. Note 1: This command is executed as Administrator and the full path of the file is specified. Note 2: Instead of Start-Process PowerShell a Start-Process pwsh can also be utilized. It was vital for one of my cases.
Essentially, a PowerShell command can be directly run at the enforce
step, nonetheless, I found it much easier to pass it first to a .ps1
file, and then just run this file. There are some restriction with the
.yaml file anywise.
PS: Passing osconfig-log-level - debug as a key-value pair as Metadata - directly to a VM or applied to all of them (Compute Engine > Setting - Metadata > EDIT > ADD ITEM) - provide some additional information and may help you on dealing with errors.
Im trying to pass my applications connectionstring from Secrets Manager into Elastic Beanstalk during deployment. If i remote desktop onto the server i am deploying to, i can run the following command -
aws secretsmanager get-secret-value --secret-id XXX-MY-SECRET-ID --version-stage AWSCURRENT --query=SecretString --output text
And it outputs correctly. All good.
However when i try and automate this through the .ebextensions folder, it always returns an empty string.
So ive tried lots of things, firstly using a container command to point to a Powershell file that i placed in the .ebextensions folder, like so -
container_commands:
00-myscript:
command : powershell.exe -ExecutionPolicy Bypass -Command ".\\.ebextensions\\BuildConnectionStrings.ps1"
Then the Powershell command would be
$response = aws secretsmanager get-secret-value --secret-id XXX-MY-SECRET-ID --version-stage AWSCURRENT --query=SecretString --output text
$FileName = "ConnectionStrings.config"
New-Item $FileName -ItemType File
Set-Content $FileName $response
If i run this directly in powershell, it all works as expected. But when i deploy the site, it just creates an empty file.
Ive tried various combinations of putting the Powershell commands directly in the .ebextensions config files, always with the same result.
I just cant figure out why the same commands work when i run them directly on the server, but not when deployed through EB.
Ive been stuck for days on this, can anyone help?
So for anyone else that has this problem...
Simply installing the aws cli on the server is not enough. The error i was getting was caused by PowerShell not recognising the aws command (even tho it had been installed).
The solution was to run the following code at the beginning of my script -
$command = "[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12"
Invoke-Expression $command
Invoke-WebRequest -Uri "https://awscli.amazonaws.com/AWSCLIV2.msi" -Outfile C:\AWSCLIV2.msi
$arguments = "/i `"C:\AWSCLIV2.msi`" /quiet"
Start-Process msiexec.exe -ArgumentList $arguments -Wait
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine")
aws --version
This ensured that the aws cli would be installed (again).
I dont know why this is necessary, but its the only solution ive found
In Build Step, I've added Send files or execute command over SSh -> SSH Publishers -> Exec command, I'm trying to run aws command to copy file from ec2 to s3. The same command runs fine when I execute it over the terminal, but via jenkins it simply returns:
bash: aws: command not found
The command is
cd ~/.local/bin/ && aws s3 cp /home/ec2-user/lambda_test/lambda_function.zip s3://temp-airflow-us/lambda_function.zip
Based on the comments.
The solution was to use the following command:
cd ~/.local/bin/ && ./aws s3 cp /home/ec2-user/lambda_test/lambda_function.zip s3://temp-airflow-us/lambda_function.zip
since aws is not available in PATH env variable.
command not found indicates that the aws utility is not on $PATH for the jenkins user.
To confirm, sudo su -l jenkins and then issue the command which aws - this will most likely return no results.
You have two options:
use the full path (likely /usr/local/bin/aws)
add /usr/local/bin to the jenkins user's $PATH
I need my Makefile to work in both Linux and Windows so the accepted answer is not an option for me.
I diagnosed the problem by adding the following to the top of my build script:
whoami
which aws
env|grep PATH
This returned:
root
which: no aws in (/sbin:/bin:/usr/sbin:/usr/bin)
PATH=/sbin:/bin:/usr/sbin:/usr/bin
Bizarrely, the path does not include /usr/local/bin, even though the interactive shell on the Jenkins host includes it. The fix is simple enough, create a symlink on the Jenkins host:
ln -s /usr/local/bin/aws /bin/aws
Now the aws command can be found by scripts running in Jenkins (in /bin).
AWS EB (Elastic Beanstalk) CLI not running in git bash (Windows 10). I have successfully installed the AWS EB CLI from AWS documentation at https://github.com/aws/aws-elastic-beanstalk-cli-setup/blob/master/README.md . At the end I have set the environment variables as mentioned in the doc. So "eb" command is working from Windows Power shell. But when I am trying to access the "eb" command from GIT Bash / IntelliJ bash prompt, it is not working.
Working fine with windows power shell:
PS C:\> eb --version
EB CLI 3.19.2 (Python 3.7.3)
Environment variable set as below under "User Variable" -> "Path":
Environment variable set windows
While trying to access the "eb" from Git Bash the error is as below:
$ eb
bash: eb: command not found
$ echo $PATH
.....
......
/c/Users/xxxxxx/.ebcli-virtual-env/executables:
Restarted the system and commandline interfaces multiple time.
Can someone please let me know if there are some issue with environment variable set, or need to configure something additional in bash environment?
After so many trial and error with different solution available in internet along with AWS doc suggestion, finally I can use "eb" from Git bash of windows 10. The problem fixed after I put the below location in my environment variable path:
C:\Users\XXXX\AppData\Roaming\Python\Python37\Scripts
The issue for me was a username with a space. The path would then look like this: C:\Users\fname lastname.ebcli-virtual-env\executables. The problem came about with the .bat files created by the AWS script did not wrap the path in double quotes. Windows then interprets it as multiple parameters.
I had to go edit eb.bat and path_exporter.bat and wrap the directives like this: (in eb.bat) CALL "C:\Users\fname lastname.ebcli-virtual-env\Scripts\activate.bat"
#start CALL "C:\Users\fname lastname.ebcli-virtual-env\Scripts\eb.exe" %args%
The EB cli seems to work properly now.
I'm customising the elastic beanstalk host machine by executing some shell commands.
I have this working fine when the commands are embedded in an ebextension config script. But I want to write my customisation stuff in normal shell script files, instead of being embedded in YAML syntax that my editor doesn't understand.
So this works:
.ebextensions/01_do_stuff.config:
commands:
01echostuff:
ignoreErrors: false
command: |
echo "Do a bunch of elaborate stuff"
But I'd rather break my customisation stuff out into a proper shell script and then just launch it with an ebextension config command.
.ebextensions/myscripts/do_stuff.sh
echo "Do a bunch of elaborate stuff"
.ebextensions/01_do_stuff.config:
commands:
01echostuff:
ignoreErrors: false
command: myscripts/do_stuff.sh
Both files (the shell and config) are packaged up in the beanstalk archive file and uploaded as part of the zip archive, I just want beanstalk to deploy my scripts along with the config files from the EB archive and allow me to execute them.
There is a difference between running commands and container_commands
You will want to trigger your shell script as container_commands which will then be able to access scripts within your application version (zip file)