I am setting up "elasticsearch" in AWS, i am trying to use AWS linux AMI. When i run the
bin/logstash -f "/path to config file"
i get error saying:
"logstash.yml" not found try using "--path.settings"
then when i use
"--path.settings="/etc/logstash"
i again get another error.
I have been following this document of AWS
https://aws.amazon.com/elasticsearch-service/resources/articles/logstash-tutorial/
The error i get after specifying
--path.settings="/etc/logstash" :
"Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}"
I have configured file logstash_simple.conf, specifying input and output.
this is the command line input in the linux ec2 instance
/usr/share/logstash/bin/logstash -f /usr/share/logstash/logstash_simple.conf
--path.settings="/etc/logstash"
Okay i had made a mistake in the config file,
i missed providing aws accesskey and secret key, dumb me!
Related
I'm trying to configure an AMI using a script that installs the unified Cloudwatch agent on both AWS and on premise Windows machines by using static IAM credentials for both of them. As part of the script, I set the credentials statically (as a test) using
$Env:AWS_ACCESS_KEY_ID="myaccesskey"
$Env:AWS_SECRET_ACCESS_KEY="mysecretkey"
$Env:AWS_DEFAULT_REGION="us-east-1"
Once I have the AMI, I create a machine and connect to it, and then verify the credentials are there by running aws configure list
Name Value Type Location
---- ----- ---- --------
profile <not set> None None
access_key ****************C6IF env
secret_key ****************SCnC env
region us-east-1 env ['AWS_REGION', 'AWS_DEFAULT_REGION']
But when I start the agent, I get the following error in the logs.
2022-12-26T17:51:49Z I! First time setting retention for log group test-cloudwatch-agent, update map to avoid setting twice
2022-12-26T17:51:49Z E! Failed to get credential from session: NoCredentialProviders: no valid providers in chain
caused by: EnvAccessKeyNotFound: failed to find credentials in the environment.
SharedCredsLoad: failed to load profile, .
EC2RoleRequestError: no EC2 instance role found
caused by: EC2MetadataError: failed to make EC2Metadata request
I'm using the Administrator user for both the installation of the agent and then when RDPing into the machine. Is there anything I'm missing?
I've already tried adding the credentials to the .aws/credentials file and modifying the common-config.toml file to use a profile. That way it works but in my case I just want to use the environment variables.
EDIT: I tested adding the credentials in the userdata script and modify a bit how they are created and now it seems to work.
$env:aws_access_key_id = "myaccesskeyid"
$env:aws_secret_access_key = "mysecretaccesskey"
[System.Environment]::SetEnvironmentVariable('AWS_ACCESS_KEY_ID',$env:aws_access_key_id,[System.EnvironmentVariableTarget]::Machine)
[System.Environment]::SetEnvironmentVariable('AWS_SECRET_ACCESS_KEY',$env:aws_secret_access_key,[System.EnvironmentVariableTarget]::Machine)
[System.Environment]::SetEnvironmentVariable('AWS_DEFAULT_REGION','us-east-1',[System.EnvironmentVariableTarget]::Machine)
Now the problem is that I'm trying to start the agent at the end of the userdata script with the command from the documentation but it does nothing (I see in the agent logs the command but there is no error). If I RDP into the machine and launch the same command in Powershell it works fine. The command is:
& "C:\Program Files\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent-ctl.ps1" -a fetch-config -m onPrem -s -c file:"C:\ProgramData\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent.json"
I finally was able to make it work but I'm not sure of why it didn't before. I was using
$env:aws_access_key_id = "accesskeyid"
$env:aws_secret_access_key = "secretkeyid"
[System.Environment]::SetEnvironmentVariable('AWS_ACCESS_KEY_ID',$env:aws_access_key_id,[System.EnvironmentVariableTarget]::Machine)
[System.Environment]::SetEnvironmentVariable('AWS_SECRET_ACCESS_KEY',$env:aws_secret_access_key,[System.EnvironmentVariableTarget]::Machine)
[System.Environment]::SetEnvironmentVariable('AWS_DEFAULT_REGION','us-east-1',[System.EnvironmentVariableTarget]::Machine)
to set the variables but then the agent was failing to initialize. I had to add
$env:aws_default_region = "us-east-1"
so it was able to run. I couldn't find the issue before because on Windows server 2022 I don't get the logs from the execution. I had to try using Windows Server 2019 to actually see the error when launching the agent.
I still don't know why the environment variables I set in the machine scope worked once logged into the machine but not when using them as part of the userdata script.
Thank you for reading.
I successfully set up the ssh config file to loggin to the AWS.
When I try to do ssh login in my local terminal, it works well, but when I try to do using my VSCode Open-SSH extension, it always fails except the first try.
The output is like this:
[18:38:25.400] Running script with connection command: ssh -T -D 53736 -o ConnectTimeout=15 -F <config> awsserver bash
[18:38:26.521] >
> SessionManagerPlugin is not found. Please refer to SessionManager Documentation here: http://docs.aws.amazon.com/console/systems-manager/session-manager-plugin-not-found
All aws commands are well reached from my terminal environment.
Thank you in advance.
I'm not familiar with the VSCode Open-SSH extension, but appears you are getting a message from Amazon's AWS CLI as if this command was being run:
aws ssm start-session --target i-0d2a6aaaaaaaa61c5
Rather than using ssh, is your extension perhaps configured to use Amazon SSM?
I have tried deploying my jhipster application in AWS Elastic BeanStalk by uploading the war directly. When the environment is created, i am getting this error.
[Instance: i-08f7c9efd8b2c5476] Command failed on instance. Return code: 1
Output: (TRUNCATED).../util/SystemPropertyUtils.class Failed to execute
'/usr/bin/unzip -o -d /var/app/staging
/opt/elasticbeanstalk/deploy/appsource/source_bundle' Failed to execute
'/usr/bin/unzip -o -d /var/app/staging
/opt/elasticbeanstalk/deploy/appsource/source_bundle'. Hook
/opt/elasticbeanstalk/hooks/restartappserver/pre/01_configure_application.sh
failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
Please suggest me what to do next.
enter image description here
I have also tried using yo jhipster:aws command as per the documentation in the jhipster page.
What i am getting is Missing credentails Config .
my question is i have added credentials.properties file in the given loaction
~/.aws/credentials...
Means .aws/credentials/credentials.properties (file). is the file extension right and the folder structure right,.
Create S3 bucket
Error jhipster:aws
Missing credentials in config
I'm not sure about your first error as you are trying to set the environment up manually and we would need information to reproduce.
In regards to yo jhipster:aws failing, the AWS credentials file should be located at ~/.aws/credentials, not ~/.aws/credentials/credentials.properties
After that create a credentials file at ~/.aws/credentials on Mac/Linux or C:\Users\USERNAME.aws\credentials on Windows.
From the docs: https://jhipster.github.io/aws/
For further clarification, use vi and create a file named "credentials" under the "~/.aws" folder.
I'm trying to deploy my code to AWS Beanstalk and get this error. I researched that it could be that the number of versions is more than 500, so I deleted a lot of versions. But, I still get this error.
eb deploy
ERROR: No Application Version named 'v0_9_2-76-gf5a4' found.
I also tried
git aws.push
Error: Failed to create the AWS Elastic Beanstalk application version
Edit:
Trying with eb deploy --debug I now get:
Instance: i-2ad238d5 Module: AWSEBAutoScalingGroup ConfigSet: null Command failed on instance. Return code: 1 Output: Error occurred during build: Command hooks failed . Script /opt/elasticbeanstalk/hooks/appdeploy/pre/10_bundle_install.sh failed with returncode 18
ebcli.objects.exceptions.ServiceError: Update environment operation is
complete, but with errors. For more information, see troubleshooting
documentation.
Did you update the file .elasticbeanstalk/config.yml ? It may have a wrong setup.
Make a backup of .elasticbeanstalk/ folder and remove it
Execute eb create
Select the same region you deployed it before. You can check the region on .elasticbeanstalk/config.yml backup
A list with the environments will appear, select the right one
Deploy now
Remove the .elasticbeanstalk/config.yml backup
Check for the .elasticbeanstalk/config.yml file
environment: CORRECT_ENV_NAME
global:
application_name: CORRECT_APP_NAME
In my case, I was doing eb deploy X where X was an environment for a different project.
When I had the error
InvalidParameterValueError: No Application Version named 'app-9f5c-180927_071528' found.
I fixed this by specifying the label I wanted to push up.
eb deploy XXX-env -l XXX.0.0.1
The -l flag is documented AWS EB Deploy Docs
most likely, the deploy is trying an incorrect Elasticbeanstalk Application. it could be because you renamed the application in the AWS console.
so double check you're pointing to the correct elasticbeanstalk Environment and Application. it could be picking out default values from your .elasticbeanstalk/config.yml file.
I just set up a free instance of an Amazon ec2 server. I'm trying to figure out how to SSH into it. I downloaded the command line tools for ec2 and, following what was written at this page: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2_GetStarted.html#EC2_LaunchInstance_Linux :
.ec2$ ec2-get-console-output [instance id]
File not found: ''
.ec2$
Where [instance id] refers to the id amazon lists in the list of instances I have. Can anyone tell me what's going on?
Edit: I might add it seems to be doing this for any binary I try to run from the command line tools... even if I call them directly.
I had the same problem until I specified the key and cert in my .bash_profile file:
export EC2_HOME=~/.ec2
export PATH=$PATH:$EC2_HOME/bin
export EC2_PRIVATE_KEY='ls $EC2_HOME/key.pem'
export EC2_CERT='ls $EC2_HOME/cert.pem'