I want to run a python script, (which I have run on a docker ubuntu installation) on AWS. It sends data to from Twitter to Elastic Search. I want to run it on Amazon Elasticsearch Service. I have set up Amazon Elasticsearch Service on AWS but I don't know how to get the script into the system and get it running.
What would the ssh be to access the Elastic Search Server?
Once I am able to access it where would I place a python script in order to feed data into the Elasticsearch server?
I tried
PS C:\Users\hpBill> ssh root#search-wbcelastic-*******.us-east-1.es.amazonaws.com/
but just get this:
ssh.exe": search-wbcelastic-**********.us-east-1.es.amazonaws.com/: no address associated with name
I have this information
Domain status
Active
Endpoint
search-wbcelastic-*********.us-east-1.es.amazonaws.com
Domain ARN
arn:aws:es:us-e******1:domain/wbcelastic
Kibana
search-wbcelastic-********.us-east-1.es.amazonaws.com/_plugin/kibana/
You cannot SSH directly into AWS Cloud Search. So that SSH command will never work. You have two option to run the Python script either launch a EC2 instance with AWS CLI or store and run the script from your local machine with AWS CLI. Here is the developer guide for the AWS CLI for Cloud Search
http://docs.aws.amazon.com/cloudsearch/latest/developerguide/using-cloudsearch-command-line-tools.html
Related
I am trying to connect to an EC2 instance from Jenkins via SSH. I always get failure in the end. I am storing the SSH key in a global credential.
This is the task and shell, using SSH agent plugin
This is how I store the key (the whole key has been pasted in)
If I am using SSH connection from my local PC, everything is fine. I am a newbie in Jenkins so this is very chaotic for me.
you need to use SSH plugin . download the plugin using Manage Jenkins and configure
the ec2 in SSH remote.
follow the steps in this link
https://www.thesunflowerlab.com/blog/jenkins-aws-ec2-instance-ssh/
So, I have created an EC2 instance and I'm running a service on port 8088. For now, I'm using SSH to connect to it, with the following command :
ssh -i "~/Downloads/my_key.pem" -L 8087:127.0.0.1:8088 ec2-user#x.xxx.xx.xx
After starting the service on port 8088, I run localhost:8087 on my browser to interact with the service.
However, I want this service to be available to everyone having access to this AWS account. Basically, one shouldn't have to SSH to this and then be able to see it on their browser. I intend to create a URL for accessing this service. How do I achieve this?
I use spark to compute parallelise tasks. In order to do it, my project is connected to a server that produces some data I need to start my spark job.
Now I would like to migrate my project to the cloud on aws.
I got my spark app on EMR and my server on EC2. How can I make my EMR spark app able to use http request on my EC2 server? Do I need something like a gateway?
Thanks,
Have a nice day.
Your EMR cluster actually runs on EC2 servers. You can always ssh to those servers. And then surely you can ssh to another ec2 server from emr ec2 server
According to my experience, you should use ssh hadoop#ec2-###-##-##-###.compute-1.amazonaws.com -i /path/mykeypair.pem instead of ssh -i /path/mykeypair.pem -ND 8157 hadoop#ec2-###-##-##-###-.compute.amazonaws.com. The second command has no response.
I created one ELB and attached a few instances to this ELB. So when I login into one of these instance, I would like to type a command or run a nodejs script that can return me the its ELB name, is it possible? I know I can look up on AWS console but I'm looking for a way to look it up programmatically. If possible, I would like to see how it is done in AWS Nodejs SDK
You do not run nodejs on an elb instance. elb is proxy to load balance client requests to your app server where you run nodejs.
You could use the aws command line tools (http://aws.amazon.com/cli/):
aws elb describe-load-balancers
Parse the JSON output for the instance ID (which you can get using this answer: Find out the instance id from within an ec2 machine) and look for whatever ELB it's attached too.
I've deployed a Node.js application using AWS Elastic Beanstalk. How do I view logs (like the output from console.logs) in real-time for the server? I'm able to SSH into the Linux instance, but am not sure what I need to do to view server logs from there if that's what's required. I know I can view the logs on the Elastic Beanstalk dashboard, but these aren't in real time and take a long time to load.
Thanks for any help!
try $tail -f /var/log/* It gives all the logs. You can further filter it . $cd /var/log and check for the folders present. If you have used apache then use $tail -f /var/log/apache2/*