I'm setting up pritunl and I want to use Amazon Document DB instead Mongo DB or Mongo DB Atlas. This is for a vpn idea that I had, server is running ubuntu 18.04 and followed the standard install guide from pritunl, I have it working correctly with MongoDB but for DR and scaling purposes I owuld like to get this into the AWS Document DB.
I have modified the /etc/pritunl.conf file and replaced the default local mongodb_uri with the document DR URI. restarted the service and I would exspect to see the pritunl login/setup page there. I have opened the retired ports on the box and run thought the pritunl setup page and it just hangs
"mongodb_uri": null,
"log_path": "/var/log/pritunl.log",
"static_cache": true,
"temp_path": "/tmp/pritunl_de511dc19aaf497dbdf67df0a0634e3d",
"bind_addr": "0.0.0.0",
"www_path": "/usr/share/pritunl/www",
"local_address_interface": "auto",
"port": 443
"mongodb_uri": mongodb://user:<***************>#vpn-pritunlleu-west-1.docdb.amazonaws.com:27017/test,
"log_path": "/var/log/pritunl.log",
"static_cache": true,
"temp_path": "/tmp/pritunl_de511dc19aaf497dbdf67df0a0634e3d",
"bind_addr": "0.0.0.0",
"www_path": "/usr/share/pritunl/www",
"local_address_interface": "auto",
"port": 443
turns out that DocumentDB does not support tailable cursors or capped collections which is a fundamental requirement for pritunl so going to used MongoDB Atlas or something like that.
A hanging connection is typically a result of the client not being able to connect to the DocumentDB cluster because the cluster's security group does not allow inbound connections on 27017 or the client is in a different VPC as the cluster.
For troubleshooting, please see: https://docs.aws.amazon.com/documentdb/latest/developerguide/troubleshooting.html#troubleshooting.cannot-connect
Related
I am trying to create a AWS RDS Sql Server database and connect to it from a local machine using SSMS. Later I'll be connecting from a web application (locally, then hosted somewhere eventually.) I am currently failing to connect to my instance (the instance is configured and running.) The error I'm getting is the network/instance related (not login.) Tried telnet and I can't even hit it that way.
Looking on the web, there seems to be a setup for network connections but it talks about EC2, VPC and things I don't think I need (or do I?)
Tried (nothing worked so far): Using the IP instead of hostname, explicitly specifying the port (1433), changing user/password, crying.
Speaking of things I hope I don't need to configure, there's also IAM authentication - didn't touch that yet.
Any input is appreciated before I open a ticket with Amazon.
UPDATE:
My scenario: Scenario
Solution - add the Inbound Rule to default Security Group: Security Groups
When you work with RDS, you need to set inbound rules; otherwise, you are unable to connect to the database. This concept is covered in this AWS tutorial. In this AWS tutorial, the database is MySQL and the app is a Java web app. However, the same concepts apply with respect to inbound rules:
Creating the Amazon Relational Database Service item tracker
One tip -- when you set an inbound rule to let your development machine connect, you can select MyIP...
Also - when you host your app (for example Elastic Beanstalk), you need to set an inbound rule for that as well (as discussed in that tutorial)
I am trying to connect to Neptune DB in AWS Instance from my local machine in office, like connecting to RDS from office. Is it possible to connect Neptune db from local machine? Is Neptune db publicly available? Is there any way a developer can connect Neptune db from office?
Neptune does not support public endpoints (endpoints that are accessible from outside the VPC). However, there are few architectural options using which you can access your Neptune instance outside your VPC. All of them have the same theme: setup a proxy (EC2 machine, or ALB, or something similar, or a combination of these) that resides inside your VPC, and make that proxy accessible from outside your VPC.
It seems like you want to talk to your instance purely for development purposes. The easiest option for that would be to spin up an ALB, and create a target group that points to your instance's IP.
Brief Steps (These are intentionally not in detail, please refer to AWS Docs for detailed instructions):
dig +short <your cluster endpoint>
This would give you the current master's IP address.
Create an ALB (See AWS Docs on how to do this).
Make your ALB's target group point to the IP Address obtained for step #1. By the end of this step, you should have an ALB listening on PORT-A, that would forward requests to IP:PORT, where IP is your database IP (from Step 1) and PORT is your database port (default is 8182).
Create a security group that allows inbound traffic from everywhere. i.e. Inbound TCP rule for 0.0.0.0 on PORT-A.
Attach the security group to your ALB
Now from your developer boxes, you can connect to your ALB endpoint at PORT-A, which would internally forward the request to your Neptune instance.
Do checkout ALB docs for details around how you can create it and the concepts around it. If you need me to elaborate any of the steps, feel free to ask.
NOTE: This is not a recommended solution for a production setup. IP's used by Neptune instances are bound to change with failovers and host replacements. Use this solution only for testing purposes. If you want a similar setup for production, feel free to ask a question and we can discuss options.
As already mentioned you can't access directly outside your VPC.
The following link describes another solution using a SSH tunnel: connecting-to-aws-neptune-from-local-environment.
I find it much easier for testing and development purpose.
You can create the SSH tunnel with Putty as well.
Reference: https://github.com/M-Thirumal/aws-cloud-tutorial/blob/main/neptune/connect_from_local.md
Connect to AWS Neptune from the local system
There are many ways to connect to Amazon Neptune from outside of the VPC, such as setting up a load balancer or VPC peering.
Amazon Neptune DB clusters can only be created in an Amazon Virtual Private Cloud (VPC). One way to connect to Amazon Neptune from outside of the VPC is to set up an Amazon EC2 instance as a proxy server within the same VPC. With this approach, you will also want to set up an SSH tunnel to securely forward traffic to the VPC.
Part 1: Set up a EC2 proxy server.
Launch an Amazon EC2 instance located in the same region as your Neptune cluster. In terms of configuration, Ubuntu can be used. Since this is a proxy server, you can choose the lowest resource settings.
Make sure the EC2 instance is in the same VPC group as your Neptune cluster. To find the VPC group for your Neptune cluster, check the console under Neptune > Subnet groups. The instance's security group needs to be able to send and receive on port 22 for SSH and port 8182 for Neptune. See below for an example security group setup.
Lastly, make sure you save the key-pair file (.pem) and note the directory for use in the next step.
Part 2: Set up an SSH tunnel.
This step can vary depending on if you are running Windows or MacOS.
Modify your hosts file to map localhost to your Neptune endpoint.
Windows: Open the hosts file as an Administrator (C:\Windows\System32\drivers\etc\hosts)
MacOS: Open Terminal and type in the command: sudo nano /etc/hosts
Add the following line to the hosts file, replacing the text with your Neptune endpoint address.
127.0.0.1 localhost YourNeptuneEndpoint
Open Command Prompt as an Administrator for Windows or Terminal for MacOS and run the following command. For Windows, you may need to run SSH from C:\Users\YourUsername\
ssh -i path/to/keypairfilename.pem ec2-user#yourec2instanceendpoint -N -L 8182:YourNeptuneEndpoint:8182
The -N flag is set to prevent an interactive bash session with EC2 and to forward ports only. An initial successful connection will ask you if you want to continue connecting? Type yes and enter.
To test the success of your local graph-notebook connection to Amazon Neptune, open a browser and navigate to:
https://YourNeptuneEndpoint:8182/status
You should see a report, similar to the one below, indicating the status and details of your specific cluster:
{
"status": "healthy",
"startTime": "Wed Nov 04 23:24:44 UTC 2020",
"dbEngineVersion": "1.0.3.0.R1",
"role": "writer",
"gremlin": {
"version": "tinkerpop-3.4.3"
},
"sparql": {
"version": "sparql-1.1"
},
"labMode": {
"ObjectIndex": "disabled",
"DFEQueryEngine": "disabled",
"ReadWriteConflictDetection": "enabled"
}
}
Close Connection
When you're ready to close the connection, use Ctrl+D to exit.
Hi you can connect NeptuneDB by using gremlin console at your local machine.
USE THIS LINK to setup your local gremlin server, it works for me gremlin 3.3.2 version
Only you have to update the remote.yaml as per your url and port
I am trying to run NiFi on an AWS machine and access the web GUI on my local computer.
I have followed guides such as: https://community.hortonworks.com/articles/47778/hdf-installation-on-ec2.html but whenever I type in the DNS:8080/nifi into my web browser I get a "connection refused" or timed out message.
I have created an AWS Red Hat machine, installed NiFi + java, and edited the nifi.properties file such that it is now:
# Site to Site properties
nifi.remote.input.host=ec2-34-224-216-146.compute-1.amazonaws.com
nifi.remote.input.secure=false
nifi.remote.input.socket.port=
I have tried leaving the port number blank, as well as other numbers such as: nifi.remote.input.socket.port=8082
but neither work when I enter
ec2-34-224-216-146.compute-1.amazonaws.com:8080/nifi into my browser.
I have also tried adding the domain to my local computer's /etc/hosts file in the form of the Public DNS as well as IPv4. I have also configured the security group on AWS such that I have a "Custom TCP Rule" with the port range 8081, 8082, etc. for the respective ports I have attempted.
I am not sure what I am doing wrong or if I am missing a step. Any help is appreciated.
The properties you are configuring are for site-to-site connections and are not related to the UI. These would be used if another NiFi or MiNiFi was making a site-to-site connection to your NiFi instance.
To control the UI you should be configuring:
nifi.web.http.host=
nifi.web.http.port=8080
nifi.web.https.host=
nifi.web.https.port=
I didn't find such guide or articles how to do it for ElasticSearch hosted on Windows server.
I have the EC2 amazon windows instance which running ElasticSearch server on port 9200, but I can't achieve it by _ec2_ip_adress:9200 outside the server.
I completely sure that all TCP ports are opened in amazon security group rules, I've turned off the firewall on the server as well.
So that is the problem in ElasticSearch configs.
Can someone help me with that?
Well but you know that then any body would be able to delete/create stuff in your index until you have shield.
If you really want to open it, also make sure that in windows firewall you opened port 9200.
So what i would do i would probably restrict in firewall on in Amazon access to this port for specific IPs (Actually in my project i am doing that :) )
There is one more thing to check on which IP is runned as soon as i remember ES will run on private IP. Look to network.host default is __local__. Try network.host: 0.0.0.0
I have set up a a micro EC2 instance on AWS. Currently, I am using the free tier in Oregon. There are two problems which I am facing.
When I try to SSH the instance using the public DNS, it says host does not exist but when I try conencting it using the public IP, it connects to it. What setting is needed to use the public DNS ?
I have opened the SSH client using the IP address. I want to set up my application which needs Node.js and MongoDB. I installed Node.js using this
Next I installed MongoDB using this
Then I connected to my instance using Filezilla and uploaded my code to it. I then start my node application which uses socket.io.
When I try to connect to socket.io server using web browser, I get a message which says connection refused "error 111". I have opened TCP port 80 in instance's security groups. In iptables, I have forwarded port 80 to 8080, but still it does not work. I have also checked that the firewall is disabled in ec2. Kindly help me to resolve this issue.
Did you check if all of the necessary ports are open on Amazon Security Policy?
What you can do is to allow all traffic on Amazon Security Policy for test and see if the connection goes well or not.
You might also check if you need access DB from outside. In that case, you also have to open the mongodb port and setup mongodb correctly as well.
Other tools that might useful to test firewall and connection issue will be tcpdump and syslog file
For the dns issue, did you try to nslookup on that name and see if the IP shown matches your server IP?
As Amazon gives a long DNS hostname for the server, I always use my own domain name. It's much easier.
example : ec2.domainname.com, which points to the Amazon IP address
Hope that help.
My problem is resolved now..
For the DNS issue, earlier I needed proxy to access internet, so I guess the DNS name was not getting resolved. When I tried using proxy free internet, I was able to ssh using public DNS.
And regarding connection to socket.io, I used port 8080 instead of 80 and used "sudo node main.js" to run my node file. Now I am able to connect to the socket.io server and MongoDB.
Another thing which I want to ask is that would running the node file with sudo rights create some security issue ?
Thanks for the answer! That also worked for me. I had the same problem trying to connect through sockets (http://myipaddress:3000) to a node.js server, i tried opening ports on the actual ec2 instance and disabling the firewall through SSH but nothing worked. Had to go to Security Groups on the ec2 console and open a new inbound tcp rule enabling that port