I have configured Hosted zones in Route 53 with external domaine.
I have upload and deploy app express with Elastic Beanstalk.
const express = require("express")
const cors = require('cors');
const app = express()
const PORT = process.env.PORT || 8000
connection()
app.use(cors({
origin: '*'
}));
app.get('/', (req, res) => {
res.send('Hello World')
})
app.listen(PORT, () => console.log(`Listen on port ${PORT}`))
module.exports = app
I have created AWS Certificate Manager with success.
In Elastic Beanstalk > Configuration > Load balancer > add listener :
443 | HTTPS : selected my certification
When i make request with http protocol (port 80) that work.
But when i make request with https, i have error timeout.
for information my app work in Heroku with https.
EDIT:
the problem came from Hosted zones. thank for your help
I am looking to deploy ECE (Elastic Cloud Enterprise) in AWS with Terraform. Reading through the documentation, I'm still not clear how this model works.
In the provider below, what is the reason for the endpoint? Is terraform connecting to this endpoint with the specified username and password? And are these credentials are being provided with the ECE license?
Hence, I'm thinking that the ECE installation endpoint can't be private. But I need to provision this privately - probably won't be able to do it via Terraform. Any experience with this?
provider "ec" {
# ECE installation endpoint
endpoint = "https://my.ece-environment.corp"
# If the ECE installation has a self-signed certificate
# you must set insecure to true.
insecure = true
username = "my-username"
password = "my-password"
}
data "ec_stack" "latest" {
version_regex = "latest"
region = "us-east-1"
}
resource "ec_deployment" "example_minimal" {
# Optional name.
name = "my_example_deployment"
# Mandatory fields
region = "us-east-1"
version = data.ec_stack.latest.version
deployment_template_id = "aws-io-optimized-v2"
elasticsearch {}
}
I want to use AWS Elastic-search service with my django application which is running on EC2 instance.
For that I use the settings -
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.elasticsearch5_backend.Elasticsearch5SearchEngine',
'URL': 'https://vpc-ES-CLUSTER.ap-south-1.es.amazonaws.com:9200/',
'INDEX_NAME': 'haystack',
'INCLUDE_SPELLING':True,
},
}
I am not even able to set the connection. Here I am getting this error -
raise ConnectionError('N/A', str(e), e)
elasticsearch.exceptions.ConnectionError: ConnectionError((, 'Connection to vpc-ES-CLUSTER.ap-south-1.es.amazonaws.com timed out. (connect timeout=10)')) caused by: ConnectTimeoutError((, 'Connection to vpc-ES-CLUSTER.ap-south-1.es.amazonaws.com timed out. (connect timeout=10)'))
I have updated the access policy to allow the user for edit and list, also in security group add the port 9200 TCP rule. How to connect ec2 with elastic search using VPC.
It is working on 443 port, use
'URL': 'https://vpc-ES-CLUSTER.ap-south-1.es.amazonaws.com:443/',
and in security groups add 443 open port.
I'm having some trouble with my AWS Kubernetes instance.
I'm trying to get my django instances to connect to the RDS service via the DB endpoint.
DATABASES = {
'default': {
'ENGINE': 'django.contrib.gis.db.backends.postgis',
'NAME': os.environ['NAME'],
'USER': os.environ['USER'],
'PASSWORD': os.environ['PASSWORD'],
'HOST': os.environ['HOST'],
'PORT': os.environ['PORT']
}
}
The host string would resemble this service.key.region.rds.amazonaws.com and is being passed to the container via env in the deploy.yml
containers:
- name: service
env:
- name: HOST
value: service.key.region.rds.amazonaws.com
This set up works locally in kubernetes but not when I put it in the cluster I have on AWS. It returns the following error instead:
django.db.utils.OperationalError: could not translate host name
Any suggestions or am I missing something in how AWS likes handling things?
Assuming your AWS deployment is now in the same VPC as your RDS, then you will need to change your host to use the private IP.
I am hoping to use Amazon's Elasticsearch server to power a search of longtext fields in a Django database. However, I also don't want to expose this search to those who don't have a log in and don't want to rely on security through obscurity or some IP restriction tactic (unless it would work well with an existing heroku app, where the Django app is deployed).
Haystack seems to go a long way toward this, but there doesn't seem to be an easy way to configure it to use Amazon's IAM credentials to access the Elasticsearch service. This functionality does exist in elasticsearch-py, whichi it uses.
https://elasticsearch-py.readthedocs.org/en/master/#running-with-aws-elasticsearch-service
from elasticsearch import Elasticsearch, RequestsHttpConnection
from requests_aws4auth import AWS4Auth
host = 'YOURHOST.us-east-1.es.amazonaws.com'
awsauth = AWS4Auth(YOUR_ACCESS_KEY, YOUR_SECRET_KEY, REGION, 'es')
es = Elasticsearch(
hosts=[{'host': host, 'port': 443}],
http_auth=awsauth,
use_ssl=True,
verify_certs=True,
connection_class=RequestsHttpConnection
)
print(es.info())
Regarding using HTTP authorization, I found this under issues at https://github.com/django-haystack/django-haystack/issues/1046
from urlparse import urlparse
parsed = urlparse('https://user:pass#host:port')
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
'URL': parsed.hostname,
'INDEX_NAME': 'haystack',
'KWARGS': {
'port': parsed.port,
'http_auth': (parsed.username, parsed.password),
'use_ssl': True,
}
}
}
I am wondering if there is a way to combine these two, something like the following (which, as expected, gives an error since it's more than just a user name and password):
from requests_aws4auth import AWS4Auth
awsauth = AWS4Auth([AACCESS_KEY],[SECRET_KEY],[REGION],'es')
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
'URL': [AWSHOST],
'INDEX_NAME': 'haystack',
'KWARGS': {
'port': 443,
'http_auth': awsauth,
'use_ssl': True,
'verify_certs': True
}
},
}
The error here:
TypeError at /admin/
must be convertible to a buffer, not AWS4Auth
Request Method: GET
Request URL: http://127.0.0.1:8000/admin/
Django Version: 1.7.7
Exception Type: TypeError
Exception Value:
must be convertible to a buffer, not AWS4Auth
Exception Location: /usr/lib/python2.7/base64.py in b64encode, line 53
Any ideas on how to accomplish this?
You are one step from success, add connection_class to KWARGS and everything should work as expected.
import elasticsearch
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
'URL': [AWSHOST],
'INDEX_NAME': 'haystack',
'KWARGS': {
'port': 443,
'http_auth': awsauth,
'use_ssl': True,
'verify_certs': True,
'connection_class': elasticsearch.RequestsHttpConnection,
}
},
}
AWS Identity and Access Management (IAM) allows you to manage users and user permissions for AWS services, to control which AWS resources users of AWS itself can access.
You cannot use IAM credentials to authorize users at the application level via http_auth, as it appears you are trying to do via Haystack here. They are different authentication schemes for different services. They are not compatible.
In your security use case, you have stated the need to 1) restrict access to your application, and 2) to secure the Elasticsearch service port from open access. These two requirements can be met using the following methods:
Restrict access to your application
I also don't want to expose this search to those who don't have a log in
For the front-end search app, you want to use a server level Basic access authentication (HTTP auth) configuration on the web server. This is where you want to control user login access to your app, via a standard http_auth username and password (again, not IAM). This will secure your app at the application level.
Secure the Elasticsearch service port
don't want to rely on security through obscurity or some
IP restriction tactic (unless it would work well with an existing
heroku app, where the Django app is deployed).
IP restriction is exactly what would work here, and consistent with AWS security best practices. You want to use security groups and security group rules as a firewall to control traffic for your EC2 instances.
Given a Haystack configuration of:
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
'URL': 'http://127.0.0.1:9200/',
'INDEX_NAME': 'haystack',
},
}
you will want to implement an IP restriction at the security group and/or ACL level on that IP and port 127.0.0.1, to restrict access from only your Django host or other authorize hosts. This will secure it from any unauthorized access at the service level.
In your implementation, the URL will likely resolve to a public or private IP, depending on your network architecture.