Hi I am trying to connect my Django application to use Redis ElastiCache and am having trouble with getting it connected using AWS. The application is published to an EC2 instance using Elastic Beanstalk and it running perfect when I am not trying to connect to my Redis cache.
From the post here (Setting up ElastiCache Redis with Elastic BeanStalk + Django) I created my ElastiCache to not use a cluster and I have set up both the EC2 instance and the Redis cache to use the same Security Group.
Here is how my cache is configured in settings.py.
CACHES = {
'default': {
'BACKEND': 'django_redis.cache.RedisCache',
'LOCATION': 'redis://my-cache.kjshd.0001.use2.cache.amazonaws.com:6379/',
'OPTIONS': {
'CLIENT_CLASS': 'django_redis.client.DefaultClient'
}
}
}
What am I missing? Are there additional settings that need changed on my cache or somewhere in AWS to open communication? Does this configuration look okay? I was previously using Redis in Azure and this configuration worked but now have the requirement to move to AWS. Is there a way to test that my EC2 instance can connect to Redis? I have the ability to SSH into the server but I was not sure what I would do once I was connected.
Thanks for any help.
After setting the security group, I found out that I needed to change the inbound settings for the security group to connect to my ElastiCache Redis node.
Documentation was found here.
https://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/GettingStarted.AuthorizeAccess.html#GettingStarted.AuthorizeAccess.VPC
Related
I have an application deployed on Cloud Run. It runs behind an HTTPS Load balancer.
I want it to be able to cache some data using memory store service. I basically followed the documentation to use a serverless vpc connector but this exception keeps poping:
Unable to create a new session key. It is likely that the cache is
unavailable.
I am guessing that my cloud run service can't access memorystore.
On Django I have:
CACHES = {
"default": {
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": f"redis://{CHANNEL_REDIS_HOST}:{CHANNEL_REDIS_PORT}/16",
"OPTIONS": {
"CLIENT_CLASS": "django_redis.client.DefaultClient",
"IGNORE_EXCEPTIONS": True,
},
"KEY_PREFIX": "api"
}
}
where CHANNEL_REDIS_HOST is the IP from my memorystore primary endpoint and CHANNEL_REDIS_PORT is the port.
When I run this command:
gcloud redis instances describe instance_name --region region --format "value(authorizedNetwork)"
it returns projects/my_project/global/networks/default.
Then, on the VPC network, I clicked on 'default' and then on 'ADD SUBNET'. I created my subnet with IP Address range 10.0.0.0/28. Maybe the problem comes from this step as I do not get a lot about this all IP Communication thing..
When I run this command:
gcloud compute networks subnets describe my subnet
purpose is PRIVATE as intended and network is https://www.googleapis.com/compute/v1/projects/my_project/global/networks/default.
So I think that my memorystore instance and my subnet are able to connect.
Then, I created a serverless VPC connector, using the same region, the default network and the subnet I just created.
Finally on my service I set the VPC connector to the one I just created and I redeploy using Only route requests to private IPs through the VPC connector option, if I choose Route all traffic through the VPC connector my deployment fails, I think probably because I am behind a load balancer, anyway I do not want to route all traffic to my connector.
And after doing this, I still receive the error mention at the beginning of the message..
Any ideas ?
Thanks
So I think my issue was using the db 16. As the maximum number of database on memorystore is 16 it must be from 0 to 15. Changing it make it works.
The question is simple,
How to install/use AWS:ElastiCache Redis session for Yii2?
Self answered. Please read below
Steps to Install and Enable Redis
[Self Managed] Install Redis-Server in your webserver, using digitalocean link or for AWS EC2
[AWS ElastiCache] AWS ElastiCache for Redis: How to Use the AWS Redis Service
Install PHP Redis extension. AWS Guide
Apply session.save_path = Endpoint ElastiCache & session.save_handler = redis in php.ini
Restart the services
Confirm the change in php.ini by phpinfo();
Apply Redis session in Yii2 Component as said here
Make sure you follow this.
Use the same security group for EC2 and ElastiCache
Add Redis Port 6379 in AWS Security Group as 0.0.0.0
While configuration In Yii2, main.php add ElastiCache endpoint as host
These steps for the session only, so modify Yii2 main component if you want to use Redis as active data or to cache
I'm deploying a Java 8 Spring Boot web app to AWS Elastic Beanstalk. I have an associated RDS MySQL instance and configured the relevant connection details.
The connection works when running the app locally, in my machine, because I set the following routing configuration for the RDS server:
As outlined, routings are also added for the security groups associated to my EC2 instances.
Therefore, running mysql on the EC2 machine works and the database can be reached.
The issue appears when deploying the app to Beanstalk, where it gets implemented into the EC2 instances. The app crashes because it gets connection refused errors when trying to connect to the MySQL RDS instance:
This doesn't seem to make any sense.
The database is accessible from both the EC2 instance (verified via the mysql command) and outside AWS, so the only remaining cause would be having misconfigured the Spring Boot app properties.
This doesn't seem to be the problem either because when running it locally, in my machine, the app has no issues connecting to the RDS instance and running normally using the production MySQL server.
I have separate application-development.properties and application-production.properties files, but I set the relevant properties to the same values:
spring.datasource.url = jdbc:mysql://XXXXXX.rds.amazonaws.com:3306/ebdb?useSSL=false&allowPublicKeyRetrieval=true&useUnicode=true&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=UTC
spring.datasource.username = XXXXXX
spring.datasource.password = XXXXXX
spring.datasource.driver-class-name = com.mysql.cj.jdbc.Driver
Any pointers as to why my app could be running locally but not when deployed to Beanstalk?
Recreating both the Beanstalk environment and RDS instance seemed to fix the issue.
I'm in the process of migrating my Heroku app database from Heroku to AWS RDS Postgres.
On my computer, I can connect to my RDS DB using:
psql -d "postgres://user:password#XXX.rds.amazonaws.com/mydb?sslrootcert=config/amazon-rds-ca-cert.pem&sslmode=require"
However, the same psql command run from within my heroku server just hangs forever.
Also, config/amazon-rds-ca-cert.pem is the RDS certificate that I added to my package as mentioned in the documentation https://devcenter.heroku.com/articles/amazon-rds#authorizing-access-to-rds-instance and here https://stackoverflow.com/a/29467638/943524 (I did combine certificates as I am using a eu-central-1 instance).
Would someone have an idea what is blocking the connection here ?
From the sound of it, your Network ACL or Security Groups are blocking your access. It looks like they allow your computer (perhaps your entire company’s IP) but not Heroku. Check out the NACLs and Security Groups and you should find your answer (i.e. add Heroku IP range to your NACLs and/or Security Groups).
What's the correct way to use Amazon's Elasticache service (with the Memcached engine) with Django's MemcachedCache backend?
I have a local Memcached service running locally, which works fine with the Django setting:
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': '127.0.0.1:11211',
}
}
I thought using Elasticache would be as simple as creating the Memcached cluster instance and then changing my setting to:
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': 'instance-name.abcdef.cfg.use1.cache.amazonaws.com:11211',
}
}
However, when I test this locally, the cache silently fails and doesn't successfully store anything.
What am I doing wrong? How do I get the MemcachedCache backend to show a real error message? Do I need to use a Elasticache-specific Django backend like this?
You're unable to connect to ElastiCache instances from outside of AWS's network. Even though your security groups might have exceptions in to allow traffic from your IP address (or the entire internet), AWS's network will not accept any traffic to it that does not originate from within their network.
This configuration is fine, however will only work from an EC2 instance.
Alternatively you can follow this guide (which also confirms my answer above) which basically involves you spinning up an EC2 instance who's IP address you will use in your CACHES configuration instead. This instance is configured to do NAT between incoming traffic on port 11211 and forward it onto your ElastiCache node. This configuration is far from ideal, and shouldn't ever be used in production though.