I'm trying to connect an ElastiCache Redis to an Express server deployed on ECS. I'm using the Official Redis package for Node.js
I get the Primary Endpoint from ElastiCache as blablabla.mccjet.ng.0001.euc1.cache.amazonaws.com:6379
In my server I try to connect like this
const { createClient } = require("redis");
const pubClient = createClient({ url: 'blablabla.mccjet.ng.0001.euc1.cache.amazonaws.com:6379' });
But when I check the ECS logs I see
/usr/src/app/node_modules/#redis/client/dist/lib/client/index.js:124
throw new TypeError('Invalid protocol');
^
TypeError: Invalid protocol
at Function.parseURL (/usr/src/app/node_modules/#redis/client/dist/lib/c...
Haven't used Redis so no idea why this is happening. Any idea how to use the endpoint properly
even tried with
const pubClient = createClient({ host: 'blablabla.mccjet.ng.0001.euc1.cache.amazonaws.com', port:6379 });
but that also didn't work
The issue was I had to add the prefix redis:// before my Primary endpoint so that it becomes redis://blablabla.mccjet.ng.0001.euc1.cache.amazonaws.com:6379
Github Issue
const pubClient = createClient({ url: 'redis://blablabla.mccjet.ng.0001.euc1.cache.amazonaws.com:6379' });
Related
I am trying to use SignalR on an application hosted in AWS EB, with an application loadbalancer in front.
It works perfectly fine when I test it locally, but when deployed, the websocket can not establish a connection and returns:
"Error: Failed to start the connection: Error: There was an error with the transport."
And
"There was an error with the transport. at WebSocket.o.onerror [as __zone_symbol__ON_PROPERTYerror]"
I have tried adding a middleware at the very start of my pipeline, that logs if i get a request for my hub, and this works, so I dont think it is the load balancer or anything AWS Related.
If i call the URL with Postman websocket, I get a status code 400.
I have followed the MS guide and added the configuration for JWT auth
jwtTokenOptions.Events = new JwtBearerEvents
{
OnMessageReceived = context =>
{
var accessToken = context.Request.Query["access_token"];
// If the request is for our hub...
var path = context.HttpContext.Request.Path;
if (!string.IsNullOrEmpty(accessToken) &&
(path.StartsWithSegments("/signalr")))
{
// Read the token out of the query string
context.Token = accessToken;
}
return Task.CompletedTask;
}
};
And as I said, it works fine locally. Any suggestions are most welcome
I fixed this by enabling web sockets for the IIS on the eb instance.
I did this by creating an eb extension with the following command
commands:
01_install_websockets_feature:
command: "%SystemRoot%\\System32\\dism.exe /online /enable-feature /featurename:IIS-WebSockets"
ignoreErrors: true
I am trying to get a .Net Core 3.1 SignalR Web API with an Angular front end to work with websockets.
The code works perfectly fine when ran locally, either from within in the IDE or via docker run. However, once the code gets deployed to an ECS instance in AWS behind an API Gateway the web sockets refuse to connect.
I setup my mappings like so:
app.UsePathBase("/ui");
app.UseEndpoints(endpoints =>
{
endpoints.MapControllerRoute(
"default",
"{controller}/{action=Index}/{id?}")
.RequireCors(PolicyName);
endpoints.MapHub<SessionHub>("/ws/session");
endpoints.MapHub<SessionsHub>("/ws/sessions");
});
And on the client I connect to the hub like so:
this.sessionsHubConnection = new HubConnectionBuilder()
.withUrl(`${window.origin}/ws/sessions`, {
skipNegotiation: true,
transport: HttpTransportType.WebSockets,
accessTokenFactory: () => this.getAccessToken()
})
.withAutomaticReconnect()
.build();
The following Fiddler trace shows the initial HTTP request to initialize the websocket connection and the error being returned by kestrel.
I tweaked my web socket middleware for handling the access token to also Console.Write some additional debugging statements that I think might prove insightful:
public async Task Invoke(HttpContext httpContext)
{
var request = httpContext.Request;
Console.WriteLine($"Starting connection id: {httpContext.Connection.Id}");
// web sockets cannot pass headers so we must take the access token from query param and
// add it to the header before authentication middleware runs
if (request.Path.StartsWithSegments("/ws", StringComparison.OrdinalIgnoreCase)
&&
request.Query.TryGetValue("access_token", out var accessToken))
{
request.Headers.Add("Authorization", $"Bearer {accessToken}");
}
try
{
var sb = new StringBuilder();
sb.AppendLine($"Connection Id: {httpContext.Connection.Id}");
Console.WriteLine(sb.ToString());
await _next(httpContext);
sb = new StringBuilder();
sb.AppendLine($"Status code {httpContext.Response.StatusCode}"); <-- this line
sb.AppendLine($"Connection Id: {httpContext.Connection.Id}"); <-- and this line
Console.WriteLine(sb.ToString());
}
catch (Exception e)
{
Console.WriteLine(e.Message);
Console.WriteLine(e.StackTrace);
throw;
}
}
And in the AWS logs you can see that the connection Id is present but is being ignored by the EndpointMiddleware(?) for some reason.
Any idea what could be causing this? Two ideas I have yet to be able to rule out are:
The HTTP/S termination at our API gateway is confusing Kestrel since the browser client is building the socket request under HTTPS, but as far as Kestrel is concerned everything is communicating over HTTP?
app.UsePathBase("/ui"); is confusing Kestrel and all the web socket paths should actually be /ui/ws/session?
I'm trying to get a Flask app to run on an EC2 instance. I have a few JS functions that send requests to the localhost in the backend to retrieve data from an API.
For instance:
if(topicQueryString != null && topicQueryString != ''){
$.ajax({
url: 'http://0.0.0.0:5000/search/t/'+topicQueryString,
type: 'GET',
dataType: 'json',
success: function(data) { //do something }
})
}
However, when deploying the app on my EC2 instance, these requests fail with ERR_CONNECTION_REFUSED on localhost:5000/search/t/topics
Is there a way to allow the EC2 instance to make requests to itself in this way?
You're trying to connect to 0.0.0.0, not localhost. The address 0.0.0.0 doesn't mean localhost. You should replace that with localhost or 127.0.0.1 (which is what localhost means, most of the time).
EDIT: actually, I'm not even sure that I understood the problem correctly. Is that JS code running in a browser? You want that code to connect to a back-end service, presumably not running on localhost? If so, you should use the address of the back-end service, rather than 0.0.0.0 (which doesn't make sense in any context as a destination address to connect to) or localhost.
Infrastructure:
cloud:
aws beanstalk
turn on nginx for container proxy server
application load balancer - https only, default process (https)
2+ instance in private subnet
enabled end to end encryption following
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/configuring-https-endtoend.html
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/https-singleinstance-docker.html
self-signed certificate on instance
instance running docker
In local, we have a 3 container to mimic the infrastructure,
1 nginx: 443 as load balancer and https reverse proxy
2 app container: 3000:3000, 3001:3001 respectively
so, not end to end encryption yet
software:
autho
passport
(https://github.com/auth0/passport-auth0)
express
react
cookie-session package
const sessionConfig = {
name: 'sessionId',
secret: uid.sync(18),
secure: true,
httpOnly: true,
secureProxy: true,
maxAge: 1800 * 1000
};
workflow:
open website, click login link, it then redirect us to auth0 login page, after input username/passport, we click submit.
We are encountering "redirect too many times" when we have more than 1 instance running. The issue goes away if I turn on sticky session on the target group in aws.
We are seeing the same when trying on the local docker environment.
In this code,
router.get('/callback', (req, res, next) => {
authenticate('auth0', (authErr, user) => {
if (authErr) {
console.error(`Error authenticating user: ${authErr}`);
return next(authErr);
}
if (!user) {
console.info(`No user data, redirecting to login page`);
return res.redirect('/login');
}
The logic always hits - if (!user), and we are not sure why this happens with multiple instance, load balancer setup.
Update:
Sorry I am new to this,
I am wondering if I can use cookie-session instead of express-session since JWT is supposed to not storing information in server.
I am asking because I have read a few tutorial of passport and Auth0, and it also mentioned about expression-session only.
Since Auth0 is using JWT, could I use cookie-session? if so, what could I do wrong?
PS.
Here is my session config:
const sessionConfig = {
name: 'sessionId',
domain: 'example.com',
secret: uid.sync(18),
secure: true,
httpOnly: true,
maxAge: 1800 * 1000
};
Please advise and help.
Thank you!
Jay
The issue is resolved.
It is because secret is random so a fixed secret was not shared between servers.
I'm setting up my first ElasticSearch app using ReactiveSearch to connect to an ElasticSearch index I created on AWS.
I'm new to Node.js and most of the technology involved here. I think I have a basic ReactiveSearch app that works but it won't connect to my AWS ElasticSearch index. When I enter a search I get no output and no errors.
I followed the ReactiveSearch Quickstart guide:
https://opensource.appbase.io/reactive-manual/getting-started/reactivesearch.html
I created the Boilerplate App with CRA:
https://github.com/facebook/create-react-app#creating-an-app
The app runs ok but there is no output when I try to search.
Then I saw the note that you have to use a proxy with AWS. I cloned https://github.com/appbaseio-apps/reactivesearch-proxy-server and got that working and I now have a proxy that runs on http://localhost:7777/
My Search App connects to the proxy like this:
<ReactiveBase
app="my-search"
url="http://localhost:7777">
This is the code that sets the target in the proxy. I commented out the authorisation because I'm not using appbase.io.
const options = {
target: 'https://search....ap-southeast-2.es.amazonaws.com',
changeOrigin: true,
onProxyReq: (proxyReq, req) => {
/* proxyReq.setHeader(
'Authorization',
`Basic ${btoa('cf7QByt5e:d2d60548-82a9-43cc-8b40-93cbbe75c34c')}`
);*/
/* transform the req body back from text */
const { body } = req;
if (body) {
if (typeof body === 'object') {
proxyReq.write(JSON.stringify(body));
} else {
proxyReq.write(body);
}
}
}
}
I can see the ReactiveSearch app on my browser at http://localhost:3000
When I type keywords into the search box I see output like this in the proxy app:
Verifying requests ✔ {"preference":"results"}
{"query":{"bool":{"must":[{"bool":{"must":[{"bool":{"should":[{"multi_match":{"query":"cables","fields":["Description"],"type":"best_fields","operator":"or","fuzziness":0}},{"multi_match":{"query":"cables","fields":["Description"],"type":"phrase_prefix","operator":"or"}}],"minimum_should_match":"1"}}]}}]}},"size":50,"_source":{"includes":["*"],"excludes":[]},"from":0}
Verifying requests ✔ {"preference":"SearchBox"}
{"query":{"bool":{"must":[{"bool":{"must":{"bool":{"should":[{"multi_match":{"query":"horse","fields":["Description"],"type":"best_fields","operator":"or","fuzziness":0}},{"multi_match":{"query":"horse","fields":["Description"],"type":"phrase_prefix","operator":"or"}}],"minimum_should_match":"1"}}}}]}},"size":20}
What am I missing to get the connection working? Do I need to add some kind of authentication in AWS and add passwords to the proxy code?
Is there a way to see some debugging info?
Thanks,
Phil