How to connect to ElasticSearch on AWS via reactivesearch-proxy-server - amazon-web-services

I'm setting up my first ElasticSearch app using ReactiveSearch to connect to an ElasticSearch index I created on AWS.
I'm new to Node.js and most of the technology involved here. I think I have a basic ReactiveSearch app that works but it won't connect to my AWS ElasticSearch index. When I enter a search I get no output and no errors.
I followed the ReactiveSearch Quickstart guide:
https://opensource.appbase.io/reactive-manual/getting-started/reactivesearch.html
I created the Boilerplate App with CRA:
https://github.com/facebook/create-react-app#creating-an-app
The app runs ok but there is no output when I try to search.
Then I saw the note that you have to use a proxy with AWS. I cloned https://github.com/appbaseio-apps/reactivesearch-proxy-server and got that working and I now have a proxy that runs on http://localhost:7777/
My Search App connects to the proxy like this:
<ReactiveBase
app="my-search"
url="http://localhost:7777">
This is the code that sets the target in the proxy. I commented out the authorisation because I'm not using appbase.io.
const options = {
target: 'https://search....ap-southeast-2.es.amazonaws.com',
changeOrigin: true,
onProxyReq: (proxyReq, req) => {
/* proxyReq.setHeader(
'Authorization',
`Basic ${btoa('cf7QByt5e:d2d60548-82a9-43cc-8b40-93cbbe75c34c')}`
);*/
/* transform the req body back from text */
const { body } = req;
if (body) {
if (typeof body === 'object') {
proxyReq.write(JSON.stringify(body));
} else {
proxyReq.write(body);
}
}
}
}
I can see the ReactiveSearch app on my browser at http://localhost:3000
When I type keywords into the search box I see output like this in the proxy app:
Verifying requests ✔ {"preference":"results"}
{"query":{"bool":{"must":[{"bool":{"must":[{"bool":{"should":[{"multi_match":{"query":"cables","fields":["Description"],"type":"best_fields","operator":"or","fuzziness":0}},{"multi_match":{"query":"cables","fields":["Description"],"type":"phrase_prefix","operator":"or"}}],"minimum_should_match":"1"}}]}}]}},"size":50,"_source":{"includes":["*"],"excludes":[]},"from":0}
Verifying requests ✔ {"preference":"SearchBox"}
{"query":{"bool":{"must":[{"bool":{"must":{"bool":{"should":[{"multi_match":{"query":"horse","fields":["Description"],"type":"best_fields","operator":"or","fuzziness":0}},{"multi_match":{"query":"horse","fields":["Description"],"type":"phrase_prefix","operator":"or"}}],"minimum_should_match":"1"}}}}]}},"size":20}
What am I missing to get the connection working? Do I need to add some kind of authentication in AWS and add passwords to the proxy code?
Is there a way to see some debugging info?
Thanks,
Phil

Related

How to troubleshoot zoom Oauth2 integration with ngrok and cookie usage in a MERN stack application?

I'm testing a local zoom app build. To be specific, zoom docs differentiate their app types, and what i want is a web view opened in the zoom client app view, therefore what ive developed is technically referred to as a "Zoom App" (more info here)
In the zoom docs, it mentions you cant setup the redirect urls to be localhost:8080 for example, it has to be set up with ngrok or a public url (more info here)
So ngrok is properly working (setup with cli command ngrok http 8080 ). I also tried this command with other values set for the --host-header flag. some attempts include --host=header=rewrite, --host-header=localhost, and --host-header=localhost:8080
Express server on :8080, react client on :3000
Express is linked into multiple oauth providers, google and zoom are 2 examples. google integration is working properly with the same oauth middleware, and route controllers on the api side (but google integration doesnt require the ngrok setup)
with zoom, and the ngrok setup, the request to the /callback route once the user confirms the zoom authorization, everything works fine, except the cookie that is returned by the server setting the header set-cookie is not set into the browsers application storage. nothing is registered in the cookies tab for oauth that goes through ngrok
the very next request confirms no cookie is stored, as there is no cookie: ... header in the request. interestingly, there are no errors on this cookie that is sent in the initial response headers of the servers /callback endpoint
Oauth Requests through Ngrok:
Oauth Requests without Ngrok:
Heres the controller that run after successful oauth verification/tokenization, triggered in both cases:
const oauth = catchAsync(async (req, res) => {
const user = req.user;
const tokens = await tokenService.generateAuthTokens(user);
res
.cookie('refreshToken', tokens.refresh.token, {
maxAge: tokens.refresh.maxAge,
httpOnly: true,
sameSite: "none",
secure: true,
// domain: "8796-2603-6011-223-7a04-2830-4c71-6f20-54c0.ngrok.io" // test
})
.redirect(`${config.clientURL}/app`)
});
I tried manually setting the domain value of the cookie config. Some values i tried include
localhost
localhost:8080
some-ngrok.io
, but to no avail
Heres the devserver webpack config, which usually has nothing extra, although i did also try with all for allowedHosts
But Im hopeful for a solution that works in both production and development
module.exports = {
// Extend/override the dev server configuration used by CRA
// See: https://github.com/timarney/react-app-rewired#extended-configuration-options
devServer: function (configFunction) {
return function (proxy, allowedHost) {
const config = configFunction(proxy, allowedHost);
// config.headers = {
// // "Cross-Origin-Embedder-Policy": "credentialless",
// // "Cross-Origin-Opener-Policy": "same-origin",
// // 'Cross-Origin-Resource-Policy': 'cross-origin',
// // 'Access-Control-Allow-Origin': '*'
// };
config.allowedHosts = ['all']
return config;
};
},
};
So maybe this is just a development environment issue? After all, google oauth works fine in prod/dev, maybe its just ngrok. So i've tested this by adding my live api url to the oauth redirect/allowedhost in zoom app web portal and ran this in production, and got the same issue.
Any one else go through this with a zoom app?

Problems with AWS Amplify, Next.js and authenticated SSR

I've got a Next.js application that uses AWS Cognito userpools for authentication. I have a custom UI and am using the aws-amplify package directly invoking signIn/signOut/etc... in my code. (I previously used the AWS Hosted UI and had the same problem set out below - I hoped switching and digging into the actual APIs who reveal my problem but it hasn't)
Everything in development (running on localhost) is working correctly - I'm able to login and get access to my current session both in a page's render function using
import { Auth } from 'aws-amplify';
...
export default const MyPage = (props) => {
useEffect(async () => {
const session = await Auth.currentSession();
...
}
...
}
and during SSR
import { withSSRContext } from 'aws-amplify';
...
export async function getServerSideProps(context) {
...
const SSR = withSSRContext(context);
const session = await SSR.Auth.currentSession();
...
}
However, when I deploy to AWS Amplify where I run my staging environment, the call to get the current session during SSR fails. This results in the page rendering as if the user is not logged in then switching when the client is able to determine that the user is in fact logged in.
Current Hypothesis - missing cookies(??):
I've checked that during the login process that the AWS cookies are being set correctly in the browser. I've also checked and devtools tells me the cookies are correctly being sent to the server with the request.
However, if I log out context.req.headers inside getServerSideProps in my staging environment, the cookie header is missing (whereas in my dev environment it appears correctly). If this is true, this would explain what I'm seeing as getServerSideProps isn't seeing my auth tokens, etc... but I can't see why the cookie headers would be stripped?
Has anyone seen anything like this before? Is this even possible? If so, why would this happen? I assume I'm missing something, e.g. config related, but I feel like I've followed the docs pretty closely - my current conf looks like this
Amplify.configure({
Auth: {...}
ssr: true
});
Next.js version is 11.1.2 (latest)
Any help very much appreciated!
You have to use Next#11.0.0 to use getServerSideProps, withSSRContext and Auth module in production.
I had same issue.
My solution was that disconnect a branch has an authentication problem once and reconnect the branch.
What are your build settings? I guess you are using next build && next export in which case this getServerSideProps shall not work. See https://nextjs.org/docs/advanced-features/static-html-export#unsupported-features
To use SSR with AWS amplify see https://docs.aws.amazon.com/amplify/latest/userguide/server-side-rendering-amplify.html#redeploy-ssg-to-ssr or consider deploying on a node server that is actually a server that you can start with next start like AWS EC2 or deploy on Vercel.
Otherwise if you use next export have to make do with client side data fetch only with client side updates only and cannot use dynamic server side features of nextjs.
One reason for context.req.headers not having any cookie in it is because CloudFront distribution is not forwarding any cookies.
This “CloudFront Behaviour” can be changed in two ways:
Forward all cookies, OR
Forward specified cookies (i.e. array of cookie names)
To change the behaviour, navigate to CloudFront on AWS console > Distributions > your_distribution > Behaviors Tab.
Then Edit existing or Create new behaviour > Change cookies settings (for example set it to "All")

SignalR can't connect when deployed behind AWS EB

I am trying to use SignalR on an application hosted in AWS EB, with an application loadbalancer in front.
It works perfectly fine when I test it locally, but when deployed, the websocket can not establish a connection and returns:
"Error: Failed to start the connection: Error: There was an error with the transport."
And
"There was an error with the transport. at WebSocket.o.onerror [as __zone_symbol__ON_PROPERTYerror]"
I have tried adding a middleware at the very start of my pipeline, that logs if i get a request for my hub, and this works, so I dont think it is the load balancer or anything AWS Related.
If i call the URL with Postman websocket, I get a status code 400.
I have followed the MS guide and added the configuration for JWT auth
jwtTokenOptions.Events = new JwtBearerEvents
{
OnMessageReceived = context =>
{
var accessToken = context.Request.Query["access_token"];
// If the request is for our hub...
var path = context.HttpContext.Request.Path;
if (!string.IsNullOrEmpty(accessToken) &&
(path.StartsWithSegments("/signalr")))
{
// Read the token out of the query string
context.Token = accessToken;
}
return Task.CompletedTask;
}
};
And as I said, it works fine locally. Any suggestions are most welcome
I fixed this by enabling web sockets for the IIS on the eb instance.
I did this by creating an eb extension with the following command
commands:
01_install_websockets_feature:
command: "%SystemRoot%\\System32\\dism.exe /online /enable-feature /featurename:IIS-WebSockets"
ignoreErrors: true

SignalR returns "Connection ID required" when ran in an AWS ECS container

I am trying to get a .Net Core 3.1 SignalR Web API with an Angular front end to work with websockets.
The code works perfectly fine when ran locally, either from within in the IDE or via docker run. However, once the code gets deployed to an ECS instance in AWS behind an API Gateway the web sockets refuse to connect.
I setup my mappings like so:
app.UsePathBase("/ui");
app.UseEndpoints(endpoints =>
{
endpoints.MapControllerRoute(
"default",
"{controller}/{action=Index}/{id?}")
.RequireCors(PolicyName);
endpoints.MapHub<SessionHub>("/ws/session");
endpoints.MapHub<SessionsHub>("/ws/sessions");
});
And on the client I connect to the hub like so:
this.sessionsHubConnection = new HubConnectionBuilder()
.withUrl(`${window.origin}/ws/sessions`, {
skipNegotiation: true,
transport: HttpTransportType.WebSockets,
accessTokenFactory: () => this.getAccessToken()
})
.withAutomaticReconnect()
.build();
The following Fiddler trace shows the initial HTTP request to initialize the websocket connection and the error being returned by kestrel.
I tweaked my web socket middleware for handling the access token to also Console.Write some additional debugging statements that I think might prove insightful:
public async Task Invoke(HttpContext httpContext)
{
var request = httpContext.Request;
Console.WriteLine($"Starting connection id: {httpContext.Connection.Id}");
// web sockets cannot pass headers so we must take the access token from query param and
// add it to the header before authentication middleware runs
if (request.Path.StartsWithSegments("/ws", StringComparison.OrdinalIgnoreCase)
&&
request.Query.TryGetValue("access_token", out var accessToken))
{
request.Headers.Add("Authorization", $"Bearer {accessToken}");
}
try
{
var sb = new StringBuilder();
sb.AppendLine($"Connection Id: {httpContext.Connection.Id}");
Console.WriteLine(sb.ToString());
await _next(httpContext);
sb = new StringBuilder();
sb.AppendLine($"Status code {httpContext.Response.StatusCode}"); <-- this line
sb.AppendLine($"Connection Id: {httpContext.Connection.Id}"); <-- and this line
Console.WriteLine(sb.ToString());
}
catch (Exception e)
{
Console.WriteLine(e.Message);
Console.WriteLine(e.StackTrace);
throw;
}
}
And in the AWS logs you can see that the connection Id is present but is being ignored by the EndpointMiddleware(?) for some reason.
Any idea what could be causing this? Two ideas I have yet to be able to rule out are:
The HTTP/S termination at our API gateway is confusing Kestrel since the browser client is building the socket request under HTTPS, but as far as Kestrel is concerned everything is communicating over HTTP?
app.UsePathBase("/ui"); is confusing Kestrel and all the web socket paths should actually be /ui/ws/session?

How to get cookies working with Nuxt Apollo and SSR correctly?

We've built a framework around Nuxt to get it to work with WordPress really well. We have been pulling our hair out trying to get post previewing working.
A common setup will be a WordPress install running on a domain like http://api.example.com and then to have Nuxt running on http://www.example.com. There is a WordPress plugin called WP-Graph-QL that creates a GraphQL endpoint like http://api.example.com/graphql, and we wrote a CORS plugin to set the correct CORS headers to work with whatever the frontend origin may be. This is that plugin if you are curious https://github.com/funkhaus/wp-graphql-cors
Our Nuxt Apollo setup is this:
export default function() {
return {
httpEndpoint: process.env.DEFAULT_ENDPOINT,
getAuth: () => process.env.BASIC_API_TOKEN || "",
httpLinkOptions: {
credentials: "include"
}
}
}
FYI sometimes the API will be hidden behind a Basic Auth UN/PW (like when on a staging site for example), that is what the getAuth function is doing.
This all seems to work client side, but it fails on SSR for some reason. It seems the cookies don't get sent in the SSR request, but they do in the client side request. Am I missing something super obvious here?
NOTE: I asked this question here 8 days, but am trying here for more attention