As a part of serverBuilder of GRPC there are couple of options where armeria server builder takes. One is channelOption and childChannelOption which takes in ChannelOption.SO_KEEPALIVE and a boolean. Even though I set these parameters i get an message saying
Unknown channel option 'SO_KEEPALIVE' for channel '[id:xxxxx]'
Can some one suggest how it can done as a part of armeria serverbuilder?
It's done like the following:
Server server =
Server.builder()
.childChannelOption(ChannelOption.SO_KEEPALIVE, true)
...
.build();
I confirmed that Unknown channel option 'SO_KEEPALIVE' error does not occur.
Related
How do you configure Javalin to change the max request size, specifically the config to increase the max size of the query parameters on the request (avoiding 414 URI too long)?
I get 414 URI is too long when I exceed what looks like a default size of 8KB so would like to configure my Javalin server to increase that slightly.
I think it uses Jetty under the hood which has a HttpConfiguration.requestHeaderSize variable that may control it. Or there's a HttpParser._maxHeaderBytes which is checked before throwing the URI_TOO_LONG_414 exception.
I can't see how it can all be wired up...
Bearing in mind all the advice and warnings listed here... Assuming you have Javalin defined as follows in your main method:
Javalin app = Javalin.create(config -> {
config.jetty.server(MyJetty::create);
}).start();
Then you can create MyJetty to customize this and any other settings you may want to use.
A very basic example:
import org.eclipse.jetty.server.HttpConfiguration;
import org.eclipse.jetty.server.HttpConnectionFactory;
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.server.ServerConnector;
public class MyJetty {
public static Server create() {
Server server = new Server();
HttpConfiguration httpConfiguration = new HttpConfiguration();
httpConfiguration.setRequestHeaderSize(8192); // use your value here
HttpConnectionFactory httpCF = new HttpConnectionFactory(httpConfiguration);
ServerConnector httpConnector = new ServerConnector(server, httpCF);
httpConnector.setPort(8080); // use your port here
server.addConnector(httpConnector);
return server;
}
}
This only sets up a simple insecure HTTP connection - but shows one way to change the HttpConfiguration value for Javalin. You can use the same approach for other connectors you may want to configure, including ones using SSL/TLS.
I am assuming the latest version of Javalin (version 5) since there were some syntax changes from Javalin 4 to 5 - and also the version of Jetty changed from 9 to 11.
If you are using Javalin 4, the config syntax is a bit different:
config.server(MyJetty::create);
But I don't think the Jetty code changes (for this specific setting, at least).
Only change the HttpConfiguration, it will inform the HttpParser.
You should be leery of doing this because ...
many browsers do not support that large of a query.
3rd party internet security software on laptops will reject that exchange.
you open yourself to various old school DoS (Denial of Service) attacks related to hashmap/hashcode abuse. (to minimize this issue, use Java 17 or newer)
If you move to HTTP/2 (or HTTP/3) many servers will reject the extension of the maximum request headers (at the HTTP level) that would be needed to support this massive request path.
Also, depending on what technology Javalin is using, you might need to also increase the ContextHandler maxFormContentSize and/or maxFormKeys.
If you have reached this need, it screams of abuse of the HTTP spec, you should investigate moving to a traditional HTTP POST with Content-Type: application/x-www-form-urlencoded instead.
I am using libwidevinecdm.so from chrome to handle DRM protected data. I am currently successfully setting the widevine server certificate I get from the license server. I can also create a session with the pssh box of the media im trying to decode. So far everything is successful (all promises resolve fine).
(session is created like this: _cdm->CreateSessionAndGenerateRequest(promise_id, cdm::SessionType::kTemporary, cdm::InitDataType::kCenc, pssh_box.data(), static_cast<uint32_t>(pssh_box.size()));)
I am then getting a session message of type kLicenseRequest which I am forwarding to the respective license server. The license server responds with a valid response and the same amount of data as I can see in the browser when using Chrome. I am then passing this to my session like this:
_cdm->UpdateSession(promise_id, session_id.data(), static_cast<uint32_t>(session_id.size()),
license_response.data(), static_cast<uint32_t>(license_response.size()));
The problem now is that this promise never resolves. It keeps posting the kLicenseRequest message over and over again to my session without ever returning. Does this mean my response is wrong? Or is this something else?
Br
Yanick
The issue is caused by the fact, that everything in CreateSessionAndGenerateRequest is done synchronous - that means by the time CreateSessionAndGenerateRequest returns your promise will always be resolved.
The CDM will emit the kLicenseRequest inside CreateSessionAndGenerateRequest and it doesn't do so in a "fire & forget" fashion, but the function waits there until you have returned from the cdm::Host_10::OnSessionMessage. Since my implementation of OnSessionMessage was creating a synchronous HTTP Request to the license server before - also synchronously - calling the UpdateSession the entire chain ended up to be blocking.
So ultimately I was calling UpdateSession while still being inside CreateSessionAndGenerateRequest and I assume the CDM cannot handle this and reacts by creating a new session with the given ID and generating a request again, which of course triggered another UpdateSession and so on.
Ultimately the simplest way to break the cycle was to make something asynchronous. I decided to launch a separate thread when receiving kLicenseRequest, wait for a few milliseconds to make sure that CreateSessionAndGenerateRequest has time to finish (not sure if that is really required) and then issue the request to the license server.
The only change I had to do was adding the surrounding std::thread:
void WidevineSession::forward_license_request(const std::vector<uint8_t> &data) {
std::thread{
[=]() {
std::this_thread::sleep_for(std::chrono::milliseconds{100});
net::HttpRequest request{"POST", _license_server_url};
request.add_header("Authorization", fmt::format("Bearer {}", _access_token))
.byte_body(data);
const auto response = _client.execute(request);
if (response.status_code() != 200) {
log->error("Widevine license request not accepted by license server: {} {} ({})", response.status_code(), response.status_text(), utils::bytes_to_utf8(response.body()));
throw std::runtime_error{"Error requesting widevine license"};
}
log->info("Successfully requested widevine license from license server");
_adapter->update_session(this, _session_id, response.body());
}
}.detach();
}
I hope I don't have misunderstood the Thrift concept, but what I see from (example) questions like this, this framework is composed by different modular layers that can be enabled or disabled.
I'm mostly interesed in the "IDL part" of Thrift, so that I can create a common interface between my C++ code and an external Javascript application. I would like to call C++ functions using JS, with Binary data transmission, and I've already used the compiler for this.
But both my C++ (the server) and JS (client) application already exchange data using a C++ Webserver with Websockets support, it is not provided by Thrift.
So I was thinking to setup the following items:
In JS (already done):
TWebSocketTransport to send data to my "Websocket server" (with host ws://xxx.xxx.xxx.xxx)
TBinaryProtocol to encapsulate the data (using this JS implementation)
The compiled Thrift JS library with the correspondent C++ functions to call (done with the JS compiler)
In C++ (partial):
TBinaryProtocol to encode/decode the data
A TProcessor with handler to get the data from the client and process it
For now, the client is already able to sent requests to my websocket server, I see receiving them in binary form and I just need Thrift to:
Decode the input
Call the appropriate C++ function
Encode the output
My webserver will send the response to the client. So no "Thrift server" is needed here. I see there is the TProcessor->process() function, I'm trying to use it when I receive the binary data but it needs an in/out TProtocol. No problem here... but in order to create the TBinaryProtocol I also need a TTransport! If no Thrift server is expected... what Transport should I use?
I tried to set TTransport to NULL in TBinaryProtocol constructor, but once I use it it gives nullptr exception.
Code is something like:
Init:
boost::shared_ptr<MySDKServiceHandler> handler(new MySDKServiceHandler());
thriftCommandProcessor = boost::shared_ptr<TProcessor>(new MySDKServiceProcessor(handler));
thriftInputProtocol = boost::shared_ptr<TBinaryProtocol>(new TBinaryProtocol(TTransport???));
thriftOutputProtocol = boost::shared_ptr<TBinaryProtocol>(new TBinaryProtocol(TTransport???));
When data arrives:
this->thriftInputProtocol->writeBinary(input); // exception here
this->thriftCommandProcessor->process(this->thriftInputProtocol, this->thriftOutputProtocol, NULL);
this->thriftOutputProtocol->readBinary(output);
I've managed to do it using the following components:
// create the Processor using my compiled Thrift class (from IDL)
boost::shared_ptr<MySDKServiceHandler> handler(new MySDKServiceHandler());
thriftCommandProcessor = boost::shared_ptr<TProcessor>(new ThriftSDKServiceProcessor(handler));
// Transport is needed, I use the TMemoryBuffer so everything is kept in local memory
boost::shared_ptr<TTransport> transport(new apache::thrift::transport::TMemoryBuffer());
// my client/server data is based on binary protocol. I pass the transport to it
thriftProtocol = boost::shared_ptr<TProtocol>(new TBinaryProtocol(transport, 0, 0, false, false));
/* .... when the message arrives through my webserver */
void parseMessage(const byte* input, const int input_size, byte*& output, int& output_size)
{
// get the transports to write and read Thrift data
boost::shared_ptr<TTransport> iTr = this->thriftProtocol->getInputTransport();
boost::shared_ptr<TTransport> oTr = this->thriftProtocol->getOutputTransport();
// "transmit" my data to Thrift
iTr->write(input, input_size);
iTr->flush();
// make the Thrift work using the Processor
this->thriftCommandProcessor->process(this->thriftProtocol, NULL);
// the output transport (oTr) contains the called procedure result
output = new byte[MAX_SDK_WS_REPLYSIZE];
output_size = oTr->read(output, MAX_SDK_WS_REPLYSIZE);
}
My webserver will send the response to the client. So no "Thrift server" is needed here. I see there is the TProcessor->process() function, I'm trying to use it when I receive the binary data but it needs an in/out TProtocol. No problem here... but in order to create the TBinaryProtocol I also need a TTransport! If no Thrift server is expected... what Transport should I use?
The usual pattern is to store the bits somewhere and use that buffer or data stream as the input, same for the output. For certain languages there is a TStreamTransport available, for C++ the TBufferBase class looks promising to me.
I'm having an unexpected issue with a c++ quickfix client application using FIX 4.4. I form marketdatarequest and populate it and then call send which returns true. The message is not found in the message or event log files.
No error seems to be reported - what could be happening?
FIX44::MarketDataRequest request(FIX::MDReqID(tmp)
, FIX::SubscriptionRequestType('1')
, FIX::MarketDepth(depth)); // 0 is full depth
FIX::SubscriptionRequestType subType(FIX::SubscriptionRequestType_SNAPSHOT);
FIX44::MarketDataRequest::NoRelatedSym symbolGroup;
symbolGroup.set(FIX::Symbol(I.subID));
request.addGroup(symbolGroup);
FIX::Header &header = request.getHeader();
header.setField(FIX::SenderCompID(sessionSenderID));
header.setField(FIX::TargetCompID(sessionTargetID));
if (FIX::Session::sendToTarget(request) == false)
return false;
My FixConfig looks like:
[DEFAULT]
HeartBtInt=30
ResetOnLogout=Y
ResetOnLogon=Y
ResetOnDisconnect=Y
ConnectionType=initiator
UseDataDictionary=Y
FileLogPath=logs
[SESSION]
FileLogPath=logs
BeginString=FIX.4.4
DataDictionary=XXXXX
ConnectionType=initiator
ReconnectInterval=60
TargetCompID=tCompID
SenderCompID=sCompID
SocketConnectPort=123456
SocketConnectHost=XX.XX.XXX.XX
SocketConnectProtocol=TCP
StartTime=01:05:00
EndTime=23:05:30
FileLogPath=logs
FileStorePath=logs
SocketUseSSL=N
thanks for any help,
Mark
Mark, just couple of notes not really related to your question but which you may found useful:
you dont have to explicitly set TargetCompId/SenderCompId for each message, engine will do it for you.
Do not place logic into callbacks(like you did with market data subscription in onLogon). Better create additional thread which will consume events from you listener, make decisions and take an action.
I'm trying to log my messages which are sent using a Metro stack into console.
Could not find any way.
Message logging to stdout (valid for METRO only!):
On the client
Java 5: Set system property
-Dcom.sun.xml.ws.transport.http.client.HttpTransportPipe.dump=true
Java 6: Set system property
-Dcom.sun.xml.internal.ws.transport.http.client.HttpTransportPipe.dump=true
On the server side
Set system property
-Dcom.sun.xml.ws.transport.http.HttpAdapter.dump=true
Here everything is explained:
https://metro.java.net/2.0/guide/Logging.html
The following options enable logging of all communication to the console (technically, you only need one of these, but that depends on the libraries you use, so setting all four is safer option).
-Dcom.sun.xml.ws.transport.http.client.HttpTransportPipe.dump=true
-Dcom.sun.xml.internal.ws.transport.http.client.HttpTransportPipe.dump=true
-Dcom.sun.xml.ws.transport.http.HttpAdapter.dump=true
-Dcom.sun.xml.internal.ws.transport.http.HttpAdapter.dump=true
Didn't mention the language but assuming Java, could you not just use something like Log4J e.g.
service = new Service();
port = service.getXxxPort();
result = port.doXxx(data);
Log.info("Result is " + result.getResult().toString());
where getResult is just a method on the return object.