I hope I don't have misunderstood the Thrift concept, but what I see from (example) questions like this, this framework is composed by different modular layers that can be enabled or disabled.
I'm mostly interesed in the "IDL part" of Thrift, so that I can create a common interface between my C++ code and an external Javascript application. I would like to call C++ functions using JS, with Binary data transmission, and I've already used the compiler for this.
But both my C++ (the server) and JS (client) application already exchange data using a C++ Webserver with Websockets support, it is not provided by Thrift.
So I was thinking to setup the following items:
In JS (already done):
TWebSocketTransport to send data to my "Websocket server" (with host ws://xxx.xxx.xxx.xxx)
TBinaryProtocol to encapsulate the data (using this JS implementation)
The compiled Thrift JS library with the correspondent C++ functions to call (done with the JS compiler)
In C++ (partial):
TBinaryProtocol to encode/decode the data
A TProcessor with handler to get the data from the client and process it
For now, the client is already able to sent requests to my websocket server, I see receiving them in binary form and I just need Thrift to:
Decode the input
Call the appropriate C++ function
Encode the output
My webserver will send the response to the client. So no "Thrift server" is needed here. I see there is the TProcessor->process() function, I'm trying to use it when I receive the binary data but it needs an in/out TProtocol. No problem here... but in order to create the TBinaryProtocol I also need a TTransport! If no Thrift server is expected... what Transport should I use?
I tried to set TTransport to NULL in TBinaryProtocol constructor, but once I use it it gives nullptr exception.
Code is something like:
Init:
boost::shared_ptr<MySDKServiceHandler> handler(new MySDKServiceHandler());
thriftCommandProcessor = boost::shared_ptr<TProcessor>(new MySDKServiceProcessor(handler));
thriftInputProtocol = boost::shared_ptr<TBinaryProtocol>(new TBinaryProtocol(TTransport???));
thriftOutputProtocol = boost::shared_ptr<TBinaryProtocol>(new TBinaryProtocol(TTransport???));
When data arrives:
this->thriftInputProtocol->writeBinary(input); // exception here
this->thriftCommandProcessor->process(this->thriftInputProtocol, this->thriftOutputProtocol, NULL);
this->thriftOutputProtocol->readBinary(output);
I've managed to do it using the following components:
// create the Processor using my compiled Thrift class (from IDL)
boost::shared_ptr<MySDKServiceHandler> handler(new MySDKServiceHandler());
thriftCommandProcessor = boost::shared_ptr<TProcessor>(new ThriftSDKServiceProcessor(handler));
// Transport is needed, I use the TMemoryBuffer so everything is kept in local memory
boost::shared_ptr<TTransport> transport(new apache::thrift::transport::TMemoryBuffer());
// my client/server data is based on binary protocol. I pass the transport to it
thriftProtocol = boost::shared_ptr<TProtocol>(new TBinaryProtocol(transport, 0, 0, false, false));
/* .... when the message arrives through my webserver */
void parseMessage(const byte* input, const int input_size, byte*& output, int& output_size)
{
// get the transports to write and read Thrift data
boost::shared_ptr<TTransport> iTr = this->thriftProtocol->getInputTransport();
boost::shared_ptr<TTransport> oTr = this->thriftProtocol->getOutputTransport();
// "transmit" my data to Thrift
iTr->write(input, input_size);
iTr->flush();
// make the Thrift work using the Processor
this->thriftCommandProcessor->process(this->thriftProtocol, NULL);
// the output transport (oTr) contains the called procedure result
output = new byte[MAX_SDK_WS_REPLYSIZE];
output_size = oTr->read(output, MAX_SDK_WS_REPLYSIZE);
}
My webserver will send the response to the client. So no "Thrift server" is needed here. I see there is the TProcessor->process() function, I'm trying to use it when I receive the binary data but it needs an in/out TProtocol. No problem here... but in order to create the TBinaryProtocol I also need a TTransport! If no Thrift server is expected... what Transport should I use?
The usual pattern is to store the bits somewhere and use that buffer or data stream as the input, same for the output. For certain languages there is a TStreamTransport available, for C++ the TBufferBase class looks promising to me.
Related
I am working with gRPC and Protobuf, using a C++ server and a C++ client, as well as a grpc-js client. Is there a way to get a read on all of the HTTP request/response headers from the transport layer in gRPC? I am looking for the sort of typical client/server HTTP headers - particularly, I would like to see what version of the protocol is being used (whether it is HTTP1.1/2). I know that gRPC is supposed to be using HTTP2, but I am trying to confirm it at a low level.
In a typical gRPC client implementation you have something like this:
class PingPongClient {
public:
PingPongClient(std::shared_ptr<Channel> channel)
: stub_(PingPong::NewStub(channel)) {}
// Assembles the client's payload, sends it and presents the response back
// from the server.
PingPongReply PingPong(PingPongRequest request) {
// Container for the data we expect from the server.
PingPongReply reply;
// Context for the client. It could be used to convey extra information to
// the server and/or tweak certain RPC behaviors.
ClientContext context;
// The actual RPC.
Status status = stub_->Ping(&context, request, &reply);
// Act upon its status.
if (status.ok()) {
return reply;
} else {
auto errorMsg = status.error_code() + ": " + status.error_message();
std::cout << errorMsg << std::endl;
throw std::runtime_error(errorMsg);
}
}
private:
std::unique_ptr<PingPong::Stub> stub_;
};
and on the serverside, something like:
class PingPongServiceImpl final : public PingPong::Service {
Status Ping(
ServerContext* context,
const PingPongRequest* request,
PingPongReply* reply
) override {
std::cout << "PingPong" << std::endl;
printContextClientMetadata(context->client_metadata());
if (request->input_msg() == "hello") {
reply->set_output_msg("world");
} else {
reply->set_output_msg("I can't pong unless you ping me 'hello'!");
}
std::cout << "Replying with " << reply->output_msg() << std::endl;
return Status::OK;
}
};
I would think that either ServerContext or the request object might have access to this information, but context seems to only provide an interface into metadata, which is custom.
None of the gRPC C++ examples give any indication that there is such an API, nor do any of the associated source/header files in the gRPC source code. I have exhausted my options here in terms of tutorials, blog posts, videos, and documentation - I asked a similar question on the grpc-io forum, but have gotten no takers. Hoping the SO crew has some insights here!
I should also note that I experimented with passing a variety of environment variables as flags to the running processes to see if I can get details about HTTP headers, but even with these flags enabled (the HTTP-related ones), I do not see basic HTTP headers.
First, the gRPC libraries absolutely do use HTTP/2. The protocol is explicitly defined in terms of HTTP/2.
The gRPC libraries do not directly expose the raw HTTP headers to the application. However, they do have trace logging options that can log a variety of information for debugging purposes, including headers. The tracers can be enabled by setting the environment variable GRPC_TRACE. The environment variable GRPC_VERBOSITY=DEBUG should also be set to make sure that all of the logs are output. More information can be found in this document describing how the library uses envinronment variables.
In the C++ library, the http tracer should log the raw headers. The grpc-js library has different internals and different tracer definitions, so you should use the call_stream tracer for that one. Those will also log other request information, but it should be pretty easy to pick out the headers.
Hi using the webrtc native c++ API I am able to establish an RTCPeerconnection from my browser to a locally running server. In the PeerConnectionObserver interface there is an OnAddStream callback that gets fired when I add a media stream from my browser to the peer connection. The server fires this callback as expected and and I now have access to the stream on the server. This is where I am stuck, what do I need to do with the stream now to get the raw binary data from it. I have seen some examples using a "VideoSink" after getting the video tracks from the stream but nothing is clear enough and there is no documentation for the c++ api header files. Anyone with webrtc experience familiar with actually doing stuff with the media stream from the native api? Here is where I am stuck
class PeerConnectionObserver : public webrtc::PeerConnectionObserver {
....
// Override adding a stream.
void OnAddStream(rtc::scoped_refptr<webrtc::MediaStreamInterface>stream ) {
//TODO now that I have the stream what do I do ?
std::cout << "OnAddStream " << stream->label();
}
....
}
I have a client and a server class in which i am sending message from client to server by making use of TCP sockets.
I have a class created in client.cpp named as Employee consisting of variables such as :
int emp_id;
char *emp_name;
float emp_weight;
My question is as follows:
1) How to send object of the employee class from client side to the server i.e how will i pass employee_object shown as follows to server:
employee_object.emp_id=10;
employee_object *emp_name=new char[30];
employee_object.emp_weight=50.2;
Any help will be of great help.I am doing this to make my self clear how to pass different objects of classes from client to server.
You have two main options: directly write the struct or class to the socket, or "serialize" it.
If you do a direct write, it's quite simple, but it requires you take care that your client and server have the same "width" (32 or 64 bit) and "endianness" (little or big). If you're dealing with regular Intel or AMD desktop or server machines only, this isn't much of an issue.
If you want to "serialize," the sky is the limit. Look up Protocol Buffers, Cap'n'Proto, JSON, etc. There are tons of libraries for this, but Stack Overflow is not the site to figure out which one you should use--you'll have to do some research. Some key considerations are whether the format is human-readable (like JSON) and whether it is fast (like Cap'n'Proto, or the direct method mentioned previously).
In my desktop application I added access to various internet resources using boost::asio. All i do is sending http requests (i.e to map tile servers) and read the results.
My code is based on the asio sync_client sample.
Now i get reports from customers who are unable to use these functions as they are running a proxy in their company. In a web browser they can enter the address of their proxy and everything is fine. Our application is unable to download data.
How can i add such support to my application?
I found the answer myself. It's quite simple:
http://www.jmarshall.com/easy/http/#proxies
gives quite a brief and clear description how http proxies work.
All i had to do is add the following code to the asio sync_client sample sample :
std::string myProxyServer = ...;
int myProxyPort = ...;
void doDownLoad(const std::string &in_server, const std::string &in_path, std::ostream &outstream)
{
std::string server = in_server;
std::string path = in_path;
char serice_port[255];
strcpy(serice_port, "http");
if(! myProxyServer.empty())
{
path = "http://" + in_server + in_path;
server = myProxyServer;
if(myProxyPort != 0)
sprintf(serice_port, "%d", myProxyPort);
}
tcp::resolver resolver(io_service);
tcp::resolver::query query(server, serice_port);
...
It seems that sample is merely a show-off of what Boost ASIO can be used for but is likely not intended to be used as-is. You should probably use a complete library that handles not only HTTP proxies, but also HTTP redirects, compression, and so on.
HTTP is a complex thing: without doing so, chances are high that you will get news from another client soon with another problem.
I found cppnetlib which looks promising and is based on Boost ASIO not sure it handles proxies though.
There is also libcurl but I don't know if it can easily be integrated with Boost ASIO.
Im developing a 'proxy' server in Thrift. My problem is, that each connection incomming to the proxy uses the same instance of the Handler. The client implementation of the proxy is in the Handler, so all the clients communicate throuh the same connection to the end server.
I have : n clients -> n sockets -> 1 handler -> 1 socket -> 1 server
What I want to implement : n clients -> n sockets -> n handlers -> n sockets -> 1 server
Now the problem is that if a client changes a 'local' parameter (something that is defined for each client independently) on the server, other clients will work with the changed environment too.
shared_ptr<CassProxyHandler> handler(new CassProxyHandler(adr_s,port_s,keyspace));
shared_ptr<TProcessor> processor(new CassandraProcessor(handler));
shared_ptr<TServerTransport> serverTransport(new TServerSocket(port));
shared_ptr<TTransportFactory> transportFactory(new TFramedTransportFactory());
shared_ptr<TProtocolFactory> protocolFactory(new TBinaryProtocolFactory());
TThreadedServer server(processor, serverTransport, transportFactory, protocolFactory);
server.serve();
Is there a way to implement a server, that creates a new instance of the Handler for each server socket instead of using the same handler?
Thanks for any suggestions or help,
#
I have managed to solve this problem. There was a solution already implemented in Java. I have used the same idea and implemented it in C++.
First thing I did is I created a TProcessorFactory instead of the TTransport class. This handles the TProcessors for each connection. It has a map structure in it, so its' get function returns the corresponding TProcessor for each TTransport. The corresponding (unique) TProcessor for each client.
I had to create a new TServer, so it would accept the newly created parameter TProcessorFactory instead of the TProcessor. In the TServer is also necessary to change a couple function calls. Your getProcessor function will no longer return a TProcessor but a TProcessorFactory (so change return type and rename).
The last thing you have to do is implement a server that allows instantiation, a derive class of TServer. I suggest using the TNonblockingServer (bit harder to implement the change) or the TThreadPoolServer. You have to change a couple function calls. Use a get function on the TProcessorFactory with a TTransport parameter to get a TProcessor where needed. The TTransport parameter is unique for each thread, each client connection is handled by one thread.
Also make sure you delete the old TProcessors, because thrift reuses (at least with the TNonblockingServer) the TTransport, so if you do not delete them and a client connects, he will probably get an inactive previous session and you probably don't want it. If you use shared pointers, just remove them from the map structure, when the client disconnects, and if there are no longer needed by thrift, they will be destructed.
I hope this helps to anyone, who encounters the same problem I did. If you don't know the inner structure of thrift, here a good guide : http://diwakergupta.github.com/thrift-missing-guide/
I hope the Thrift developers are going to implement something similar, but more sophisticated and abstract solution in the near future.
#
I know this is an old thread, but in case it's ever of use to anyone - I have contributed a change to the C# implementation of Thrift to solve this problem...
https://issues.apache.org/jira/browse/THRIFT-3397
In addition to the old method of passing a TProcessor as the first argument to the threaded servers, one can now set up something like
new ThreadPoolServer(processorFactory,serverTransport,
transportFactory,protocolFactory);
Where 'processorFactory' is a TProcessorFactory.
I've created TPrototypeProcessorFactory<TProcessor,Handler>(object[] handlerArgs) which would be set up like so:
TProcessorFactory processorFactory =
new TPrototypeProcessorFactory<ThriftGenerated.Processor, MyHandlerClass>();
The 'MyHandlerClass' implements your ThriftGenerated.Iface. Optionally, if this class takes arguments, they can be added as an array of objects to the processor factory.
Internally - For each new client connection, this processor factory will:
Create a new instance of 'MyHandlerClass' using any arguments
supplied (using Activator.CreateInstance)
If 'MyHandlerClass' implements 'TControllingHandler' it will set its
'server' property to the parent TServer (e.g. to allow control of
the TServer using a thift client)
Return a new instance of ThriftGenerated.Processor(handler)
Therefore for C# you get n clients -> n sockets -> n handlers -> n sockets -> 1 server
I hope this becomes useful to other people - it's certainly solved a problem for me.
Instead of making your proxy server talk thrift, you could just make it a generic TCP proxy that opens a new TCP connection for each incoming connection.