Im developing a 'proxy' server in Thrift. My problem is, that each connection incomming to the proxy uses the same instance of the Handler. The client implementation of the proxy is in the Handler, so all the clients communicate throuh the same connection to the end server.
I have : n clients -> n sockets -> 1 handler -> 1 socket -> 1 server
What I want to implement : n clients -> n sockets -> n handlers -> n sockets -> 1 server
Now the problem is that if a client changes a 'local' parameter (something that is defined for each client independently) on the server, other clients will work with the changed environment too.
shared_ptr<CassProxyHandler> handler(new CassProxyHandler(adr_s,port_s,keyspace));
shared_ptr<TProcessor> processor(new CassandraProcessor(handler));
shared_ptr<TServerTransport> serverTransport(new TServerSocket(port));
shared_ptr<TTransportFactory> transportFactory(new TFramedTransportFactory());
shared_ptr<TProtocolFactory> protocolFactory(new TBinaryProtocolFactory());
TThreadedServer server(processor, serverTransport, transportFactory, protocolFactory);
server.serve();
Is there a way to implement a server, that creates a new instance of the Handler for each server socket instead of using the same handler?
Thanks for any suggestions or help,
#
I have managed to solve this problem. There was a solution already implemented in Java. I have used the same idea and implemented it in C++.
First thing I did is I created a TProcessorFactory instead of the TTransport class. This handles the TProcessors for each connection. It has a map structure in it, so its' get function returns the corresponding TProcessor for each TTransport. The corresponding (unique) TProcessor for each client.
I had to create a new TServer, so it would accept the newly created parameter TProcessorFactory instead of the TProcessor. In the TServer is also necessary to change a couple function calls. Your getProcessor function will no longer return a TProcessor but a TProcessorFactory (so change return type and rename).
The last thing you have to do is implement a server that allows instantiation, a derive class of TServer. I suggest using the TNonblockingServer (bit harder to implement the change) or the TThreadPoolServer. You have to change a couple function calls. Use a get function on the TProcessorFactory with a TTransport parameter to get a TProcessor where needed. The TTransport parameter is unique for each thread, each client connection is handled by one thread.
Also make sure you delete the old TProcessors, because thrift reuses (at least with the TNonblockingServer) the TTransport, so if you do not delete them and a client connects, he will probably get an inactive previous session and you probably don't want it. If you use shared pointers, just remove them from the map structure, when the client disconnects, and if there are no longer needed by thrift, they will be destructed.
I hope this helps to anyone, who encounters the same problem I did. If you don't know the inner structure of thrift, here a good guide : http://diwakergupta.github.com/thrift-missing-guide/
I hope the Thrift developers are going to implement something similar, but more sophisticated and abstract solution in the near future.
#
I know this is an old thread, but in case it's ever of use to anyone - I have contributed a change to the C# implementation of Thrift to solve this problem...
https://issues.apache.org/jira/browse/THRIFT-3397
In addition to the old method of passing a TProcessor as the first argument to the threaded servers, one can now set up something like
new ThreadPoolServer(processorFactory,serverTransport,
transportFactory,protocolFactory);
Where 'processorFactory' is a TProcessorFactory.
I've created TPrototypeProcessorFactory<TProcessor,Handler>(object[] handlerArgs) which would be set up like so:
TProcessorFactory processorFactory =
new TPrototypeProcessorFactory<ThriftGenerated.Processor, MyHandlerClass>();
The 'MyHandlerClass' implements your ThriftGenerated.Iface. Optionally, if this class takes arguments, they can be added as an array of objects to the processor factory.
Internally - For each new client connection, this processor factory will:
Create a new instance of 'MyHandlerClass' using any arguments
supplied (using Activator.CreateInstance)
If 'MyHandlerClass' implements 'TControllingHandler' it will set its
'server' property to the parent TServer (e.g. to allow control of
the TServer using a thift client)
Return a new instance of ThriftGenerated.Processor(handler)
Therefore for C# you get n clients -> n sockets -> n handlers -> n sockets -> 1 server
I hope this becomes useful to other people - it's certainly solved a problem for me.
Instead of making your proxy server talk thrift, you could just make it a generic TCP proxy that opens a new TCP connection for each incoming connection.
Related
In the unary RPC example provided in the grpc Github (client) and (server), is there any way to detect client's closed connection?
For example, in server.cc file:
std::string prefix("Hello ");
reply_.set_message(prefix + request_.name());
// And we are done! Let the gRPC runtime know we've finished, using the
// memory address of this instance as the uniquely identifying tag for
// the event.
status_ = FINISH;
int p = 0,i=0;
while(i++ < 1000000000) { // some dummy work
p = p + 10;
}
responder_.Finish(reply_, Status::OK, this);
With this dummy task before sending the response back to the client, server will take a few seconds. If we close the client (for example say with Ctrl+C), the server does not throw any error. It simply calls Finish and then deallocates the object as if the Finish is successful.
Is there any async feature (handler function) on the server-side to get us notified that the client has closed the connection or client is terminated?
Thank You!
Unfortunately, no.
But now guys from gRPC team works hard to implement callback mechanism into C++ implementation. As I understand it will work the same way as on Java implementation( https://youtu.be/5tmPvSe7xXQ?t=1843 ).
You can see how to work with future API with next examples: client_callback.cc and server_callback.cc
And the point of your interest there is ServerBidiReactor class from ::grpc::experimental namespace for server side. It have OnDone and OnCancel notification methods that maybe can help you.
Another interesting point there is that you can store a pointers to connection object and send notifications to client at any time.
But it still have many issue and I don't recommend to use this API in production code.
Current progress of C++ callbacks implementation you can see there: https://github.com/grpc/grpc/projects/12#card-12554506
I hope I don't have misunderstood the Thrift concept, but what I see from (example) questions like this, this framework is composed by different modular layers that can be enabled or disabled.
I'm mostly interesed in the "IDL part" of Thrift, so that I can create a common interface between my C++ code and an external Javascript application. I would like to call C++ functions using JS, with Binary data transmission, and I've already used the compiler for this.
But both my C++ (the server) and JS (client) application already exchange data using a C++ Webserver with Websockets support, it is not provided by Thrift.
So I was thinking to setup the following items:
In JS (already done):
TWebSocketTransport to send data to my "Websocket server" (with host ws://xxx.xxx.xxx.xxx)
TBinaryProtocol to encapsulate the data (using this JS implementation)
The compiled Thrift JS library with the correspondent C++ functions to call (done with the JS compiler)
In C++ (partial):
TBinaryProtocol to encode/decode the data
A TProcessor with handler to get the data from the client and process it
For now, the client is already able to sent requests to my websocket server, I see receiving them in binary form and I just need Thrift to:
Decode the input
Call the appropriate C++ function
Encode the output
My webserver will send the response to the client. So no "Thrift server" is needed here. I see there is the TProcessor->process() function, I'm trying to use it when I receive the binary data but it needs an in/out TProtocol. No problem here... but in order to create the TBinaryProtocol I also need a TTransport! If no Thrift server is expected... what Transport should I use?
I tried to set TTransport to NULL in TBinaryProtocol constructor, but once I use it it gives nullptr exception.
Code is something like:
Init:
boost::shared_ptr<MySDKServiceHandler> handler(new MySDKServiceHandler());
thriftCommandProcessor = boost::shared_ptr<TProcessor>(new MySDKServiceProcessor(handler));
thriftInputProtocol = boost::shared_ptr<TBinaryProtocol>(new TBinaryProtocol(TTransport???));
thriftOutputProtocol = boost::shared_ptr<TBinaryProtocol>(new TBinaryProtocol(TTransport???));
When data arrives:
this->thriftInputProtocol->writeBinary(input); // exception here
this->thriftCommandProcessor->process(this->thriftInputProtocol, this->thriftOutputProtocol, NULL);
this->thriftOutputProtocol->readBinary(output);
I've managed to do it using the following components:
// create the Processor using my compiled Thrift class (from IDL)
boost::shared_ptr<MySDKServiceHandler> handler(new MySDKServiceHandler());
thriftCommandProcessor = boost::shared_ptr<TProcessor>(new ThriftSDKServiceProcessor(handler));
// Transport is needed, I use the TMemoryBuffer so everything is kept in local memory
boost::shared_ptr<TTransport> transport(new apache::thrift::transport::TMemoryBuffer());
// my client/server data is based on binary protocol. I pass the transport to it
thriftProtocol = boost::shared_ptr<TProtocol>(new TBinaryProtocol(transport, 0, 0, false, false));
/* .... when the message arrives through my webserver */
void parseMessage(const byte* input, const int input_size, byte*& output, int& output_size)
{
// get the transports to write and read Thrift data
boost::shared_ptr<TTransport> iTr = this->thriftProtocol->getInputTransport();
boost::shared_ptr<TTransport> oTr = this->thriftProtocol->getOutputTransport();
// "transmit" my data to Thrift
iTr->write(input, input_size);
iTr->flush();
// make the Thrift work using the Processor
this->thriftCommandProcessor->process(this->thriftProtocol, NULL);
// the output transport (oTr) contains the called procedure result
output = new byte[MAX_SDK_WS_REPLYSIZE];
output_size = oTr->read(output, MAX_SDK_WS_REPLYSIZE);
}
My webserver will send the response to the client. So no "Thrift server" is needed here. I see there is the TProcessor->process() function, I'm trying to use it when I receive the binary data but it needs an in/out TProtocol. No problem here... but in order to create the TBinaryProtocol I also need a TTransport! If no Thrift server is expected... what Transport should I use?
The usual pattern is to store the bits somewhere and use that buffer or data stream as the input, same for the output. For certain languages there is a TStreamTransport available, for C++ the TBufferBase class looks promising to me.
I have a client and a server class in which i am sending message from client to server by making use of TCP sockets.
I have a class created in client.cpp named as Employee consisting of variables such as :
int emp_id;
char *emp_name;
float emp_weight;
My question is as follows:
1) How to send object of the employee class from client side to the server i.e how will i pass employee_object shown as follows to server:
employee_object.emp_id=10;
employee_object *emp_name=new char[30];
employee_object.emp_weight=50.2;
Any help will be of great help.I am doing this to make my self clear how to pass different objects of classes from client to server.
You have two main options: directly write the struct or class to the socket, or "serialize" it.
If you do a direct write, it's quite simple, but it requires you take care that your client and server have the same "width" (32 or 64 bit) and "endianness" (little or big). If you're dealing with regular Intel or AMD desktop or server machines only, this isn't much of an issue.
If you want to "serialize," the sky is the limit. Look up Protocol Buffers, Cap'n'Proto, JSON, etc. There are tons of libraries for this, but Stack Overflow is not the site to figure out which one you should use--you'll have to do some research. Some key considerations are whether the format is human-readable (like JSON) and whether it is fast (like Cap'n'Proto, or the direct method mentioned previously).
i'm studying this source base. Basically this is an Anim server client for Symbian 3rd edition for the purpose of grabbing input events without consuming them in a reliable way.
If you spot this line of the server, here it is basically setting the RProperty value (apparently to an increasing counter); it seems no actual processing of the input is done.
inside this client line, the client is supposed to be receiving the notification data, but it only calls Attach.
my understanding is that Attach is only required to be called once, but is not clear in the client what event is triggered every time the server sets the RProperty
How (and where) is the client supposed to access the RProperty value?
After Attaching the client will somewhere Subscribe to the property where it passes a TRequestStatus reference. The server will signal the request status property via the kernel when the asynchronous event has happened (in your case the property was changed). If your example source code is implemented in the right way, you will find an active object (AO; CActive derived class) hanging around and the iStatus of this AO will be passed to the RProperty API. In this case the RunL function of the AO will be called when the property has been changed.
It is essential in Symbian to understand the active object framework and quite few people do it actually. Unfortunately I did not find a really good description online (they are explained quite well in Symbian OS Internals book) but this page at least gives you a quick example.
Example
In the ConstructL of your CMyActive subclass of CActive:
CKeyEventsClient* iClient;
RProperty iProperty;
// ...
void CMyActive::ConstructL()
{
RProcess myProcess;
TSecureId propertyCategory = myProcess.SecureId();
// avoid interference with other properties by defining the category
// as a secure ID of your process (perhaps it's the only allowed value)
TUint propertyKey = 1; // whatever you want
iClient = CKeyEventsClient::NewL(propertyCategory, propertyKey, ...);
iClient->OpenNotificationPropertyL(&iProperty);
// ...
CActiveScheduler::Add(this);
iProperty.Subscribe(iStatus);
SetActive();
}
Your RunL will be called when the property has been changed:
void CMyActive::RunL()
{
if (iStatus.Int() != KErrCancel) User::LeaveIfError(iStatus.Int());
// forward the error to RunError
// "To ensure that the subscriber does not miss updates, it should
// re-issue a subscription request before retrieving the current value
// and acting on it." (from docs)
iProperty.Subscribe(iStatus);
TInt value; // this type is passed to RProperty::Define() in the client
TInt err = iProperty.Get(value);
if (err != KErrNotFound) User::LeaveIfError(err);
SetActive();
}
i'm sure this must be really simple or i'm missing the point, but how do you disconnect from Mongo using the C++ driver and DBClientConnection? DBClient has a public member of 'connect' but no disconnect/kill/drop etc that I can find.
There is some talk (in stack overflow and on the web) of using ScopedDBConnection which does seem to be able to allow me to drop my connection - but there are very few examples of how it would be used - or info on when I should use that class over the DBClientConnection class.
Any ideas?
If you're using a DBClientConnection, it has one connection, and you aren't supposed to disconnect/reconnect. I guess it kills the connection when it calls the destructors. You can set it up to automatically reconnect so you can keep using it if it loses its connection.
If you want to have connection pooling and multiple connections, you want to use ScopedDBConnection. You can see some examples here: https://github.com/mongodb/mongo/blob/master/src/mongo/client/model.cpp
Here's the gist:
ScopedDbConnection conn("localhost");
mongo::BSONObjBuilder obj;
obj.append( "name" , "asd" );
conn->insert("test.test", obj);
conn.done();
Basically, you can do anything with conn that you can do with a DBClientConnection, but when you're done you call done().