I have a continuos WebJob using the ServiceBusTrigger. I also set a timeout of 10 minutes.
If the function finishes before, it sends a response to another queue.
But in case of a timeout, it just exits. Is it possible to run some code when timeout is reached? Like a callback function. So I can send a proper message that the process timed out.
[Timeout("00:10:00")]
public static async Task ProcessPreflightQueueMessage([ServiceBusTrigger("%preflightQueue%")] string message, CancellationToken token, ILogger logger)
{ ... }
Yes you could, while the function timeout, it will throw TaskCanceledException, you could catch it and do your stuff.
Here is my test.
public class Functions
{
[Timeout("00:00:30")]
public static async Task TimeoutJob(
[QueueTrigger("myqueue")] string message,
CancellationToken token,
TextWriter log)
{
try
{
await log.WriteLineAsync(message);
log.WriteLine($"-- [{DateTime.Now.ToString()}] Processing Begin --");
await Task.Delay(TimeSpan.FromMinutes(1), token);
log.WriteLine($"-- [{DateTime.Now.ToString()}] Complete Time-consuming jobs --");
}
catch (TaskCanceledException)
{
log.WriteLine("timeout");
}
log.WriteLine($"-- [{DateTime.Now.ToString()}] Processing End -- ");
}
}
Related
I am using a very simple proto where the Message contains only 1 string field. Like so:
service LongLivedConnection {
// Starts a grpc connection
rpc Connect(Connection) returns (stream Message) {}
}
message Connection{
string userId = 1;
}
message Message{
string serverMessage = 1;
}
The use case is that the client should connect to the server, and the server will use this grpc for push messages.
Now, for the client code, assuming that I am already in a worker thread, how do I properly set it up so that I can continuously receive messages that come from server at random times?
void StartConnection(const std::string& user) {
Connection request;
request.set_userId(user);
Message message;
ClientContext context;
stub_->Connect(&context, request, &reply);
// What should I do from now on?
// notify(serverMessage);
}
void notify(std::string message) {
// generate message events and pass to main event loop
}
I figured out how to used the api. Looks like it is pretty flexible, but still a little bit weird given that I typically just expect the async api to receive some kind of lambda callback.
The code below is blocking, you'll have to run this in a different thread so it doesn't block your application.
I believe you can have multiple thread accessing the CompletionQueue, but in my case I just had one single thread handling this grpc connection.
GrpcConnection.h file:
public:
void StartGrpcConnection();
private:
std::shared_ptr<grpc::Channel> m_channel;
std::unique_ptr<grpc::ClientReader<push_notifications::Message>> m_reader;
std::unique_ptr<push_notifications::PushNotificationService::Stub> m_stub;
GrpcConnection.cpp files:
...
void GrpcConnectionService::StartGrpcConnection()
{
m_channel = grpc::CreateChannel("localhost:50051",grpc::InsecureChannelCredentials());
LongLiveConnection::Connect request;
request.set_user_id(12345);
m_stub = LongLiveConnection::LongLiveConnectionService::NewStub(m_channel);
grpc::ClientContext context;
grpc::CompletionQueue cq;
std::unique_ptr<grpc::ClientAsyncReader<LongLiveConnection::Message>> reader =
m_stub->PrepareAsyncConnect(&context, request, &cq);
void* got_tag;
bool ok = false;
LongLiveConnection::Message reply;
reader->StartCall((void*)1);
cq.Next(&got_tag, &ok);
if (ok && got_tag == (void*)1)
{
// startCall() is successful if ok is true, and got_tag is void*1
// start the first read message with a different hardcoded tag
reader->Read(&reply, (void*)2);
while (true)
{
ok = false;
cq.Next(&got_tag, &ok);
if (got_tag == (void*)2)
{
// this is the message from server
std::string body = reply.server_message();
// do whatever you want with body, in my case i push it to my applications' event stream to be processed by other components
// lastly, initialize another read
reader->Read(&reply, (void*)2);
}
else if (got_tag == (void*)3)
{
// if you do something else, such as listening to GRPC channel state change, in your call, you can pass a different hardcoded tag, then, in here, you will be notified when the result is received from that call.
}
}
}
}
I am trying to write a long running Subscriber service in Java. I have set up the Listeners to listen to any failures inside the Subscriber service. I am trying to make this fault tolerant and I do not quite understand few things, Below are my doubts/questions.
I have followed the basic setup shown here https://github.com/googleapis/google-cloud-java/blob/master/google-cloud-examples/src/main/java/com/google/cloud/examples/pubsub/snippets/SubscriberSnippets.java. Specifically, I have setup addListener as shown below.
As shown in the following code, initializeSubscriber acts a state variable which will determine if the Subscriber service should restart. Inside the while loop, this variable is continuously monitored to determine if the restart is required.
My question here is,
1. How do I raise an exception inside Subscriber.Listener's failed method and capture it in the main while loop. I tried throwing a new Exception() in failed method and catching it in catch block inside while, However, I am unable to compile the code as it is a checked exception.
2. As shown here, I use Java Executor thread to run the Listener. How do I handle the Listener failures ? Will I able to catch Listener failures under general Exception catch block as shown here ?
try {
boolean initializeSubscriber = true;
while (true) {
try {
if (initializeSubscriber) {
createSingleThreadedSubscriber();
addErrorListenerToSubscriber();
subscriber.startAsync().awaitRunning();
initializeSubscriber = false;
}
// Checks the status of subscriber service every minute
Thread.sleep(60000);
} catch (Exception ex) {
LOGGER.error("Could not start the Subscriber service", ex);
cleanupSubscriber();
initializeSubscriber = true;
}
}
} catch (RuntimeException e) {
} finally {
shutdown();
}
private void addErrorListenerToSubscriber() {
subscriber.addListener(
new Subscriber.Listener() {
#Override
public void failed(Subscriber.State from, Throwable failure) throws RuntimeException {
LOGGER.info("Subscriber reached a failed state due to " + failure.getMessage()
+ ",Restarting Subscriber service");
initializeSubscriber = true;
}
},
Executors.newSingleThreadExecutor());
}
private void cleanupSubscriber() {
try {
if (subscriber != null) {
subscriber.stopAsync().awaitTerminated();
}
if (!subscriptionListener.isShutdown()) {
subscriptionListener.shutdown();
}
} catch (Exception ex) {
LOGGER.error("Error in cleaning up Subscriber thread " + ex);
}
}
It should not be necessary to add a listener to the subscriber if you just want to recreate the subscriber on a failure. You could instead catch the exception on awaitTerminated:
try {
boolean initializeSubscriber = true;
while (initializeSubscriber) {
try {
createSingleThreadedSubscriber();
subscriber.startAsync().awaitRunning();
initializeSubscriber = false;
subscriber.awaitTerminated();
} catch (Exception ex) {
LOGGER.error("Error in the Subscriber service", ex);
cleanupSubscriber();
initializeSubscriber = true;
}
}
} catch (RuntimeException e) {
} finally {
shutdown();
}
If the subscriber shutdown successfully because of a call to stopAsync, then awaitTerminated will not throw an exception. If there was some kind of exception, then awaitTerminated will throw an IllegalStateException because the state will be FAILED instead of TERMINATED.
Note that transient errors are handled by the library itself. For example, if the server become briefly unavailable due to a network hiccup, the library will seamlessly reconnect and continue to deliver messages. Failures that result in a change in state for the subscriber are likely permanent failures such as permission issues (where the account running the subscriber does not have permission to subscribe to the subscription) or resource issues (such as the subscription having been deleted). In these permanent failure cases, recreating the subscriber will likely just result in the same error unless one takes manual steps to intervene and fix the problem.
I am working with the boost::asio tcp, version 1.57, creating a custom server/client, roughly following this example: Async_Tcp_Client , but I'm running the io_service run() in it's own thread per server/client. Also, there can be multiple server/clients per application.
Following the example I put my await_output function to sleep when I DON'T want to send a Message, and waking it up when I do want to send one (via async_write). After a varying amount of send-operations (sometimes less then 10, sometimes several thousand) I run into strange behaviour of my await_output Deadline (a boost deadline timer).
At some point, the async_wait against the timer just "disappears" and doesn't return when I cancel the deadline to send a message.
The transmit function, that is called by the Application owning the Client/Server (only by the application though, I guess it is not very threadsafe);
The await_output function that is waiting on the mOutputQueueDeadline;
And the handle_write function:
void SocketTcp::transmit(std::string pMsg) {
if (mStopped)
{ return; }
mOutputQueue.push(pMsg); // a global queue
// Signal that the output queue contains messages. Modifying the expiry
// will wake the output actor, if it is waiting on the timer.
size_t quits = mOutputQueueDeadline.expires_at(boost::posix_time::neg_infin);
//this returns '0' when the error occurs
}
void SocketTcp::await_output(const boost::system::error_code& ec)
{
if (mStopped)
{ return; }
if (mOutputQueue.empty())
{
size_t quits = mOutputQueueDeadline.expires_at(boost::posix_time::pos_infin);
mOutputQueueDeadline.async_wait(boost::bind(&SocketTcp::await_output, this, _1));
//this async_wait starts a wait on the deadline, that sometimes never returns!
}
else
{
boost::asio::async_write(mSocket,
boost::asio::buffer(mOutputQueue.front()),
boost::bind(&SocketTcp::handle_write, this, _1));
}
}
void SocketTcp::handle_write(const boost::system::error_code& ec)
{
if (mStopped)
{ return; }
if(!ec)
{
mOutputQueue.pop(); //remove sent element from queue
boost::system::error_code errcode;
await_output(errcode); //start the waiting actor for outgoing messages
}
else
{
mConnected = false; //update the connection status
this->stop();
}
}
I tried implementing a workaround, restarting the await_output in transmit() when expire_at returns 0, but that leads to TWO actors beeing awakened the next time I send a message, and then running into a crash (String iterator not dereferencable - the design doesn't allow for parallel send OP, much less trying to send the same message...)
I tried debugging with the BOOST_ASIO_ENABLE_HANDLER_TRACKING option, and found the error here:
#asio|1468415460.456019|0|deadline_timer#000000000050AB88.cancel //transmit cancels the timer
#asio|1468415460.456019|>474|ec=system:995 //await_output is called
#asio|1468415460.456019|474*479|socket#000000000050A9D8.async_send //starts the async send
#asio|1468415460.457019|<474|
#asio|1468415460.457019|>479|ec=system:0,bytes_transferred=102 //async send returns to it's handler
#asio|1468415460.457019|479|deadline_timer#000000000050AB88.cancel
//this cancel op is the only difference to the 'normal' order,
//not sure where it originates though!!
#asio|1468415460.457019|479*480|deadline_timer#000000000050AB88.async_wait //the handler starts the new async wait
//handler 480 never gets called when the deadline is canceled the next time
#asio|1468415460.457019|<479|
I'm pretty new to c++ as well as the stackoverflow (even though it has already safed me multiple times!) so please tell me if I can improve my question somehow!
I have set up a multiplayer dedicated server that runs using PHP scripts and MySQL databases. I am trying to access the server via HTTP to send/receive game data, starting with something as simple as getting the server status.
I have been able to contact the server successfully using the Unreal Doc's IHttpRequest:
void ANetwork::getContentsOfURL(FString URL)
{
serverResponse = NULL;
TSharedRef<IHttpRequest> HttpRequest = FHttpModule::Get().CreateRequest();
HttpRequest->SetHeader(TEXT("Content-Type"), TEXT("application/json"));
HttpRequest->SetURL(URL);
HttpRequest->SetVerb(TEXT("POST"));
//Creating JSON Object
FString json = "{\"auth\":\"" + authenticator = "\"";
json += "}";
HttpRequest->SetContentAsString(json);
HttpRequest->OnProcessRequestComplete().BindUObject(this, &ANetwork::OnResponseReceived);
HttpRequest->ProcessRequest();
}
void ANetwork::OnResponseReceived(FHttpRequestPtr Request, FHttpResponsePtr Response, bool bWasSuccessful)
{
GEngine->AddOnScreenDebugMessage(-1, 15.0f, FColor::Red, Response->GetContentAsString());
if (!Response.IsValid())
{
serverResponse = "FAIL";
}
else
{
serverResponse = Response->GetContentAsString();
}
}
And this echoes the proper codes to the debugger, so I know the server is working and the code is in fact getting what it needs to get. However, I need to be able to get the HTTP response as an FString and return it to the caller so that I can use it in-game. Right now this method is asynchronous, which prevents me from returning the response.
How can I make a synchronous HTTP Request so that I can return the response as a string to the caller?
i.e.
FString ANetwork::getContentsOfURL(FString URL)
Reset (unsignal) an event at the bottom of getContentsOfUrl.
Await for it to become signaled.
Signal the event from OnResponseReceived.
CreateEvent https://msdn.microsoft.com/en-us/library/windows/desktop/ms682396(v=vs.85).aspx ResetEvent https://msdn.microsoft.com/en-us/library/windows/desktop/ms685081(v=vs.85).aspx WaitForSingleObject https://msdn.microsoft.com/en-us/library/windows/desktop/ms687032(v=vs.85).aspx SetEvent (signals it) https://msdn.microsoft.com/en-us/library/windows/desktop/ms686211(v=vs.85).aspx
HANDLE hEvent = CreateEvent(NULL, FALSE, FALSE, NULL);
// bottom getCongentsOfUrl:
ResetEvent(hEvent); // optional because inital state is unsignaled
WaitForSingleObject(hEvent);
// OnResponseReceived
SetEvent(hEvent)
I have Server A that receive's updates from Server B. I would like to add functionality to Server A where if it does not receive a message(server B will send update and ping messages) in 1 minutes time, Server A will go into a paused state and wait for messages to come in again.
I was looking into a boost::asio::deadline_timer, but I cannot figure out if it is possible, or if you can run this asynchronously. I tried a class that runs in its own thread and uses a deadline timer, but I am unable to cancel and restart the deadline timer. Here is some example code I used for that.
The implementation:
void ping_timeout::reset_timer()
{
ping_timeout_.cancel();
ping_timeout_.expires_from_now(boost::posix_time::seconds(60));
//Call to clear the cache of a static class, which is the paused state I would like
ping_timeout_.async_wait(boost::bind(&cache::empty_cache));
io_.run();
}
I am unable to cancel the deadline timer from my main thread of execution by calling reset timer, I am guessing because io_.run() is waiting for the 60 seconds to expire.
Is there any modification I can do, any any libraries out there that I can us to achieve the results I would like? Any help would be appreciated.
Thank you
Edit:
Main Loop:
ping_timeout timeout;
boost::thread(boost::bind(&cache::run_io,boost::ref(service)));
while(true)
{
std::string message = s_recv(subscriber);
}
if(message.compare("UPDATE") == 0)
{
//Process update
}
else if(message.compare("PING") == 0)
{
timeout.reset_timer();
}
}
Edit 2:
Working code:
void cache::process_cache()
{
boost::asio::io_service service;
boost::asio::io_service::work work(service);
boost::thread(boost::bind(&cache::run_io,boost::ref(service)));
boost::asio::deadline_timer timer(service,boost::posix_time::seconds(60));
timer.async_wait(boost::bind(&cache::empty_cache,boost::asio::placeholders::error));
while(true)
{
std::string message = s_recv(subscriber);
if(message.compare("UPDATE") == 0)
{
//Process update
}
else if(message.compare("PING") == 0)
{
timer.cancel();
timer.expires_from_now(boost::posix_time::seconds(60));
timer.async_wait(boost::bind(&cache::empty_cache,boost::asio::placeholders::error));
}
}
}
void cache::empty_cache(const boost::system::error_code& e)
{
if(e.value() == 0)
{
//Clear cache
}
}
void cache::run_io(boost::asio::io_service& io)
{
io.run();
}
boost::asio::io_service::run() is a blocking call. In your specific case, you should avoid calling that in your main thread.
Note: In a typical async-driven app, you should build your app around the run method.
As for the timer code logic, something like that should work :
boost::asio::io_service service;
// Creates a work object to prevent the thread from exiting after the first job is done
boost::asio::io_service::work work(service);
// Creates the timer and post the aync wait now, will only start when service.run() is called
boost::asio::deadline_timer timer(service, boost::posix_time::seconds(60));
timer.async_wait(boost::bind(&cache::empty_cache, ...));
// Starts the worker thread to allow the timer to asynchronously waits
boost::thread ping_thread(boost::bind(&boost::asio::io_service::run, &service));
while (true) // you should add a condition in order to leave if the timer expires
{
std::string message = s_recv(subscriber);
/**/ if (message == "UPDATE")
{
// Process update
}
else if (message == "PING")
{
// Cancel the current timer
timer.cancel();
// Start another async wait
timer.async_wait(boost::bind(&cache::empty_cache, ...));
}
}