I'm working on creating a simple email server status page that calls two different CFCs.
The status page requirements:
Query a MariaDB database table via a CFC and return data from two fields: server_name (ie. MyServerName) & server_domain (ie. mail.domain.com). Currently, there are 4 rows in the database table to pull.
Hand the database data from step 1 to a CFC that checks if port 25 is listening. If the CFC can reach port 25 the result is true, if not the result is false. This step needs to be threaded.
Hand the boolean result from step 2 through a loop to print the server_name and boolean result.
Output something similar to this:
MyServerName - <up arrow>
MyServerName2 - <up arrow>
MyServerName3 - <up arrow>
MyServerName4 - <down arrow>
The code:
RetrieveEmailServers = APPLICATION.selectQueries.RetrieveEmailServers()
if (RetrieveEmailServers.recordCount) {
for(i = 1; i <= RetrieveEmailServers.recordCount(); i++) {
LOCAL.theDomains = RetrieveEmailServers.check_servers_domain[i];
LOCAL.theNames = RetrieveEmailServers.check_servers_name[i];
thread action="run" name="thread#i#" theDomains="#LOCAL.theDomains#" theNames="#LOCAL.theNames#" {
VARIABLES.theServers = APPLICATION.emailCheck.checkSMTPServer('#domains#',25,'','');
}
}
thread action="join" timeout="6000"{}
for(i = 1; i <= RetrieveEmailServers.recordCount(); i++) {
VARIABLES.theResult = cfthread["thread#i#"];
if (VARIABLES.theResult.theServers) {
LOCAL.theStatus = "<i class='fad fa-angle-double-up text-success fs-1'></i>"
}
else {
LOCAL.theStatus = "<i class='fad fa-angle-double-down text-danger fs-1'></i>"
}
writeOutput(ATTRIBUTES.theNames & " - " & LOCAL.theStatus & "<br>");
}
}
else {
writeOutput("No servers listed at this time.")
}
The error: The key [THESERVERS] does not exist, the structure is empty
For consideration:
I know my code is not great and I know it could be written better. I'm working hard to improve.
I'm not a full time coder but I have been coding off and on for many
years. I still consider myself a newbie with CFML so a lot of the methodology goes over my head.
The above code mostly works but I'm having difficulty understanding
how to pass information outside of a CFTHREAD to be used in the rest
of the page, especially when dealing with CFLOOP.
I have read many times, but still don't completely understand, how
to correctly use the thread-local scope, the Thread scope and the
Attributes scope.
The code above has a simple task, to check a port, but the end goal is to use similar code for other parts of my application. I'm aware there are better monitoring tools available; this is an exercise to help me understand and learn.
Specific to Lucee, I'm aware that threadData()['thread#i#'].status; or similar may be a required modification to cfthread[].
Attributes
The Attributes scope is only for holding values passed into a thread. Therefore, the scope is short-lived and only exists within a thread. Each thread has its own "attributes" scope, which doesn't exist before that thread runs or after it completes.
For example, this snippet passes in an attribute named "theDomains". The variable Attributes.theDomains only exists inside the thread.
thread action="run" name="thread1" theDomains="example.com" {
writeDump( attributes.theDomains );
}
thread action="join" name="thread1" {};
writeOutput( thread1.output );
Thread-local
"Thread-local" is another short-lived scope whose purpose is to hold variables used only within a thread. Each thread has its own private "local" scope, separate from all other threads. Like the attributes scope, it only exists while a thread is executing and is wiped out when the thread completes.
For example, this snippet creates a local variable named "MyLocalVar". Displaying the thread output demonstrates the variable exists within the thread
thread action="run" name="thread1" {
// Un-scoped v
myLocalVar = "foo";
writeOutput( "myLocalVar ="& myLocalVar );
}
thread action="join" name="thread1" {};
writeOutput( thread1.output );
But attempting to access it after the thread completes will cause an error
// fails with error "key [MYLOCALVAR] doesn't exist"
writeOutput( "myLocalVar ="& thread1.myLocalVar );
Thread Scope
The Thread scope has a longer life-span. It's designed to store "..thread-specific variables and metadata about the thread...". More importantly, this scope can be used to pass information back to the calling page (or even other threads).
For example, this snippet creates a thread scoped variable that's visible to the calling page, even after the thread completes its execution:
thread action="run" name="thread1" {
// use scope prefix "thread."
thread.myThreadVar = "foo";
}
thread action="join" name="thread1" {};
writeOutput( "thread1.myThreadVar="& thread1.myThreadVar );
writeDump( thread1 );
The Problem: key [THESERVERS] does not exist
When you've been looking at an error for what feels like days, it's easy to forget the basics :) First thing to do with undefined errors is dump the object and see if actually contains what you expected before trying to use it.
for(i = 1; i <= RetrieveEmailServers.recordCount(); i++) {
VARIABLES.theResult = cfthread["thread#i#"];
writeDump( variables.theResult );
/* temporarily comment out rest of code
...
*/
}
A dump of VARIABLES.theResult shows the threads are actually failing with a different error
Due to the wrong attribute name within the thread. It should be attributes.theDomains, not domains.
thread ...{
APPLICATION.emailCheck.checkSMTPServer( attributes.theDomains, ... );
}
Okay, I fixed it. Still getting "key [THESERVERS] does not exist", now what?
Another dump of the threads reveals the error message wasn't lying. The threads really DON'T contain a variable named theServers, due to incorrect scoping. Use the thread scope, not `variables.
thread ...{
thread.theServers = ....;
}
Yet another error?! variable [ATTRIBUTES] doesn't exist
Even after fixing the previous two issues, you'll still get yet another error here
for(i = 1; i <= RetrieveEmailServers.recordCount(); i++) {
...
writeOutput(ATTRIBUTES.theNames & " - " & LOCAL.theStatus & "<br>");
}
Remember the attributes scope only exists within a thread. So obviously it can't be used once the thread is completed. Either store theNames in the thread scope too, or since you're looping through a query, use the query column value, RetrieveEmailServers.the_query_column_name[ i ].
Joining threads
One last potential problem. The join statement isn't actually waiting for the threads you created. It's just waiting for 6000 ms. If for some reason any of the threads take longer than that, you'll get an error when trying to retrieve the thread results. To actually wait for the threads created, you must use thread action="join" name=(list of thread names) {}. I'll leave that as an exercise for the reader (:
Tbh, there are other things that could be cleaned up and/or improved, but hopefully this long rambling thread explains WHY the errors occurred in the first place AND how to avoid them when working with threads in future
Related
I understand how RAP creates scopes have a specific thread for each client and so on. I also understand how the application scope is unique among several clients, however I don't know how to access that specific scope in a single thread manner.
I would like to have a server side (with access to databases and stuff) that is a single execution to ensure it has a global knowledge of all transaction and that requests from clients are executed in sequence instead of parallel.
Currently I am accessing the application context as follows from the UI:
synchronized( MyServer.class ) {
ApplicationContext appContext = RWT.getApplicationContext();
MyServer myServer = (MyServer) appContext.getAttribute("myServer");
if (myServer == null){
myServer = new MyServer();
appContext.setAttribute("myServer", myServer);
}
myServer.doSomething(RWTUtils.getSessionID());
}
Even if I access myServer object there and trigger requests, the execution will still be running in the UI thread.
For now the only way to ensure the sequence is to use synchronized as follows on my server
public class MyServer {
String text = "";
public void doSomething(String string) {
try {
synchronized (this) {
System.out.println("doSomething - start :" + string);
text += "[" + string + "]";
System.out.println("text: " + (text));
Thread.sleep(10000);
System.out.println("text: " + (text));
System.out.println("doSomething - stop :" + string);
}
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
Is there a better way to not have to manage the thread synchronization myself?
Any help is welcome
EDIT:
To better explain myself, here is what I mean. Either I trust the database to handle multiple request properly and I have to handle also some other knowledge in a synchronized manner to share information between clients (example A) or I find a solution where another thread handles both (example B), the knowledge and the database. Of course, the problem here is that one client may block the others, but this is can be managed with background threads for long actions, most of them will be no problem. My initial question was, is there maybe already some specific thread of the application scope that does Example B or is Example A actually the way to go?
Conclusion (so far)
Basically, option A) is the way to go. For database access it will require connection pooling and for shared information it will require thoughtful synchronization of key objects. Main attention has to be done in the database design and the synchronization of objects to ensure that two clients cannot write incompatible data at the same time (e.g. write contradicting entries that make the result dependent of the write order).
First of all, the way that you create MyServer in the first snippet is not thread safe. You are likely to create more than one instance of MyServer.
You need to synchronize the creation of MyServer, like this for example:
synchronized( MyServer.class ) {
MyServer myServer = (MyServer) appContext.getAttribute("myServer");
if (myServer == null){
myServer = new MyServer();
appContext.setAttribute("myServer", myServer);
}
}
See also this post How to implement thread-safe lazy initialization? for other possible solutions.
Furthermore, your code is calling doSomething() on the client thread (i.e. the UI thread) which will cause each client to wait until pending requests of other clients are processed. The client UI will become unresponsive.
To solve this problem your code should call doSomething() (or any other long-running operation for that matter) from a background thread (see also
Threads in RAP)
When the background thread has finished, you should use Server Push to update the UI.
I want to read elements from the database and return them as JSON objects.
Scalatra is set up to return JSON.
Databaseschema is created.
Players are added.
The following code seems to be the main problem:
get("/") {
inTransaction {
List(from(MassTournamentSchema.players)(s => select(s)))
}
}
I get the following error:
"No session is bound to current thread, a session must be created via Session.create and bound to the thread via 'work' or 'bindToCurrentThread' Usually this error occurs when a statement is executed outside of a transaction/inTrasaction block "
I want to do it right so simply adding something like "Session.create" may not really be the right way.
Can anyone help a scalatra-noob? :-)
I think that your comment is on the right track. The inTransaction block will bind a JDBC connection to a thread local variable and start the connection on it. If the select doesn't occur on the same thread, you'll see an error like the one your received. There are two things I would suggest that you try:
Start your transaction later
List(inTransaction {
from(MassTournamentSchema.players)(s => select(s))
})
I'm not familiar with Scalatra's List, but it's possible that it's accepting a by-name parameter and executing it later on a different thread.
Force an eager evaluation of the query
inTransaction {
List(from(MassTournamentSchema.players)(s => select(s)).toList)
}
Here the .toList call will turn the Query object Squeryl returns into a Scala List immediately and guard against any lazy evaluation errors caused by later iteration.
I need to call an Async method within a method I declared. The method should return a value. I'm trying to wrap calls to the Windows Store into an easy to use class. My method should look like this:
bool Purchase(enum_InAppOption optionToPurchase);
enum_InAppOption is an enum consisting of all In-App options to purchase. At some point I need to call RequestProductPurchaseAsync. The result of this call determines if the method should return trueor false. I'm new to c++/cx (or at least I have a long history between now and the last time I used c++), so maybe this is easier as I think.
The create_task looks like this:
create_task(CurrentAppSimulator::RequestProductPurchaseAsync(this->_LastProductId, false))
The options I considered / tried:
returning the task would not abstract the store
tried to call wait on the task. I've got the exception An invalid parameter was passed to a function that considers invalid parameters fatal.
tried to use structured_task_group but it seems this does not allow for non void returning methods or I'm trying to provide a wrong interpretation. Compiler returns error C2064 (have googled but I can't get the point what to change)
Using an array of tasks and when_all
Found the following code on http://msdn.microsoft.com/en-us/library/dd492427.aspx#when_all in the middle of the page:
array<task<void>, 3> tasks =
{
create_task([] { wcout << L"Hello from taskA." << endl; }),
create_task([] { wcout << L"Hello from taskB." << endl; }),
create_task([] { wcout << L"Hello from taskC." << endl; })
};
auto joinTask = when_all(begin(tasks), end(tasks));
// Print a message from the joining thread.
wcout << L"Hello from the joining thread." << endl;
// Wait for the tasks to finish.
joinTask.wait();
So I tried to translate it into the following code:
array<task<Platform::String^>,1> tasks = {
create_task(CurrentAppSimulator::RequestProductPurchaseAsync(this->_LastProductId, false))
};
Even though I included the compiler throws C2065 ('array': undeclared identifier), C2275 ('Concurrency::task<_ReturnType>': illegal use of this type as an expression and some errors that seem to be errors following up on those two.
To sum up: How to make the method return after the async task has completed, so I can return a meaningful result based on the stuff going on asynchronously?
How to make the method return after the async task has completed, so I can return a meaningful result based on the stuff going on asynchronously?
This doesn't make much sense: the "stuff" isn't asynchronous if you want to wait for it to complete before returning. That's the definition of synchronous.
When using C++/CX, you cannot wait on a not-yet-completed task on an STA. Any attempt to do so will result in an exception being thrown. If you are going to call Purchase() on an STA and if it starts an asynchronous operation, you cannot wait for that operation to complete before returning.
Instead, you can use .then to perform another operation when the asynchronous operation completes. If the continuation needs to be performed on the invoking thread, make sure to pass the use_current() continuation context to ensure that the continuation is executed in the correct context.
Sascha,
Returning a task would abstract out the store, and I think that would be the most reasonable decision, since you are not restricting the users of your helper class to get the results straight away, but also allowing them to handle the results in their own way and asynchronously.
As #James correctly mentioned, you are not allowed to wait in the UI thread, then you will make the app unresponsive, there are different ways to avoid waiting:
create a continuation with concurrency::task::then;
you cannot wait in the UI thread, but you can wait for an operation on the UI thread to complete, that means you can wrap the future result of the task running on UI in a task_completion_event and then wait on the event in another (background) thread and handle the result;
concurrency::task_completion_event<Platform::String^> purchaseCompleted;
create_task(CurrentAppSimulator::RequestProductPurchaseAsync(
this->_LastProductId, false)).then(
[purchaseCompleted](concurrency::task<Platform::String^> task)
{
try
{
purchaseCompleted.set(task.get());
}
catch(Platform::Exception^ exception)
{
purchaseCompleted.set_exception(exception);
}
});
// and somewhere on non-UI thread you can do
Platform::String^ purchaseResult = create_task(purchaseCompleted).get();
you can achieve the previous trick using more WinRT-specific facilities rather than Concurrency Runtime, more precisely, IAsyncOperation<T>::Completed and IAsyncOperation<T>::GetResults;
and
seem irrelevant here, since you have only 1 real task, which is make a purchase.
Before implementing this I would like to check if this will lead to undefined behaviour or race conditions.
When uploading files to asure, this must be done in blocks. I want to upload 5 blocks in parallel and they all get their data from the same file. This would happen like this:
char *currentDataChunk;
int currentDataChunkSize;
connect(_blobStorageProvider, SIGNAL(putBlockSucceded(int)), this, SLOT(finalizeAndUploadNextBlock(int)));
int parallelUploads = ((_item->size() / MAX_BLOCK_SIZE) >= MAX_PARALLEL_BLOCKUPLOADS) ? MAX_PARALLEL_BLOCKUPLOADS : (_item->size() / MAX_BLOCK_SIZE);
_latestProcessedBlockId = (parallelUploads - 1);
for(int i = 0; i < parallelUploads; i++) {
currentDataChunkSize = _item->read(currentDataChunk, MAX_BLOCK_SIZE);
...
uploader->putBlock(_container, _blobName, currentDataChunk, i);
}
In the putBlock function in the uploader, it calls the QNetworkAccessManager with the call. When it's done it sends back a signal if it failed, succeded or got canceled, along with the blockId so that I know which one of the blocks that was uploaded.
void BigBlobUploader::finalizeAndUploadNextBlock(int blockId) {
// FINALIZE BY ADDING SUCCESSFUL BLOCK TO FUTURE BLOCKLIST
QByteArray temp;
for(int i = 0; i != sizeof(blockId); i++) {
temp.append((char)(blockId >> (i * 8)));
}
_uploadedBlockIds.insert(blockId, QString(temp.toBase64()));
this->uploadNextBlock();
}
void BigBlobUploader::uploadNextBlock() {
char *newDataChunk;
int newDataChunkSize = _item->read(newDataChunk, MAX_BLOCK_SIZE);
...
_latestProcessedBlockId++;
uploader->putBlock(_container, _blobName, newDataChunk, _latestProcessedBlockId);
}
My plan now is to fetch these signals to a slot which should take note that this block was uploaded (put it in a list to be able to put a block list to finalize this blob), increase the index by one (which starts at 5) and fetch a new chunk of data and redo the whole process.
My issue now is, what if two of them finishes at the EXACT same time? I'm not dealing with threads here but since the HTTP requests are threaded by default, what is the case here? Are the signals queued (or should I use QueuedConnection)? Can a slot be called in parallel? Is there a better way of doing this?
Sorry for the inconvenience, I assumed you were using .NET since you added the Windows Azure tag to this thread. I'm familar with Windows Azure, but my understanding about Qt is limited. However, it would not be different from using signals/slots in other concurrent scenarios. This document may help: http://qt-project.org/doc/qt-4.8/signalsandslots.html.
Best Regards,
Ming Xu.
I am not familiar with QNetworkAccessManager. But in general, to deal with race conditions, please use locks. Usually, the way to use locks in C# is leveraging the lock keyword. Something like:
private object lockingObject = new object();
In a method:
lock
{
// If a thread acquires a lock, another thread is blocked here until the lock is released.
}
In addition, you can refer to http://msdn.microsoft.com/en-us/library/c5kehkcz(v=vs.100).aspx for more information.
Best Regards,
Ming Xu.
I made a class that has an asynchronous OpenWebPage() function. Once you call OpenWebPage(someUrl), a handler gets called - OnPageLoad(reply). I have been using a global variable called lastAction to take care of stuff once a page is loaded - handler checks what is the lastAction and calls an appropriate function. For example:
this->lastAction == "homepage";
this->OpenWebPage("http://www.hardwarebase.net");
void OnPageLoad(reply)
{
if(this->lastAction == "homepage")
{
this->lastAction = "login";
this->Login(); // POSTs a form and OnPageLoad gets called again
}
else if(this->lastAction == "login")
{
this->PostLogin(); // Checks did we log in properly, sets lastAction as new topic and goes to new topic URL
}
else if(this->lastAction == "new topic")
{
this->WriteTopic(); // Does some more stuff ... you get the point
}
}
Now, this is rather hard to write and keep track of when we have a large number of "actions". When I was doing stuff in Python (synchronously) it was much easier, like:
OpenWebPage("http://hardwarebase.net") // Stores the loaded page HTML in self.page
OpenWebpage("http://hardwarebase.net/login", {"user": username, "pw": password}) // POSTs a form
if(self.page == ...): // now do some more checks etc.
// do something more
Imagine now that I have a queue class which holds the actions: homepage, login, new topic. How am I supposed to execute all those actions (in proper order, one after one!) via the asynchronous callback? The first example is totally hard-coded obviously.
I hope you understand my question, because frankly I fear this is the worst question ever written :x
P.S. All this is done in Qt.
You are inviting all manner of bugs if you try and use a single member variable to maintain state for an arbitrary number of asynchronous operations, which is what you describe above. There is no way for you to determine the order that the OpenWebPage calls complete, so there's also no way to associate the value of lastAction at any given time with any specific operation.
There are a number of ways to solve this, e.g.:
Encapsulate web page loading in an immutable class that processes one page per instance
Return an object from OpenWebPage which tracks progress and stores the operation's state
Fire a signal when an operation completes and attach the operation's context to the signal
You need to add "return" statement in the end of every "if" branch: in your code, all "if" branches are executed in the first OnPageLoad call.
Generally, asynchronous state mamangment is always more complicated that synchronous. Consider replacing lastAction type with enumeration. Also, if OnPageLoad thread context is arbitrary, you need to synchronize access to global variables.