Should the same UIManagedDocument be open on both of my devices, and I save (using the following code):
[self.documentDatabase.managedObjectContext performBlockAndWait:^{
STNoteLabelCell *cell = (STNoteLabelCell *)[self.noteTableView cellForRowAtIndexPath:indexPath];
[cell setNote:newNote animated:YES];
}];
I am told that the UIManagedDocuments documentState is changed to UIDocumentStateSavingError then I get this error:
CoreData: error: (1) I/O error for database at /var/mobile/Applications/some-long-id/Documents/Read.dox/StoreContent.nosync/persistentStore. SQLite error code:1, 'cannot rollback - no transaction is active'
2013-05-14 16:30:09.062 myApp[11711:4d23] -[_PFUbiquityRecordImportOperation main](312): CoreData: Ubiquity: Threw trying to get the knowledge vector from the store: <NSSQLCore: 0x1e9e2680> (URL: file://localhost/var/mobile/Applications/some-long-id/Documents/Read.dox/StoreContent.nosync/persistentStore)
Does anybody know why this error happens?
A couple of things...
I think the saving the same document on two different devices is a red herring, as it will actually be working on a local copy of your database on each device - only the transaction logs get uploaded to iCloud.
Secondly, I may be confused, but I don't see anything in the above code snippet that indicates you are performing a save (unless it is triggered by one of those calls, or autosave happens).
What that code snippet does seem to be doing is:
On your database document's child MOC thread, run the following block
of code
And that block of code is doing purely UI related stuff,
nothing to actually do with the database? The only thing that might
be going out to the DB is cellForRowAtIndexPath - and this usually
would be expected to only be doing read operations, not something
that needs to save.
The only other thing.... if the above code does trigger a save - you might have an issue with performing that as a performBlockAndWait. The UIManagedDocument save routines do stuff asynchronously - but they need the run loop to execute before they actually get a chance to run the async part... So by blocking before continuing you may actually be preventing the actual save from being executed or something.
I'm guessing wildly here, but have seen enough with saving managed documents to know to be really careful with which thread things are actually being called from, and that after a save request has been made, to allow the run loop to have a chance to actually do it. To be fair, this has only ever been an issue with calling saveToURL: repeatedly within one method, or in a loop, in which case all the async parts of the saves get queued up and executed at the end, usually to great comical effect.
Related
I have a class UserInterface containing a list of Items (whatever they represent). The content of these Items is too expensive to keep in memory due to possibly big size, so each Item only stores some metadata (description, preview...) for the related data, the data itself is stored on the disk in binary files. There is another class - FileHandler, that is responsible for managing these files. Each Item has id_ associated with the related File. UserInterface has a member reference to FileHandler to be able to get the list of Files from it and represent them as Items on the screen, but FileHandler does not even know about the existence of UserInterface. When the user selects an Item in the UI, UserInerface asks FileHandler to find a File with the same id_ (as the selected Item has), and then this File can be read from the disk for further usage of its content.
But here are some drawbacks of this design:
Let's say the user wants to delete some Items. They should be deleted from both UI and disk storage:
UserInterface::deleteItems(/*list of id's*/)
{
fileHandler_.deleteFiles(/*list of id's*/);
// What if a power failure or unexpected crash happens here?
// Delete items from UI...
}
If something bad happens during the execution of the following method, the next time the user runs the program they will get a broken state - the UI will contain corrupted items linked to files that have been deleted. I can still check if all files listed in the UI exist, and if some of them don't, I can mark corrupted items as invalid/broken, but I would rather prefer to avoid such situations completely.
Are there any good design patterns/techniques aiming to solve such problems?
Your program can stop at any time when the power is lost or the operating system crashes, and there is nothing you can do to avoid that.
However, it can also stop due to a controlled operating system shutdown or when your own code crashes. You can design your software so that it can close gracefully in some of these cases, by handling the shutdown message from the operating system, catching unhandled exceptions, handling std::terminate, handling close signals, etc.
If you use a database such as SQLite, and use transactions to write data, it can handle many of these situations for you. However, even SQLite database can get corrupted in some cases. For more information.
Because you cannot control the situation when the operating system crashes, you also need to fix any problems during your startup sequence. You should design the program so that if cannot fix corrupted database files (or other problems), it can at least detect them. In SQLite, you can execute PRAGMA integrity_check to see whether the database is corrupted. If your program must be able to recover automatically, you could take backups and recover the most recent backup, or you could restore the default settings.
Hard to write a good title for this question. I am developing a performance test in Gatling for a SOAP Webservice. I'm not very experienced with Gatling so I'm learning things as I go, but this conundrum has me entirely stumped.
One of the scenarios I am implementing a test for is an order-process consisting of several unique consecutive calls to the webservice, one of which is a polling call that returns the current status of the ordering process. Simplified, this call gets a SOAP Response with a status that can be of three types:
PROCESSING - Signifying the order is still processing.
ORDER_OK - Order completed without errors.
EVERYTHING_ELSE - A group of varying error-statuses and other results.
What I want to do, is have Gatling continuously poll the webservice until the processing-status changes - and then check that the status says it completed successfully. Polling continuously is easily implemented, but performing the check after it completes is turning out to be a far greater challenge than it has any business being.
So far, this is what I've done to solve the polling:
exec { session => session.set("status", "PROCESSING") }
.asLongAs(session => session("status").as[String].equals("PROCESSING")) {
exec(http("Poll order")
.post("/MyWebService")
.body(ELFileBody("bodies/ws/pollOrder.xml"))
.check(
status.is(200),
regex("soapFault").notExists,
regex("pollResponse").exists,
xpath("//*[local-name(.)='result']").exists.saveAs("status")
)
).exitHereIfFailed.pause(5 seconds)
}
This snip appears to be performing the polling correctly, it continues to poll until the orderStatus changes from processing to something else. I need to check the status to see if it changed to the response I am interested in however, because I don't know what it is, and only one of the many results it can be should cause the scenario to continue for that user.
A potential fix would be to add more checks in that call that go something like this:
.check(regex("EVERYTHING_ELSE_XYZ")).notExists
The service can return a LOT of different "not a happy day" messages however and I'm only really interested in the two other ones, so it would be preferable for me to be able to do a check only for the two valid happy-day responses. Checking if one exact thing exists seems far more sensible than checking that dozens of things don't.
What I thought I would be able to do was performing a check on the status variable in the users session when the step exits the asLongAs-loop, and continue/exit the scenario for that user. As it's a session-variable I could probably do this in the next step of the total scenario and break the run for that user there, but that would also mean the error is reported in the wrong place, and the next calls fault-% would be polluted by errors from the previous call.
Using pseudocode, being able to do something like this immediately after it exits the asLongAs loop would have been perfect:
if (session("status").as[String].equals("ORDER_OK")) ? continueTheScenario : failTheScenario
but I've not been able to do anything similar to that inside a gatling-chain. It's almost starting to appear impossible to do something like that, but can anyone see a solution to this that I'm not seeing?
Instead of "exists", use "in" to check that the result is one of the 2 valid values.
I'm creating a few simple helper classes and methods for working with libpq, and am wondering if I receive an error from the database - (e.g. SQL error), how should I handle it?
At the moment, each method returns a bool depending on whether the operation was a success, and so is up to the user to check before continuing with new operations.
However, after reading the libpq docs, if an error occurs the best I can come up with is that I should log the error message / status and otherwise ignore. For example, if the application is in the middle of a transaction, then I believe it can still continue (Postgresql won't cancel the transaction as far as I know).
Is there something I can do with PostgreSQL / libpq to make the consequences of such errors safe regarding the database server, or is ignorance the better policy?
You should examine the SQLSTATE in the error and make handling decisions based on that and that alone. Never try to make decisions in code based on the error message text.
An application should simply retry transactions for certain kinds of errors:
Serialization failures
Deadlock detection transaction aborts
For connection errors, you should reconnect then re-try the transaction.
Of course you want to set a limit on the number of retries, so you don't loop forever if the issue doesn't clear up.
Other kinds of errors aren't going to be resolved by trying again, so the app should report an error to the client. Syntax error? Unique violation? Check constraint violation? Running the statement again won't help.
There is a list of error codes in the documentation but the docs don't explain much about each error, but the preamble is quite informative.
On a side note: One trap to avoid falling into is "testing" connections with a trivial query before using them, and assuming that means the real query can't fail. That's a race condition. Don't bother testing connections; simply run the real query and handle any error.
The details of what exactly to do depend on the error and on the application. If there was a single always-right answer, libpq would already do it for you.
My suggestions:
Always keep a record of the transaction until you've got a confirmed commit from the DB, in case you have to re-run. Don't just fire-and-forget SQL statements.
Retry the transaction without a disconnect and reconnect for SQLSTATEs 40001 (serialization_failure) and 40P01 (deadlock_detected), as these are transient conditions generally resolved by re-trying. You should log them, as they're opportunities to improve how the app interacts with the DB and if they happen a lot they're a performance problem.
Disconnect, reconnect, and retry the transaction at least once for error class 08 (connection exceptions).
Handle 53300 (too_many_connections) and 53400 (connection limit exceeded) with specific and informative errors to the user. Same with the other 53 class entries.
Handle class 57's entries with specific and informative errors to the user. Do not retry if you get a query_cancelled (57014), it'll make sysadmins very angry.
Handle 25006 (read_only_sql_transaction) by reporting a different error, telling the user you tried to write to a read-only database or using a read-only transaction.
Report a different error for 23505 (UNIQUE violation), indicating that there's a conflict in a unique constraint or primary key constraint. There's no point retrying.
Error class 01 should never produce an exception.
Treat other cases as errors and report them to the caller, with details from the problem - most importantly SQLSTATE. Log all the details if you return a simplified error.
Hope that's useful.
I have a multithreaded application that uses sqlite (3.7.3)
I'm hitting the database locked error that seems to be quite prevalent.
I'm wondering how to avoid it in my case.
Let me describe what I'm building. Sorry, no code it's too large and complex.
I have around 8 threads that simultaneously access the database. Any one of those threads can either read or write at the same time.
Each row in a table in the database has a file path that points to a resource + other attributes related to that resource.
3 fields of note are readers, status and del.
Readers is incremented each time a thread reads from the resource, but only if status > 0 and del = 0.
So I have some SQL that does
UPDATE resource set readers=readers+1 where id=? AND del=0 AND status>0
After that, I check the number of rows updated. It should only be 1.
After that I try to read the row back with a select. I do that even if it failed
to update because I need to know the reason that it failed.
I tried wrapping both the update and the select in in a transaction but that didn't help.
I've checked that I'm calling finalize on my statements too.
Now, I thought that sqlite serializes by default. I've tried a couple of open modes but I still get the same error.
And before you ask, no I'm not intending to go to mysql. I absolutely need zero config.
Can someone provide some pointers on how to avoid this type of problem? should I move the readers lock out of the DB? If I do that what mechanism should I replace it with? I'm using Linux under C++ and with the boost library available.
EDIT:
Interestingly, adding COMMIT after my updated call improved things dramatically.
When you open the db, you should configure the 'busy timeout'
int sqlite3_busy_timeout(sqlite3*, int ms);
http://www.sqlite.org/c3ref/busy_timeout.html
First question: are you trying to use one connection with all eight threads? If so, make sure each thread has their own connection. I don't know of any database that likes that.
Also check out the FAQ: http://www.sqlite.org/faq.html
Apparently SQLite has to be compiled with a SQLITE_THREADSAFE preprocessor option set to 1. They do have a method to determine if that's your problem.
Another issue is that writes can only happen from one process safely.
I'm writing a program that among other things needs to download a file given its URL. I'm too lazy to implement the Http/Https protocols manually, so that I needed some library/object/function that'll do the job.
Critical requirement: The download must be asynchronous. That is, the thread that issued the download must be able to do something else "while" downloading the file, plus the download must be able to be aborted anytime without any barbaric side effects (such as internal call to TerminateThread).
Nice-to-have requirements:
Should be able to download the file "into memory". Means - read the contents of the file as they arrive, not necessarily save it into some "file system" file.
It'd be nice to have some convenient Win32 progress notification mechanism (waitable event, semahpore, completion port, etc.), rather than just periodically polling the download status.
I've chosen the XmlHttpRequest COM object to do the work. It seemed to work fine enough, plus it supported asynchronous mode.
However I noticed that after some period it just stops working.
That is, after several successful file downloads it stops downloading anything.
I periodically poll it to get its status, it reports "in-progress", but nothing actually happens, and there's no network activity. Moreover, when the same process creates another instance of XmlHttpRequest object to perform new downloads - the effect is the same. The object reports "in progress", whereas it doesn't even try to connect to the server (according to network sniffers and system TCP state).
The only way to make this object work back is to restart the process. This makes me suspect that there's a sort of a bug (sorry, I meant undocumented feature) in the object. Also it's not a bug at the level of an individual object, since the problem persists when the object is destroyed and another one is created. It's probably some global state of the DLL that implements this object.
Does anyone know something about this? Is this a known bug?
I'm pretty sure there's no chance that I have another bug in my code, because of which it seems to me to be the bug is in the XmlHttpRequest. I've done enoughtests and spent time with the debugger to conclude without reasonable doubt that it's just the object stops working.
BTW, while the object should work, I do all the waiting via MsgWaitXXXX API calls. So that if this object needs the message loop to work properly (for instance, it may create a hidden notification window and bind it to a socket via WSAAsyncSelect) - I give it the opportunity.
I know from my own experiences that the Microsoft implementation of the XmlHttpRequest falls short of full compliance with the draft standard. In particular the standard mandates that streamed data should be able to be extracted in ready state '3' (Receiving) which IE deliberately ignores.
Unfortunately I have not seen what you are describing despite using XmlHttpRequest objects extensively for long polling purposes.