Large-scale database corruption on El Capitan‏

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Large-scale database corruption on El Capitan‏

deon
This post has NOT been accepted by the mailing list yet.
With the El Capitan rollout we're now seeing very large scale SQLite database corruption.
 
Anecdotally, what it looks like is that El Capitan has a lot of hard hang bugs that causes users to reboot their machines while our app is running. Trying this ourselves, we can corrupt a database about 25% of the time by hard-rebooting the machine. It seems the easiest way to hard-hang OSX is to plug/unplug a USB device - this has been starting to happen frequently in the last few months.
 
Is there something in the mean time we can do in the way we use SQLite to make it more resilient against hard hangs?
 
Currently our shipping settings are:
* SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE | SQLITE_OPEN_NOMUTEX | SQLITE_OPEN_PRIVATECACHE
* PRAGMA page_size=4096
* PRAGMA cache_size=16384
* PRAGMA JOURNAL_MODE = WAL
* PRAGMA synchronous = NORMAL

I've also just tried the following, but no luck (it actually seems to INCREASE the chance of corruption):
* SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE | SQLITE_OPEN_NOMUTEX | SQLITE_OPEN_PRIVATECACHE
* PRAGMA JOURNAL_MODE = WAL
* PRAGMA synchronous = FULL
 
We use SQLite from multiple threads (one connection per thread), so we can't use exclusive mode to open it - not without 6 months of re-engineering at least.
 
I can also supply a corrupted database if that will be useful, or run with debug options - for better or worse, we can reproduce the corruption in minutes.