Quantcast

SQLiteDiskIOException: disk I/O error (code 1034)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

SQLiteDiskIOException: disk I/O error (code 1034)

Philip Warner
I have an Android app that does work in the database every five minutes (at
least). Every few days I am seeing:

     org.sqlite.database.sqlite.SQLiteDiskIOException: disk I/O error (code 1034)

Code 1034 is SQLITE_IOERR_FSYNC.

This is using a custom build of the Sqlite bindings but I should note that this
also occurred under the vanilla build as well as in SqlCipher.

This leads me to believe it's a problem with the underlying code (or my use of it).

The app has multiple threads. It's always a background thread that throws this
error.

I believe that the specific Sqlite3 api being called in sqlite3_step, but can
not be sure. The android stack trace is:

Caused by: org.sqlite.database.sqlite.SQLiteDiskIOException: disk I/O error
(code 1034)
at
org.sqlite.database.sqlite.SQLiteConnection.nativeExecuteForChangedRowCount(Native
Method)
at
org.sqlite.database.sqlite.SQLiteConnection.executeForChangedRowCount(SQLiteConnection.java:776)
at
org.sqlite.database.sqlite.SQLiteSession.executeForChangedRowCount(SQLiteSession.java:759)
at
org.sqlite.database.sqlite.SQLiteStatement.executeUpdateDelete(SQLiteStatement.java:68)
at net.philipwarner.platforms.android
<http://net.philipwarner.platforms.android>.sqlitenative.NativeStatement.execute(NativeStatement.java:74)

Any comments or thoughts on what this might mean? Fsync in multiple threads?
Fysnc while doing IO elsewhere?

The app continues to work after the crash caused by this error; but it does kill
the entire app.


_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SQLiteDiskIOException: disk I/O error (code 1034)

Dan Kennedy-4
On 05/06/2017 02:13 PM, Philip Warner wrote:

> I have an Android app that does work in the database every five
> minutes (at least). Every few days I am seeing:
>
>     org.sqlite.database.sqlite.SQLiteDiskIOException: disk I/O error
> (code 1034)
>
> Code 1034 is SQLITE_IOERR_FSYNC.
>
> This is using a custom build of the Sqlite bindings but I should note
> that this also occurred under the vanilla build as well as in SqlCipher.
>
> This leads me to believe it's a problem with the underlying code (or
> my use of it).
>
> The app has multiple threads. It's always a background thread that
> throws this error.
>
> I believe that the specific Sqlite3 api being called in sqlite3_step,
> but can not be sure. The android stack trace is:
>
> Caused by: org.sqlite.database.sqlite.SQLiteDiskIOException: disk I/O
> error (code 1034)
> at
> org.sqlite.database.sqlite.SQLiteConnection.nativeExecuteForChangedRowCount(Native
> Method)
> at
> org.sqlite.database.sqlite.SQLiteConnection.executeForChangedRowCount(SQLiteConnection.java:776)
> at
> org.sqlite.database.sqlite.SQLiteSession.executeForChangedRowCount(SQLiteSession.java:759)
> at
> org.sqlite.database.sqlite.SQLiteStatement.executeUpdateDelete(SQLiteStatement.java:68)
> at net.philipwarner.platforms.android
> <http://net.philipwarner.platforms.android>.sqlitenative.NativeStatement.execute(NativeStatement.java:74)
>
> Any comments or thoughts on what this might mean? Fsync in multiple
> threads? Fysnc while doing IO elsewhere?

Is the storage media very close to full?

If you are able to enable sqlite3_log() output (see docs for
SQLITE_CONFIG_LOG) it will log a message identifying the file that the
fsync() failed on. And the output of strerror().

Dan.



_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SQLiteDiskIOException: disk I/O error (code 1034)

Philip Warner
Not full, several gb free.

Will turn on logging and hope it plays well with Android.

Is it likely to be a threading issue? If so, will the logging show other accesses to the file?

On 7 May 2017 2:13:28 am AEST, Dan Kennedy <[hidden email]> wrote:

>On 05/06/2017 02:13 PM, Philip Warner wrote:
>> I have an Android app that does work in the database every five
>> minutes (at least). Every few days I am seeing:
>>
>>     org.sqlite.database.sqlite.SQLiteDiskIOException: disk I/O error
>> (code 1034)
>>
>> Code 1034 is SQLITE_IOERR_FSYNC.
>>
>> This is using a custom build of the Sqlite bindings but I should note
>
>> that this also occurred under the vanilla build as well as in
>SqlCipher.
>>
>> This leads me to believe it's a problem with the underlying code (or
>> my use of it).
>>
>> The app has multiple threads. It's always a background thread that
>> throws this error.
>>
>> I believe that the specific Sqlite3 api being called in sqlite3_step,
>
>> but can not be sure. The android stack trace is:
>>
>> Caused by: org.sqlite.database.sqlite.SQLiteDiskIOException: disk I/O
>
>> error (code 1034)
>> at
>>
>org.sqlite.database.sqlite.SQLiteConnection.nativeExecuteForChangedRowCount(Native
>
>> Method)
>> at
>>
>org.sqlite.database.sqlite.SQLiteConnection.executeForChangedRowCount(SQLiteConnection.java:776)
>> at
>>
>org.sqlite.database.sqlite.SQLiteSession.executeForChangedRowCount(SQLiteSession.java:759)
>> at
>>
>org.sqlite.database.sqlite.SQLiteStatement.executeUpdateDelete(SQLiteStatement.java:68)
>> at net.philipwarner.platforms.android
>>
><http://net.philipwarner.platforms.android>.sqlitenative.NativeStatement.execute(NativeStatement.java:74)
>>
>> Any comments or thoughts on what this might mean? Fsync in multiple
>> threads? Fysnc while doing IO elsewhere?
>
>Is the storage media very close to full?
>
>If you are able to enable sqlite3_log() output (see docs for
>SQLITE_CONFIG_LOG) it will log a message identifying the file that the
>fsync() failed on. And the output of strerror().
>
>Dan.
>
>
>
>_______________________________________________
>sqlite-users mailing list
>[hidden email]
>http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

--
Sent from my Android device with K-9 Mail. Please excuse my dreadful spell-checker and my brevity.
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SQLiteDiskIOException: disk I/O error (code 1034)

Dan Kennedy-4
On 05/07/2017 12:21 AM, Philip Warner wrote:
> Not full, several gb free.
>
> Will turn on logging and hope it plays well with Android.
>
> Is it likely to be a threading issue? If so, will the logging show other accesses to the file?



The log most likely won't show too much other than the strerror() text.

It might be a threads-related error I suppose. I wouldn't jump straight
to that though.

SQLITE_IOERR_FSYNC means just that - an fsync() call failed. There are
only a couple of reasons for that - invalid file descriptor (some other
module calling close() on our fd?), a real IO error (dodgy phone/SD
card, have you seen this on more than one device?), or perhaps the fd is
actually a socket (hardly seems likely if you got all the way to fsync()
without some other error).

Dan.





>
> On 7 May 2017 2:13:28 am AEST, Dan Kennedy <[hidden email]> wrote:
>> On 05/06/2017 02:13 PM, Philip Warner wrote:
>>> I have an Android app that does work in the database every five
>>> minutes (at least). Every few days I am seeing:
>>>
>>>      org.sqlite.database.sqlite.SQLiteDiskIOException: disk I/O error
>>> (code 1034)
>>>
>>> Code 1034 is SQLITE_IOERR_FSYNC.
>>>
>>> This is using a custom build of the Sqlite bindings but I should note
>>> that this also occurred under the vanilla build as well as in
>> SqlCipher.
>>> This leads me to believe it's a problem with the underlying code (or
>>> my use of it).
>>>
>>> The app has multiple threads. It's always a background thread that
>>> throws this error.
>>>
>>> I believe that the specific Sqlite3 api being called in sqlite3_step,
>>> but can not be sure. The android stack trace is:
>>>
>>> Caused by: org.sqlite.database.sqlite.SQLiteDiskIOException: disk I/O
>>> error (code 1034)
>>> at
>>>
>> org.sqlite.database.sqlite.SQLiteConnection.nativeExecuteForChangedRowCount(Native
>>
>>> Method)
>>> at
>>>
>> org.sqlite.database.sqlite.SQLiteConnection.executeForChangedRowCount(SQLiteConnection.java:776)
>>> at
>>>
>> org.sqlite.database.sqlite.SQLiteSession.executeForChangedRowCount(SQLiteSession.java:759)
>>> at
>>>
>> org.sqlite.database.sqlite.SQLiteStatement.executeUpdateDelete(SQLiteStatement.java:68)
>>> at net.philipwarner.platforms.android
>>>
>> <http://net.philipwarner.platforms.android>.sqlitenative.NativeStatement.execute(NativeStatement.java:74)
>>> Any comments or thoughts on what this might mean? Fsync in multiple
>>> threads? Fysnc while doing IO elsewhere?
>> Is the storage media very close to full?
>>
>> If you are able to enable sqlite3_log() output (see docs for
>> SQLITE_CONFIG_LOG) it will log a message identifying the file that the
>> fsync() failed on. And the output of strerror().
>>
>> Dan.
>>
>>
>>
>> _______________________________________________
>> sqlite-users mailing list
>> [hidden email]
>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SQLiteDiskIOException: disk I/O error (code 1034)

Rowan Worth-2
On 7 May 2017 at 01:39, Dan Kennedy <[hidden email]> wrote:

> The log most likely won't show too much other than the strerror() text.
>
> It might be a threads-related error I suppose. I wouldn't jump straight to
> that though.
>
> SQLITE_IOERR_FSYNC means just that - an fsync() call failed. There are
> only a couple of reasons for that - invalid file descriptor (some other
> module calling close() on our fd?), a real IO error (dodgy phone/SD card,
> have you seen this on more than one device?), or perhaps the fd is actually
> a socket (hardly seems likely if you got all the way to fsync() without
> some other error).
>

This is a massive tangent, but for me sqlite's treatment of i/o errors is
my least favourite part of the overall design. Because while the extended
error code tells you _what_ failed, it doesn't give any information
regarding _why_ it failed.

The why appears in the log at least (assuming you've configured a
callback), but that codepath is logically very separate from the one
handling the error which makes it awkward to programatically make decisions
about the underlying cause (eg. to give the user an accurate diagnostic --
the difference between "your disk is full" vs "your disk appears broke").

These days (the past 8 years?) at least there's the SQLITE_FCNTL_LAST_ERRNO
parameter to sqlite3_file_control() allowing the underlying cause to be
introspected, I just feel like it was a mistake to ever mask that cause.

I realise no one is asking for my opinion and hindsight is 20/20 so feel
free to ignore my musings :)

-Rowan
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SQLiteDiskIOException: disk I/O error (code 1034)

Clemens Ladisch
Rowan Worth wrote:
> These days (the past 8 years?) at least there's the SQLITE_FCNTL_LAST_ERRNO
> parameter to sqlite3_file_control() allowing the underlying cause to be
> introspected, I just feel like it was a mistake to ever mask that cause.

That error code is behind a FCNTL because it is not portable; it could
never be a part of the public API.


Regards,
Clemens
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SQLiteDiskIOException: disk I/O error (code 1034)

Rowan Worth-2
On 8 May 2017 at 14:35, Clemens Ladisch <[hidden email]> wrote:

> Rowan Worth wrote:
> > These days (the past 8 years?) at least there's the
> SQLITE_FCNTL_LAST_ERRNO
> > parameter to sqlite3_file_control() allowing the underlying cause to be
> > introspected, I just feel like it was a mistake to ever mask that cause.
>
> That error code is behind a FCNTL because it is not portable; it could
> never be a part of the public API.
>

Yeah, that's entirely fair. And platform-agnostic error code categories are
perhaps too much to expect, especially since this mechanism enables people
who care (eg. myself) to design said categories according to their needs
(at least if my java bindings actually covered sqlite3_file_control >_<).

Preserving the underlying cause is still possible to do portably; something
like sqlite3_os_errmsg() would provide an "opaque" string suitable for
user/developer consumption.

-Rowan
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SQLiteDiskIOException: disk I/O error (code 1034)

Philip Warner
Interesting idea; when I've finished diagnosing my current problem, it might be
worth expanding the repertoire of the Android library.

I presume that the sqlite logging function could be used to get the message at
least, is that correct? IIRC, it's called on an error and the OS file-related
errors are often still available, but I am very rusty on this stuff.


On 8/05/2017 5:32 PM, Rowan Worth wrote:
> Preserving the underlying cause is still possible to do portably; something
> like sqlite3_os_errmsg() would provide an "opaque" string suitable for
> user/developer consumption.

_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SQLiteDiskIOException: disk I/O error (code 1034)

Rowan Worth-2
AFAIK The sqlite logging function already includes the OS error message for
i/o errors, so you'll have it there in the log already. It would be
difficult to capture that and plumb it back to the callsite though, since
the logging function provides you with zero context as to which connection
is responsible for its invocation (the callback is configured globally).
Probably simpler to call sqlite3_file_control() to get the last error code
and convert to the relevant message according to platform.

-Rowan

On 10 May 2017 at 17:32, Philip Warner <[hidden email]> wrote:

> Interesting idea; when I've finished diagnosing my current problem, it
> might be worth expanding the repertoire of the Android library.
>
> I presume that the sqlite logging function could be used to get the
> message at least, is that correct? IIRC, it's called on an error and the OS
> file-related errors are often still available, but I am very rusty on this
> stuff.
>
>
> On 8/05/2017 5:32 PM, Rowan Worth wrote:
>
>> Preserving the underlying cause is still possible to do portably;
>> something
>> like sqlite3_os_errmsg() would provide an "opaque" string suitable for
>> user/developer consumption.
>>
>
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Loading...