BUG: Async IO module works incorrectly with large database files

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

BUG: Async IO module works incorrectly with large database files

Pavel Ivanov-2
Hi!

As anonymous users are not allowed to file bug reports on SQLite site
anymore so I'm sending it here.

I've encountered segmentation fault at sqlite3async.c:715. The problem
is on sqlite3async.c:712:

        nCopy = (int)MIN(pWrite->nByte-iBeginIn, iAmt-iBeginOut);

My actual numbers: iAmt = 16, iBeginOut = 2147844072. After
subtraction we're getting -2147844056 or FFFFFFFF7FFA8028 in hex which
after truncating to int gives 7FFA8028, a positive number. And it
blows away the next check if( nCopy>0 ).


Pavel
_______________________________________________
sqlite-users mailing list
[hidden email]
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: BUG: Async IO module works incorrectly with large database files

Dan Kennedy-4

On Oct 7, 2009, at 6:42 PM, Pavel Ivanov wrote:

> Hi!
>
> As anonymous users are not allowed to file bug reports on SQLite site
> anymore so I'm sending it here.
>
> I've encountered segmentation fault at sqlite3async.c:715. The problem
> is on sqlite3async.c:712:
>
>       nCopy = (int)MIN(pWrite->nByte-iBeginIn, iAmt-iBeginOut);
>
> My actual numbers: iAmt = 16, iBeginOut = 2147844072. After
> subtraction we're getting -2147844056 or FFFFFFFF7FFA8028 in hex which
> after truncating to int gives 7FFA8028, a positive number. And it
> blows away the next check if( nCopy>0 ).

Thanks. Ticket now posted here:

  http://www.sqlite.org/src/info/94c04eaadb

Dan.

_______________________________________________
sqlite-users mailing list
[hidden email]
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: BUG: Async IO module works incorrectly with large database files

Pavel Ivanov-2
Hi, Dan!

I've found another bug in async io module. It happens only
occasionally in my application but I've found how to perfectly
reproduce it. You need to:
- not call sqlite3async_run();
- open new not yet existing database;
- execute PRAGMA journal_mode = PERSIST;
- execute any 2 statements needing a transaction, e.g. some CREATE TABLE

In this scenario (when connection uses async io module) last statement
returns error SQLITE_CANTOPEN.

I've investigated why it's happening. In first transaction when SQLite
opens journal file async module determines that this opening should
happen asynchronously and puts ASYNC_OPENEXCLUSIVE into the queue.
Then at the end of transaction journal file is closed and everything
is ok. When SQLite starts 2nd transaction it checks (inside
hasHotJournal() ) for journal existence and async module returns "yes"
because of ASYNC_OPENEXCLUSIVE in the event queue. Then SQLite tries
to open journal for reading and async module determines that it should
be done immediately. It try to physically open journal and fail
because file does not exist yet. So SQLite decides that there is hot
journal tries to open it again for read/write (inside
sqlite3PageSharedLock() ) and fails again because file doesn't exist
yet. And this error is already goes as a return code from
sqlite3_step().


Pavel

On Wed, Oct 7, 2009 at 9:57 AM, Dan Kennedy <[hidden email]> wrote:

>
> On Oct 7, 2009, at 6:42 PM, Pavel Ivanov wrote:
>
>> Hi!
>>
>> As anonymous users are not allowed to file bug reports on SQLite site
>> anymore so I'm sending it here.
>>
>> I've encountered segmentation fault at sqlite3async.c:715. The problem
>> is on sqlite3async.c:712:
>>
>>       nCopy = (int)MIN(pWrite->nByte-iBeginIn, iAmt-iBeginOut);
>>
>> My actual numbers: iAmt = 16, iBeginOut = 2147844072. After
>> subtraction we're getting -2147844056 or FFFFFFFF7FFA8028 in hex which
>> after truncating to int gives 7FFA8028, a positive number. And it
>> blows away the next check if( nCopy>0 ).
>
> Thanks. Ticket now posted here:
>
>  http://www.sqlite.org/src/info/94c04eaadb
>
> Dan.
>
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: BUG: Async IO module works incorrectly with large database files

Dan Kennedy-4

On Oct 7, 2009, at 11:21 PM, Pavel Ivanov wrote:

> Hi, Dan!
>
> I've found another bug in async io module. It happens only
> occasionally in my application but I've found how to perfectly
> reproduce it. You need to:
> - not call sqlite3async_run();
> - open new not yet existing database;
> - execute PRAGMA journal_mode = PERSIST;
> - execute any 2 statements needing a transaction, e.g. some CREATE  
> TABLE

Thanks again. Now here:

  http://www.sqlite.org/src/info/897b96d49d

Dan.


>
> In this scenario (when connection uses async io module) last statement
> returns error SQLITE_CANTOPEN.
>
> I've investigated why it's happening. In first transaction when SQLite
> opens journal file async module determines that this opening should
> happen asynchronously and puts ASYNC_OPENEXCLUSIVE into the queue.
> Then at the end of transaction journal file is closed and everything
> is ok. When SQLite starts 2nd transaction it checks (inside
> hasHotJournal() ) for journal existence and async module returns "yes"
> because of ASYNC_OPENEXCLUSIVE in the event queue. Then SQLite tries
> to open journal for reading and async module determines that it should
> be done immediately. It try to physically open journal and fail
> because file does not exist yet. So SQLite decides that there is hot
> journal tries to open it again for read/write (inside
> sqlite3PageSharedLock() ) and fails again because file doesn't exist
> yet. And this error is already goes as a return code from
> sqlite3_step().
>
>
> Pavel
>
> On Wed, Oct 7, 2009 at 9:57 AM, Dan Kennedy <[hidden email]>  
> wrote:
>>
>> On Oct 7, 2009, at 6:42 PM, Pavel Ivanov wrote:
>>
>>> Hi!
>>>
>>> As anonymous users are not allowed to file bug reports on SQLite  
>>> site
>>> anymore so I'm sending it here.
>>>
>>> I've encountered segmentation fault at sqlite3async.c:715. The  
>>> problem
>>> is on sqlite3async.c:712:
>>>
>>>      nCopy = (int)MIN(pWrite->nByte-iBeginIn, iAmt-iBeginOut);
>>>
>>> My actual numbers: iAmt = 16, iBeginOut = 2147844072. After
>>> subtraction we're getting -2147844056 or FFFFFFFF7FFA8028 in hex  
>>> which
>>> after truncating to int gives 7FFA8028, a positive number. And it
>>> blows away the next check if( nCopy>0 ).
>>
>> Thanks. Ticket now posted here:
>>
>> http://www.sqlite.org/src/info/94c04eaadb
>>
>> Dan.
>>
>> _______________________________________________
>> sqlite-users mailing list
>> [hidden email]
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

_______________________________________________
sqlite-users mailing list
[hidden email]
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users