sqlite3_expanded_sql is reading freed heap memory

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

sqlite3_expanded_sql is reading freed heap memory

Jens Alfke-2
I've got a repeatable situation in my library's unit tests wherein the Clang Address Sanitizer catches sqlite3_expanded_sql() reading from a freed heap block. This is with SQLite 3.22 on MacOS 10.13.3.

The background: I've added some code to my library to log warnings if the database is closed (via sqlite3_close_v2) while there are still open statements. So just before calling sqlite3_close_v2, I'm using sqlite3_next_stmt to iterate them, like so:

    sqlite3_stmt *stmt = nullptr;
    while (nullptr != (stmt = sqlite3_next_stmt(mpSQLite, stmt))) {
        callback(sqlite3_expanded_sql(stmt), sqlite3_stmt_busy(stmt));
    }

In one of my unit tests, the call to sqlite3_expanded_sql() results in a breakpoint by the address sanitizer where it reports a read of a freed heap block, at sqlite3.c:79051:
        for(i=0; i<nOut; i++){
          sqlite3XPrintf(&out, "%02x", pVar->z[i]&0xff); // ⟵ here; the variable 'i' is 0.
        }
This is writing a blob value as hex, so it looks like the actual blob data was already freed (by fts3SegWriterFree, according to the dump.)

The string already accumulated into `out` is:
        REPLACE INTO 'main'.'kv_default::byStreet_segdir' VALUES(0,0,0,0,'0 1671',x'
where `kv_default::byStreet` is the name of an FTS4 table.

Let me know if there's more information I can provide.

—Jens

=================================================================
==5150==ERROR: AddressSanitizer: heap-use-after-free on address 0x621002808d00 at pc 0x0001025ef097 bp 0x7ffeefbfcc30 sp 0x7ffeefbfcc28
READ of size 1 at 0x621002808d00 thread T0
    #0 0x1025ef096 in sqlite3VdbeExpandSql sqlite3.c:79051
    #1 0x1025edc9f in sqlite3_expanded_sql sqlite3.c:78609
    #2 ...

0x621002808d00 is located 0 bytes inside of 4064-byte region [0x621002808d00,0x621002809ce0)
freed by thread T0 here:
    #0 0x100caefa4 in __sanitizer_mz_free (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x59fa4)
    #1 0x10287cdcb in sqlite3MemFree sqlite3.c:21428
    #2 0x1025cc9c2 in sqlite3_free sqlite3.c:25151
    #3 0x1028e1cf4 in fts3SegWriterFree sqlite3.c:159856
    #4 0x1028de3b1 in fts3SegmentMerge sqlite3.c:160727
    #5 0x1028fa66e in sqlite3Fts3PendingTermsFlush sqlite3.c:160741
    #6 0x1028a6174 in fts3SyncMethod sqlite3.c:150819
    #7 0x10268a702 in sqlite3VtabSync sqlite3.c:127211
    #8 0x10268553a in vdbeCommit sqlite3.c:74426
    #9 0x102682f80 in sqlite3VdbeHalt sqlite3.c:74890
    #10 0x1026a6aab in sqlite3VdbeExec sqlite3.c:82236
    #11 0x1025e8c41 in sqlite3Step sqlite3.c:77535
    #12 0x1025e7b79 in sqlite3_step sqlite3.c:77598
    #13 0x1025f8c3f in sqlite3_exec sqlite3.c:112187
    #14 ...

previously allocated by thread T0 here:
    #0 0x100caea3c in __sanitizer_mz_malloc (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x59a3c)
    #1 0x7fff716b6200 in malloc_zone_malloc (libsystem_malloc.dylib:x86_64+0x2200)
    #2 0x10287cd7a in sqlite3MemMalloc sqlite3.c:21396
    #3 0x102615efb in mallocWithAlarm sqlite3.c:25040
    #4 0x1025cc812 in sqlite3Malloc sqlite3.c:25070
    #5 0x1025cc6a5 in sqlite3_malloc sqlite3.c:25088
    #6 0x1028df1b5 in fts3SegWriterAdd sqlite3.c:159702
    #7 0x1028de046 in fts3SegmentMerge sqlite3.c:160705
    #8 0x1028fa66e in sqlite3Fts3PendingTermsFlush sqlite3.c:160741
    #9 0x1028a6174 in fts3SyncMethod sqlite3.c:150819
    #10 0x10268a702 in sqlite3VtabSync sqlite3.c:127211
    #11 0x10268553a in vdbeCommit sqlite3.c:74426
    #12 0x102682f80 in sqlite3VdbeHalt sqlite3.c:74890
    #13 0x1026a6aab in sqlite3VdbeExec sqlite3.c:82236
    #14 0x1025e8c41 in sqlite3Step sqlite3.c:77535
    #15 0x1025e7b79 in sqlite3_step sqlite3.c:77598
    #16 0x1025f8c3f in sqlite3_exec sqlite3.c:112187
    #17 ...
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: sqlite3_expanded_sql is reading freed heap memory

J Decker
 can you use sqlite3_sql  instead; it won't be complete information... but
ya, the time that expanded_sql is valid is really only while the bound
parameters are still valid.   (could re-bind parameters when done I guess)


On Tue, Feb 6, 2018 at 1:07 PM, Jens Alfke <[hidden email]> wrote:

> I've got a repeatable situation in my library's unit tests wherein the
> Clang Address Sanitizer catches sqlite3_expanded_sql() reading from a freed
> heap block. This is with SQLite 3.22 on MacOS 10.13.3.
>
> The background: I've added some code to my library to log warnings if the
> database is closed (via sqlite3_close_v2) while there are still open
> statements. So just before calling sqlite3_close_v2, I'm using
> sqlite3_next_stmt to iterate them, like so:
>
>     sqlite3_stmt *stmt = nullptr;
>     while (nullptr != (stmt = sqlite3_next_stmt(mpSQLite, stmt))) {
>         callback(sqlite3_expanded_sql(stmt), sqlite3_stmt_busy(stmt));
>     }
>
> In one of my unit tests, the call to sqlite3_expanded_sql() results in a
> breakpoint by the address sanitizer where it reports a read of a freed heap
> block, at sqlite3.c:79051:
>         for(i=0; i<nOut; i++){
>           sqlite3XPrintf(&out, "%02x", pVar->z[i]&0xff);                //
> ⟵ here; the variable 'i' is 0.
>         }
> This is writing a blob value as hex, so it looks like the actual blob data
> was already freed (by fts3SegWriterFree, according to the dump.)
>
> The string already accumulated into `out` is:
>         REPLACE INTO 'main'.'kv_default::byStreet_segdir'
> VALUES(0,0,0,0,'0 1671',x'
> where `kv_default::byStreet` is the name of an FTS4 table.
>
> Let me know if there's more information I can provide.
>
> —Jens
>
> =================================================================
> ==5150==ERROR: AddressSanitizer: heap-use-after-free on address
> 0x621002808d00 at pc 0x0001025ef097 bp 0x7ffeefbfcc30 sp 0x7ffeefbfcc28
> READ of size 1 at 0x621002808d00 thread T0
>     #0 0x1025ef096 in sqlite3VdbeExpandSql sqlite3.c:79051
>     #1 0x1025edc9f in sqlite3_expanded_sql sqlite3.c:78609
>     #2 ...
>
> 0x621002808d00 is located 0 bytes inside of 4064-byte region
> [0x621002808d00,0x621002809ce0)
> freed by thread T0 here:
>     #0 0x100caefa4 in __sanitizer_mz_free (libclang_rt.asan_osx_dynamic.
> dylib:x86_64h+0x59fa4)
>     #1 0x10287cdcb in sqlite3MemFree sqlite3.c:21428
>     #2 0x1025cc9c2 in sqlite3_free sqlite3.c:25151
>     #3 0x1028e1cf4 in fts3SegWriterFree sqlite3.c:159856
>     #4 0x1028de3b1 in fts3SegmentMerge sqlite3.c:160727
>     #5 0x1028fa66e in sqlite3Fts3PendingTermsFlush sqlite3.c:160741
>     #6 0x1028a6174 in fts3SyncMethod sqlite3.c:150819
>     #7 0x10268a702 in sqlite3VtabSync sqlite3.c:127211
>     #8 0x10268553a in vdbeCommit sqlite3.c:74426
>     #9 0x102682f80 in sqlite3VdbeHalt sqlite3.c:74890
>     #10 0x1026a6aab in sqlite3VdbeExec sqlite3.c:82236
>     #11 0x1025e8c41 in sqlite3Step sqlite3.c:77535
>     #12 0x1025e7b79 in sqlite3_step sqlite3.c:77598
>     #13 0x1025f8c3f in sqlite3_exec sqlite3.c:112187
>     #14 ...
>
> previously allocated by thread T0 here:
>     #0 0x100caea3c in __sanitizer_mz_malloc (libclang_rt.asan_osx_dynamic.
> dylib:x86_64h+0x59a3c)
>     #1 0x7fff716b6200 in malloc_zone_malloc (libsystem_malloc.dylib:x86_
> 64+0x2200)
>     #2 0x10287cd7a in sqlite3MemMalloc sqlite3.c:21396
>     #3 0x102615efb in mallocWithAlarm sqlite3.c:25040
>     #4 0x1025cc812 in sqlite3Malloc sqlite3.c:25070
>     #5 0x1025cc6a5 in sqlite3_malloc sqlite3.c:25088
>     #6 0x1028df1b5 in fts3SegWriterAdd sqlite3.c:159702
>     #7 0x1028de046 in fts3SegmentMerge sqlite3.c:160705
>     #8 0x1028fa66e in sqlite3Fts3PendingTermsFlush sqlite3.c:160741
>     #9 0x1028a6174 in fts3SyncMethod sqlite3.c:150819
>     #10 0x10268a702 in sqlite3VtabSync sqlite3.c:127211
>     #11 0x10268553a in vdbeCommit sqlite3.c:74426
>     #12 0x102682f80 in sqlite3VdbeHalt sqlite3.c:74890
>     #13 0x1026a6aab in sqlite3VdbeExec sqlite3.c:82236
>     #14 0x1025e8c41 in sqlite3Step sqlite3.c:77535
>     #15 0x1025e7b79 in sqlite3_step sqlite3.c:77598
>     #16 0x1025f8c3f in sqlite3_exec sqlite3.c:112187
>     #17 ...
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: sqlite3_expanded_sql is reading freed heap memory

Richard Hipp-3
In reply to this post by Jens Alfke-2
On 2/6/18, Jens Alfke <[hidden email]> wrote:

> I've got a repeatable situation in my library's unit tests wherein the Clang
> Address Sanitizer catches sqlite3_expanded_sql() reading from a freed heap
> block. This is with SQLite 3.22 on MacOS 10.13.3.
>
> The background: I've added some code to my library to log warnings if the
> database is closed (via sqlite3_close_v2) while there are still open
> statements. So just before calling sqlite3_close_v2, I'm using
> sqlite3_next_stmt to iterate them, like so:
>
>     sqlite3_stmt *stmt = nullptr;
>     while (nullptr != (stmt = sqlite3_next_stmt(mpSQLite, stmt))) {
>         callback(sqlite3_expanded_sql(stmt), sqlite3_stmt_busy(stmt));
>     }
>
> In one of my unit tests, the call to sqlite3_expanded_sql() results in a
> breakpoint by the address sanitizer where it reports a read of a freed heap
> block, at sqlite3.c:79051:
>         for(i=0; i<nOut; i++){
>           sqlite3XPrintf(&out, "%02x", pVar->z[i]&0xff); // ⟵ here; the
> variable 'i' is 0.
>         }
> This is writing a blob value as hex, so it looks like the actual blob data
> was already freed (by fts3SegWriterFree, according to the dump.)
>
> The string already accumulated into `out` is:
> REPLACE INTO 'main'.'kv_default::byStreet_segdir' VALUES(0,0,0,0,'0
> 1671',x'
> where `kv_default::byStreet` is the name of an FTS4 table.
>
> Let me know if there's more information I can provide.

The line numbers in your ASAN output below seem incorrect.  The
SHA3-256 hash for sqlite3.c version 3.22.0 should be
206df47ebc49cd1710ac0dd716ce5de5854826536993f4feab7a49d136b85069.
What is the hash on the "sqlite3.c" file you are using?  Can you post
the text of the sqlite3.c file that you are using?


>
> —Jens
>
> =================================================================
> ==5150==ERROR: AddressSanitizer: heap-use-after-free on address
> 0x621002808d00 at pc 0x0001025ef097 bp 0x7ffeefbfcc30 sp 0x7ffeefbfcc28
> READ of size 1 at 0x621002808d00 thread T0
>     #0 0x1025ef096 in sqlite3VdbeExpandSql sqlite3.c:79051
>     #1 0x1025edc9f in sqlite3_expanded_sql sqlite3.c:78609
>     #2 ...
>
> 0x621002808d00 is located 0 bytes inside of 4064-byte region
> [0x621002808d00,0x621002809ce0)
> freed by thread T0 here:
>     #0 0x100caefa4 in __sanitizer_mz_free
> (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x59fa4)
>     #1 0x10287cdcb in sqlite3MemFree sqlite3.c:21428
>     #2 0x1025cc9c2 in sqlite3_free sqlite3.c:25151
>     #3 0x1028e1cf4 in fts3SegWriterFree sqlite3.c:159856
>     #4 0x1028de3b1 in fts3SegmentMerge sqlite3.c:160727
>     #5 0x1028fa66e in sqlite3Fts3PendingTermsFlush sqlite3.c:160741
>     #6 0x1028a6174 in fts3SyncMethod sqlite3.c:150819
>     #7 0x10268a702 in sqlite3VtabSync sqlite3.c:127211
>     #8 0x10268553a in vdbeCommit sqlite3.c:74426
>     #9 0x102682f80 in sqlite3VdbeHalt sqlite3.c:74890
>     #10 0x1026a6aab in sqlite3VdbeExec sqlite3.c:82236
>     #11 0x1025e8c41 in sqlite3Step sqlite3.c:77535
>     #12 0x1025e7b79 in sqlite3_step sqlite3.c:77598
>     #13 0x1025f8c3f in sqlite3_exec sqlite3.c:112187
>     #14 ...
>
> previously allocated by thread T0 here:
>     #0 0x100caea3c in __sanitizer_mz_malloc
> (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x59a3c)
>     #1 0x7fff716b6200 in malloc_zone_malloc
> (libsystem_malloc.dylib:x86_64+0x2200)
>     #2 0x10287cd7a in sqlite3MemMalloc sqlite3.c:21396
>     #3 0x102615efb in mallocWithAlarm sqlite3.c:25040
>     #4 0x1025cc812 in sqlite3Malloc sqlite3.c:25070
>     #5 0x1025cc6a5 in sqlite3_malloc sqlite3.c:25088
>     #6 0x1028df1b5 in fts3SegWriterAdd sqlite3.c:159702
>     #7 0x1028de046 in fts3SegmentMerge sqlite3.c:160705
>     #8 0x1028fa66e in sqlite3Fts3PendingTermsFlush sqlite3.c:160741
>     #9 0x1028a6174 in fts3SyncMethod sqlite3.c:150819
>     #10 0x10268a702 in sqlite3VtabSync sqlite3.c:127211
>     #11 0x10268553a in vdbeCommit sqlite3.c:74426
>     #12 0x102682f80 in sqlite3VdbeHalt sqlite3.c:74890
>     #13 0x1026a6aab in sqlite3VdbeExec sqlite3.c:82236
>     #14 0x1025e8c41 in sqlite3Step sqlite3.c:77535
>     #15 0x1025e7b79 in sqlite3_step sqlite3.c:77598
>     #16 0x1025f8c3f in sqlite3_exec sqlite3.c:112187
>     #17 ...
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>


--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: sqlite3_expanded_sql is reading freed heap memory

Jens Alfke-2


> On Feb 6, 2018, at 6:20 PM, Richard Hipp <[hidden email]> wrote:
>
> The line numbers in your ASAN output below seem incorrect.  The
> SHA3-256 hash for sqlite3.c version 3.22.0 should be
> 206df47ebc49cd1710ac0dd716ce5de5854826536993f4feab7a49d136b85069.
> What is the hash on the "sqlite3.c" file you are using?  Can you post
> the text of the sqlite3.c file that you are using?

I'm sorry — the version is actually 3.21, not 3.22:

#define SQLITE_VERSION        "3.21.0"
#define SQLITE_VERSION_NUMBER 3021000
#define SQLITE_SOURCE_ID      "2017-10-24 18:55:49 1a584e499906b5c87ec7d43d4abce641fdf017c42125b083109bc77c4de48827"

—Jens

_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users