SQLite bug: Malformed UTF16 leads to a 2-byte buffer overread

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

SQLite bug: Malformed UTF16 leads to a 2-byte buffer overread

Kelvin Jiang
Hi there,

I'm using SQlite 3.12.1 on Android device and we found this bug on most
Android devices.

The problem can be reproduced by following procedures:

1. Prepare a malformed Java String, like "123😂", then cut it right from
the middle of the surrogates, like "123😂".substring(0, 4), and append
other strings. The code (in Java) may result in:

String value = "123😂".substring(0, 4)+"\", \"another_key\":\"by_plus\"";

2. and then put it in SQLite db:

SQLiteOpenHelper sqliteHelper = new {some implement class};

ContentValues values = new ContentValues();
values.put("other_key", "other_values");
values.put("some_key", value);
values.put("other_key2", "other_values2");
sqliteHelper.getWritableDatabase().replace("some_table_name", null, values);

And after writing, we can see the wrong value, like:

123🐢, "another_key":"by_plus"

in this record.


Simply put, from the 4th byte to the 5th byte, we put data like
 11011000 00111101 00000000 00100010
in the db, but result in :
11110000 10011111 10010000 10100010
. We get different data before and after write into the db.

And I found another similar bug report before in
https://www.sqlite.org/src/tktview?name=3fe897352e , which says that
problem has been fixed, but this problem is still able to be reproduced on
version 3.12.1. Maybe we can learn from it to fix this problem.

Best,

Kelvin
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: SQLite bug: Malformed UTF16 leads to a 2-byte buffer overread

Richard Hipp-3
On 9/29/18, Kelvin Jiang <[hidden email]> wrote:
> Hi there,
>
> I'm using SQlite 3.12.1 on Android device and we found this bug on most
> Android devices.

Maybe this is a bug in Android or in the SQLite interface that are
built into Android.  But SQLite itself seems to be working correct.
Or, at least, I am unable to get it to malfunction.

I wrote the test program seen at

    https://sqlite.org/temp/utf16-test1.c

And ran that test program using 3.12.2 and the latest trunk version of
SQLite.  I ran the program with various arguments, and I ran it under
valgrind to confirm that there were no buffer overruns.  In every
case, the malformed UTF16 string value returned by SQLite was exactly
the same string value that was input to SQLite.  This demonstrates
that even if you hand a malformed UTF16 string to SQLite, SQLite will
store that string for you (exactly as you handed it to SQLite) and
will return it back to you as the exact same sequence of bytes that
you originally handed it.  This is the contract that SQLite provides.

Hence, I cannot reproduce any problems like this in the core SQLite.

If I have misunderstood the nature of the problem you are having,
please clarify and I will look again.


>
> The problem can be reproduced by following procedures:
>
> 1. Prepare a malformed Java String, like "123😂", then cut it right from
> the middle of the surrogates, like "123😂".substring(0, 4), and append
> other strings. The code (in Java) may result in:
>
> String value = "123😂".substring(0, 4)+"\", \"another_key\":\"by_plus\"";
>
> 2. and then put it in SQLite db:
>
> SQLiteOpenHelper sqliteHelper = new {some implement class};
>
> ContentValues values = new ContentValues();
> values.put("other_key", "other_values");
> values.put("some_key", value);
> values.put("other_key2", "other_values2");
> sqliteHelper.getWritableDatabase().replace("some_table_name", null, values);
>
> And after writing, we can see the wrong value, like:
>
> 123🐢, "another_key":"by_plus"
>
> in this record.
>
>
> Simply put, from the 4th byte to the 5th byte, we put data like
>  11011000 00111101 00000000 00100010
> in the db, but result in :
> 11110000 10011111 10010000 10100010
> . We get different data before and after write into the db.
>
> And I found another similar bug report before in
> https://www.sqlite.org/src/tktview?name=3fe897352e , which says that
> problem has been fixed, but this problem is still able to be reproduced on
> version 3.12.1. Maybe we can learn from it to fix this problem.
>
> Best,
>
> Kelvin
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>


--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users