Claimed vulnerability in SQLite: Info or Intox?

classic Classic list List threaded Threaded
34 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Claimed vulnerability in SQLite: Info or Intox?

Dominique Devienne
https://blade.tencent.com/magellan/index_en.html

Sounds to me it more related to a "remote callable" program like Chrome,
than SQLite proper, but I'd like an official stance on SQLite itself please.

Thanks, --DD
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Claimed vulnerability in SQLite: Info or Intox?

Clemens Ladisch
Dominique Devienne wrote:
> I'd like an official stance on SQLite itself please.

<https://news.ycombinator.com/item?id=18685296>


Regards,
Clemens
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Claimed vulnerability in SQLite: Info or Intox?

Peter da Silva-2
I have to say I'm pretty boggled that Chrome allows hostile users to feed
code directly into an SQL interpreter that wasn't written from the ground
up to be secure. Secure interpreters are *hard* even when you're designing
them from scratch (see also, the whole history of web-based
vulnerabilities). That seems to be dancing with the screwup fairy to me.
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Claimed vulnerability in SQLite: Info or Intox?

Simon Slavin-3
On 18 Dec 2018, at 9:00pm, Peter da Silva <[hidden email]> wrote:

> I have to say I'm pretty boggled that Chrome allows hostile users to feed code directly into an SQL interpreter that wasn't written from the ground up to be secure.

Chrome has problems far more serious than that.  And one can do all sorts of nasty things in Chrome extensions.  It's difficult for the developers of Chrome to both prevent exploits by webmaster and extension writers, and also allow those people to do wonderful, entirely legitimate, things.  At the level of making an API call it's not possible for the called function to work out whether it's being used legitimately or not without a lot of extra processing which would make it so slow nobody would use it.

The tencent.com report is not entirely straightforward about precisely where the problem lies, and what an exploit could do.  It would be just as useful a report if it mentioned the problem in Chrome and avoided all mention of SQLite.  And implying that SQLite ever had a remote code execution problem is incorrect.

Simon.
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Claimed vulnerability in SQLite: Info or Intox?

Keith Medcalf
In reply to this post by Peter da Silva-2

Why shocked?  

Chrome allows direct execution of untrusted and likely malicious code that it gets over the network.  It is called JavaScript.  That a new method for execution of untrusted remote malicious code has been created is completely unsurprising since the whole point of Chrome is to permit local execution of remotely obtained and possibly malicious code.

---
The fact that there's a Highway to Hell but only a Stairway to Heaven says a lot about anticipated traffic volume.

>-----Original Message-----
>From: sqlite-users [mailto:sqlite-users-
>[hidden email]] On Behalf Of Peter da Silva
>Sent: Tuesday, 18 December, 2018 14:00
>To: SQLite mailing list
>Subject: Re: [sqlite] Claimed vulnerability in SQLite: Info or Intox?
>
>I have to say I'm pretty boggled that Chrome allows hostile users to
>feed
>code directly into an SQL interpreter that wasn't written from the
>ground
>up to be secure. Secure interpreters are *hard* even when you're
>designing
>them from scratch (see also, the whole history of web-based
>vulnerabilities). That seems to be dancing with the screwup fairy to
>me.
>_______________________________________________
>sqlite-users mailing list
>[hidden email]
>http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users



_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Claimed vulnerability in SQLite: Info or Intox?

Peter da Silva-2
Javascript was designed from the start to safely execute malicious code.
That doesn't mean it is safe, it just means it might be. There have been
all kinds of javascript-based exploits, after all.

But an interpreter that was not originally designed to be safe in the face
of malicious code? I can't understand the confusion in the mind that would
lead one to expect miracles of it. This is not a criticism of sqlite, by
any means. Safe languages are rare.
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Claimed vulnerability in SQLite: Info or Intox?

Nathan Green
In reply to this post by Simon Slavin-3
On Tue, Dec 18, 2018 at 3:14 PM Simon Slavin <[hidden email]> wrote:

> On 18 Dec 2018, at 9:00pm, Peter da Silva <[hidden email]> wrote:
>
> > I have to say I'm pretty boggled that Chrome allows hostile users to
> feed code directly into an SQL interpreter that wasn't written from the
> ground up to be secure.
>
> Chrome has problems far more serious than that.  And one can do all sorts
> of nasty things in Chrome extensions.  It's difficult for the developers of
> Chrome to both prevent exploits by webmaster and extension writers, and
> also allow those people to do wonderful, entirely legitimate, things.  At
> the level of making an API call it's not possible for the called function
> to work out whether it's being used legitimately or not without a lot of
> extra processing which would make it so slow nobody would use it.
>
> The tencent.com report is not entirely straightforward about precisely
> where the problem lies, and what an exploit could do.  It would be just as
> useful a report if it mentioned the problem in Chrome and avoided all
> mention of SQLite.  And implying that SQLite ever had a remote code
> execution problem is incorrect.
>
> Simon.
>
>
Except the problem isn't just in Chrome. Apparently, any system that allows
SQL injection is vulnerable. Since SQLite can be used as a file format to
transport application data (https://www.sqlite.org/appfileformat.html),
other applications might be also be vulnerable. It's not hard to conceive
of exploiting an application with a "restore from backup" feature. How
"remote" the RCE is depends on the application architecture. I'm thankful
that SQLite works really well for my use cases, and also that I have
sandboxed all of my code to run in unprivileged accounts.

Nathan
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Claimed vulnerability in SQLite: Info or Intox?

Peter da Silva-2
On Tue, Dec 18, 2018 at 3:49 PM Nathan Green <[hidden email]> wrote:

> Except the problem isn't just in Chrome. Apparently, any system that allows
> SQL injection is vulnerable.
>

That's kind of a tautology isn't it? Isn't there some kind of Godwin's Law
variant for XKCD 327?

I notice that the 12 points on https://www.sqlite.org/appfileformat.html
don't include "secure".

I mean, sure, we used to distribute software on Usenet as shell scripts
(look up "shar archive") but it's not 1984 any more.
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Claimed vulnerability in SQLite: Info or Intox?

Nathan Green
On Tue, Dec 18, 2018 at 4:00 PM Peter da Silva <[hidden email]> wrote:

> On Tue, Dec 18, 2018 at 3:49 PM Nathan Green <[hidden email]> wrote:
>
> > Except the problem isn't just in Chrome. Apparently, any system that
> allows
> > SQL injection is vulnerable.
> >
>
> That's kind of a tautology isn't it? Isn't there some kind of Godwin's Law
> variant for XKCD 327?
>
> I notice that the 12 points on https://www.sqlite.org/appfileformat.html
> don't include "secure".
>
> I mean, sure, we used to distribute software on Usenet as shell scripts
> (look up "shar archive") but it's not 1984 any more.
>
>
SQL injection in the generic sense isn't exactly RCE because SQL is
declarative. Arbitrarily messing up things in a database is not the same as
running any executable code that the database process might have permission
to execute.

Nathan
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Claimed vulnerability in SQLite: Info or Intox?

Richard Hipp-3
In reply to this post by Dominique Devienne
On 12/18/18, Dominique Devienne <[hidden email]> wrote:
> https://blade.tencent.com/magellan/index_en.html
>
> Sounds to me it more related to a "remote callable" program like Chrome,
> than SQLite proper, but I'd like an official stance on SQLite itself please.
>

There was a bug in FTS3 (not in the SQLite core) that allowed an SQL
Injection to escalate into a Remote Code Execution (RCE).  By making
malicious changes to the shadow tables that FTS3 uses and then running
FTS3 queries that used those tables, an integer overflow could cause a
buffer overrun, which if carefully managed might lead to an RCE.

This is only a problem for application that enable FTS3 (using the
SQLITE_ENABLE_FTS3 or SQLITE_ENABLE_FTS4 compile-time options) and
which allow potential attackers to run arbitrary SQL.  Contrary to
published reports, there are probably *not* millions of application
that meet those circumstances.  In fact, I only know of one:  Chrome.
Chrome has FTS3 enabled, and allows arbitrary SQL through the WebSQL
interface.  On the other hand, Chrome only allows this within a
sandbox, so as far as I know, the problem did not lead to an exploit
even in Chrome.  Perhaps there were other techniques used for escaping
the sandbox after executing the RCE against FTS3, but if not then the
attack against Chrome was also benign, as far as I know.

I believe that Safari also enables WebSQL, but unless I am mistaken,
Safari has disabled FTS3 and so it is probably not vulnerable.

The Fossil Version Control System can be configured to allow anonymous
users to enter SQL to generate bug-report summaries.  However, that
SQL is very restricted.  Only SELECT statements are allowed, and they
are only allowed to access specific tables.  So Fossil is not
vulnerable.

I am not aware of any other applications that deliberately run SQL
from anonymous sources

In any application that enables FTS3 and also has an SQL Injection
bugs, the magellan problem allows those SQL Injection bugs to escalate
to an RCE bug.   And so, the FTS3 bug makes a preexisting SQL
Injection bug worse, but does not introduce any new bugs.

The bug in FTS3 was fixed in version 3.26.0.  So if you are using
SQLite 3.26.0, then you are not vulnerable to this RCE even if you do
enable FTS3 and do allow SQL Injections.

Our policy with respect to bugs is to try to not only fix the bug, but
also fix the process.  Hence, when this issue was discovered in FTS3,
we tried to figure out what we could have done differently from the
beginning to have prevented this bug in the first place, and what we
could do differently moving forward to prevent future similar bugs.
As part of that analysis, we came up with the new
SQLITE_DBCONFIG_DEFENSIVE option.  Applications that do deliberately
allow anonymously-sourced SQL can enable SQLITE_DBCONFIG_DEFENSIVE to
prevent the SQL from deliberately corrupting the database file.  By
preventing corrupt database files, the attack surface is reduced.  Had
this option been available before Magellan, and had Chrome used it
(the latest Chrome releases *do* use it, I am told) then the Magellan
bug would have never happened.  But SQLITE_DBCONFIG_DEFENSIVE is not
the "fix" for the bug.  It is defense-in-depth against similar future
bugs.

We are hard at work on additional defense-in-depth measures now.  I do
not know of any other exploits against FTS3 or SQLite or any other
common SQLite extensions.  But we are working to make sure no new ones
are discovered.

--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Claimed vulnerability in SQLite: Info or Intox?

Keith Medcalf
In reply to this post by Nathan Green
On Tuesday, 18 December, 2018 14:50, Nathan Green <[hidden email]> wrote:

>Except the problem isn't just in Chrome. Apparently, any system that
>allows SQL injection is vulnerable. Since SQLite can be used as a file
>format to transport application data
>(https://www.sqlite.org/appfileformat.html),
>other applications might be also be vulnerable. It's not hard to
>conceive of exploiting an application with a "restore from backup"
>feature.

But this is not an SQLite3 problem.  This is a crap design of the application problem as the precondition "any system that allows SQL injection is vulnerable" is an absolute requirement.  If you prohibit that precondition, the issue cannot exist (though I suppose it would be possible for a malicious application to deliberately send malicious SQL, but again, this is an application problem, not an SQLite3 problem).

>How "remote" the RCE is depends on the application architecture. I'm
>thankful that SQLite works really well for my use cases, and also that I have
>sandboxed all of my code to run in unprivileged accounts.

Allāhu Akbar!  (Facing north cuz, well, some other people failed spherical geometry in grade school)

Hanging curtains (and closing them) on the bedroom windows to prevent the neighbours over the way from peeping in through the one-way glass and taking pictures of your naughty bits is the prudent thing to do.  You can giggle and "move along, nothing to see here" when the peeper over the way puts pictures of your neighbour in the local newspaper because he believed the glass vendors claim of the one-way-ness of the glazing and ignore all the kerfufle that ensues (with a nice glass of single malt and a bag of popcorn).

I have been told that it is "not fair" to implement proper security and protect oneself in advance and that one should follow the "Best Practice" and view the world though the fog of short-sightedness so induced in such a manner as to create the most "Oh Shit" moments possible and to avoid giggling when something that cannot possibly affect me affects my lesser prepared neighbours ...
 
---
The fact that there's a Highway to Hell but only a Stairway to Heaven says a lot about anticipated traffic volume.





_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Claimed vulnerability in SQLite: Info or Intox?

Dominique Devienne
In reply to this post by Richard Hipp-3
On Tue, Dec 18, 2018 at 11:13 PM Richard Hipp <[hidden email]> wrote:

> On 12/18/18, Dominique Devienne <[hidden email]> wrote:
> > https://blade.tencent.com/magellan/index_en.html
> >
> > Sounds to me it's more related to a "remote callable" program like
> Chrome,
> > than SQLite proper, but I'd like an official stance on SQLite itself
> please.
>
> There was a bug in FTS3 (not in the SQLite core) that allowed an SQL
> Injection to escalate into a Remote Code Execution (RCE).


Thanks. Good to know. Understanding how it happened is as important
as knowing this is fixed in SQLite already (in due time of course, when it's
"safe" to share information). Much appreciated.


> By making malicious changes to the shadow tables that FTS3 uses and then
> running
> FTS3 queries that used those tables, an integer overflow could cause a
> buffer overrun, which if carefully managed might lead to an RCE.

[... more details]

We are hard at work on additional defense-in-depth measures now.
>

Could there be a way to make shadow tables off-limit to arbitrary SQL?
By explicitly taking them as such, and creating them with a "secret"
associated
to them, so that preparing a statement against them also needs that
"secret"?
The goal being to restrict access to those tables to only the
vtable-related code
that is the "front-end" to those shadow tables. Only that code should have
direct
access to those tables, at least on the write-side of them.

I understand that they are persistent, so what would be relatively "easy"
to do
at runtime for the connection, becomes harder w/o changing the file-format
or
introducing new sqlite_... reserved tables older versions of SQLite would
ignore.

I hope you can think of ways to persist more meta-data about tables in
general,
be it that FKs should be enforced by default on them when new connections
are
created by them, or to persist the fact a table is a shadow tables
associated to a
given virtual-table module, such that only that module (somehow) can write
them.

My $0.02. --DD
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Claimed vulnerability in SQLite: Info or Intox?

Richard Hipp-3
>
> Could there be a way to make shadow tables off-limit to arbitrary SQL?

That is one of the things that the new SQLITE_DBCONFIG_DEFENSIVE
option does - it makes shadow tables read-only so that they cannot be
corrupted by SQL.

However, it is off by default, since some application make use of the
ability to write directly into shadow tables.  For example, when you
restore a database using the output of the ".dump" command, that
requires writing directly into shadow tables.


--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Claimed vulnerability in SQLite: Info or Intox?

Dominique Devienne
On Wed, Dec 19, 2018 at 11:14 AM Richard Hipp <[hidden email]> wrote:

> > Could there be a way to make shadow tables off-limit to arbitrary SQL?
>
> That is one of the things that the new SQLITE_DBCONFIG_DEFENSIVE
> option does - it makes shadow tables read-only so that they cannot be
> corrupted by SQL.
>

May I please know how SQLite knows a table is a shadow table?
This is not something I've run across yet. Good to know BTW, of course.
Thanks.

Not knowing how this is done/known, I wonder whether this applies to other
"bits" of information about tables, notably "custom" ones from a given
application.
(answers seems to be no, given what's below).

Ah, I googled SQLITE_DBCONFIG_DEFENSIVE and found the 3.26 release notes,
which pointed me to [1] and then [2]. I had missed [2]. Sending anyway,
might be
useful to someone else too.

[1]
https://www.sqlite.org/draft/c3ref/c_dbconfig_defensive.html#sqlitedbconfigdefensive
[2] https://www.sqlite.org/draft/vtab.html#xshadowname


> However, it is off by default, since some application make use of the
> ability to write directly into shadow tables.  For example, when you
> restore a database using the output of the ".dump" command, that
> requires writing directly into shadow tables.
>

Good point. Didn't think of that. Thanks again, --DD
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Claimed vulnerability in SQLite: Info or Intox?

Jens Alfke-2
In reply to this post by Richard Hipp-3


> On Dec 18, 2018, at 2:13 PM, Richard Hipp <[hidden email]> wrote:
>
> I am not aware of any other applications that deliberately run SQL
> from anonymous sources

In applications that use SQLite databases as a file format, couldn’t a malicious document be created that uses a trigger to run SQL that triggers an exploit when the document/database is edited? In other words:

1. Mallory creates or obtains some innocuous document.
2. Mallory uses something like the ’sqlite3’ tool to open the database and execute a CREATE TRIGGER statement whose trigger SQL exploits a vulnerability to do something nasty like remote code execution.
3. Mallory passes the document to Alice.
4. Alice opens the document and makes a change that causes SQLite to update a table in a way that activates the trigger.
5. The malicious SQL runs in the application process on Alice’s computer and does its business.

—Jens
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Claimed vulnerability in SQLite: Info or Intox?

Simon Slavin-3
On 19 Dec 2018, at 6:19pm, Jens Alfke <[hidden email]> wrote:

> 2. Mallory uses something like the ’sqlite3’ tool to open the database and execute a CREATE TRIGGER statement whose trigger SQL exploits a vulnerability to do something nasty like remote code execution.

I'm not sure how you would do that purely inside a trigger.  You can't just specially craft a BLOB with bad content.  I think it would need participation from the software making the call to the API.

Simon.
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Claimed vulnerability in SQLite: Info or Intox?

Peter da Silva-2
sqlite is not immune to wandering through bad pointers, because code
coverage tests don't test for malicious data... I found a null pointer
crash in sqlite earlier this year. I could see Mallory crafting a database
that had carefully corrupted structures in it that smashed the stack.
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Claimed vulnerability in SQLite: Info or Intox?

Jens Alfke-2


> On Dec 19, 2018, at 4:03 PM, Peter da Silva <[hidden email]> wrote:
>
> sqlite is not immune to wandering through bad pointers, because code
> coverage tests don't test for malicious data..

Fuzz testing does, though [implicitly].
https://www.sqlite.org/testing.html#sql_fuzz_using_the_american_fuzzy_lop_fuzzer

—Jens
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Claimed vulnerability in SQLite: Info or Intox?

Jens Alfke-2
In reply to this post by Simon Slavin-3


> On Dec 19, 2018, at 10:32 AM, Simon Slavin <[hidden email]> wrote:
>
> I'm not sure how you would do that purely inside a trigger.  You can't just specially craft a BLOB with bad content.  I think it would need participation from the software making the call to the API.

Can’t you put [nearly] any SQL statement in the body of a trigger? Including one with an x’…’ blob literal?

—Jens
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Claimed vulnerability in SQLite: Info or Intox?

Peter da Silva-2
In reply to this post by Jens Alfke-2
Fuzz testing would be extremely unlikely to have caught the original
attack. Nor would fuzz testing on input be likely to hit all corrupt
database attacks. Fuzz testing using fuzzed corrupted databases might.

On Thu., 20 Dec. 2018, 11:26 Jens Alfke <[hidden email] wrote:

>
>
> > On Dec 19, 2018, at 4:03 PM, Peter da Silva <[hidden email]> wrote:
> >
> > sqlite is not immune to wandering through bad pointers, because code
> > coverage tests don't test for malicious data..
>
> Fuzz testing does, though [implicitly].
>
> https://www.sqlite.org/testing.html#sql_fuzz_using_the_american_fuzzy_lop_fuzzer
>
> —Jens
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
12