sqlite 3.20.1: Core dump when running query with json functions in subquery

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

sqlite 3.20.1: Core dump when running query with json functions in subquery

Martin Thierer
I'm consistently getting a core dump for a query that has json
functions in a subquery. The same functions work fine in other
queries.

I came up with this small example which triggers the crash for me when
executed from the sqlite3 shell for a new, empty database:

CREATE TABLE dataset (
  doc TEXT
);
INSERT INTO dataset VALUES ('{"field1": [{"value": "val1"}, {"value":
"val2"}]');
INSERT INTO dataset VALUES ('{"field1": [{"value": "val3"}, {"value":
"val2"}]');
INSERT INTO dataset VALUES ('{"field1": [{"value": "val4"}, {"value":
"val1"}]');
SELECT * FROM dataset WHERE 'val1' IN
  (SELECT json_extract(json_each.value, '$.value')
    FROM json_each(json_extract(dataset.doc, '$.field')));

As I got syntax errors when copy-pasting even from my own email draft,
I also put it here:

https://gist.github.com/thierer/f4367232f58a452ff960fe61c3250fbe

This is with sqlite 3.20.1 on Arch Linux x86_64.

Thanks! (both for looking into this issue and sqlite in general...)

Martin
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: sqlite 3.20.1: Core dump when running query with json functions in subquery

Richard Hipp-3
On 9/3/17, Martin Thierer <[hidden email]> wrote:
> I'm consistently getting a core dump for a query that has json
> functions in a subquery. The same functions work fine in other
> queries.

Thanks for the concise bug report.  A ticket is here:
https://sqlite.org/src/tktview/b899b6042f97f

I'm away from the office at a database conference (in Tokyo - all
y'all Tokyo-based SQLite users come say "hello" to me at
http://www.db-tech-showcase.com/dbts/tokyo) so I have limited cycles
available to work on this.  Dan or Joe might be able to fix this when
they get to the office in about 12 hours if I can't get to it before
then.

--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: sqlite 3.20.1: Core dump when running query with json functions in subquery

Richard Hipp-3
A proposed fix for this problem is now on trunk.


On 9/3/17, Richard Hipp <[hidden email]> wrote:

> On 9/3/17, Martin Thierer <[hidden email]> wrote:
>> I'm consistently getting a core dump for a query that has json
>> functions in a subquery. The same functions work fine in other
>> queries.
>
> Thanks for the concise bug report.  A ticket is here:
> https://sqlite.org/src/tktview/b899b6042f97f
>
> I'm away from the office at a database conference (in Tokyo - all
> y'all Tokyo-based SQLite users come say "hello" to me at
> http://www.db-tech-showcase.com/dbts/tokyo) so I have limited cycles
> available to work on this.  Dan or Joe might be able to fix this when
> they get to the office in about 12 hours if I can't get to it before
> then.
>
> --
> D. Richard Hipp
> [hidden email]
>


--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: sqlite 3.20.1: Core dump when running query with json functions in subquery

Bart Smissaert
Trying this out, so I edited the source and indeed now there is no crash.
I get however a message:

malformed JSON

This is the dump of that testing table, dataset:

doc
{"field1": [{"value": "val1"}, {"value": "val2"}]
{"field1": [{"value": "val3"}, {"value": "val2"}]
{"field1": [{"value": "val4"}, {"value": "val1"}]

Is there anything wrong with this JSON data?


RBS

On Mon, Sep 4, 2017 at 1:35 AM, Richard Hipp <[hidden email]> wrote:

> A proposed fix for this problem is now on trunk.
>
>
> On 9/3/17, Richard Hipp <[hidden email]> wrote:
> > On 9/3/17, Martin Thierer <[hidden email]> wrote:
> >> I'm consistently getting a core dump for a query that has json
> >> functions in a subquery. The same functions work fine in other
> >> queries.
> >
> > Thanks for the concise bug report.  A ticket is here:
> > https://sqlite.org/src/tktview/b899b6042f97f
> >
> > I'm away from the office at a database conference (in Tokyo - all
> > y'all Tokyo-based SQLite users come say "hello" to me at
> > http://www.db-tech-showcase.com/dbts/tokyo) so I have limited cycles
> > available to work on this.  Dan or Joe might be able to fix this when
> > they get to the office in about 12 hours if I can't get to it before
> > then.
> >
> > --
> > D. Richard Hipp
> > [hidden email]
> >
>
>
> --
> D. Richard Hipp
> [hidden email]
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: sqlite 3.20.1: Core dump when running query with json functions in subquery

Rowan Worth-2
On 4 September 2017 at 15:37, Bart Smissaert <[hidden email]>
wrote:

> Trying this out, so I edited the source and indeed now there is no crash.
> I get however a message:
>
> malformed JSON
>
> This is the dump of that testing table, dataset:
>
> doc
> {"field1": [{"value": "val1"}, {"value": "val2"}]
> {"field1": [{"value": "val3"}, {"value": "val2"}]
> {"field1": [{"value": "val4"}, {"value": "val1"}]
>
> Is there anything wrong with this JSON data?
>

Yes, it's malformed.


... :P
Specifically, there's no closing curly brace for the top-level object.

{"field1":
  [
    {"value": "val1"},
    {"value": "val2"}
  ]
*** MISSING CURLY BRACE HERE ***

-Rowan
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: sqlite 3.20.1: Core dump when running query with json functions in subquery

Richard Hipp-3
In reply to this post by Bart Smissaert
On 9/4/17, Bart Smissaert <[hidden email]> wrote:
>
> malformed JSON
>

SQLite should never segfault, regardless of whether or not the JSON is
malformed.

The ticket (https://sqlite.org/src/info/b899b6042f97f5) shows a
modified test case that uses no JSON at all.

--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: sqlite 3.20.1: Core dump when running query with json functions in subquery

Bart Smissaert
In reply to this post by Rowan Worth-2
Thanks, this runs OK now.
I am not familiar with JSON, but was just trying this out and see if I
could make it work in SQLite.

RBS



On Mon, Sep 4, 2017 at 8:40 AM, Rowan Worth <[hidden email]> wrote:

> On 4 September 2017 at 15:37, Bart Smissaert <[hidden email]>
> wrote:
>
> > Trying this out, so I edited the source and indeed now there is no crash.
> > I get however a message:
> >
> > malformed JSON
> >
> > This is the dump of that testing table, dataset:
> >
> > doc
> > {"field1": [{"value": "val1"}, {"value": "val2"}]
> > {"field1": [{"value": "val3"}, {"value": "val2"}]
> > {"field1": [{"value": "val4"}, {"value": "val1"}]
> >
> > Is there anything wrong with this JSON data?
> >
>
> Yes, it's malformed.
>
>
> ... :P
> Specifically, there's no closing curly brace for the top-level object.
>
> {"field1":
>   [
>     {"value": "val1"},
>     {"value": "val2"}
>   ]
> *** MISSING CURLY BRACE HERE ***
>
> -Rowan
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: sqlite 3.20.1: Core dump when running query with json functions in subquery

Bart Smissaert
In reply to this post by Richard Hipp-3
Thanks that runs fine now.

RBS

On Mon, Sep 4, 2017 at 8:45 AM, Richard Hipp <[hidden email]> wrote:

> On 9/4/17, Bart Smissaert <[hidden email]> wrote:
> >
> > malformed JSON
> >
>
> SQLite should never segfault, regardless of whether or not the JSON is
> malformed.
>
> The ticket (https://sqlite.org/src/info/b899b6042f97f5) shows a
> modified test case that uses no JSON at all.
>
> --
> D. Richard Hipp
> [hidden email]
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users