Unexpected results from <, <= and BETWEEN with a CTE in 3.18.0

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Unexpected results from <, <= and BETWEEN with a CTE in 3.18.0

Keith Maxwell
sqlite-users,

Can anyone please explain the results of the query below? I have tried
create a very simple test case. In brief returning 9 rows each starting
with an integer between zero and nine. Then check if it less than or equal
to five, between zero and five and then less than six. I expect all of
these comparisons to have the same result. I can't understand the results,
and '.explain' isn't any help.

    SQLite version 3.18.0 2017-03-28 18:48:43
    Enter ".help" for usage hints.
    Connected to a transient in-memory database.
    Use ".open FILENAME" to reopen on a persistent database.
    sqlite>
    sqlite> WITH
    ...>     t1(X) AS (
    ...>         SELECT 1
    ...>         UNION ALL
    ...>         SELECT X+1 FROM t1
    ...>         LIMIT 9
    ...>         )
    ...>     ,t2(Y) AS (
    ...>         SELECT abs(random() % 10) FROM t1
    ...>         )
    ...> SELECT Y, Y <= 5, Y BETWEEN 0 AND 5, Y < 6 FROM t2;
    7|1|1|0
    3|1|0|0
    2|1|1|1
    2|0|0|1
    0|1|1|1
    3|1|1|0
    5|1|1|0
    6|1|1|0
    0|1|1|1
    sqlite>

Thanks in advance for your assistance.

Kind Regards

Keith Maxwell
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Unexpected results from <, <= and BETWEEN with a CTE in 3.18.0

Hick Gunter
My guess ist hat random() is being called once for each expression containing Y. Using constant values from a regular table works as expected. It is exactly the bytecode output by EXPLAIN <query> that could help to determine what happens, but unfortunately you did not post it.

asql> insert into t values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9);
asql> select y, y<=5, y between 0 and 5, y<6 from t;
y           y<=5  y between 0 and 5  y<6
----------  ----  -----------------  ---
0           1     1                  1
1           1     1                  1
2           1     1                  1
3           1     1                  1
4           1     1                  1
5           1     1                  1
6           0     0                  0
7           0     0                  0
8           0     0                  0
9           0     0                  0

-----Urspr√ľngliche Nachricht-----
Von: sqlite-users [mailto:[hidden email]] Im Auftrag von Keith Maxwell
Gesendet: Dienstag, 11. April 2017 10:30
An: SQLite mailing list <[hidden email]>
Betreff: [sqlite] Unexpected results from <, <= and BETWEEN with a CTE in 3.18.0

sqlite-users,

Can anyone please explain the results of the query below? I have tried create a very simple test case. In brief returning 9 rows each starting with an integer between zero and nine. Then check if it less than or equal to five, between zero and five and then less than six. I expect all of these comparisons to have the same result. I can't understand the results, and '.explain' isn't any help.

    SQLite version 3.18.0 2017-03-28 18:48:43
    Enter ".help" for usage hints.
    Connected to a transient in-memory database.
    Use ".open FILENAME" to reopen on a persistent database.
    sqlite>
    sqlite> WITH
    ...>     t1(X) AS (
    ...>         SELECT 1
    ...>         UNION ALL
    ...>         SELECT X+1 FROM t1
    ...>         LIMIT 9
    ...>         )
    ...>     ,t2(Y) AS (
    ...>         SELECT abs(random() % 10) FROM t1
    ...>         )
    ...> SELECT Y, Y <= 5, Y BETWEEN 0 AND 5, Y < 6 FROM t2;
    7|1|1|0
    3|1|0|0
    2|1|1|1
    2|0|0|1
    0|1|1|1
    3|1|1|0
    5|1|1|0
    6|1|1|0
    0|1|1|1
    sqlite>

Thanks in advance for your assistance.

Kind Regards

Keith Maxwell
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


___________________________________________
 Gunter Hick
Software Engineer
Scientific Games International GmbH
FN 157284 a, HG Wien
Klitschgasse 2-4, A-1130 Vienna, Austria
Tel: +43 1 80100 0
E-Mail: [hidden email]

This communication (including any attachments) is intended for the use of the intended recipient(s) only and may contain information that is confidential, privileged or legally protected. Any unauthorized use or dissemination of this communication is strictly prohibited. If you have received this communication in error, please immediately notify the sender by return e-mail message and delete all copies of the original communication. Thank you for your cooperation.


_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Unexpected results from <, <= and BETWEEN with a CTE in 3.18.0

Clemens Ladisch
In reply to this post by Keith Maxwell
Keith Maxwell wrote:

> Can anyone please explain the results of the query below?
>
>     sqlite> WITH
>     ...>     t1(X) AS (
>     ...>         SELECT 1
>     ...>         UNION ALL
>     ...>         SELECT X+1 FROM t1
>     ...>         LIMIT 9
>     ...>         )
>     ...>     ,t2(Y) AS (
>     ...>         SELECT abs(random() % 10) FROM t1
>     ...>         )
>     ...> SELECT Y, Y <= 5, Y BETWEEN 0 AND 5, Y < 6 FROM t2;
>     7|1|1|0
>     3|1|0|0
>     2|1|1|1
>     2|0|0|1
>     0|1|1|1
>     3|1|1|0
>     5|1|1|0
>     6|1|1|0
>     0|1|1|1

You have four separate calls to random() in each row.

This is because of subquery flattening.
http://www.sqlite.org/optoverview.html#flattening

You can force the database to create a temporary result for t2
by violating at least one of these rules, e.g., add "LIMIT 9" to
t2 and "WHERE 1" to the final SELECT.


Regards,
Clemens
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Unexpected results from <, <= and BETWEEN with a CTE in 3.18.0

Keith Maxwell
Thanks both that is really helpful!

I hadn't come across the query flattening optimisation before. The
documentation is helpful as is prepending EXPLAIN.

Thanks again.

Kind Regards

Keith

On 11 April 2017 at 09:55, Clemens Ladisch <[hidden email]> wrote:

> Keith Maxwell wrote:
> > Can anyone please explain the results of the query below?
> >
> >     sqlite> WITH
> >     ...>     t1(X) AS (
> >     ...>         SELECT 1
> >     ...>         UNION ALL
> >     ...>         SELECT X+1 FROM t1
> >     ...>         LIMIT 9
> >     ...>         )
> >     ...>     ,t2(Y) AS (
> >     ...>         SELECT abs(random() % 10) FROM t1
> >     ...>         )
> >     ...> SELECT Y, Y <= 5, Y BETWEEN 0 AND 5, Y < 6 FROM t2;
> >     7|1|1|0
> >     3|1|0|0
> >     2|1|1|1
> >     2|0|0|1
> >     0|1|1|1
> >     3|1|1|0
> >     5|1|1|0
> >     6|1|1|0
> >     0|1|1|1
>
> You have four separate calls to random() in each row.
>
> This is because of subquery flattening.
> http://www.sqlite.org/optoverview.html#flattening
>
> You can force the database to create a temporary result for t2
> by violating at least one of these rules, e.g., add "LIMIT 9" to
> t2 and "WHERE 1" to the final SELECT.
>
>
> Regards,
> Clemens
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Loading...