Re: SQLite 3.20.0 postponed

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SQLite 3.20.0 postponed

petern
Richard.  Here is food for thought.  Below is a contrived but realistic
example of an embedded application wire format which demonstrates some
expected extension function pointer allocations and another recent topic.

SELECT DoTrackingRecalibrate(column1,CommandPacket(column2,column3)) FROM
(VALUES
  ('Winnipeg','field36',
      (SELECT TrimCommand(column1,column2) FROM (VALUES
                ('field1',1.33),
                ('field2','radians')
            ))),
    ('Winnipeg','field37','Apogee'),
    ('Minneapolis','field36',
      (SELECT TrimCommand(column1,column2) FROM (VALUES
                ('field1',4.33),
                ('field2','radians')
            ))),
    ('Minneapolis','field37','Perigee')
)
GROUP BY column1;

DoTrackingRecalibrate() is an ordinary extension function wrapping a host
function.  It takes arguments of a TEXT value and a BLOB pointer, the
output of host object wrapper CommandPacket().  CommandPacket() and
TrimCommand() are both aggregate extension functions which allocate their
underlying host object on the first step and supply a destructor callback,
on the final step, to sqlite3_result_blob().

This SQL embodies both sending the two commands and the complete
storage/wire format if these exact commands must be marshaled remotely or
stored for playback.  One of the strengths of SQLite is how it can support
a straightforward and portable serialization format like this.

This example also demonstrates a natural and convenient pattern for
expressing initialization values of host objects nested to arbitrary
depth.  Notice here how introducing WITH clauses to name the initialization
value columns would produce redundant table names and unintuitive statement
order.

Now I'll go a bit further with a suggestion.  Consider the improvement in
clarity and readability of this wire format after a minor change in VALUES
clause syntax to directly specify column names.  Here I'll introduce a
hypothetical AS sub-clause but any other proposal which affords specifying
column names within the VALUES clause scope would work equally well.

Hypothetically naming columns thusly:

   VALUES (),()...  AS ("colname1","colname2",..."columnN")

The above example can now read even more clearly and intuitively as follows.

SELECT DoTrackingRecalibrate(node,CommandPacket(packetField,packetValue))
FROM (VALUES
  ('Winnipeg','field36',
      (SELECT TrimCommand(trimField,trimValue) FROM (VALUES
                ('field1',1.33),
                ('field2','radians')
            AS ("trimField","trimValue")))),
    ('Winnipeg','field37','Apogee'),
    ('Minneapolis','field36',
      (SELECT TrimCommand(trimField,trimValue) FROM (VALUES
                ('field1',4.33),
                ('field2','radians')
            AS ("trimField","trimValue")))),
    ('Minneapolis','field37','Perigee')
AS ("node","packetField","packetValue"))
GROUP BY node;



On Mon, Jul 17, 2017 at 11:54 AM, Richard Hipp <[hidden email]> wrote:

> The 3.20.0 release will be delayed.  Some concerns came up over the
> new sqlite3_value_pointer() interface.  Interface chagnes were made
> over the weekend.  But there are still concerns.  So the decision has
> been made to back off and give the current design a few weeks to soak
> before trying to press forward with a release which will commit us to
> a particular design.
>
> The draft website is still up at https://sqlite.org/draft - note that
> the change log at https://sqlite.org/draft/releaselog/3_20_0.html now
> identifies three (obscure) backwards compatibility breaks.  Your input
> on these changes is requested.
>
> The "prelease snapshot" on https://sqlite.org/download.html is
> up-to-date for people who want to try out the new code.  Please do so.
> Report any concerns here or via direct email to me.
>
> I will try provide a write-up on the motivations, use, and limitations
> of sqlite3_value_pointer() soon.
>
> All of the changes that where queued for release on branch-3.20 are on
> trunk.  Development will continue on trunk until we restart the
> release cycle.
>
> The check-list (https://www.sqlite.org/checklists/3200000/index) has
> been reset to its initial state.  It will start over again in a week
> or two.
>
> --
> D. Richard Hipp
> [hidden email]
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SQLite 3.20.0 postponed

Gwendal Roué-2

> Le 17 juil. 2017 à 20:54, Richard Hipp <[hidden email]> a écrit :
>
> The 3.20.0 release will be delayed.  Some concerns came up over the
> new sqlite3_value_pointer() interface.  Interface chagnes were made
> over the weekend.  But there are still concerns.  So the decision has
> been made to back off and give the current design a few weeks to soak
> before trying to press forward with a release which will commit us to
> a particular design.
>
> The draft website is still up at https://sqlite.org/draft - note that
> the change log at https://sqlite.org/draft/releaselog/3_20_0.html now
> identifies three (obscure) backwards compatibility breaks.  Your input
> on these changes is requested.

Hello,

When I read the documentation for sqlite3_bind_pointer, I read:

> The T parameter should be a static string

The reason is pretty clear: this T parameter will be used later by sqlite3_value_pointer, for a string comparison with strcmp(). It hence has to remain is memory forever - and static strings are good at that.

I could test it and make it work reliably in Swift for custom FTS5 tokenisers.

Here is my comment: I wonder if the key comparison with strcmp() is really necessary.

First, this strcmp() give a lot of work to languages that wrap SQLite and lack support for "static strings". Building a global \0-terminated buffer that never gets deallocated is not always that easy :-)

Next, there are techniques for building unique "keys" that hold in a machine word, and can simply be compared with ==. For example:

    typedef void *sqlite3_pointer_key_t;   // defined in sqlite3.h
    sqlite3_pointer_key_t key1 = "my_key"; // nice for debugging
    sqlite3_pointer_key_t key2 = &key2;    // hard core but still valid

Maybe this is considered awful practices - I'm certainly not a C expert.

And this would also force functions that use the new pointer APIs to expose those keys in some header (such as FTS5()). You may have chosen the current technique precisely because you don't want such API pollution.

What were your rationale behind this choice?

Thanks in advance,
Gwendal Roué

_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SQLite 3.20.0 postponed

Richard Hipp-3
On 7/21/17, Gwendal Roué <[hidden email]> wrote:
>
> First, this strcmp() give a lot of work to languages that wrap SQLite and
> lack support for "static strings".

But sqlite3_result_pointer() and sqlite3_bind_pointer() are not
invoked from those languages.  The _pointer() routines are invoked
from C, and C does easily support string literals that are static
strings.

A C-language wrapper around sqlite3_result_pointer() and
sqlite3_bind_pointer() that interfaces to the non-static-string
language can simply insert the required static string.

We do not want the static string to be a parameter to a generic
higher-level interface.  That defeats the purpose of the static
string.  Remember, the string is a "pointer type".  We do not want to
support interfaces that provide access to pointers of any type the
user wants.  We are not trying to recreate C++ templates or other
interfaces that work with arbitrary types.  Each use of _pointer() is
intended to be used for a single narrowly defined purpose.

--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Loading...