API, sqlite3_value_bytes and sqlite3_value_blob...

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

API, sqlite3_value_bytes and sqlite3_value_blob...

dave
I have a question regarding the API documention at
http://sqlite.org/c3ref/value_blob.html, which states:
"... the pointer returned from sqlite3_value_blob(), .. can be invalidated
by a subsequent call to sqlite3_value_bytes(), ..."
Is that statement still true?  I ask because I notice that the source of
many of the extensions in 'sqlite/ext' seem to violate that advice.
 
I first noticed this when I was recently working on fileio.c (e.g. line 73
vs 77), but grepping through the source I find many other cases where the
pointer is retrieved via  *_blob() or *.text() BEFORE invoking
sqlite3_value_bytes().  E.g these source and line numbers:
fts2_tokenizer.c:71, 72
fts3_expr.c:1248, 1249
fts3_tokenizer.c:78, 79
fts3_tokenize_vtab.c:347, 348
fts3_write.c:5290, 5291
fts5_index.c:6270, 6271
fts5_storage.c:735, 736
fts5_tcl.c:547
fts5_test_tok.c:375, 376
fts5_vocab.c:607, 608; 612, 613; 616, 617
(I stopped grepping at this point; this list is not comprehensive).
 
Anyway, just wondered if the api documentation's advice is maybe out-of-date
with current reality.  Thoughts/comments?
 
Cheers!
-dave
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: API, sqlite3_value_bytes and sqlite3_value_blob...

Clemens Ladisch
dave wrote:
> "... the pointer returned from sqlite3_value_blob(), .. can be invalidated
> by a subsequent call to sqlite3_value_bytes(), ..."
> Is that statement still true?

The returned pointer is invalidated when you call a function that forces
SQLite to convert the value into another type.


Regards,
Clemens
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: API, sqlite3_value_bytes and sqlite3_value_blob...

petern
In reply to this post by dave
Dave. The documentation contains many such catch-all statements which do
not reflect a full decision tree.  The usual cover story will either be (I
paraphrase) : 1. "that's an implementation detail" or 2. "it might change
later, so the documentation can only make a short blanket statement".

It is far more likely that spelling and grammatical errors you report of
the documentation will be corrected.

The current decision tree of the particular catch-all documentation comment
you found is in vdbemem.c at the comment and function body listing below.
Ultimately there is only the source code.  Getting used to reading it for
yourself will probably save you a lot of time.

int sqlite3ValueBytes(sqlite3_value *pVal, u8 enc){
  Mem *p = (Mem*)pVal;
  assert( (p->flags & MEM_Null)==0 || (p->flags & (MEM_Str|MEM_Blob))==0 );
  if( (p->flags & MEM_Str)!=0 && pVal->enc==enc ){
    return p->n;
  }
  if( (p->flags & MEM_Blob)!=0 ){
    if( p->flags & MEM_Zero ){
      return p->n + p->u.nZero;
    }else{
      return p->n;
    }
  }
  if( p->flags & MEM_Null ) return 0;
  return valueBytes(pVal, enc);
}

Peter

On Wed, Dec 13, 2017 at 8:38 AM, dave <[hidden email]> wrote:

> I have a question regarding the API documention at
> http://sqlite.org/c3ref/value_blob.html, which states:
> "... the pointer returned from sqlite3_value_blob(), .. can be invalidated
> by a subsequent call to sqlite3_value_bytes(), ..."
> Is that statement still true?  I ask because I notice that the source of
> many of the extensions in 'sqlite/ext' seem to violate that advice.
>
> I first noticed this when I was recently working on fileio.c (e.g. line 73
> vs 77), but grepping through the source I find many other cases where the
> pointer is retrieved via  *_blob() or *.text() BEFORE invoking
> sqlite3_value_bytes().  E.g these source and line numbers:
> fts2_tokenizer.c:71, 72
> fts3_expr.c:1248, 1249
> fts3_tokenizer.c:78, 79
> fts3_tokenize_vtab.c:347, 348
> fts3_write.c:5290, 5291
> fts5_index.c:6270, 6271
> fts5_storage.c:735, 736
> fts5_tcl.c:547
> fts5_test_tok.c:375, 376
> fts5_vocab.c:607, 608; 612, 613; 616, 617
> (I stopped grepping at this point; this list is not comprehensive).
>
> Anyway, just wondered if the api documentation's advice is maybe
> out-of-date
> with current reality.  Thoughts/comments?
>
> Cheers!
> -dave
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: API, sqlite3_value_bytes and sqlite3_value_blob...

J Decker
On Wed, Dec 13, 2017 at 1:25 PM, petern <[hidden email]> wrote:

> Dave. The documentation contains many such catch-all statements which do
> not reflect a full decision tree.  The usual cover story will either be (I
> paraphrase) : 1. "that's an implementation detail" or 2. "it might change
> later, so the documentation can only make a short blanket statement".
>
> It is far more likely that spelling and grammatical errors you report of
> the documentation will be corrected.
>
> The current decision tree of the particular catch-all documentation comment
> you found is in vdbemem.c at the comment and function body listing below.
> Ultimately there is only the source code.  Getting used to reading it for
> yourself will probably save you a lot of time.
>
> int sqlite3ValueBytes(sqlite3_value *pVal, u8 enc){
>   Mem *p = (Mem*)pVal;
>   assert( (p->flags & MEM_Null)==0 || (p->flags & (MEM_Str|MEM_Blob))==0 );
>   if( (p->flags & MEM_Str)!=0 && pVal->enc==enc ){
>     return p->n;
>   }
>   if( (p->flags & MEM_Blob)!=0 ){
>     if( p->flags & MEM_Zero ){
>       return p->n + p->u.nZero;
>     }else{
>       return p->n;
>     }
>   }
>   if( p->flags & MEM_Null ) return 0;
>   return valueBytes(pVal, enc);
>

it's valueBytes that has the biggest impact

sqlite3_value_blob
sqlite3_value_text
sqlite3_value_text16
sqlite3_value_text16be
sqlite3_value_text16le

sqlite3_value_bytes
   - invalidates sqlite3_value_text16*() result  (always)

sqlite3_value_bytes16
  - invalidates sqlite3_value_text() result (always)
  - invalidates sqlite3_value_text16*() result if the format does not match
defined SQLITE_UTF16NATIVE

if the value type is a blob, sqlite3_value_bytes() and
sqlite3_value_bytes16() will do no conversion, and will not invalidate blob
result.

for both; if the value was NULL there is no invalidation; and 0 will be
returned

if sqlite3_value_blob is used and the original value type is NOT a blob, it
returns sqlite3_value_text(); so bytes16 will invalidate the result.
(rephrased) sqlite3_value_blob() on a column that is not a blob has the
potential of being invalidated by bytes16().



}

>
> Peter
>
> On Wed, Dec 13, 2017 at 8:38 AM, dave <[hidden email]> wrote:
>
> > I have a question regarding the API documention at
> > http://sqlite.org/c3ref/value_blob.html, which states:
> > "... the pointer returned from sqlite3_value_blob(), .. can be
> invalidated
> > by a subsequent call to sqlite3_value_bytes(), ..."
> > Is that statement still true?  I ask because I notice that the source of
> > many of the extensions in 'sqlite/ext' seem to violate that advice.
> >
> > I first noticed this when I was recently working on fileio.c (e.g. line
> 73
> > vs 77), but grepping through the source I find many other cases where the
> > pointer is retrieved via  *_blob() or *.text() BEFORE invoking
> > sqlite3_value_bytes().  E.g these source and line numbers:
> > fts2_tokenizer.c:71, 72
> > fts3_expr.c:1248, 1249
> > fts3_tokenizer.c:78, 79
> > fts3_tokenize_vtab.c:347, 348
> > fts3_write.c:5290, 5291
> > fts5_index.c:6270, 6271
> > fts5_storage.c:735, 736
> > fts5_tcl.c:547
> > fts5_test_tok.c:375, 376
> > fts5_vocab.c:607, 608; 612, 613; 616, 617
> > (I stopped grepping at this point; this list is not comprehensive).
> >
> > Anyway, just wondered if the api documentation's advice is maybe
> > out-of-date
> > with current reality.  Thoughts/comments?
> >
> > Cheers!
> > -dave
> > _______________________________________________
> > sqlite-users mailing list
> > [hidden email]
> > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> >
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: API, sqlite3_value_bytes and sqlite3_value_blob...

dave
> -----Original Message-----
> Behalf Of J Decker
...
> <[hidden email]> wrote:
>
> > Dave. The documentation contains many such catch-all
> statements which do
...
> > The current decision tree of the particular catch-all
> documentation comment
...
> > int sqlite3ValueBytes(sqlite3_value *pVal, u8 enc){
...
> it's valueBytes that has the biggest impact
>
...
> sqlite3_value_bytes
>    - invalidates sqlite3_value_text16*() result  (always)
>
> sqlite3_value_bytes16
>   - invalidates sqlite3_value_text() result (always)
>   - invalidates sqlite3_value_text16*() result if the format
> does not match
> defined SQLITE_UTF16NATIVE
>
...

Now I can see it!

Thanks, guys - Clemens, Peter, d3ck0r - for your perspicacious insights!  I
am slightly aglow with the enlightenment; I think I'll unplug the Xmas tree
for the rest of the evening, and bask instead in the illumnation your have
so kindly imbued upon me.  I think there will be enough light for me to do
some reading of vdbemem.c!

-dave


_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users