Upgrade to SQLite 3.20.0 im Mozilla - Consequences for Mozilla Thunderbird

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Upgrade to SQLite 3.20.0 im Mozilla - Consequences for Mozilla Thunderbird

Jörg Knobloch
Dear SQLite developer,

Mozilla have upgraded to SQLite 3.20.0 and that has caused the
Thunderbird test suite some test failures [1] as follows:

Assertion failed: (p->flags & MEM_Dyn)==0 || p->szMalloc==0, file
sqlite3.c, line 70285

One of the Mozilla developers was of the opinion that the custom
tokenizer and a custom ranking function we're using might be at the root
of the problem and mentioned that the example which we copied
(https://sqlite.org/fts3.html#appendix_a, very end, aMatchinfo =
(unsigned int *)sqlite3_value_blob(apVal[0]);) might need to be updated
in view of the pointer interfaces mentioned here:
https://sqlite.org/bindptr.html.

Any help would be appreciated.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1392937

--
Jörg Knobloch - [hidden email] <mailto:[hidden email]>- www.jorgk.com
<http://www.jorgk.com>
Thunderbird Developer
<http://www.jorgk.com/misc/Mozilla-work.pdf>(Thunderbird
<https://wiki.mozilla.org/Modules/Thunderbird#Thunderbird>, Compose
<https://wiki.mozilla.org/Modules/Thunderbird#Compose>and Mailnews
Editor <https://wiki.mozilla.org/Modules/MailNews_Core#Editor>and MIME
<https://wiki.mozilla.org/Modules/MailNews_Core#MIME_Parser>peer) -
Member of the Thunderbird Council
<https://wiki.mozilla.org/Modules/Thunderbird#Thunderbird_Council>

_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Upgrade to SQLite 3.20.0 im Mozilla - Consequences for Mozilla Thunderbird

Clemens Ladisch
Jörg Knobloch wrote:
> Mozilla have upgraded to SQLite 3.20.0 and that has caused the
> Thunderbird test suite some test failures [1] as follows:
>
> Assertion failed: (p->flags & MEM_Dyn)==0 || p->szMalloc==0, file sqlite3.c, line 70285

Hmm, is there a custom memory allocator somewhere?
And one can never rule out memory corruption ...

> One of the Mozilla developers was of the opinion that the custom
> tokenizer and a custom ranking function we're using might be at the
> root of the problem [...] might need to be updated in view of the
> pointer interfaces mentioned here: https://sqlite.org/bindptr.html.

That is a new interface; the old method still works.


Regards,
Clemens
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Upgrade to SQLite 3.20.0 im Mozilla - Consequences for Mozilla Thunderbird

Jörg Knobloch
On 24/08/2017 11:40, Clemens Ladisch wrote:
> Jörg Knobloch wrote:
>> Mozilla have upgraded to SQLite 3.20.0 and that has caused the
>> Thunderbird test suite some test failures [1] as follows:
>>
>> Assertion failed: (p->flags & MEM_Dyn)==0 || p->szMalloc==0, file sqlite3.c, line 70285
> Hmm, is there a custom memory allocator somewhere?
> And one can never rule out memory corruption ...
>
Yes, as far as I'm aware Mozilla have their own (twisted and
complicated) memory management, let me see:

https://dxr.mozilla.org/mozilla-central/source/memory

My position is a little difficult, since I'm the Thunderbird maintainer,
not a Mozilla core developer. Thunderbird has a small team using Mozilla
core technology (frequently referred to a "Gecko"). Mozilla core is
backed by hundreds of employees, some look after the memory management
and also a component called mozStorage which interfaces with SQLite. The
maintainers of that component sent me here.

Jörg.


_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Upgrade to SQLite 3.20.0 im Mozilla - Consequences for Mozilla Thunderbird

Richard Hipp-3
On 8/24/17, Jörg Knobloch <[hidden email]> wrote:

> On 24/08/2017 11:40, Clemens Ladisch wrote:
>> Jörg Knobloch wrote:
>>> Mozilla have upgraded to SQLite 3.20.0 and that has caused the
>>> Thunderbird test suite some test failures [1] as follows:
>>>
>>> Assertion failed: (p->flags & MEM_Dyn)==0 || p->szMalloc==0, file
>>> sqlite3.c, line 70285
>> Hmm, is there a custom memory allocator somewhere?
>> And one can never rule out memory corruption ...
>>
> Yes, as far as I'm aware Mozilla have their own (twisted and
> complicated) memory management, let me see:
>
> https://dxr.mozilla.org/mozilla-central/source/memory
>
> My position is a little difficult, since I'm the Thunderbird maintainer,
> not a Mozilla core developer. Thunderbird has a small team using Mozilla
> core technology (frequently referred to a "Gecko"). Mozilla core is
> backed by hundreds of employees, some look after the memory management
> and also a component called mozStorage which interfaces with SQLite. The
> maintainers of that component sent me here.

I think the problem is a bug in the new sqlite3_result_pointer()
interface.  I added a proposed patch to the ticket.  But we are still
working on getting a test case here.

The problem, if my analysis is correct, should be obscure and require
a very specific query to hit.

--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Upgrade to SQLite 3.20.0 im Mozilla - Consequences for Mozilla Thunderbird

Jörg Knobloch
On 24/08/2017 13:02, Richard Hipp wrote:
> The problem, if my analysis is correct, should be obscure and require
> a very specific query to hit.

Thanks, your patch works. I believe our use case is as obscure as you
can think ;-(

Jörg.

_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users