Re: sqlite-users Digest, Vol 111, Issue 2

Previous Topic Next Topic
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Re: sqlite-users Digest, Vol 111, Issue 2

Simone Mosciatti
Hi Dan,

actually I was using the version 3.15.1 so at first I didn't investigate
too carefully.

However, the TCL script should be using the 3.15.2.

To dispel every doubt I updated to the latest 3.17.0 and I am seeing
performance way more reasonable, and the bottleneck that I am seeing are
from redis and not SQLite now.

It is worth to investigate further, I will keep the mail list updated
and I will update the github issue.

Thanks for now.



On 02/03/2017 13:00, [hidden email] wrote:

> Message: 10 Date: Wed, 1 Mar 2017 23:08:06 +0700 From: Dan Kennedy
> <[hidden email]> To: [hidden email]
> Subject: Re: [sqlite] Performance difference in running SQLite embed
> or in a TCL script Message-ID:
> <[hidden email]> Content-Type:
> text/plain; charset=utf-8; format=flowed On 03/01/2017 09:53 PM,
> Simone Mosciatti wrote:
>> Hi all,
>> tl;dr:
>> What are the difference between running SQLite inside TCL and running
>> it embed in a shared object module? Why I am seeing such big
>> difference in performance?
>> I finally got some time to work again on my redis module rediSQL,
>> which basically embed SQLite inside redis.
>> I already asked in this same mail list help about performance, and it
>> was showed to me that the poor performance of the modules are not
>> because of SQLite and that I should look at something else.
>> I have a couple of benchmark where I insert triple of integers inside
>> a single table, using straight TCL my machine reach a stable 240k
>> insert per second.
>> Using the redis module and doing the same kind of operation I got way
>> worse performance that degrade with time. The insert per second start
>> at roughly 24k and then decrease down to 1k.
>> What I did to investigate the different behaviour was to use perf on
>> both the script TCL and the redis server running the module.
>> Since the SQLite performances are order of magnitude better than the
>> redis module performances I was expecting that something redis related
>> was throttling down the insert per second. I was wrong.
>> The perf of the TCL script show that most of the time is spent in
>> libpthread, libc, libtcl and only after in libsqlite in the symbol
>> sqlite3_randomness.
>> The perf ot the redis module, instead show that most of the time is
>> spent in |sqlite3PcacheTruncate | (52%) and then on
>> |sqlite3PcacheClearWritable |(30%) I must be doing something totally
>> wrong, but what?
> I forget exactly when it was introduced, but there was a bug causing
> those routines to use excessive CPU on systems with really big page
> caches. Fixed for 3.14.1. Are you using something earlier than 3.14.1?
> If so, it's worth trying a newer version.
> Dan.
>> You can find more details on the issues I opened here:
>> To sum up I am asking what are the difference in running redis from
>> TCL and from running it embed in a shared object.
>> Thanks for your time.
>> Simone
>> _______________________________________________
>> sqlite-users mailing list
>> [hidden email]

sqlite-users mailing list
[hidden email]