Performance difference in running SQLite embed or in a TCL script

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Performance difference in running SQLite embed or in a TCL script

Simone Mosciatti
Hi all,


tl;dr:

What are the difference between running SQLite inside TCL and running it
embed in a shared object module? Why I am seeing such big difference in
performance?

https://github.com/RedBeardLab/rediSQL/issues/13#issuecomment-283309641


I finally got some time to work again on my redis module rediSQL, which
basically embed SQLite inside redis.

I already asked in this same mail list help about performance, and it
was showed to me that the poor performance of the modules are not
because of SQLite and that I should look at something else.

I have a couple of benchmark where I insert triple of integers inside a
single table, using straight TCL my machine reach a stable 240k insert
per second.

Using the redis module and doing the same kind of operation I got way
worse performance that degrade with time. The insert per second start at
roughly 24k and then decrease down to 1k.

What I did to investigate the different behaviour was to use perf on
both the script TCL and the redis server running the module.

Since the SQLite performances are order of magnitude better than the
redis module performances I was expecting that something redis related
was throttling down the insert per second. I was wrong.

The perf of the TCL script show that most of the time is spent in
libpthread, libc, libtcl and only after in libsqlite in the symbol
sqlite3_randomness.

The perf ot the redis module, instead show that most of the time is
spent in |sqlite3PcacheTruncate | (52%) and then on
|sqlite3PcacheClearWritable |(30%) I must be doing something totally
wrong, but what?

You can find more details on the issues I opened here:
https://github.com/RedBeardLab/rediSQL/issues/13#issuecomment-283309641

To sum up I am asking what are the difference in running redis from TCL
and from running it embed in a shared object.

Thanks for your time.

Simone

_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Performance difference in running SQLite embed or in a TCL script

Dan Kennedy-4
On 03/01/2017 09:53 PM, Simone Mosciatti wrote:

> Hi all,
>
>
> tl;dr:
>
> What are the difference between running SQLite inside TCL and running
> it embed in a shared object module? Why I am seeing such big
> difference in performance?
>
> https://github.com/RedBeardLab/rediSQL/issues/13#issuecomment-283309641
>
>
> I finally got some time to work again on my redis module rediSQL,
> which basically embed SQLite inside redis.
>
> I already asked in this same mail list help about performance, and it
> was showed to me that the poor performance of the modules are not
> because of SQLite and that I should look at something else.
>
> I have a couple of benchmark where I insert triple of integers inside
> a single table, using straight TCL my machine reach a stable 240k
> insert per second.
>
> Using the redis module and doing the same kind of operation I got way
> worse performance that degrade with time. The insert per second start
> at roughly 24k and then decrease down to 1k.
>
> What I did to investigate the different behaviour was to use perf on
> both the script TCL and the redis server running the module.
>
> Since the SQLite performances are order of magnitude better than the
> redis module performances I was expecting that something redis related
> was throttling down the insert per second. I was wrong.
>
> The perf of the TCL script show that most of the time is spent in
> libpthread, libc, libtcl and only after in libsqlite in the symbol
> sqlite3_randomness.
>
> The perf ot the redis module, instead show that most of the time is
> spent in |sqlite3PcacheTruncate | (52%) and then on
> |sqlite3PcacheClearWritable |(30%) I must be doing something totally
> wrong, but what?


I forget exactly when it was introduced, but there was a bug causing
those routines to use excessive CPU on systems with really big page
caches. Fixed for 3.14.1. Are you using something earlier than 3.14.1?
If so, it's worth trying a newer version.

   http://sqlite.org/releaselog/3_14_1.html

Dan.


>
> You can find more details on the issues I opened here:
> https://github.com/RedBeardLab/rediSQL/issues/13#issuecomment-283309641
>
> To sum up I am asking what are the difference in running redis from
> TCL and from running it embed in a shared object.
>
> Thanks for your time.
>
> Simone
>
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Loading...