SQLite between forks

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

SQLite between forks

Simone Mosciatti
Hi all,

it is suggested in several place to don't share a connection between forks.

However I fail to see how this can be a problem for purely in-memory
database.

If my understanding of fork and sqlite are correct I don't see issues in
having a forked child reading a database connection opened before the
fork itself.

The address space of the two database are different and, at this point,
we are already talking about two completely different database.

Of course each write in the child (parent) won't be visible by the
parent (child), but this is not my concern.

Is my reasoning sound and correct?

Different point if the database is backed by a regular file, however, in
such case, couldn't I simply close the connection in the child and
reopen it (maybe in read only mode)?

Sorry for this question that may be trivial!

Best Regards,

Simone
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: SQLite between forks

Clemens Ladisch
Simone Mosciatti wrote:
> it is suggested in several place to don't share a connection between forks.

Because of how locking and file handles interact.

> However I fail to see how this can be a problem for purely in-memory database.

In-memory databases do not use a file handle or file locking.

> Different point if the database is backed by a regular file, however,
> in such case, couldn't I simply close the connection in the child

fork(2) says that
| the child does not inherit process-associated record locks from its
| parent (fcntl(2)).  (On the other hand, it does inherit fcntl(2) open
| file description locks and flock(2) locks from its parent.)

So closing the child's handle can break the parent's locks.


Regards,
Clemens
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: SQLite between forks

Roger Binns
In reply to this post by Simone Mosciatti
On 26/03/18 13:30, Simone Mosciatti wrote:
> However I fail to see how this can be a problem for purely in-memory
> database.

When a process forks, only the thread that called fork is kept in the
new child process.  Also note that semaphores (and locks in general) are
left in the same state as at the time of the fork call, and are not
reset in the child (forked) process.

A very simple example of a problem would be if a thread in the parent
process is inside a SQLite call and another thread calls fork().  The
currently active locks will remain in the child so SQLite operations
will just block forever in the child.

That can be avoided providing you use no threads, the libraries you use
have no threads, and the libraries those libraries use have no threads.
That will be increasingly unlikely over time.  If you do use threads,
then you would have to make things safe by controlling exactly when
forks can happen (assuming no libraries thwart you).

This is very tedious work that is very difficult to statically check,
let alone verify everything is correct in every possible code path.
Hence the recommendation to not use SQLite across forks.  I address that
by making sure only leaf processes do SQLite work.

For my Python SQLite wrapper, I added a fork checker to help with
testing and make sure you don't use SQLite across forks.  The way it is
implemented is replacing the SQLite mutex operations (there is a table
of them) with ones that check the process id, and then call the original
mutex operation.  In my test of a benchmark where 100% of the code was
doing SQLite operations, there was a slowdown of 1%.

https://rogerbinns.github.io/apsw/apsw.html#apsw.fork_checker

Roger


_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

signature.asc (169 bytes) Download Attachment