1 Writer and 1 Reader in WAL mode in two different processes, but as soon reading is started writer crashes

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

1 Writer and 1 Reader in WAL mode in two different processes, but as soon reading is started writer crashes

avinash.jha2493

The issue appears similar to what was mentioned here:
https://github.com/tensorflow/tensorboard/issues/1467


Multiple traces are seen, one of which is as follows:

(gdb) bt
#0  walRestartLog (pWal=0x2001f78) at sqlite3.c:51561
#1  sqlite3WalFrames (sync_flags=2, isCommit=1, nTruncate=68381,
pList=0x2002660, szPage=1024, pWal=0x2001f78)
    at sqlite3.c:51702
#2  pagerWalFrames (pPager=0x1fefc58, pList=0x0, nTruncate=68381,
isCommit=1) at sqlite3.c:44709
#3  0x000000000046fd02 in sqlite3PagerCommitPhaseOne (pPager=0x1fefc58,
zMaster=0x0, noSync=0) at sqlite3.c:47779
#4  0x00000000004701a8 in sqlite3PagerCommitPhaseOne (noSync=0, zMaster=0x0,
pPager=<optimized out>)
    at sqlite3.c:47758
#5  sqlite3BtreeCommitPhaseOne (p=0x1fc7758, zMaster=0x0) at sqlite3.c:56539
#6  0x00000000004816eb in vdbeCommit (p=0x7f2120001808, db=0x1fc7138) at
sqlite3.c:66795
#7  sqlite3VdbeHalt (p=0x7f2120001808) at sqlite3.c:1667
#8  0x00000000004852fd in sqlite3VdbeExec (p=0x7f2120001808) at
sqlite3.c:73820
#9  0x000000000048cf29 in sqlite3Step (p=0x7f2120001808) at sqlite3.c:69487
#10 sqlite3_step (pStmt=0x7f2120001808) at sqlite3.c:4017
#11 0x0000000000491462 in sqlite3_exec (db=0x1fc7138, zSql=<optimized out>,
xCallback=0x0, pArg=0x0, pzErrMsg=0x0)
    at sqlite3.c:101758

Could someone guide on the next steps, have been playing with pragmas for a
while now, but didn't succeed.



--
Sent from: http://sqlite.1065341.n5.nabble.com/
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: 1 Writer and 1 Reader in WAL mode in two different processes, but as soon reading is started writer crashes

Richard Hipp-3
On 12/2/19, avinash.jha2493 <[hidden email]> wrote:
>
> Could someone guide on the next steps, have been playing with pragmas for a
> while now, but didn't succeed.
>

One thing you could do is tell us exactly what version of SQLite you
are using so that the line-number information in your stack trace can
be decoded.

--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: 1 Writer and 1 Reader in WAL mode in two different processes, but as soon reading is started writer crashes

Warren Young
In reply to this post by avinash.jha2493
On Dec 2, 2019, at 3:02 AM, avinash.jha2493 <[hidden email]> wrote:
>
> The issue appears similar to what was mentioned here:
> https://github.com/tensorflow/tensorboard/issues/1467

Yes, and OP posted a reply to his own issue a few days later giving the correct answer: TensorFlow needs to pick one of the two versions of SQLite and use it exclusively within that process.  This is not a SQLite issue, it’s a SQLite *usage* issue.  You can’t have two different versions of a library both using the same RAM data structures but with different assumptions about their binary layout.

They’ve known about this problem for over a year, and they’re not moving on it.  Apparently they want someone to contribute a fix for it.

Maybe you?

> Could someone guide on the next steps, have been playing with pragmas for a while now, but didn't succeed.

It’s not a configuration issue.  They’re basically begging for data corruption with that setup.

Mind, SQLite does allow multiple versions to access the same DB from *different* processes.  The problem in TensorFlow is that they’ve got two different versions linked into the same binary, and both are trying to access the same DB.

To avoid problems with this, you’d basically have to get rid of all globals in SQLite.  If I’m interpreting the GCC map file correctly, there’s about 32 of them in recent versions of SQLite.  For two versions of *any* library — not just SQLite — to be linked to the same program and work correctly together, they’d both have to have the same binary interpretation for each global variable, at the least.
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users