Segfault when query again in-memory db

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Segfault when query again in-memory db

Wang, Peter (Xu)
Hi
I am trying to provide a asychnonous queue based on sqlite3 in Python
Current, I already passted the file based tests with my queue
When switching file to in-memory db, I keep meeting a "segfault" issue when running the same tests test suite

Can anyone help me out of this situation?


I tested the in-memory with multi-thread (but inserts/deletes are in a lock)
Here is the trace of the core file:

(gdb) bt
#0  sqlite3_value_type (pVal=0x0) at sqlite3.c:72512
#1  0x00007fda2a34fd86 in sqlite3_column_type (pStmt=0x7fda08004178, i=<optimized out>) at sqlite3.c:73318
#2  0x00007fda2a607987 in _pysqlite_fetch_one_row (self=self@entry=0x7fda2a204b20) at /build/python2.7-1tJBSB/python2.7-2.7.12/Modules/_sqlite/cursor.c:352
#3  0x00007fda2a6089c1 in _pysqlite_query_execute (self=0x7fda2a204b20, multiple=<optimized out>, args=<optimized out>)
    at /build/python2.7-1tJBSB/python2.7-2.7.12/Modules/_sqlite/cursor.c:711
#4  0x00000000004b0cb3 in PyObject_Call () at ../Objects/abstract.c:2546
#5  0x00000000004ce5d0 in PyEval_CallObjectWithKeywords () at ../Python/ceval.c:4219
#6  0x00007fda2a609106 in pysqlite_connection_execute (self=<optimized out>, args=('SELECT _id, data FROM queue_default ORDER BY _id ASC LIMIT 1', ()))
    at /build/python2.7-1tJBSB/python2.7-2.7.12/Modules/_sqlite/connection.c:1262
#7  0x00000000004cb945 in call_function (oparg=<optimized out>, pp_stack=0x7fda19ff9fc0) at ../Python/ceval.c:4350
#8  PyEval_EvalFrameEx () at ../Python/ceval.c:2987
#9  0x00000000004c2765 in PyEval_EvalCodeEx () at ../Python/ceval.c:3582
#10 0x00000000004ca8d1 in fast_function (nk=0, na=<optimized out>, n=<optimized out>, pp_stack=0x7fda19ffa1d0, func=<function at remote 0x7fda2a83c050>)
    at ../Python/ceval.c:4445
#11 call_function (oparg=<optimized out>, pp_stack=0x7fda19ffa1d0) at ../Python/ceval.c:4370
#12 PyEval_EvalFrameEx () at ../Python/ceval.c:2987
#13 0x00000000004c9d8f in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fda19ffa320,
    func=<function at remote 0x7fda2a8416e0>) at ../Python/ceval.c:4435
#14 call_function (oparg=<optimized out>, pp_stack=0x7fda19ffa320) at ../Python/ceval.c:4370
#15 PyEval_EvalFrameEx () at ../Python/ceval.c:2987
#16 0x00000000004c2765 in PyEval_EvalCodeEx () at ../Python/ceval.c:3582
#17 0x00000000004ca099 in fast_function (nk=1, na=<optimized out>, n=<optimized out>, pp_stack=0x7fda19ffa530, func=<function at remote 0x7fda2a841758>)
    at ../Python/ceval.c:4445
#18 call_function (oparg=<optimized out>, pp_stack=0x7fda19ffa530) at ../Python/ceval.c:4370
#19 PyEval_EvalFrameEx () at ../Python/ceval.c:2987
#20 0x00000000004c2765 in PyEval_EvalCodeEx () at ../Python/ceval.c:3582
#21 0x00000000004de8b8 in function_call.lto_priv () at ../Objects/funcobject.c:523
#22 0x00000000004b0cb3 in PyObject_Call () at ../Objects/abstract.c:2546
#23 0x00000000004c6ad1 in ext_do_call (nk=<optimized out>, na=<optimized out>, flags=<optimized out>, pp_stack=0x7fda19ffa7e8,
    func=<function at remote 0x7fda29fd6c80>) at ../Python/ceval.c:4664
#24 PyEval_EvalFrameEx () at ../Python/ceval.c:3026
#25 0x00000000004c9d8f in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fda19ffa930,
    func=<function at remote 0x7fda2da20758>) at ../Python/ceval.c:4435
#26 call_function (oparg=<optimized out>, pp_stack=0x7fda19ffa930) at ../Python/ceval.c:4370
#27 PyEval_EvalFrameEx () at ../Python/ceval.c:2987
#28 0x00000000004c9d8f in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fda19ffaa80,
---Type <return> to continue, or q <return> to quit---
    func=<function at remote 0x7fda2da208c0>) at ../Python/ceval.c:4435
#29 call_function (oparg=<optimized out>, pp_stack=0x7fda19ffaa80) at ../Python/ceval.c:4370
#30 PyEval_EvalFrameEx () at ../Python/ceval.c:2987
#31 0x00000000004c2765 in PyEval_EvalCodeEx () at ../Python/ceval.c:3582
#32 0x00000000004de6fe in function_call.lto_priv () at ../Objects/funcobject.c:523
#33 0x00000000004b0cb3 in PyObject_Call () at ../Objects/abstract.c:2546
#34 0x00000000004f492e in instancemethod_call.lto_priv () at ../Objects/classobject.c:2602
#35 0x00000000004b0cb3 in PyObject_Call () at ../Objects/abstract.c:2546
#36 0x00000000004ce5d0 in PyEval_CallObjectWithKeywords () at ../Python/ceval.c:4219
#37 0x0000000000597e32 in t_bootstrap () at ../Modules/threadmodule.c:620
#38 0x00007fda2d77f6ba in start_thread (arg=0x7fda19ffb700) at pthread_create.c:333
#39 0x00007fda2d4b53dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Here is the python trace:

(gdb) py-bt
Traceback (most recent call first):
  File "/home/user/Documents/persist-queue/persistqueue/sqlbase.py", line 140, in _select
    def _select(self, *args):
  File "/home/user/Documents/persist-queue/persistqueue/sqlqueue.py", line 49, in _pop
    row = self._select()
  File "/home/wangp11/Documents/persist-queue/persistqueue/sqlqueue.py", line 67, in get
    pickled = self._pop()
  File "/home/user/Documents/persist-queue/tests/test_sqlqueue.py", line 105, in consumer
    x = m_queue.get(block=True)
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 774, in __bootstrap
    self.__bootstrap_inner()
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Segfault when query again in-memory db

Richard Hipp-3
On 10/31/17, Wang, Peter (Xu) <[hidden email]> wrote:
> Hi
> I am trying to provide a asychnonous queue based on sqlite3 in Python
> Current, I already passted the file based tests with my queue
> When switching file to in-memory db, I keep meeting a "segfault" issue when
> running the same tests test suite
>
> Can anyone help me out of this situation?

Can you get us a reproducible test case?

Can you tell us what version of SQLite you are using?

Can you recompile the SQLite shared library using -DSQLITE_DEBUG and
-DSQLITE_ENABLE_API_ARMOR and see how that changes the outcome?

>
>
> I tested the in-memory with multi-thread (but inserts/deletes are in a lock)
> Here is the trace of the core file:
>
> (gdb) bt
> #0  sqlite3_value_type (pVal=0x0) at sqlite3.c:72512
> #1  0x00007fda2a34fd86 in sqlite3_column_type (pStmt=0x7fda08004178,
> i=<optimized out>) at sqlite3.c:73318
> #2  0x00007fda2a607987 in _pysqlite_fetch_one_row
> (self=self@entry=0x7fda2a204b20) at
> /build/python2.7-1tJBSB/python2.7-2.7.12/Modules/_sqlite/cursor.c:352
> #3  0x00007fda2a6089c1 in _pysqlite_query_execute (self=0x7fda2a204b20,
> multiple=<optimized out>, args=<optimized out>)
>     at /build/python2.7-1tJBSB/python2.7-2.7.12/Modules/_sqlite/cursor.c:711
> #4  0x00000000004b0cb3 in PyObject_Call () at ../Objects/abstract.c:2546
> #5  0x00000000004ce5d0 in PyEval_CallObjectWithKeywords () at
> ../Python/ceval.c:4219
> #6  0x00007fda2a609106 in pysqlite_connection_execute (self=<optimized out>,
> args=('SELECT _id, data FROM queue_default ORDER BY _id ASC LIMIT 1', ()))
>     at
> /build/python2.7-1tJBSB/python2.7-2.7.12/Modules/_sqlite/connection.c:1262
> #7  0x00000000004cb945 in call_function (oparg=<optimized out>,
> pp_stack=0x7fda19ff9fc0) at ../Python/ceval.c:4350
> #8  PyEval_EvalFrameEx () at ../Python/ceval.c:2987
> #9  0x00000000004c2765 in PyEval_EvalCodeEx () at ../Python/ceval.c:3582
> #10 0x00000000004ca8d1 in fast_function (nk=0, na=<optimized out>,
> n=<optimized out>, pp_stack=0x7fda19ffa1d0, func=<function at remote
> 0x7fda2a83c050>)
>     at ../Python/ceval.c:4445
> #11 call_function (oparg=<optimized out>, pp_stack=0x7fda19ffa1d0) at
> ../Python/ceval.c:4370
> #12 PyEval_EvalFrameEx () at ../Python/ceval.c:2987
> #13 0x00000000004c9d8f in fast_function (nk=<optimized out>, na=<optimized
> out>, n=<optimized out>, pp_stack=0x7fda19ffa320,
>     func=<function at remote 0x7fda2a8416e0>) at ../Python/ceval.c:4435
> #14 call_function (oparg=<optimized out>, pp_stack=0x7fda19ffa320) at
> ../Python/ceval.c:4370
> #15 PyEval_EvalFrameEx () at ../Python/ceval.c:2987
> #16 0x00000000004c2765 in PyEval_EvalCodeEx () at ../Python/ceval.c:3582
> #17 0x00000000004ca099 in fast_function (nk=1, na=<optimized out>,
> n=<optimized out>, pp_stack=0x7fda19ffa530, func=<function at remote
> 0x7fda2a841758>)
>     at ../Python/ceval.c:4445
> #18 call_function (oparg=<optimized out>, pp_stack=0x7fda19ffa530) at
> ../Python/ceval.c:4370
> #19 PyEval_EvalFrameEx () at ../Python/ceval.c:2987
> #20 0x00000000004c2765 in PyEval_EvalCodeEx () at ../Python/ceval.c:3582
> #21 0x00000000004de8b8 in function_call.lto_priv () at
> ../Objects/funcobject.c:523
> #22 0x00000000004b0cb3 in PyObject_Call () at ../Objects/abstract.c:2546
> #23 0x00000000004c6ad1 in ext_do_call (nk=<optimized out>, na=<optimized
> out>, flags=<optimized out>, pp_stack=0x7fda19ffa7e8,
>     func=<function at remote 0x7fda29fd6c80>) at ../Python/ceval.c:4664
> #24 PyEval_EvalFrameEx () at ../Python/ceval.c:3026
> #25 0x00000000004c9d8f in fast_function (nk=<optimized out>, na=<optimized
> out>, n=<optimized out>, pp_stack=0x7fda19ffa930,
>     func=<function at remote 0x7fda2da20758>) at ../Python/ceval.c:4435
> #26 call_function (oparg=<optimized out>, pp_stack=0x7fda19ffa930) at
> ../Python/ceval.c:4370
> #27 PyEval_EvalFrameEx () at ../Python/ceval.c:2987
> #28 0x00000000004c9d8f in fast_function (nk=<optimized out>, na=<optimized
> out>, n=<optimized out>, pp_stack=0x7fda19ffaa80,
> ---Type <return> to continue, or q <return> to quit---
>     func=<function at remote 0x7fda2da208c0>) at ../Python/ceval.c:4435
> #29 call_function (oparg=<optimized out>, pp_stack=0x7fda19ffaa80) at
> ../Python/ceval.c:4370
> #30 PyEval_EvalFrameEx () at ../Python/ceval.c:2987
> #31 0x00000000004c2765 in PyEval_EvalCodeEx () at ../Python/ceval.c:3582
> #32 0x00000000004de6fe in function_call.lto_priv () at
> ../Objects/funcobject.c:523
> #33 0x00000000004b0cb3 in PyObject_Call () at ../Objects/abstract.c:2546
> #34 0x00000000004f492e in instancemethod_call.lto_priv () at
> ../Objects/classobject.c:2602
> #35 0x00000000004b0cb3 in PyObject_Call () at ../Objects/abstract.c:2546
> #36 0x00000000004ce5d0 in PyEval_CallObjectWithKeywords () at
> ../Python/ceval.c:4219
> #37 0x0000000000597e32 in t_bootstrap () at ../Modules/threadmodule.c:620
> #38 0x00007fda2d77f6ba in start_thread (arg=0x7fda19ffb700) at
> pthread_create.c:333
> #39 0x00007fda2d4b53dd in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
>
> Here is the python trace:
>
> (gdb) py-bt
> Traceback (most recent call first):
>   File "/home/user/Documents/persist-queue/persistqueue/sqlbase.py", line
> 140, in _select
>     def _select(self, *args):
>   File "/home/user/Documents/persist-queue/persistqueue/sqlqueue.py", line
> 49, in _pop
>     row = self._select()
>   File "/home/wangp11/Documents/persist-queue/persistqueue/sqlqueue.py",
> line 67, in get
>     pickled = self._pop()
>   File "/home/user/Documents/persist-queue/tests/test_sqlqueue.py", line
> 105, in consumer
>     x = m_queue.get(block=True)
>   File "/usr/lib/python2.7/threading.py", line 754, in run
>     self.__target(*self.__args, **self.__kwargs)
>   File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
>     self.run()
>   File "/usr/lib/python2.7/threading.py", line 774, in __bootstrap
>     self.__bootstrap_inner()
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>


--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Segfault when query again in-memory db

Keith Medcalf
In reply to this post by Wang, Peter (Xu)

>I tested the in-memory with multi-thread (but inserts/deletes are in
>a lock)

What does "with multi-thread" mean (to you)?

---
The fact that there's a Highway to Hell but only a Stairway to Heaven says a lot about anticipated traffic volume.

>-----Original Message-----
>From: sqlite-users [mailto:sqlite-users-
>[hidden email]] On Behalf Of Wang, Peter (Xu)
>Sent: Tuesday, 31 October, 2017 02:21
>To: [hidden email]
>Cc: [hidden email]
>Subject: [sqlite] Segfault when query again in-memory db
>
>Hi
>I am trying to provide a asychnonous queue based on sqlite3 in Python
>Current, I already passted the file based tests with my queue
>When switching file to in-memory db, I keep meeting a "segfault"
>issue when running the same tests test suite
>
>Can anyone help me out of this situation?
>
>
>I tested the in-memory with multi-thread (but inserts/deletes are in
>a lock)
>Here is the trace of the core file:
>
>(gdb) bt
>#0  sqlite3_value_type (pVal=0x0) at sqlite3.c:72512
>#1  0x00007fda2a34fd86 in sqlite3_column_type (pStmt=0x7fda08004178,
>i=<optimized out>) at sqlite3.c:73318
>#2  0x00007fda2a607987 in _pysqlite_fetch_one_row
>(self=self@entry=0x7fda2a204b20) at /build/python2.7-
>1tJBSB/python2.7-2.7.12/Modules/_sqlite/cursor.c:352
>#3  0x00007fda2a6089c1 in _pysqlite_query_execute
>(self=0x7fda2a204b20, multiple=<optimized out>, args=<optimized out>)
>    at /build/python2.7-1tJBSB/python2.7-
>2.7.12/Modules/_sqlite/cursor.c:711
>#4  0x00000000004b0cb3 in PyObject_Call () at
>../Objects/abstract.c:2546
>#5  0x00000000004ce5d0 in PyEval_CallObjectWithKeywords () at
>../Python/ceval.c:4219
>#6  0x00007fda2a609106 in pysqlite_connection_execute
>(self=<optimized out>, args=('SELECT _id, data FROM queue_default
>ORDER BY _id ASC LIMIT 1', ()))
>    at /build/python2.7-1tJBSB/python2.7-
>2.7.12/Modules/_sqlite/connection.c:1262
>#7  0x00000000004cb945 in call_function (oparg=<optimized out>,
>pp_stack=0x7fda19ff9fc0) at ../Python/ceval.c:4350
>#8  PyEval_EvalFrameEx () at ../Python/ceval.c:2987
>#9  0x00000000004c2765 in PyEval_EvalCodeEx () at
>../Python/ceval.c:3582
>#10 0x00000000004ca8d1 in fast_function (nk=0, na=<optimized out>,
>n=<optimized out>, pp_stack=0x7fda19ffa1d0, func=<function at remote
>0x7fda2a83c050>)
>    at ../Python/ceval.c:4445
>#11 call_function (oparg=<optimized out>, pp_stack=0x7fda19ffa1d0) at
>../Python/ceval.c:4370
>#12 PyEval_EvalFrameEx () at ../Python/ceval.c:2987
>#13 0x00000000004c9d8f in fast_function (nk=<optimized out>,
>na=<optimized out>, n=<optimized out>, pp_stack=0x7fda19ffa320,
>    func=<function at remote 0x7fda2a8416e0>) at
>../Python/ceval.c:4435
>#14 call_function (oparg=<optimized out>, pp_stack=0x7fda19ffa320) at
>../Python/ceval.c:4370
>#15 PyEval_EvalFrameEx () at ../Python/ceval.c:2987
>#16 0x00000000004c2765 in PyEval_EvalCodeEx () at
>../Python/ceval.c:3582
>#17 0x00000000004ca099 in fast_function (nk=1, na=<optimized out>,
>n=<optimized out>, pp_stack=0x7fda19ffa530, func=<function at remote
>0x7fda2a841758>)
>    at ../Python/ceval.c:4445
>#18 call_function (oparg=<optimized out>, pp_stack=0x7fda19ffa530) at
>../Python/ceval.c:4370
>#19 PyEval_EvalFrameEx () at ../Python/ceval.c:2987
>#20 0x00000000004c2765 in PyEval_EvalCodeEx () at
>../Python/ceval.c:3582
>#21 0x00000000004de8b8 in function_call.lto_priv () at
>../Objects/funcobject.c:523
>#22 0x00000000004b0cb3 in PyObject_Call () at
>../Objects/abstract.c:2546
>#23 0x00000000004c6ad1 in ext_do_call (nk=<optimized out>,
>na=<optimized out>, flags=<optimized out>, pp_stack=0x7fda19ffa7e8,
>    func=<function at remote 0x7fda29fd6c80>) at
>../Python/ceval.c:4664
>#24 PyEval_EvalFrameEx () at ../Python/ceval.c:3026
>#25 0x00000000004c9d8f in fast_function (nk=<optimized out>,
>na=<optimized out>, n=<optimized out>, pp_stack=0x7fda19ffa930,
>    func=<function at remote 0x7fda2da20758>) at
>../Python/ceval.c:4435
>#26 call_function (oparg=<optimized out>, pp_stack=0x7fda19ffa930) at
>../Python/ceval.c:4370
>#27 PyEval_EvalFrameEx () at ../Python/ceval.c:2987
>#28 0x00000000004c9d8f in fast_function (nk=<optimized out>,
>na=<optimized out>, n=<optimized out>, pp_stack=0x7fda19ffaa80,
>---Type <return> to continue, or q <return> to quit---
>    func=<function at remote 0x7fda2da208c0>) at
>../Python/ceval.c:4435
>#29 call_function (oparg=<optimized out>, pp_stack=0x7fda19ffaa80) at
>../Python/ceval.c:4370
>#30 PyEval_EvalFrameEx () at ../Python/ceval.c:2987
>#31 0x00000000004c2765 in PyEval_EvalCodeEx () at
>../Python/ceval.c:3582
>#32 0x00000000004de6fe in function_call.lto_priv () at
>../Objects/funcobject.c:523
>#33 0x00000000004b0cb3 in PyObject_Call () at
>../Objects/abstract.c:2546
>#34 0x00000000004f492e in instancemethod_call.lto_priv () at
>../Objects/classobject.c:2602
>#35 0x00000000004b0cb3 in PyObject_Call () at
>../Objects/abstract.c:2546
>#36 0x00000000004ce5d0 in PyEval_CallObjectWithKeywords () at
>../Python/ceval.c:4219
>#37 0x0000000000597e32 in t_bootstrap () at
>../Modules/threadmodule.c:620
>#38 0x00007fda2d77f6ba in start_thread (arg=0x7fda19ffb700) at
>pthread_create.c:333
>#39 0x00007fda2d4b53dd in clone () at
>../sysdeps/unix/sysv/linux/x86_64/clone.S:109
>
>Here is the python trace:
>
>(gdb) py-bt
>Traceback (most recent call first):
>  File "/home/user/Documents/persist-queue/persistqueue/sqlbase.py",
>line 140, in _select
>    def _select(self, *args):
>  File "/home/user/Documents/persist-queue/persistqueue/sqlqueue.py",
>line 49, in _pop
>    row = self._select()
>  File "/home/wangp11/Documents/persist-
>queue/persistqueue/sqlqueue.py", line 67, in get
>    pickled = self._pop()
>  File "/home/user/Documents/persist-queue/tests/test_sqlqueue.py",
>line 105, in consumer
>    x = m_queue.get(block=True)
>  File "/usr/lib/python2.7/threading.py", line 754, in run
>    self.__target(*self.__args, **self.__kwargs)
>  File "/usr/lib/python2.7/threading.py", line 801, in
>__bootstrap_inner
>    self.run()
>  File "/usr/lib/python2.7/threading.py", line 774, in __bootstrap
>    self.__bootstrap_inner()
>_______________________________________________
>sqlite-users mailing list
>[hidden email]
>http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users



_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Segfault when query again in-memory db

Peter Wang
In reply to this post by Richard Hipp-3
Hi Richard

Thanks for taking care of the issue.

I was using the the ubuntu 16.04.3 with
/usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6,
it's shipped byubuntu.

I compiled the latest sqlite3 with the suggested params, I force the python
to load the debug sqlite3.so and my app is not crashing now,

Unfortunately, my tests are blocked, by a lock in sqlite3.c "#2
0x00007fde23735089 in pthreadMutexEnter (p=0x285d220) at sqlite3.c:24023"


from trace of my python app, an "INSERT"(thread 3) and an "SELECT" (thread
2) failed into a dead lock and never had a chance to break up.

::
(gdb) thread apply all py-bt

Thread 3 (Thread 0x7fde232a6700 (LWP 4135)):
Traceback (most recent call first):
  Waiting for the GIL
  <built-in method execute of sqlite3.Cursor object at remote
0x7fde236fa490>
  <built-in method execute of sqlite3.Connection object at remote
0x7fde23c799d0>
  File "/home/user/Documents/persist-queue/persistqueue/sqlbase.py", line
17, in _execute
    tran.execute(stat, param)
  File "/home/user/Documents/persist-queue/persistqueue/sqlqueue.py", line
39, in put
    self._insert_into(obj, _time.time())
  File "/home/user/Documents/persist-queue/tests/test_sqlqueue.py", line
99, in producer
    m_queue.put('var%d' % i)
  File "/usr/lib/python3.5/threading.py", line 862, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.5/threading.py", line 882, in _bootstrap
    self._bootstrap_inner()

Thread 2 (Thread 0x7fde222a4700 (LWP 4134)):
Traceback (most recent call first):
  <built-in method execute of sqlite3.Cursor object at remote
0x7fde236fa340>
  <built-in method execute of sqlite3.Connection object at remote
0x7fde23c799d0>
  File "/home/user/Documents/persist-queue/persistqueue/sqlbase.py", line
149, in _select
    return self._getter.execute(self._sql_select, args).fetchone()
  File "/home/user/Documents/persist-queue/persistqueue/sqlqueue.py", line
49, in _pop
    row = self._select()
  File "/home/user/Documents/persist-queue/persistqueue/sqlqueue.py", line
71, in get
    pickled = self._pop()
  File "/home/user/Documents/persist-queue/tests/test_sqlqueue.py", line
105, in consumer
    x = m_queue.get(block=True)
  File "/usr/lib/python3.5/threading.py", line 862, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.5/threading.py", line 882, in _bootstrap
---Type <return> to continue, or q <return> to quit---
    self._bootstrap_inner()

Thread 1 (Thread 0x7fde276f4700 (LWP 4121)):
Traceback (most recent call first):
  <built-in method acquire of _thread.lock object at remote 0x7fde23c3e0d0>
  File "/usr/lib/python3.5/threading.py", line 1070, in
_wait_for_tstate_lock
    elif lock.acquire(block, timeout):
  File "/usr/lib/python3.5/threading.py", line 1054, in join
    self._wait_for_tstate_lock()
  File "/home/user/Documents/persist-queue/tests/test_sqlqueue.py", line
112, in test_multi_threaded_parallel
    p.join()
  File "/usr/lib/python3.5/unittest/case.py", line 600, in run
    testMethod()
  File "/usr/lib/python3.5/unittest/case.py", line 648, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib/python3.5/unittest/suite.py", line 122, in run
    test(result)
  File "/usr/lib/python3.5/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib/python3.5/unittest/suite.py", line 122, in run
    test(result)
  File "/usr/lib/python3.5/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib/python3.5/unittest/suite.py", line 122, in run
    test(result)
  File "/usr/lib/python3.5/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
  File
"/home/user/Documents/persist-queue/.tox/py35/lib/python3.5/site-packages/nose2/runner.py",
line 41, in <lambda>
    executor = lambda suite, result: suite(result)
  File
"/home/user/Documents/persist-queue/.tox/py35/lib/python3.5/site-packages/nose2/runner.py",
line 53, in run
    executor(test, result)
  File
"/home/user/Documents/persist-queue/.tox/py35/lib/python3.5/site-packages/nose2/main.py",
line 271, in runTests
    self.result = runner.run(self.test)
  File "/usr/lib/python3.5/unittest/main.py", line 94, in __init__
    self.runTests()
  File
"/home/user/Documents/persist-queue/.tox/py35/lib/python3.5/site-packages/nose2/main.py",
line 100, in __init__
---Type <return> to continue, or q <return> to quit---
    super(PluggableTestProgram, self).__init__(**kw)
  File
"/home/user/Documents/persist-queue/.tox/py35/lib/python3.5/site-packages/nose2/main.py",
line 306, in discover
    return main(*args, **kwargs)
  File "/home/user/Documents/persist-queue/.tox/py35/bin/nose2", line 11,
in <module>
    sys.exit(discover())

====

Reproduce:

1. checkout my repo:
$ git clone https://github.com/peter-wangxu -b memory_shared_cache
2. run the tox
 $ tox -e py35 -- --verbose tests.test_sqlqueue.SQLite3QueueInMemory

Above will reproduce the hung issue.



Thanks
Peter

On Tue, Oct 31, 2017 at 10:28 PM, Richard Hipp <[hidden email]> wrote:

> On 10/31/17, Wang, Peter (Xu) <[hidden email]> wrote:
> > Hi
> > I am trying to provide a asychnonous queue based on sqlite3 in Python
> > Current, I already passted the file based tests with my queue
> > When switching file to in-memory db, I keep meeting a "segfault" issue
> when
> > running the same tests test suite
> >
> > Can anyone help me out of this situation?
>
> Can you get us a reproducible test case?
>
> Can you tell us what version of SQLite you are using?
>
> Can you recompile the SQLite shared library using -DSQLITE_DEBUG and
> -DSQLITE_ENABLE_API_ARMOR and see how that changes the outcome?
>
> >
> >
> > I tested the in-memory with multi-thread (but inserts/deletes are in a
> lock)
> > Here is the trace of the core file:
> >
> > (gdb) bt
> > #0  sqlite3_value_type (pVal=0x0) at sqlite3.c:72512
> > #1  0x00007fda2a34fd86 in sqlite3_column_type (pStmt=0x7fda08004178,
> > i=<optimized out>) at sqlite3.c:73318
> > #2  0x00007fda2a607987 in _pysqlite_fetch_one_row
> > (self=self@entry=0x7fda2a204b20) at
> > /build/python2.7-1tJBSB/python2.7-2.7.12/Modules/_sqlite/cursor.c:352
> > #3  0x00007fda2a6089c1 in _pysqlite_query_execute (self=0x7fda2a204b20,
> > multiple=<optimized out>, args=<optimized out>)
> >     at /build/python2.7-1tJBSB/python2.7-2.7.12/Modules/_
> sqlite/cursor.c:711
> > #4  0x00000000004b0cb3 in PyObject_Call () at ../Objects/abstract.c:2546
> > #5  0x00000000004ce5d0 in PyEval_CallObjectWithKeywords () at
> > ../Python/ceval.c:4219
> > #6  0x00007fda2a609106 in pysqlite_connection_execute (self=<optimized
> out>,
> > args=('SELECT _id, data FROM queue_default ORDER BY _id ASC LIMIT 1',
> ()))
> >     at
> > /build/python2.7-1tJBSB/python2.7-2.7.12/Modules/_
> sqlite/connection.c:1262
> > #7  0x00000000004cb945 in call_function (oparg=<optimized out>,
> > pp_stack=0x7fda19ff9fc0) at ../Python/ceval.c:4350
> > #8  PyEval_EvalFrameEx () at ../Python/ceval.c:2987
> > #9  0x00000000004c2765 in PyEval_EvalCodeEx () at ../Python/ceval.c:3582
> > #10 0x00000000004ca8d1 in fast_function (nk=0, na=<optimized out>,
> > n=<optimized out>, pp_stack=0x7fda19ffa1d0, func=<function at remote
> > 0x7fda2a83c050>)
> >     at ../Python/ceval.c:4445
> > #11 call_function (oparg=<optimized out>, pp_stack=0x7fda19ffa1d0) at
> > ../Python/ceval.c:4370
> > #12 PyEval_EvalFrameEx () at ../Python/ceval.c:2987
> > #13 0x00000000004c9d8f in fast_function (nk=<optimized out>,
> na=<optimized
> > out>, n=<optimized out>, pp_stack=0x7fda19ffa320,
> >     func=<function at remote 0x7fda2a8416e0>) at ../Python/ceval.c:4435
> > #14 call_function (oparg=<optimized out>, pp_stack=0x7fda19ffa320) at
> > ../Python/ceval.c:4370
> > #15 PyEval_EvalFrameEx () at ../Python/ceval.c:2987
> > #16 0x00000000004c2765 in PyEval_EvalCodeEx () at ../Python/ceval.c:3582
> > #17 0x00000000004ca099 in fast_function (nk=1, na=<optimized out>,
> > n=<optimized out>, pp_stack=0x7fda19ffa530, func=<function at remote
> > 0x7fda2a841758>)
> >     at ../Python/ceval.c:4445
> > #18 call_function (oparg=<optimized out>, pp_stack=0x7fda19ffa530) at
> > ../Python/ceval.c:4370
> > #19 PyEval_EvalFrameEx () at ../Python/ceval.c:2987
> > #20 0x00000000004c2765 in PyEval_EvalCodeEx () at ../Python/ceval.c:3582
> > #21 0x00000000004de8b8 in function_call.lto_priv () at
> > ../Objects/funcobject.c:523
> > #22 0x00000000004b0cb3 in PyObject_Call () at ../Objects/abstract.c:2546
> > #23 0x00000000004c6ad1 in ext_do_call (nk=<optimized out>, na=<optimized
> > out>, flags=<optimized out>, pp_stack=0x7fda19ffa7e8,
> >     func=<function at remote 0x7fda29fd6c80>) at ../Python/ceval.c:4664
> > #24 PyEval_EvalFrameEx () at ../Python/ceval.c:3026
> > #25 0x00000000004c9d8f in fast_function (nk=<optimized out>,
> na=<optimized
> > out>, n=<optimized out>, pp_stack=0x7fda19ffa930,
> >     func=<function at remote 0x7fda2da20758>) at ../Python/ceval.c:4435
> > #26 call_function (oparg=<optimized out>, pp_stack=0x7fda19ffa930) at
> > ../Python/ceval.c:4370
> > #27 PyEval_EvalFrameEx () at ../Python/ceval.c:2987
> > #28 0x00000000004c9d8f in fast_function (nk=<optimized out>,
> na=<optimized
> > out>, n=<optimized out>, pp_stack=0x7fda19ffaa80,
> > ---Type <return> to continue, or q <return> to quit---
> >     func=<function at remote 0x7fda2da208c0>) at ../Python/ceval.c:4435
> > #29 call_function (oparg=<optimized out>, pp_stack=0x7fda19ffaa80) at
> > ../Python/ceval.c:4370
> > #30 PyEval_EvalFrameEx () at ../Python/ceval.c:2987
> > #31 0x00000000004c2765 in PyEval_EvalCodeEx () at ../Python/ceval.c:3582
> > #32 0x00000000004de6fe in function_call.lto_priv () at
> > ../Objects/funcobject.c:523
> > #33 0x00000000004b0cb3 in PyObject_Call () at ../Objects/abstract.c:2546
> > #34 0x00000000004f492e in instancemethod_call.lto_priv () at
> > ../Objects/classobject.c:2602
> > #35 0x00000000004b0cb3 in PyObject_Call () at ../Objects/abstract.c:2546
> > #36 0x00000000004ce5d0 in PyEval_CallObjectWithKeywords () at
> > ../Python/ceval.c:4219
> > #37 0x0000000000597e32 in t_bootstrap () at ../Modules/threadmodule.c:620
> > #38 0x00007fda2d77f6ba in start_thread (arg=0x7fda19ffb700) at
> > pthread_create.c:333
> > #39 0x00007fda2d4b53dd in clone () at
> > ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
> >
> > Here is the python trace:
> >
> > (gdb) py-bt
> > Traceback (most recent call first):
> >   File "/home/user/Documents/persist-queue/persistqueue/sqlbase.py",
> line
> > 140, in _select
> >     def _select(self, *args):
> >   File "/home/user/Documents/persist-queue/persistqueue/sqlqueue.py",
> line
> > 49, in _pop
> >     row = self._select()
> >   File "/home/wangp11/Documents/persist-queue/persistqueue/sqlqueue.py",
> > line 67, in get
> >     pickled = self._pop()
> >   File "/home/user/Documents/persist-queue/tests/test_sqlqueue.py", line
> > 105, in consumer
> >     x = m_queue.get(block=True)
> >   File "/usr/lib/python2.7/threading.py", line 754, in run
> >     self.__target(*self.__args, **self.__kwargs)
> >   File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
> >     self.run()
> >   File "/usr/lib/python2.7/threading.py", line 774, in __bootstrap
> >     self.__bootstrap_inner()
> > _______________________________________________
> > sqlite-users mailing list
> > [hidden email]
> > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> >
>
>
> --
> D. Richard Hipp
> [hidden email]
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users