Quantcast

using the same sqlite parameter more than once causes premature memory deallocation

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

using the same sqlite parameter more than once causes premature memory deallocation

abbood
if create an sqlite statement that uses the same parameter more than once ie

    NSString* updateStmt = @"INSERT INTO search_email(..., subject, ...)"
    " SELECT ..., :subject, ...,"
    " coalesce((SELECT search_email.threadID "
    " FROM search_email "
    " WHERE search_email.subject MATCH :subject2 "
    " ),"
    " :uid"
    " )";

int subjectIndex = sqlite3_bind_parameter_index(searchEmailInsertStmt,":subject");
int subjectIndex2 = sqlite3_bind_parameter_index(searchEmailInsertStmt,":subject2");

...    
sqlite3_bind_text(searchEmailInsertStmt, subjectIndex, [subject UTF8String], -1, SQLITE_TRANSIENT);        // subject
sqlite3_bind_text(searchEmailInsertStmt, subjectIndex2, [subjectCopy UTF8String], -1, SQLITE_TRANSIENT);        // search_email.subject
then it crashes with the following error: malloc: *** error for object 0x9b6350: pointer being freed was not allocated *** set a breakpoint in malloc_error_break to debug

any idea why?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: using the same sqlite parameter more than once causes premature memory deallocation

Roger Binns
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 24/01/13 00:24, abbood wrote:
> then it crashes with the following error: malloc: *** error for object
> 0x9b6350: pointer being freed was not allocated *** set a breakpoint
> in malloc_error_break to debug

Coincidentally enough I am debugging this exact same issue right now.  Are
you sure the error has anything to do with your binds?  In my case I did
as it said and have found that ARC has added something to the autorelease
pool.  This has nothing directly to do with SQLite and everything to do
with the boundary between Objective C/ARC and C.

Roger

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAlEBu6sACgkQmOOfHg372QSR5gCfU+FAhhp8QuzAB6q1sP8xq2T4
j3wAoMUx3i4dFzzIldgyGJghNtXmsJR6
=+wFc
-----END PGP SIGNATURE-----
_______________________________________________
sqlite-users mailing list
[hidden email]
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: using the same sqlite parameter more than once causes premature memory deallocation

abbood

Well I wish i could say the same but my app isn't even using arc.. Actually some files do others don't.. But the file that contains this code doesn't..

I'm 100%that it has something to do with the binds.. But I can't point out which exactly.. Btw I'm curious how did you find out about this auto release thing? Can you add more detail about that?

Ie I put a break point in malloc_error_break.. But then I just jump to the internals of sqlite.. And basically within the salite internals it's freeing an operation that doesn't exist.. I thought of editing the internals of sqlite.c and capturing the error and bubbling it up all the way to my objective c code and catching it there.. But im not sure if it's a good idea to modify the guts of sqlite.. Is it?

For the time being I broke the sql statement down into two parts to simply narrow down the source of the problem.. And then I made the second part use in line variables as opposed to bindings.. Ie NSString stringwithformat.. That worked but it kept on giving me the same error with the same sql row all the time.. So I deleted that row from the dbase and it works fine.. I couldn't find anything different on that row..

So I'm still not satisfied because I couldn't identify the cause of the problem..

On Jan 25, 2013 12:54 AM, "Roger Binns [via SQLite]" <[hidden email]> wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 24/01/13 00:24, abbood wrote:
> then it crashes with the following error: malloc: *** error for object
> 0x9b6350: pointer being freed was not allocated *** set a breakpoint
> in malloc_error_break to debug

Coincidentally enough I am debugging this exact same issue right now.  Are
you sure the error has anything to do with your binds?  In my case I did
as it said and have found that ARC has added something to the autorelease
pool.  This has nothing directly to do with SQLite and everything to do
with the boundary between Objective C/ARC and C.

Roger

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAlEBu6sACgkQmOOfHg372QSR5gCfU+FAhhp8QuzAB6q1sP8xq2T4
j3wAoMUx3i4dFzzIldgyGJghNtXmsJR6
=+wFc
-----END PGP SIGNATURE-----
_______________________________________________
sqlite-users mailing list
[hidden email]
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users



To unsubscribe from using the same sqlite parameter more than once causes premature memory deallocation, click here.
NAML
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: using the same sqlite parameter more than once causes premature memory deallocation

Roger Binns
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 24/01/13 17:37, abbood wrote:
> Btw I'm curious how did you find out about this auto release thing? Can
> you add more detail about that?

I used a debugger to set a breakpoint in malloc_error_debug as the message
says.  That gave a stack trace of an autorelease pool being drained.

Since most frees happen in the autorelease pool drain it doesn't
particularly help with telling you which item is the problem.  I asked on
the valgrind list and got a way of keeping the allocation stack trace
instead of the free one.  (The next valgrind release will allow you to
print both.)

  http://article.gmane.org/gmane.comp.debugging.valgrind/12755

The cause of that original issue in my code was NSInvocation and dealing
with returned objects.  The cause of the most recent issue was because
NSData was owning a buffer passed to it that I didn't want it to.

> Ie I put a break point in malloc_error_break.. But then I just jump to
> the internals of sqlite.. And basically within the salite internals
> it's freeing an operation that doesn't exist..

Using valgrind will narrow down the problem.  What you are seeing is the
consequence of earlier memory errors.

> But im not sure if it's a good idea to modify the guts of sqlite.. Is
> it?

SQLite is *extremely* unlikely to have a bug.  Some other piece of code
has the bug, and SQLite is the victim.  Remember that virtually every web
browser on virtually every platform is using SQLite - an error would show
up for someone.

  http://www.sqlite.org/testing.html

> And then I made the second part use in line variables as opposed to
> bindings.. Ie NSString stringwithformat..

Do remember sqlite3_mprintf for that sort of thing, especially if strings
are involved to avoid sql injection attacks/bugs:

  http://www.sqlite.org/c3ref/mprintf.html

> So I deleted that row from the dbase and it works fine.. I couldn't
> find anything different on that row..

Memory allocators typically have buckets for different sized allocations
often in powers of two.  It could be that row had a string needing 33
bytes while others needed less which then caused memory to come out of a
different bucket which then changes where the victim of the actual bug
shows up.

Roger
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAlEC2o8ACgkQmOOfHg372QQQgACg2u97/wfBys3ryf/EZphv0R43
hjUAoLh2/anUjqGWa+GxC+7GO5tt3D0L
=X0kB
-----END PGP SIGNATURE-----
_______________________________________________
sqlite-users mailing list
[hidden email]
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: using the same sqlite parameter more than once causes premature memory deallocation

abbood
i fixed it!! you were right! it's not to do with the guts of sql.. rather it's to do with my incorrect sql statement..

i used MATCH instead of =.. for more details see the stack over flow response i made here http://stackoverflow.com/a/14534957/766570

A


On Fri, Jan 25, 2013 at 9:18 PM, Roger Binns [via SQLite] <[hidden email]> wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 24/01/13 17:37, abbood wrote:
> Btw I'm curious how did you find out about this auto release thing? Can
> you add more detail about that?

I used a debugger to set a breakpoint in malloc_error_debug as the message
says.  That gave a stack trace of an autorelease pool being drained.

Since most frees happen in the autorelease pool drain it doesn't
particularly help with telling you which item is the problem.  I asked on
the valgrind list and got a way of keeping the allocation stack trace
instead of the free one.  (The next valgrind release will allow you to
print both.)

  http://article.gmane.org/gmane.comp.debugging.valgrind/12755

The cause of that original issue in my code was NSInvocation and dealing
with returned objects.  The cause of the most recent issue was because
NSData was owning a buffer passed to it that I didn't want it to.

> Ie I put a break point in malloc_error_break.. But then I just jump to
> the internals of sqlite.. And basically within the salite internals
> it's freeing an operation that doesn't exist..

Using valgrind will narrow down the problem.  What you are seeing is the
consequence of earlier memory errors.

> But im not sure if it's a good idea to modify the guts of sqlite.. Is
> it?

SQLite is *extremely* unlikely to have a bug.  Some other piece of code
has the bug, and SQLite is the victim.  Remember that virtually every web
browser on virtually every platform is using SQLite - an error would show
up for someone.

  http://www.sqlite.org/testing.html


> And then I made the second part use in line variables as opposed to
> bindings.. Ie NSString stringwithformat..

Do remember sqlite3_mprintf for that sort of thing, especially if strings
are involved to avoid sql injection attacks/bugs:

  http://www.sqlite.org/c3ref/mprintf.html


> So I deleted that row from the dbase and it works fine.. I couldn't
> find anything different on that row..

Memory allocators typically have buckets for different sized allocations
often in powers of two.  It could be that row had a string needing 33
bytes while others needed less which then caused memory to come out of a
different bucket which then changes where the victim of the actual bug
shows up.

Roger
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAlEC2o8ACgkQmOOfHg372QQQgACg2u97/wfBys3ryf/EZphv0R43
hjUAoLh2/anUjqGWa+GxC+7GO5tt3D0L
=X0kB
-----END PGP SIGNATURE-----
_______________________________________________
sqlite-users mailing list
[hidden email]
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users



To unsubscribe from using the same sqlite parameter more than once causes premature memory deallocation, click here.
NAML

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: using the same sqlite parameter more than once causes premature memory deallocation

Roger Binns
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 25/01/13 23:45, abbood wrote:
> i fixed it!! you were right! it's not to do with the guts of sql..
> rather it's to do with my incorrect sql statement..

Huh?  There is no SQL statement, valid or not, that can cause memory
errors.  It looks like SQLite is the victim of some other memory
mismanagement in your app and changing the SQL has just changed what code
will fall victim to it.

It is a very good idea to run valgrind before proceeding.

Roger
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAlEEA6AACgkQmOOfHg372QSswACgoIg1jidTzar4EVVfQmFZlDwb
ZuAAn1iIuUz84T4Gpzgv2Q58U1zYq9MR
=z+X5
-----END PGP SIGNATURE-----
_______________________________________________
sqlite-users mailing list
[hidden email]
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: using the same sqlite parameter more than once causes premature memory deallocation

abbood
humm... i think you are right.. the last time i spoke with you the error did indeed disappear.. but now it came back (embarrassed me infront of my client lol).. 

i tried using the memory diagnostics tools in Xcode/Instruments -- Zombies, GuardMalloc, and Malloc Stack Logging.. but they didn't tell me much.. i'm assuming valgrind is better? 

i took a sneak peak at valgrind.. and it looked really complicated.. is it? do you have pointers about some light weight tutorials or intros? i found their documentation and tutorials a bit heavy handed.

A


On Sat, Jan 26, 2013 at 6:26 PM, Roger Binns [via SQLite] <[hidden email]> wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 25/01/13 23:45, abbood wrote:
> i fixed it!! you were right! it's not to do with the guts of sql..
> rather it's to do with my incorrect sql statement..

Huh?  There is no SQL statement, valid or not, that can cause memory
errors.  It looks like SQLite is the victim of some other memory
mismanagement in your app and changing the SQL has just changed what code
will fall victim to it.

It is a very good idea to run valgrind before proceeding.

Roger
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAlEEA6AACgkQmOOfHg372QSswACgoIg1jidTzar4EVVfQmFZlDwb
ZuAAn1iIuUz84T4Gpzgv2Q58U1zYq9MR
=z+X5
-----END PGP SIGNATURE-----
_______________________________________________
sqlite-users mailing list
[hidden email]
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users



To unsubscribe from using the same sqlite parameter more than once causes premature memory deallocation, click here.
NAML

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: using the same sqlite parameter more than once causes premature memory deallocation

Roger Binns
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 08/02/13 06:01, abbood wrote:
> i tried using the memory diagnostics tools in Xcode/Instruments --
> Zombies, GuardMalloc, and Malloc Stack Logging.. but they didn't tell
> me much.. i'm assuming valgrind is better?

They are fairly lightweight and only catch the more simple errors.
valgrind is perfect - it runs your application in such a way that every
single access (read or write) of any byte is tracked as well as the status
of every byte of memory.  This makes things run considerably slower, but
the resulting notifications are top notch.

> i took a sneak peak at valgrind.. and it looked really complicated.. is
> it?

Not in my opinion, and no matter what this is the only tool that will help
you so you'll just have to figure it out.  You just run your app prefixed
with valgrind.  eg if you did this:

  $ myapp arg1 arg2

Then you do this:

  $ valgrind myapp arg1 arg2

It is recommended you compile your app with debugging. Chances are highly
likely that you will then be shown your issue.  It is possible for some
issues to be missed.  One example is that after memory is freed it is kept
in a pool for a while, and then released for reuse making it valid again.
 I have a massive pool so memory is not reused with the option (5GB pool):

  --freelist-vol=50000000000

I also care about memory leaks which valgrind will show on exit.  These
options cause it to show a lot of detail:

  --leak-check=full --leak-resolution=high --show-reachable=yes

Roger
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAlEWueoACgkQmOOfHg372QQtaQCeNfCzwMp2UzNslMjoI538tjBf
0SEAoNEfzvG70jqtWeY7T2LcVfjcFYAM
=jmof
-----END PGP SIGNATURE-----
_______________________________________________
sqlite-users mailing list
[hidden email]
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Loading...