Question about Memsys5 Internals

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Question about Memsys5 Internals

Philip Bennefall
Hi all,

First, if this is the wrong place to ask this question, please let me
know and I will ask elsewhere.

I have a question about memsys5, which I am using as a general purpose
memory pool in my application. I am allocating fixed size objects, and I
have set mReq to the size of that object (it is a power of 2). I know
that I will only be using N objects at any given time. Is there an easy
way to calculate how much space I need to reserve in order to hold N
objects simultaneously? If I allocate a total of N*sizeof(object) bytes,
it doesn't seem to let me store N objects at the same time. Am I doing
something wrong or is this expected?

Thanks in advance for any tips!

Kind regards,

Philip Bennefall
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Question about Memsys5 Internals

Hick Gunter
This should be computable from the base addresses returned when allocating successive objects.


-----Urspr√ľngliche Nachricht-----
Von: sqlite-users [mailto:[hidden email]] Im Auftrag von Philip Bennefall
Gesendet: Montag, 05. September 2016 08:22
An: General Discussion of SQLite Database <[hidden email]>
Betreff: [sqlite] Question about Memsys5 Internals

Hi all,

First, if this is the wrong place to ask this question, please let me know and I will ask elsewhere.

I have a question about memsys5, which I am using as a general purpose memory pool in my application. I am allocating fixed size objects, and I have set mReq to the size of that object (it is a power of 2). I know that I will only be using N objects at any given time. Is there an easy way to calculate how much space I need to reserve in order to hold N objects simultaneously? If I allocate a total of N*sizeof(object) bytes, it doesn't seem to let me store N objects at the same time. Am I doing something wrong or is this expected?

Thanks in advance for any tips!

Kind regards,

Philip Bennefall
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


___________________________________________
 Gunter Hick
Software Engineer
Scientific Games International GmbH
FN 157284 a, HG Wien
Klitschgasse 2-4, A-1130 Vienna, Austria
Tel: +43 1 80100 0
E-Mail: [hidden email]

This communication (including any attachments) is intended for the use of the intended recipient(s) only and may contain information that is confidential, privileged or legally protected. Any unauthorized use or dissemination of this communication is strictly prohibited. If you have received this communication in error, please immediately notify the sender by return e-mail message and delete all copies of the original communication. Thank you for your cooperation.


_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Question about Memsys5 Internals

Richard Hipp-3
In reply to this post by Philip Bennefall
On 9/5/16, Philip Bennefall <[hidden email]> wrote:

> Hi all,
>
> First, if this is the wrong place to ask this question, please let me
> know and I will ask elsewhere.
>
> I have a question about memsys5, which I am using as a general purpose
> memory pool in my application. I am allocating fixed size objects, and I
> have set mReq to the size of that object (it is a power of 2). I know
> that I will only be using N objects at any given time. Is there an easy
> way to calculate how much space I need to reserve in order to hold N
> objects simultaneously? If I allocate a total of N*sizeof(object) bytes,
> it doesn't seem to let me store N objects at the same time. Am I doing
> something wrong or is this expected?
>

Let sz be the number of bytes of memory you provide to the memory
allocator.  The number of minimum-size objects that can be stored is
sz/(mnReq+1).  Not sz/mnReq.  The extra +1 is some space taken from
the front of the provided memory and used for internal bookkeeping.

--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Question about Memsys5 Internals

Philip Bennefall
Thanks, Richard. I actually arrived at that by experimenting, but I had
no idea whether I was doing it right. Thanks for the explanation - it
confirms that I've got it set up correctly.

Kind regards,

Philip Bennefall

On 9/5/2016 12:13 PM, Richard Hipp wrote:

> On 9/5/16, Philip Bennefall <[hidden email]> wrote:
>> Hi all,
>>
>> First, if this is the wrong place to ask this question, please let me
>> know and I will ask elsewhere.
>>
>> I have a question about memsys5, which I am using as a general purpose
>> memory pool in my application. I am allocating fixed size objects, and I
>> have set mReq to the size of that object (it is a power of 2). I know
>> that I will only be using N objects at any given time. Is there an easy
>> way to calculate how much space I need to reserve in order to hold N
>> objects simultaneously? If I allocate a total of N*sizeof(object) bytes,
>> it doesn't seem to let me store N objects at the same time. Am I doing
>> something wrong or is this expected?
>>
> Let sz be the number of bytes of memory you provide to the memory
> allocator.  The number of minimum-size objects that can be stored is
> sz/(mnReq+1).  Not sz/mnReq.  The extra +1 is some space taken from
> the front of the provided memory and used for internal bookkeeping.
>
> --
> D. Richard Hipp
> [hidden email]
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Question about Memsys5 Internals

Richard Hipp-3
In reply to this post by Philip Bennefall
On 9/5/16, Philip Bennefall <[hidden email]> wrote:
>
> I am using [memsys5] as a general purpose
> memory pool in my application.

There is a strong feeling in some parts of the open-source community
that nobody should ever attempt to build their own memory allocator.
The malloc/free from the standard library is believed to be good
enough for every situation and any attempt to work around standard
malloc/free is discouraged.

SQLite uses system malloc/free by default, as you know.  But even the
presence of memsys5 in the source tree as a compile-time option is
controversial in some circles.  I put memsys5 there for good reasons,
but they are obscure reasons, which is why memsys5 is usually omitted
from the build.

So I'm wondering:  Why are *you* using memsys5 rather than malloc/free
from the standard library?
--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Question about Memsys5 Internals

Philip Bennefall
Hi Richard,

Our library runs in video games (on both consoles and mobile devices),
and many game developers detest calls to malloc/free at run-time but
they are happy to give me a sizable chunk in one go. So I'm using
memsys5 as an optional memory pool in an attempt to reduce the number of
*user* allocations. The user is able to plug in their own malloc/free
replacements as well in which case the memory pool is redundant, but the
pool is there as a shield in the default configuration.

Kind regards,

Philip Bennefall
On 9/5/2016 7:39 PM, Richard Hipp wrote:

> On 9/5/16, Philip Bennefall <[hidden email]> wrote:
>> I am using [memsys5] as a general purpose
>> memory pool in my application.
> There is a strong feeling in some parts of the open-source community
> that nobody should ever attempt to build their own memory allocator.
> The malloc/free from the standard library is believed to be good
> enough for every situation and any attempt to work around standard
> malloc/free is discouraged.
>
> SQLite uses system malloc/free by default, as you know.  But even the
> presence of memsys5 in the source tree as a compile-time option is
> controversial in some circles.  I put memsys5 there for good reasons,
> but they are obscure reasons, which is why memsys5 is usually omitted
> from the build.
>
> So I'm wondering:  Why are *you* using memsys5 rather than malloc/free
> from the standard library?

_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users