Size of the SQLite library

classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Size of the SQLite library

Richard Hipp-3
For many years, we have boasted that the size of the SQLite library is
"less than half a megabyte".  That will likely still be true in the
3.24.0 release, though just barely.  Compiling with gcc 5.4.0 and -Os
on ubuntu, I get 499,709 bytes.  With gcc 7.1.0 and -Os I get 496,399
bytes.

The library is, of course, larger if you enable optional features such
as full-text search and/or use compiler optimizations like -O3 which
enable loop unrolling and function inlining.  And most people do
compile it that way.  So "less than a megabyte" might be a more
accurate description of SQLite in practice.  But the default
configuration compiled with -Os is a good metric for comparison.

See https://sqlite.org/tmp/size-20180531.jpg for the library size
trend over 5 years.  The measurements in the graph were done with gcc
5.4.0 and -Os on ubuntu.  As you can see, we have held the line below
500,000 bytes for a long time.  But the recent addition of new
features (ex: UPSERT) has caused a slight uptick in the library size.
As further new features are in the pipeline, the upcoming 3.24.0
release will probably be the last for which the library size comes in
at less than 500,000 bytes.  For this reason, I will probably change
the size bullet point to say "less than 500 kibibytes (KiB)" or "less
than 0.5 mebibytes (MiB)", as "less than 600KB" does not have quite
the same emotional impact.  You will notice that the graph linked
above is calibrated in mebibytes.
--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Size of the SQLite library

R Smith-2

On 2018/05/31 3:44 PM, Richard Hipp wrote:

> For many years, we have boasted that the size of the SQLite library is
> "less than half a megabyte".  That will likely still be true in the
> 3.24.0 release, though just barely.  Compiling with gcc 5.4.0 and -Os
> on ubuntu, I get 499,709 bytes.  With gcc 7.1.0 and -Os I get 496,399
> bytes.
>
> The library is, of course, larger if you enable optional features such
> as full-text search and/or use compiler optimizations like -O3 which
> enable loop unrolling and function inlining.  And most people do
> compile it that way.  So "less than a megabyte" might be a more
> accurate description of SQLite in practice.  But the default
> configuration compiled with -Os is a good metric for comparison.
>
> See https://sqlite.org/tmp/size-20180531.jpg for the library size
> trend over 5 years.  The measurements in the graph were done with gcc
> 5.4.0 and -Os on ubuntu.  As you can see, we have held the line below
> 500,000 bytes for a long time.  But the recent addition of new
> features (ex: UPSERT) has caused a slight uptick in the library size.
> As further new features are in the pipeline, the upcoming 3.24.0
> release will probably be the last for which the library size comes in
> at less than 500,000 bytes.  For this reason, I will probably change
> the size bullet point to say "less than 500 kibibytes (KiB)" or "less
> than 0.5 mebibytes (MiB)", as "less than 600KB" does not have quite
> the same emotional impact.  You will notice that the graph linked
> above is calibrated in mebibytes.

Nice idea, but to be honest, I can't remember when last someone cared
about "Kilobytes", and I mean embedded people, not big OSes.

The measure of importance is how expensive the DATA storing is, both in
size and write-frequency, when committed to some hardware NANDs. The
code store section of even the smallest modern embedded system will be
designed to fit things many megabytes more than SQLite requires
(exceptions may exist, but are really thin on the ground). So then,
whether the operating code is given in KB or MiB or KiB is, to my mind,
not very relevant - and it too will become untrue in a non-too-distant
future.

May I propose, if changing is on the table, to rather update to a more
current universal notion (and modern embedded capacities) and make it
leaner by removing one word and simply call it:

"less than a megabyte"

This has much the same emotional impact, still is downright amazing,
doesn't require naming shenanigans, is very TRUE, even with a few funny
switches compiled-in, AND will remain true for many years to come,
possibly to the end of the SQLite lifecycle ~30 years hence.


My 2c...
Ryan


_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Size of the SQLite library

Bob Friesenhahn
On Thu, 31 May 2018, R Smith wrote:
>
> Nice idea, but to be honest, I can't remember when last someone cared about
> "Kilobytes", and I mean embedded people, not big OSes.

I work on embedded projects and we do definitely worry about
"kilobytes".  This is even though our embedded projects have large
resources compared with many other embedded projects.  The firmware
image for some of our products is consuming all available Flash pages,
(except for spares for wear-leveling/repair).

Many embedded projects are very cost-sensitive since they sell into
hyper-competitive markets where being a bit more expensive than the
competition results in a lack of sales.

> The measure of importance is how expensive the DATA storing is, both in size
> and write-frequency, when committed to some hardware NANDs. The code store
> section of even the smallest modern embedded system will be designed to fit
> things many megabytes more than SQLite requires (exceptions may exist, but
> are really thin on the ground). So then, whether the operating code is given
> in KB or MiB or KiB is, to my mind, not very relevant - and it too will
> become untrue in a non-too-distant future.

Your experience is different than mine.  What NOR or NAND Flash chip
are you using on your PCB?  If you are not using a single soldered
chip with a specialized flash filesystem (e.g. JFFS2, UBIFS, squashfs
on UBI or bare) then perhaps you are just using a small form factor
PC which uses components common in laptop PCs.

Bob
--
Bob Friesenhahn
[hidden email], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Size of the SQLite library

veneff
I have to agree with Bob!

We have considered SQLITE for our project.  Going over 500Kbytes puts it
just beyond the size of our Flash - the current Firmware.

Vance

On 2018-05-31 11:04, Bob Friesenhahn wrote:

> On Thu, 31 May 2018, R Smith wrote:
>
>> Nice idea, but to be honest, I can't remember when last someone cared about "Kilobytes", and I mean embedded people, not big OSes.
>
> I work on embedded projects and we do definitely worry about "kilobytes".  This is even though our embedded projects have large resources compared with many other embedded projects.  The firmware image for some of our products is consuming all available Flash pages, (except for spares for wear-leveling/repair).
>
> Many embedded projects are very cost-sensitive since they sell into hyper-competitive markets where being a bit more expensive than the competition results in a lack of sales.
>
>> The measure of importance is how expensive the DATA storing is, both in size and write-frequency, when committed to some hardware NANDs. The code store section of even the smallest modern embedded system will be designed to fit things many megabytes more than SQLite requires (exceptions may exist, but are really thin on the ground). So then, whether the operating code is given in KB or MiB or KiB is, to my mind, not very relevant - and it too will become untrue in a non-too-distant future.
>
> Your experience is different than mine.  What NOR or NAND Flash chip are you using on your PCB?  If you are not using a single soldered chip with a specialized flash filesystem (e.g. JFFS2, UBIFS, squashfs on UBI or bare) then perhaps you are just using a small form factor PC which uses components common in laptop PCs.
>
> Bob
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Size of the SQLite library

Richard Hipp-3
On 5/31/18, [hidden email] <[hidden email]> wrote:
>
> We have considered SQLITE for our project.  Going over 500Kbytes puts it
> just beyond the size of our Flash - the current Firmware.

By using multiple SQLITE_OMIT compile-time options to leave out
features, I can get the size down to 308,189 bytes using gcc-7 -Os
-m32.
--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Size of the SQLite library

Christian Schmitz
In reply to this post by Richard Hipp-3
>
> See https://sqlite.org/tmp/size-20180531.jpg for the library size
> trend over 5 years.

Maybe your graph should have three lines.

1. Minimum SQLite with all off
2. Default SQLite
3. Maximum SQLite with all on

Sincerely
Christian

--
Read our blog about news on our plugins:

http://www.mbsplugins.de/


_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Size of the SQLite library

R Smith-2
In reply to this post by veneff

On 2018/05/31 5:17 PM, [hidden email] wrote:
> I have to agree with Bob!
>
> We have considered SQLITE for our project.  Going over 500Kbytes puts it
> just beyond the size of our Flash - the current Firmware.

I stand corrected! It seems the embedded systems with still an extremely
limited memory footprint size may not be as thin on the ground as I
imagined, and I regret trying to categorize all embedded systems under
the same ideal - apologies for that.

Towards my point though, both Bob and Vance, would you be especially
swayed if the marketing slogan had said "under a megabyte" as opposed to
"under half a megabyte"?  I still feel that this level of embedded
system is not common, and even where it might be common, I bet that
slogan is not the catch phrase that got you interested in SQLite (or
would sway you from choosing it).

It's however clear my view may not be 100% representative, so perhaps
the  KiB or 0.5 MiB route has its place.


_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Size of the SQLite library

Chris Brody
In reply to this post by Richard Hipp-3
On Thu, May 31, 2018 at 11:38 AM, Richard Hipp <[hidden email]> wrote:
> [...]
> By using multiple SQLITE_OMIT compile-time options to leave out
> features, I can get the size down to 308,189 bytes using gcc-7 -Os
> -m32.

@Richard can you elaborate some more on how you make this kind of a build?

I wouldn't mind if we drop some more less-used options from the
default build to keep the standard size "less than half a megabyte".
Also -1 for kibibyte/mebibyte wording on my part.

On Thu, May 31, 2018 at 11:57 AM, Christian Schmitz
<[hidden email]> wrote:
> [...]
> Maybe your graph should have three lines.

+1 would be nice, not major though (I think)

On Thu, May 31, 2018 at 11:58 AM, R Smith <[hidden email]> wrote:
> [...]
> Towards my point though, both Bob and Vance, would you be especially swayed
> if the marketing slogan had said "under a megabyte" as opposed to "under
> half a megabyte"?

I think Vance already gave the answer (negative). Would it be an idea
to have size slogans for both regular and embedded builds?
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Size of the SQLite library

Chris Smith
In reply to this post by R Smith-2
"You are soooooo, bloated," said Java.

On Thu, May 31, 2018, 11:58 R Smith <[hidden email]> wrote:

>
> On 2018/05/31 5:17 PM, [hidden email] wrote:
> > I have to agree with Bob!
> >
> > We have considered SQLITE for our project.  Going over 500Kbytes puts it
> > just beyond the size of our Flash - the current Firmware.
>
> I stand corrected! It seems the embedded systems with still an extremely
> limited memory footprint size may not be as thin on the ground as I
> imagined, and I regret trying to categorize all embedded systems under
> the same ideal - apologies for that.
>
> Towards my point though, both Bob and Vance, would you be especially
> swayed if the marketing slogan had said "under a megabyte" as opposed to
> "under half a megabyte"?  I still feel that this level of embedded
> system is not common, and even where it might be common, I bet that
> slogan is not the catch phrase that got you interested in SQLite (or
> would sway you from choosing it).
>
> It's however clear my view may not be 100% representative, so perhaps
> the  KiB or 0.5 MiB route has its place.
>
>
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
--
Cheers,
Chris
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Size of the SQLite library

Dominique Devienne
In reply to this post by Richard Hipp-3
On Thu, May 31, 2018 at 3:44 PM Richard Hipp <[hidden email]> wrote:

> For many years, we have boasted that the size of the SQLite library is
> "less than half a megabyte".
>

Given where the conversation is going, let me point out that many do not
care one bit about the lib's size :)

I'd much rather have an SQLite with tons of features, than forego those in
the name saving a few bytes,
to save a few bucks on the embedded chip and flash for commercial products
that don't even pay for SQLite.

SQLite is already amazingly small for the value it brings. And if people
want "smaller", they can still stick
with older leaner versions of SQLite too. My $0.02... --DD
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Size of the SQLite library

J. King-3
On May 31, 2018 12:18:51 PM EDT, Dominique Devienne <[hidden email]> wrote:

>On Thu, May 31, 2018 at 3:44 PM Richard Hipp <[hidden email]> wrote:
>
>> For many years, we have boasted that the size of the SQLite library
>is
>> "less than half a megabyte".
>>
>
>Given where the conversation is going, let me point out that many do
>not
>care one bit about the lib's size :)
>
>I'd much rather have an SQLite with tons of features, than forego those
>in
>the name saving a few bytes,
>to save a few bucks on the embedded chip and flash for commercial
>products
>that don't even pay for SQLite.
>
>SQLite is already amazingly small for the value it brings. And if
>people
>want "smaller", they can still stick
>with older leaner versions of SQLite too. My $0.02... --DD
>_______________________________________________
>sqlite-users mailing list
>[hidden email]
>http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

I agree with this sentiment. I mostly use SQLite in PHP, where it is awkward to customize SQLite, and all but impossible to rely on features not included in the standard build when distributing to others. A more powerful default configuration would be very beneficial, and a less powerful one possibly crippling.
--
J. King
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Size of the SQLite library

Simon Slavin-3
In reply to this post by Dominique Devienne
On 31 May 2018, at 5:18pm, Dominique Devienne <[hidden email]> wrote:

> On Thu, May 31, 2018 at 3:44 PM Richard Hipp <[hidden email]> wrote:
>
>> For many years, we have boasted that the size of the SQLite library is
>> "less than half a megabyte".
>
> Given where the conversation is going, let me point out that many do not
> care one bit about the lib's size :)

Did you know that less than half of SQLite installations are on desktop computers ?  My guess is that mobile phones are now the biggest category of devices.  They run off battery power.  They have firmware on chips.  The more chips they have to keep powered-up, the more battery power they use, the physically bigger the phone has to be to hold not just the chips but the bigger battery, the heavier and more expensive it is.

Not to mention a whole fleet of handheld safety-testing equipment with different configurations for different installations, and those gadgets parking inspectors carry around to keep track of which cars parked when.

Might be interesting to find out what proportion of SQLite devices are mains-powered vs. battery-powered.  Although whether one should class an Airbus A350 XWB as "battery-powered" I am not certain.

Simon.
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Size of the SQLite library

Richard Hipp-3
On 5/31/18, Simon Slavin <[hidden email]> wrote:
>
> Did you know that less than half of SQLite installations are on desktop
> computers ?  My guess is that mobile phones are now the biggest category of
> devices.  They run off battery power.  They have firmware on chips.  The
> more chips they have to keep powered-up, the more battery power they use,
> the physically bigger the phone has to be to hold not just the chips but the
> bigger battery, the heavier and more expensive it is.
>

Back in the day, Motorola and Symbian used to really pressure us to
keep the library footprint down.  Every byte mattered.

But more recently, mobile phone designers are telling me things like
"try to keep the size under 5 megabytes, if you can, please."

Based on those more recent conversations, I'm thinking that we have
more headroom that we have had historically, and so I have recently
been allowing new features to start creeping into the core.

Size is still important.  But having useful features is important too.
I'm continuing to work to find the right balance between these
competing goals.
--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Size of the SQLite library

Thomas Kurz
In reply to this post by Dominique Devienne
I totally agree with that. On most systems it is much more important to have a feature-rich library than a very small one. Any application where a few bytes more or less matter should be written in pure assembler anyway.


----- Original Message -----
From: Dominique Devienne <[hidden email]>
To: General Discussion of SQLite Database <[hidden email]>
Sent: Thursday, May 31, 2018, 18:18:51
Subject: [sqlite] Size of the SQLite library

On Thu, May 31, 2018 at 3:44 PM Richard Hipp <[hidden email]> wrote:

> For many years, we have boasted that the size of the SQLite library is
> "less than half a megabyte".


Given where the conversation is going, let me point out that many do not
care one bit about the lib's size :)

I'd much rather have an SQLite with tons of features, than forego those in
the name saving a few bytes,
to save a few bucks on the embedded chip and flash for commercial products
that don't even pay for SQLite.

SQLite is already amazingly small for the value it brings. And if people
want "smaller", they can still stick
with older leaner versions of SQLite too. My $0.02... --DD
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Size of the SQLite library

Keith Medcalf
In reply to this post by Dominique Devienne

>On Thursday, 31 May, 2018 10:19, Dominique Devienne said:

>Given where the conversation is going, let me point out that many do
>not care one bit about the lib's size :)

>I'd much rather have an SQLite with tons of features, than forego
>those in the name saving a few bytes, to save a few bucks on the
>embedded chip and flash for commercial products that don't even
>pay for SQLite.

>SQLite is already amazingly small for the value it brings. And if
>people want "smaller", they can still stick
>with older leaner versions of SQLite too. My $0.02... --DD

The custom version of the library that I build which contains *all* features and extensions built-in, and then some, compiled with MinGW-w64 GCC 7.1.0 on Windows using -m64 -O3, and statically linked with all the GCC libraries (__float128, runtime, threading, etc., so no dependancies other than to the subsystem runtime (MSVCRT) and the Windows DLLs actually used) comes in around 2 MB ... APSW a wee bit bigger ...

2018-05-30  16:40         2,173,456 sqlite3.dll
2018-05-30  16:43         2,326,544 apsw.cp36-win_amd64.pyd

Of course, this includes almost all the extensions, all the math library, many Windows APIs, and a bunch of aggregates/functions/collations to handle IP Addresses (v4 and v6), some running statistics, proper rounding (Half-Even), all the hash functions (MD4/MD5/SHA/SHA1/SHA2(256/384/512)/SHA3(224/256/384/512)) and a few other odds and sods.

If it were for a computer with "limited resources" I would par it down a lot, but having everything automatically available is very nice ... and I am not CPU/Memory/IO constained (though having all NVMe drives does make me have to "fix" things from time to time to stay within the constraints of spinning rust (3 GB/s I/O can become very addictive).

---
The fact that there's a Highway to Hell but only a Stairway to Heaven says a lot about anticipated traffic volume.




_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Size of the SQLite library

Roger Binns
In reply to this post by Richard Hipp-3
On 31/05/18 10:15, Richard Hipp wrote:
> Size is still important.  But having useful features is important too.
> I'm continuing to work to find the right balance between these
> competing goals.

A pattern used in other projects is to have standard downloads, as well
as custom ones.  With the latter you can include or exclude additional
components and features.  You can already do this with SQLite, but it
requires several more command line tools, programming languages, and
comprehensive reading of the documentation.

Perhaps a custom download web page that gives you some presets
(smallest, default, everything) or lets you choose your own settings.
It would then produce known good source files, and users would be happy.

Here is an example page for a Javascript project:

  https://jqueryui.com/download/

On the balance side, STAT4 is a good example.  I think it would benefit
the majority of SQLite users if it was enabled by default.  But making
only that change could change query plans for existing users.  (Many
users also don't compile SQLite itself - they get binaries from the
platform, or language bindings.)

Roger


_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

signature.asc (201 bytes) Download Attachment
dmp
Reply | Threaded
Open this post in threaded view
|

Re: Size of the SQLite library

dmp
In reply to this post by Richard Hipp-3
1. Define in documentation as < 1Mb. (Don't have to visit again.)

2. Continue to strive to keep in the 0.5-1MB range.

3. Add some information on building a MINIMUM size for those
   concerned that is relatively easy to accomplish without
   a lot of expertise if possible.

danap.

_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Size of the SQLite library

Warren Young
In reply to this post by Roger Binns
On May 31, 2018, at 6:32 PM, Roger Binns <[hidden email]> wrote:
>
> On 31/05/18 10:15, Richard Hipp wrote:
>> Size is still important.  But having useful features is important too.
>> I'm continuing to work to find the right balance between these
>> competing goals.
>
> A pattern used in other projects is to have standard downloads, as well
> as custom ones.

Your jQuery example later on doesn’t much apply here, for several reasons:

1. JavaScript is a dynamic language, while C is a statically-compiled language.  That means that all of the symbols needed to link the program need to be available at link time in order to produce a binary.  To achieve that with C, you’d have to do things like create mock modules that export an API but don’t implement it, merely to placate the linker.

Contrast a language like JavaScript, where you can ship a program that has calls to functions that don’t exist, and as long as you continue to not call those functions, the JS VM won’t balk.

2. There are ways around this with C, such as with the plugin pattern — which is in fact already being used in SQLite’s VFS layer — but it carries an indirection overhead.

jQuery is a bad exemplar here because it’s a solution for implementing user-facing actions, which means that as long as each group of actions implemented using it take under 100 ms or so, it’s fast enough.  For SQLite, though, adding layers of abstraction merely for the programmer’s convenience means slowing it down materially, which can turn a viable solution into failure.

Some sage once opined that any problem in computer science can be solved by adding another layer of abstraction, but there is one that can’t: “The software is too slow, and we’re unwilling to buy more hardware.”

3. SQLite already has a way to generate multiple versions of the source code in a programmatic way: #ifdefs.  That’s precisely what the C preprocessor does.  JavaScript has no equivalent mechanism, so someone had to go an factor jQuery into modules by hand and rely on the user to provide the correct subset of modules needed by their software.

One could write a variant of cpp that would run on the sqlite3.c amalgamation to produce predigested subsets without bringing in all of the #includes, but all that’s really needed here are curated sets of -DSQLITE_* flags, since then the end user can use them with the C preprocessor they already have.



I’ve a feeling that I’m missing more reasons, but those will suffice.
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Size of the SQLite library

Robert M. Münch
In reply to this post by Richard Hipp-3
On 31 May 2018, at 19:15, Richard Hipp wrote:

> But more recently, mobile phone designers are telling me things like
> "try to keep the size under 5 megabytes, if you can, please."
>
> Based on those more recent conversations, I'm thinking that we have
> more headroom that we have had historically, and so I have recently
> been allowing new features to start creeping into the core.

Size matters IMO, it’s a sign of good design and less is more WRT errors etc.


> Size is still important.  But having useful features is important too.

True, and we all know that most features are not used. Do you have an idea what features are used by ratio? Maybe adding a „report back feature collector“ might be an idea, for those wanting to support the feature selection process.


> I'm continuing to work to find the right balance between these
> competing goals.

Keeping things configurable as it is, is a very good approach, please keep it.

--

Robert M. Münch, CEO
M: +41 79 65 11 49 6

Saphirion AG
smarter | better | faster

http://www.saphirion.com
http://www.nlpp.ch

_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

signature.asc (537 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Size of the SQLite library

Roger Binns
In reply to this post by Warren Young
On 01/06/18 13:46, Warren Young wrote:
> Your jQuery example later on doesn’t much apply here, for several reasons:

Note that I was showing how the site let you choose whatever features
you want, and then gave you a download matching exactly that.

> 1. JavaScript is a dynamic language, while C is a statically-compiled language.  

Your comments while correct don't actually apply to SQLite.  SQLite is
not a C file.  It is many C source files, many headers, a grammar (not
in C), and various tools (typically TCL).

For example to exclude virtual tables from SQLite, you can't just add a
compile time option and be done.  You have to regenerate from  the
grammar (so it is no longer valid SQL syntax and no longer has calls to
virtual table relevant functions).  And then you almost certainly want
to use the tool to make the amalgamation from the updated grammar.  And
then you need to make sure your Makefile or equivalent passes in the
omit flag too.

The web site doing all that work for you, and getting it right every
time does have value IMHO.  It also makes it easier for SQLite to have
bigger or smaller presets to address the varying developer needs.  And
the team will have some idea of what OMITs are used, where testing
should check etc.

> That means that all of the symbols needed to link the program ...

That was nothing to do with the issue.  To be very clear:

* SQLite has a way of omiting functionality

* Other than a few special cases, you must use the SQLite source (not
the amalgamation) to regenerate what you finally use

* Doing this is difficult and error prone

> Contrast a language like JavaScript, where you can ship a program that has calls to functions that don’t exist, and as long as you continue to not call those functions, the JS VM won’t balk.

You can do lazy runtime linking in some operating systems so functions
to calls that don't exist are ok (until you call them).

But in any event JS code is not distributed how you think.  Minified
source is usually used, and works best if run through dead code
elimination first (called "tree shaking" in the JS world).  ie the
distribution isn't that different to SQLite (amalgamation).

> 2. There are ways around this with C,

My point is that it isn't.  You cannot add / remove defines to the
amalgamation to omit most features.  Heck if you try it just won't
compile.  More work has to be done.  The mailing list archives have many
messages where people tried a few compile flags and it didn't work.

> One could write a variant of cpp that would run on the sqlite3.c amalgamation ...

It won't work for anything grammar related.  And the project has tools
like you describe (eg it is how the amalgamation is produced).  Those
tools are in TCL and know about the structure and coding patterns of
SQLite.

Roger


_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

signature.asc (201 bytes) Download Attachment
12