Appropriate Uses For SQLite

classic Classic list List threaded Threaded
32 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Appropriate Uses For SQLite

Richard Hipp-3
In a feeble effort to do "marketing", I have revised the "Appropriate
Uses For SQLite" webpage to move trendy buzzwords like "Internet of
Things" and "Edge of the Network" above the break.  See:

    https://www.sqlite.org/whentouse.html

Please be my "focus group", and provide feedback, comments,
suggestions, and/or criticism about the revised document.   Send your
remarks back to this mailing list, or directly to me at the email in
the signature.

Thank you for your help.

--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Appropriate Uses For SQLite

Jim Callahan
I would mention the open source statistical language R in the "data
analysis" section. The interface in the RSqlite package is much better and
faster than any of the Python interfaces in that the interface fully
understands queries as tables and that the looping for the return of rows
is done in compiled code rather than at an interpreted command line.
On Feb 18, 2015 9:34 AM, "Richard Hipp" <[hidden email]> wrote:

> In a feeble effort to do "marketing", I have revised the "Appropriate
> Uses For SQLite" webpage to move trendy buzzwords like "Internet of
> Things" and "Edge of the Network" above the break.  See:
>
>     https://www.sqlite.org/whentouse.html
>
> Please be my "focus group", and provide feedback, comments,
> suggestions, and/or criticism about the revised document.   Send your
> remarks back to this mailing list, or directly to me at the email in
> the signature.
>
> Thank you for your help.
>
> --
> D. Richard Hipp
> [hidden email]
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Appropriate Uses For SQLite

Richard Hipp-3
On 2/18/15, Jim Callahan <[hidden email]> wrote:
> I would mention the open source statistical language R in the "data
> analysis" section.

I've heard of R but never tried to use it myself.  Is an SQLite
interface built into R, sure enough?  Or is that something that has to
be added in separately?


> The interface in the RSqlite package is much better and
> faster than any of the Python interfaces in that the interface fully
> understands queries as tables and that the looping for the return of rows
> is done in compiled code rather than at an interpreted command line.
> On Feb 18, 2015 9:34 AM, "Richard Hipp" <[hidden email]> wrote:
>
>> In a feeble effort to do "marketing", I have revised the "Appropriate
>> Uses For SQLite" webpage to move trendy buzzwords like "Internet of
>> Things" and "Edge of the Network" above the break.  See:
>>
>>     https://www.sqlite.org/whentouse.html
>>
>> Please be my "focus group", and provide feedback, comments,
>> suggestions, and/or criticism about the revised document.   Send your
>> remarks back to this mailing list, or directly to me at the email in
>> the signature.
>>
>> Thank you for your help.
>>
>> --
>> D. Richard Hipp
>> [hidden email]
>> _______________________________________________
>> sqlite-users mailing list
>> [hidden email]
>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>>
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>


--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Appropriate Uses For SQLite

Rich Shepard
In reply to this post by Richard Hipp-3
On Wed, 18 Feb 2015, Richard Hipp wrote:

> Please be my "focus group", and provide feedback, comments, suggestions,
> and/or criticism about the revised document. Send your remarks back to
> this mailing list, or directly to me at the email in the signature.

Richard,

   It is clear and well organized. The one difference between SQLite and
a client-server rdbms that could be explicit is single-user versus
multi-user applications and situations. That difference is implied in the
descriptions but likely not noticed by your target market audience.

Rich
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Appropriate Uses For SQLite

John McKown
In reply to this post by Richard Hipp-3
On Wed, Feb 18, 2015 at 8:53 AM, Richard Hipp <[hidden email]> wrote:

> On 2/18/15, Jim Callahan <[hidden email]> wrote:
> > I would mention the open source statistical language R in the "data
> > analysis" section.
>
> I've heard of R but never tried to use it myself.  Is an SQLite
> interface built into R, sure enough?  Or is that something that has to
> be added in separately?
>
>
​It is not a part of the base language, no. It is an add-on module on CRAN
by Hadley Wickham, who is very expert in R. CRAN is to R what CPAN is to
Perl and CTAN is to TeX.
http://cran.revolutionanalytics.com/
http://cran.revolutionanalytics.com/web/packages/RSQLite/index.html



--
He's about as useful as a wax frying pan.

10 to the 12th power microphones = 1 Megaphone

Maranatha! <><
John McKown
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Appropriate Uses For SQLite

Marcus Grimm
In reply to this post by Richard Hipp-3
We use sqlite as the db engine inside a server application
with a number of clients that connect to the server.
Sqlite works just beatiful here and I wish these statements
"sqlite shall not be used for client/server things" would be
worded less generally. In fact when we mention sqlite as our
db engine customer point to this restriction and we run into
an excuse sort of arguments.
On the bottom line: Sqlite CAN very well serve as the DB
engine for client/server applications, it just depend how
the api is used.

Marcus

Am 2015-02-18 15:34, schrieb Richard Hipp:

> In a feeble effort to do "marketing", I have revised the "Appropriate
> Uses For SQLite" webpage to move trendy buzzwords like "Internet of
> Things" and "Edge of the Network" above the break.  See:
>
>     https://www.sqlite.org/whentouse.html
>
> Please be my "focus group", and provide feedback, comments,
> suggestions, and/or criticism about the revised document.   Send your
> remarks back to this mailing list, or directly to me at the email in
> the signature.
>
> Thank you for your help.
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Appropriate Uses For SQLite

Darko Volaric
I second this notion. I think SQLite is uniquely suited to server based
applications of all kinds. Its light footprint and the fact that it's a
library rather than a full system gives it a flexibility and raw
performance that other systems cannot. We use it at the core of each node
in a distributed and parallel system.

When using SQLite the architecture of your database system is not
preordained by designers who could not foresee novel designs and
approaches. SQLite is like a systems programing language: It's lean and
mean and a powerful tool that gives full control to the systems designer
and programmer.

The only thing I'd change about SQLite is the SQL bit. To me it's an
anachronism and a mess and needs to be factored further out of the SQLite
core, with a more rigorous formalism providing an interface (with an
exposed and supported API) to the database engine, but at a higher level
than say the virtual machine.

On Wed, Feb 18, 2015 at 9:12 AM, Marcus Grimm <[hidden email]>
wrote:

> We use sqlite as the db engine inside a server application
> with a number of clients that connect to the server.
> Sqlite works just beatiful here and I wish these statements
> "sqlite shall not be used for client/server things" would be
> worded less generally. In fact when we mention sqlite as our
> db engine customer point to this restriction and we run into
> an excuse sort of arguments.
> On the bottom line: Sqlite CAN very well serve as the DB
> engine for client/server applications, it just depend how
> the api is used.
>
> Marcus
>
> Am 2015-02-18 15:34, schrieb Richard Hipp:
>
>> In a feeble effort to do "marketing", I have revised the "Appropriate
>> Uses For SQLite" webpage to move trendy buzzwords like "Internet of
>> Things" and "Edge of the Network" above the break.  See:
>>
>>     https://www.sqlite.org/whentouse.html
>>
>> Please be my "focus group", and provide feedback, comments,
>> suggestions, and/or criticism about the revised document.   Send your
>> remarks back to this mailing list, or directly to me at the email in
>> the signature.
>>
>> Thank you for your help.
>>
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Appropriate Uses For SQLite

Richard Hipp-3
On 2/18/15, Darko Volaric <[hidden email]> wrote:
> The only thing I'd change about SQLite is the SQL bit.

Most people agree that the SQL language is a bit of a mess.  But so is
the Qwerty keyboard layout.  The problem is that the improvement you
get by moving to something else is less than the pain of making the
move.  Everybody agrees that Dvorak is a better keyboard layout, and
yet nobody uses Dvorak even though it has been widely available on
computer keyboards since the Apple-II,  Not intending to rain on your
parade, but I think the truth is we are probably stuck with SQL for a
while yet.

--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Appropriate Uses For SQLite

Roger Binns
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 02/18/2015 11:43 AM, Richard Hipp wrote:
> but I think the truth is we are probably stuck with SQL for a while
> yet.

In theory there could be an intermediate representation form (like
compilers do) that is publicly available, with the (now optional) SQL
part producing IR, as well as any other query language
implementations.  The LLVM project is an example of doing a design
like this.

In practise the result wouldn't be very Lite, and would constrain
future development.

Roger

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iEYEARECAAYFAlTlAGsACgkQmOOfHg372QRP+ACgqoP3Ss8ZgvO95M8IVHhLRDbo
itEAoMhJKWIKiiYjsAqNUGl/cpv/e+fp
=Z+np
-----END PGP SIGNATURE-----
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Appropriate Uses For SQLite

Jay Kreibich

On Feb 18, 2015, at 3:13 PM, Roger Binns <[hidden email]> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 02/18/2015 11:43 AM, Richard Hipp wrote:
>> but I think the truth is we are probably stuck with SQL for a while
>> yet.
>
> In theory there could be an intermediate representation form (like
> compilers do) that is publicly available, with the (now optional) SQL
> part producing IR, as well as any other query language
> implementations.  The LLVM project is an example of doing a design
> like this.

SQLite kind of already does this, if you consider VDBE instructions to be an IR.

I’ve often wondering how difficult it would be to put a new front-end on SQLite to parse Tutorial D (or some other “true relational” language) and generate VDBE instructions.

 -j

--  
Jay A. Kreibich < J A Y @ K R E I B I.C H >

"Intelligence is like underwear: it is important that you have it, but showing it to the wrong people has the tendency to make them feel uncomfortable." -- Angela Johnson





_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Appropriate Uses For SQLite

Darko Volaric
In reply to this post by Richard Hipp-3
I agree with you, and am not suggesting getting rid of it, but rather
making it "pluggable" like many parts of the back end.

Right now, roughly speaking, I'm doing:  logical form -> SQL -> execution
of logical form, and SQL seems to me to just be an arbitrary hoop that I
have to jump through, complicating things along the way.

Obviously this is not terribly mainstream, but I think would greatly
improve the usefulness and structure of SQLite.

Just my 2 cents on the subject...

On Wed, Feb 18, 2015 at 11:43 AM, Richard Hipp <[hidden email]> wrote:

> On 2/18/15, Darko Volaric <[hidden email]> wrote:
> > The only thing I'd change about SQLite is the SQL bit.
>
> Most people agree that the SQL language is a bit of a mess.  But so is
> the Qwerty keyboard layout.  The problem is that the improvement you
> get by moving to something else is less than the pain of making the
> move.  Everybody agrees that Dvorak is a better keyboard layout, and
> yet nobody uses Dvorak even though it has been widely available on
> computer keyboards since the Apple-II,  Not intending to rain on your
> parade, but I think the truth is we are probably stuck with SQL for a
> while yet.
>
> --
> D. Richard Hipp
> [hidden email]
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Appropriate Uses For SQLite

Darko Volaric
In reply to this post by Roger Binns
I think that IR would be something like first order predicate logic, to
which SQL and the relational calculus is closely related. Now that we have
WITH and recursive queries, you've basically got a bottom-up evaluation of
the declarative subset of Prolog (if you ignore issues relating to logic
variables).

On Wed, Feb 18, 2015 at 1:13 PM, Roger Binns <[hidden email]> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 02/18/2015 11:43 AM, Richard Hipp wrote:
> > but I think the truth is we are probably stuck with SQL for a while
> > yet.
>
> In theory there could be an intermediate representation form (like
> compilers do) that is publicly available, with the (now optional) SQL
> part producing IR, as well as any other query language
> implementations.  The LLVM project is an example of doing a design
> like this.
>
> In practise the result wouldn't be very Lite, and would constrain
> future development.
>
> Roger
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1
>
> iEYEARECAAYFAlTlAGsACgkQmOOfHg372QRP+ACgqoP3Ss8ZgvO95M8IVHhLRDbo
> itEAoMhJKWIKiiYjsAqNUGl/cpv/e+fp
> =Z+np
> -----END PGP SIGNATURE-----
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Appropriate Uses For SQLite

Adam Kennedy-3
In reply to this post by Darko Volaric
But the great thing about SQLite is you don't have to go logical form in
light weight apps.

http://search.cpan.org/dist/ORLite/lib/ORLite.pm

ORLite does a half-and-half approach that generates the easy parts of the
SQL with very little code, but avoids the code weight needed to generate
all of it.

my @users = DB::User->select( 'where name = ? order by name', $name );

It's great to have the flexibility to go all logical form, partly, or none,
as best suits the nature of the database you are embedding in your larger
application.

Adam

On 18 February 2015 at 15:44, Darko Volaric <[hidden email]> wrote:

> Right now, roughly speaking, I'm doing:  logical form -> SQL -> execution
> of logical form, and SQL seems to me to just be an arbitrary hoop that I
> have to jump through, complicating things along the way.
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Appropriate Uses For SQLite

Olivier Vidal
In reply to this post by Richard Hipp-3
Hello all,

https://www.sqlite.org/whentouse.html :
"The amount of web traffic that SQLite can handle depends on how heavily
the website uses its database. Generally speaking, any site that gets
fewer than 100K hits/day should work fine with SQLite. The 100K hits/day
figure is a conservative estimate, not a hard upper bound. SQLite has
been demonstrated to work with 10 times that amount of traffic.

The SQLite website (https://www.sqlite.org/) uses SQLite itself, of
course, and as of this writing (2015) it handles about 400K to 500K HTTP
requests per day, about 15-20% of which are dynamic pages touching the
database. Each dynamic page does roughly 200 SQL statements. This setup
runs on a single VM that shares a physical server with 23 others and yet
still keeps the load average of below 0.1 most of the time."

------

it would be interesting to put *all* sqlite.org pages in the database,
even if it is useless. This would test with 500K HTTP requests per day.
It will then be possible to modify this paragraph and indicate that
Sqlite smoothly manages the 500K HTTP requests per day of this website,
thus about 100 000K SQL statements per day.

And why not test with writing on each visit, and even every page visit?
If Sqlite accept the charge, it would be impressive. it would also
demonstrate the interest of WAL mode.

With the evolution of Sqlite and materials evolution (SSD,
microprocessors ...), it might be possible.

Olivier

> Richard Hipp <mailto:[hidden email]>
> 18 février 2015 15:34
> In a feeble effort to do "marketing", I have revised the "Appropriate
> Uses For SQLite" webpage to move trendy buzzwords like "Internet of
> Things" and "Edge of the Network" above the break. See:
>
> https://www.sqlite.org/whentouse.html
>
> Please be my "focus group", and provide feedback, comments,
> suggestions, and/or criticism about the revised document. Send your
> remarks back to this mailing list, or directly to me at the email in
> the signature.
>
> Thank you for your help.
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Appropriate Uses For SQLite

Simon Slavin-3

On 19 Feb 2015, at 7:31am, Olivier <[hidden email]> quoted:

> it would be interesting to put *all* sqlite.org pages in the database, even if it is useless. This would test with 500K HTTP requests per day. It will then be possible to modify this paragraph and indicate that Sqlite smoothly manages the 500K HTTP requests per day of this website, thus about 100 000K SQL statements per day.

It may be a note in whentouse.html should distinguish between situations where the database is frequently written-to and situations where you have data which is rarely changed.  The lack of writes means that a lot of advantages of client/server systems give little advantage.  The SQLite web site is a good example.

I have an example where I have a completely static 40 Gigabyte (sic.) SQLite web-facing database which is accessed using PHP.  It doesn't have as many reads as the SQLite web site but it's a great test of a 40 Gig database file and shows no bad affects related to its size.  And SQLite searches billions (sic.) of rows so quickly that one frequent user suspected it wasn't working properly and I had to explain tree-based indexes to him.

Simon.
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Appropriate Uses For SQLite

Olivier Vidal


> Simon Slavin <mailto:[hidden email]>
> 19 février 2015 08:54
>
> It may be a note in whentouse.html should distinguish between
> situations where the database is frequently written-to and situations
> where you have data which is rarely changed. The lack of writes means
> that a lot of advantages of client/server systems give little
> advantage. The SQLite web site is a good example.

yes, that's why it would be interesting that there is a writing in the
database at each visit of page, to see if SQLite can handle it. With WAL
mode and good material, this may be possible.

This can be done by step, so as not to bring down the site. For example,
at first, a write every 100 page visits, then if there is no problem,
80, etc.

Sqlite is known and famous now. But his multi-user capabilities, WAL
mode, are *much* less known. On the side of marketing, it is its weak point.

I think it is important to raise awareness of the possibilities of WAL
mode. And what better way than to show a real website with thousands of
writing every day at the same time that hundreds of thousands of reading?

Many developers (there are idlers everywhere!) need to show to their
hierarchy of actual usage examples. Otherwise, the hierarchy will take a
decision on the only reputation of the product: very good single-user
database. They will be very reluctant to any other  type of use.

> I have an example where I have a completely static 40 Gigabyte (sic.)
> SQLite web-facing database which is accessed using PHP. It doesn't
> have as many reads as the SQLite web site but it's a great test of a
> 40 Gig database file and shows no bad affects related to its size. And
> SQLite searches billions (sic.) of rows so quickly that one frequent
> user suspected it wasn't working properly and I had to explain
> tree-based indexes to him.
Interestingly, thank you Simon!

Maybe on the SQLite website, a page might list these cases of actual
use, to show that SQLite can handle a variety of situations.

Olivier

>
> Simon.
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> Olivier <mailto:[hidden email]>
> 19 février 2015 08:31
> Hello all,
>
> https://www.sqlite.org/whentouse.html :
> "The amount of web traffic that SQLite can handle depends on how
> heavily the website uses its database. Generally speaking, any site
> that gets fewer than 100K hits/day should work fine with SQLite. The
> 100K hits/day figure is a conservative estimate, not a hard upper
> bound. SQLite has been demonstrated to work with 10 times that amount
> of traffic.
>
> The SQLite website (https://www.sqlite.org/) uses SQLite itself, of
> course, and as of this writing (2015) it handles about 400K to 500K
> HTTP requests per day, about 15-20% of which are dynamic pages
> touching the database. Each dynamic page does roughly 200 SQL
> statements. This setup runs on a single VM that shares a physical
> server with 23 others and yet still keeps the load average of below
> 0.1 most of the time."
>
> ------
>
> it would be interesting to put *all* sqlite.org pages in the database,
> even if it is useless. This would test with 500K HTTP requests per
> day. It will then be possible to modify this paragraph and indicate
> that Sqlite smoothly manages the 500K HTTP requests per day of this
> website, thus about 100 000K SQL statements per day.
>
> And why not test with writing on each visit, and even every page
> visit? If Sqlite accept the charge, it would be impressive. it would
> also demonstrate the interest of WAL mode.
>
> With the evolution of Sqlite and materials evolution (SSD,
> microprocessors ...), it might be possible.
>
> Olivier
>
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Appropriate Uses For SQLite

Eric Grange-3
> https://www.sqlite.org/whentouse.html :

I am currently relying on SQLite as main database engine for blockchain
explorers, that's usually between 1 and 2 millions http queries per day,
pretty much all of them hitting more than 60 GB of SQLite databases. There
is a mix of simple table queries, complex joins and extensive
aggregation/statistics queries.

This is kind of a sweet spot for SQLite WAL: the data is queried and
updated all the time, but with only one updater task. Performance is very
good, better than what I see from client/server DB under much lighter
loads. And the latency is just excellent: simple queries run in a matter of
microseconds.

And while it may not be fashionable to say so, I really like the SQL in
SQLite :)
There is no query that cannot be performed efficiently and with little
code. Worst that happens is you need a new index or a few temporary tables
to breakup a complex query in several simpler steps.

I would say SQLite is perfectly suited for "front line" web servers that
serve, present and generally make accessible live data coming from other
systems.

Eric
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Appropriate Uses For SQLite

Jonathan Moules-2
In reply to this post by Richard Hipp-3
Hi Richard,
How about mentioning extensions as a whole? I can't seem to find a list of SQLite extensions on sqlite.org, but it seems like it'd be useful information, and not just for those deciding on whether the language is right for them.
(When I use the word "extensions", I'm referring to things like Spatialite).

I appreciate the extensions are separate projects, but a list would probably be useful (PostGres has one, but it seems short - http://www.postgresql.org/download/products/6-postgresql-extensions/ ). It might be worth creating such a page for SQLite too.


Another thought - the rich ecosystem of administrative GUI's (Both open source and commercial). Given most folks on this list appear to be Guru's who breathe SQL, I can see why it was missed, but they're important to us lay-users.

Cheers,
Jonathan



-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Richard Hipp
Sent: Wednesday, February 18, 2015 2:34 PM
To: General Discussion of SQLite Database
Subject: [sqlite] Appropriate Uses For SQLite

In a feeble effort to do "marketing", I have revised the "Appropriate Uses For SQLite" webpage to move trendy buzzwords like "Internet of Things" and "Edge of the Network" above the break.  See:

    https://www.sqlite.org/whentouse.html

Please be my "focus group", and provide feedback, comments,
suggestions, and/or criticism about the revised document.   Send your
remarks back to this mailing list, or directly to me at the email in the signature.

Thank you for your help.

--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


This message has been scanned for viruses by MailControl - www.mailcontrol.com



Click https://www.mailcontrol.com/sr/YnvGRjO1h!fGX2PQPOmvUkWM85sEKD4+AxKwKR2OO3rUCZRh4ynTU2SGHzny8KZl+wI!vZXG5UPCP4z!bwErJQ== to report this email as spam.

________________________________

HR Wallingford and its subsidiaries uses faxes and emails for confidential and legally privileged business communications. They do not of themselves create legal commitments. Disclosure to parties other than addressees requires our specific consent. We are not liable for unauthorised disclosures nor reliance upon them.
If you have received this message in error please advise us immediately and destroy all copies of it.

HR Wallingford Limited
Howbery Park, Wallingford, Oxfordshire, OX10 8BA, United Kingdom
Registered in England No. 02562099

________________________________
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Appropriate Uses For SQLite

Lindsay Lawrence
In reply to this post by Darko Volaric
I have found SQLite is uniquely suited to server-based applications of all
kinds. Wrapped in a simple custom server it is an outstandingly performant
ultra-lightweight engine that can easily service multiple clients. The
ability to spin up multiple instances of the library against the same file
is invaluable. It is the high performance core of a distributed system we
use. Zero configuration is a beautiful thing.

We have found it to be suitable for quite large data; recently loaded the
full Census 2010 and ACS survey datasets, over 0.5 billion rows, into a
single file totalling over 100GB. Properly indexed it is a instantly
queryable allowing some fascinating data exploration and visualization. We
have also used it in scenarios where the ability to 'attach' distinct
database files allows the system to load only what it needs to service
particular queries and makes it simple to clone core tables and distribute
them over multiple systems for combining with related data.

Finally, it has replaced many traditional text-based data exploration tools
I once used... sed, awk, grep etc. Server logs, data exports from
'enterprise' systems, all manner of regularly structured data, are easily
ingested into a sqlite database file and with the addition of simple
dynamically loaded plugins + SQL can be quickly explored, data summarized,
etc. Creating views off of the raw data, I can deliver a single compact
database file to a report writer who using the ODBC driver can make eye
candy reports off of the summarized data with Excel, Access or other UI
based tools.

I will say, the recent addition of the 'WITH' clause to the language as
brought consider expressive power to what I can write directly in SQL. I
love the fact that I can do so much with the SQL language itself now
without writing a line of low-level code and the SQLite API makes it easy
to extend the functional api as needed.

In short, the zero configuration, high performance and lightweight
characteristics of SQLite have made it uniquely suited for the distributed
'cloud' environment we run in where relational data can be fragmented and
distributed over many VMs and still be accessible with a common interface
and query language.

Many thanks.

Best Regards
Lindsay


On Wed, Feb 18, 2015 at 11:11 AM, Darko Volaric <[hidden email]> wrote:

> I second this notion. I think SQLite is uniquely suited to server based
> applications of all kinds. Its light footprint and the fact that it's a
> library rather than a full system gives it a flexibility and raw
> performance that other systems cannot. We use it at the core of each node
> in a distributed and parallel system.
>
> When using SQLite the architecture of your database system is not
> preordained by designers who could not foresee novel designs and
> approaches. SQLite is like a systems programing language: It's lean and
> mean and a powerful tool that gives full control to the systems designer
> and programmer.
>
> The only thing I'd change about SQLite is the SQL bit. To me it's an
> anachronism and a mess and needs to be factored further out of the SQLite
> core, with a more rigorous formalism providing an interface (with an
> exposed and supported API) to the database engine, but at a higher level
> than say the virtual machine.
>
> On Wed, Feb 18, 2015 at 9:12 AM, Marcus Grimm <[hidden email]>
> wrote:
>
> > We use sqlite as the db engine inside a server application
> > with a number of clients that connect to the server.
> > Sqlite works just beatiful here and I wish these statements
> > "sqlite shall not be used for client/server things" would be
> > worded less generally. In fact when we mention sqlite as our
> > db engine customer point to this restriction and we run into
> > an excuse sort of arguments.
> > On the bottom line: Sqlite CAN very well serve as the DB
> > engine for client/server applications, it just depend how
> > the api is used.
> >
> > Marcus
> >
> > Am 2015-02-18 15:34, schrieb Richard Hipp:
> >
> >> In a feeble effort to do "marketing", I have revised the "Appropriate
> >> Uses For SQLite" webpage to move trendy buzzwords like "Internet of
> >> Things" and "Edge of the Network" above the break.  See:
> >>
> >>     https://www.sqlite.org/whentouse.html
> >>
> >> Please be my "focus group", and provide feedback, comments,
> >> suggestions, and/or criticism about the revised document.   Send your
> >> remarks back to this mailing list, or directly to me at the email in
> >> the signature.
> >>
> >> Thank you for your help.
> >>
> > _______________________________________________
> > sqlite-users mailing list
> > [hidden email]
> > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> >
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Appropriate Uses For SQLite

Hick Gunter
In reply to this post by Richard Hipp-3
We are using SQLite as the catch-all data access method (via custom extensions) for

- Oracle tables and views
- Faircom CTree files
- Shared memory record stores ("Data Dictionary")
- Log file access
- Blob to record translation (TLV structures)
- Partitioned data stores (CTree and Data Dictionary; content-based partitioning)
- Report generation
- User-Defined tables (script output)


-----Ursprüngliche Nachricht-----
Von: Richard Hipp [mailto:[hidden email]]
Gesendet: Mittwoch, 18. Februar 2015 15:34
An: General Discussion of SQLite Database
Betreff: [sqlite] Appropriate Uses For SQLite

In a feeble effort to do "marketing", I have revised the "Appropriate Uses For SQLite" webpage to move trendy buzzwords like "Internet of Things" and "Edge of the Network" above the break.  See:

    https://www.sqlite.org/whentouse.html

Please be my "focus group", and provide feedback, comments,
suggestions, and/or criticism about the revised document.   Send your
remarks back to this mailing list, or directly to me at the email in the signature.

Thank you for your help.

--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


___________________________________________
 Gunter Hick
Software Engineer
Scientific Games International GmbH
FN 157284 a, HG Wien
Klitschgasse 2-4, A-1130 Vienna, Austria
Tel: +43 1 80100 0
E-Mail: [hidden email]

This communication (including any attachments) is intended for the use of the intended recipient(s) only and may contain information that is confidential, privileged or legally protected. Any unauthorized use or dissemination of this communication is strictly prohibited. If you have received this communication in error, please immediately notify the sender by return e-mail message and delete all copies of the original communication. Thank you for your cooperation.


_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
12