what is server-process-edition?

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

what is server-process-edition?

Gelin Yan
Hi All

   I noticed there is a tag called server-process-edition in the timeline
but failed to find any related documents. I am curious what it is.

Regards

gelin yan
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: what is server-process-edition?

Rowan Worth-2
https://sqlite.org/src/artifact/0c6bc6f55191b690

(it was linked recently by Richard Hipp in another thread, to pre-empt
questions of "how did you find that" :) )
-Rowan

On 22 August 2017 at 10:22, Gelin Yan <[hidden email]> wrote:

> Hi All
>
>    I noticed there is a tag called server-process-edition in the timeline
> but failed to find any related documents. I am curious what it is.
>
> Regards
>
> gelin yan
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: what is server-process-edition?

Gelin Yan
Hi Rowan

   Thanks for your hints. The README also mentions begin-concurrent. Do you
know what it is?

Regards

gelin yan
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: what is server-process-edition?

Olivier Mascia
> Le 22 août 2017 à 12:05, Gelin Yan <[hidden email]> a écrit :
>
> Hi Rowan
>
>   Thanks for your hints. The README also mentions begin-concurrent. Do you
> know what it is?
>
> Regards
>
> gelin yan

The recent post by Richard Hipp to this mailing-list (August, 4th), covers this.
Copy below.

--
Best Regards, Meilleures salutations, Met vriendelijke groeten,
Olivier Mascia, http://integral.software

> De: Richard Hipp <[hidden email]>
> Objet: Rép : [sqlite] Thinking about a way to extend the number of writers in WAL mode
> Date: 4 août 2017 à 14:15:25 UTC+2
> À: SQLite mailing list <[hidden email]>
> Répondre à: SQLite mailing list <[hidden email]>
>
> On 8/4/17, Luc DAVID <[hidden email]> wrote:
>> Hello,
>>
>> I was thinking about a possible solution for sqlite "only single writer
>> is allowed at the same time" and database lock.
>>
>> sqlite has WAL mode for better concurrency and this could maybe be used
>> to extend the number of writters:
>
> The begin-concurrent branch
> (https://sqlite.org/src/timeline?r=begin-concurrent&n=all) allows you
> to say:
>
>     BEGIN CONCURRENT;
>     -- various database reads and updates
>     COMMIT;
>
> And to do that simultaneously in two or more database connections, and
> have them all work.  Except, the concurrent transactions may not
> overlap.  That is to say, content written by one may not be read or
> written by another.  If the transactions do overlap, the second one to
> try to COMMIT will get an SQLITE_BUSY_SNAPSHOT error and will be
> forced to abandon its transaction and start over.
>
> The begin-concurrent branch is in production use in high-stress
> environments.  We have not merged that branch to trunk (yet) because
> it currently imposes extra overhead on all applications, even
> applications that do not use BEGIN CONCURRENT.
>
> Another alternative is the newer server-process-edition branch
> (https://sqlite.org/src/timeline?n=all&r=server-process-edition) which
> you can read about here:
> https://sqlite.org/src/artifact/0c6bc6f55191b690
>
> --
> D. Richard Hipp
> [hidden email]

_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: what is server-process-edition?

Marco Bambini
Is there a better formal description about the "transactions may not overlap" sentence?
Is there any example about overlapping transactions?

--
Marco Bambini
http://www.sqlabs.com
http://twitter.com/sqlabs



>> The begin-concurrent branch
>> (https://sqlite.org/src/timeline?r=begin-concurrent&n=all) allows you
>> to say:
>>
>>    BEGIN CONCURRENT;
>>    -- various database reads and updates
>>    COMMIT;
>>
>> And to do that simultaneously in two or more database connections, and
>> have them all work.  Except, the concurrent transactions may not
>> overlap.  That is to say, content written by one may not be read or
>> written by another.  If the transactions do overlap, the second one to
>> try to COMMIT will get an SQLITE_BUSY_SNAPSHOT error and will be
>> forced to abandon its transaction and start over.
>>
>> The begin-concurrent branch is in production use in high-stress
>> environments.  We have not merged that branch to trunk (yet) because
>> it currently imposes extra overhead on all applications, even
>> applications that do not use BEGIN CONCURRENT.
>>
>> Another alternative is the newer server-process-edition branch
>> (https://sqlite.org/src/timeline?n=all&r=server-process-edition) which
>> you can read about here:
>> https://sqlite.org/src/artifact/0c6bc6f55191b690
>>
>> --
>> D. Richard Hipp
>> [hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: what is server-process-edition?

Simon Slavin-3
On 26 Mar 2018, at 8:09am, Marco Bambini <[hidden email]> wrote:

> Is there a better formal description about the "transactions may not overlap" sentence?
> Is there any example about overlapping transactions?

Overlapping transactions occur when a second connection does a BEGIN before the first connection does its COMMIT.  It's difficult to present it well without colours or fixed-width fonts, but try this:

connection 1: BEGIN CONCURRENT
connection 1: -- various database reads and updates
connection 2:                 BEGIN CONCURRENT
connection 2:                 -- various database reads and updates
connection 1: COMMIT
connection 2:                 COMMIT

(Alternatively the COMMIT lines could occur in the other order.  Either way, the transactions are overlapping.)

Using normal BEGIN variants, the thread executing the third or fourth lines would be the one which noticed the problem.  The thread would be delayed and may eventually return SQLITE_. This new variant BEGIN CONCURRENT ensures that the thread executing the second BEGIN is not delayed, and that instead the error is returned when that connection executes the COMMIT.

Simon.
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: what is server-process-edition?

Marco Bambini
So it has nothing to do with which table/row the transaction is modifying?
In your example connection 2 always returns with an error on COMMIT?

Seems like the improvement is only on when the error occurs and not on concurrency or am I missing something?
--
Marco Bambini
http://www.sqlabs.com
http://twitter.com/sqlabs



> On 26 Mar 2018, at 09:41, Simon Slavin <[hidden email]> wrote:
>
> On 26 Mar 2018, at 8:09am, Marco Bambini <[hidden email]> wrote:
>
>> Is there a better formal description about the "transactions may not overlap" sentence?
>> Is there any example about overlapping transactions?
>
> Overlapping transactions occur when a second connection does a BEGIN before the first connection does its COMMIT.  It's difficult to present it well without colours or fixed-width fonts, but try this:
>
> connection 1: BEGIN CONCURRENT
> connection 1: -- various database reads and updates
> connection 2:                 BEGIN CONCURRENT
> connection 2:                 -- various database reads and updates
> connection 1: COMMIT
> connection 2:                 COMMIT
>
> (Alternatively the COMMIT lines could occur in the other order.  Either way, the transactions are overlapping.)
>
> Using normal BEGIN variants, the thread executing the third or fourth lines would be the one which noticed the problem.  The thread would be delayed and may eventually return SQLITE_. This new variant BEGIN CONCURRENT ensures that the thread executing the second BEGIN is not delayed, and that instead the error is returned when that connection executes the COMMIT.
>
> Simon.
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: [EXTERNAL] Re: what is server-process-edition?

Hick Gunter
In reply to this post by Marco Bambini
Think of transactions as cars and the rows as the paving stones of a paved car park, and writing data as parking your car. You cannot park two cars on the same paving stones at the same time without creating a collision. The second car will have to leave the car park and try again.

-----Ursprüngliche Nachricht-----
Von: sqlite-users [mailto:[hidden email]] Im Auftrag von Marco Bambini
Gesendet: Montag, 26. März 2018 09:10
An: SQLite mailing list <[hidden email]>
Betreff: [EXTERNAL] Re: [sqlite] what is server-process-edition?

Is there a better formal description about the "transactions may not overlap" sentence?
Is there any example about overlapping transactions?

--
Marco Bambini
http://www.sqlabs.com
http://twitter.com/sqlabs



>> The begin-concurrent branch
>> (https://sqlite.org/src/timeline?r=begin-concurrent&n=all) allows you
>> to say:
>>
>>    BEGIN CONCURRENT;
>>    -- various database reads and updates
>>    COMMIT;
>>
>> And to do that simultaneously in two or more database connections,
>> and have them all work.  Except, the concurrent transactions may not
>> overlap.  That is to say, content written by one may not be read or
>> written by another.  If the transactions do overlap, the second one
>> to try to COMMIT will get an SQLITE_BUSY_SNAPSHOT error and will be
>> forced to abandon its transaction and start over.
>>
>> The begin-concurrent branch is in production use in high-stress
>> environments.  We have not merged that branch to trunk (yet) because
>> it currently imposes extra overhead on all applications, even
>> applications that do not use BEGIN CONCURRENT.
>>
>> Another alternative is the newer server-process-edition branch
>> (https://sqlite.org/src/timeline?n=all&r=server-process-edition)
>> which you can read about here:
>> https://sqlite.org/src/artifact/0c6bc6f55191b690
>>
>> --
>> D. Richard Hipp
>> [hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


___________________________________________
 Gunter Hick | Software Engineer | Scientific Games International GmbH | Klitschgasse 2-4, A-1130 Vienna | FN 157284 a, HG Wien, DVR: 0430013 | (O) +43 1 80100 - 0

May be privileged. May be confidential. Please delete if not the addressee.
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: what is server-process-edition?

Simon Slavin-3
In reply to this post by Marco Bambini
On 26 Mar 2018, at 8:57am, Marco Bambini <[hidden email]> wrote:

> So it has nothing to do with which table/row the transaction is modifying?

Correct.  SQLite does not have table-locking or row-locking.  Any locks in a SQLite database lock the entire database.  This is a fundamental aspect of SQLite and one of the reasons it's so fast and simple.

> In your example connection 2 always returns with an error on COMMIT?

Assuming I understand the documentation correctly ...
If the commands are executed in the order I showed, then the second COMMIT will always return SQLITE_BUSY_SNAPSHOT instead of SQLITE_OKAY.

> Seems like the improvement is only on when the error occurs and not on concurrency or am I missing something?

The BEGIN CONCURRENT command is inconvenient for most SQLite users and they should avoid it.  It's an improvement for just a very few programs which depend on threads executing in predictable amounts of time.

Simon.
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: what is server-process-edition?

Olivier Mascia
> Le 26 mars 2018 à 10:20, Simon Slavin <[hidden email]> a écrit :
>
> On 26 Mar 2018, at 8:57am, Marco Bambini <[hidden email]> wrote:
>
>> So it has nothing to do with which table/row the transaction is modifying?
>
> Correct.  SQLite does not have table-locking or row-locking.  Any locks in a SQLite database lock the entire database.  This is a fundamental aspect of SQLite and one of the reasons it's so fast and simple.

Simon, if this discussion is really around the branch 'server-process-edition', it was my (possibly wrong) understanding that this is not really true. This branch does apply page-level locking, or I got it all wrong.

See: https://sqlite.org/src/artifact/0c6bc6f55191b690

> The "server-process-edition" branch contains two modifications to stock SQLite that work together to provide concurrent read/write transactions using page-level-locking provided that:
>
> • All clients are in the same process, and
> • The application uses "PRAGMA synchronous=off.

--
Best Regards, Meilleures salutations, Met vriendelijke groeten,
Olivier Mascia


_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: what is server-process-edition?

Simon Slavin-3
On 26 Mar 2018, at 9:26am, Olivier Mascia <[hidden email]> wrote:

> Simon, if this discussion is really around the branch 'server-process-edition', it was my (possibly wrong) understanding that this is not really true. This branch does apply page-level locking, or I got it all wrong.

Whoops.  Thanks for the correction, Olivier.  OP please check the page Olivier cited:

<https://sqlite.org/src/artifact/0c6bc6f55191b690>

I don't know how this information modifies the answers I gave.  Perhaps lock conflicts occur only when two connections try to access the same page of the database file.  One page of the database file might contain the data for part of one row, one row, or more than one row of a table.

Simon.
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users