Is it possible to change "synchronous" flag of ZIPVFS pager?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Is it possible to change "synchronous" flag of ZIPVFS pager?

Tomasz.Maj
This post has NOT been accepted by the mailing list yet.

Hi,


According to my observations, "PRAGMA synchronous=...;" query affects only the standard sqlite pager. But for zipped databases sqlite uses additional ZIPVFS pager layer which actually decide whether and when to sync content of files associated with a database. It looks like "PRAGMA synchronous=...;" query doesn't have any effect on zipped databases. Are my observations right? If so, is it possible to manipulate "synchronous" flag of ZIPVFS pager?


Cheers,

Tomasz M

Reply | Threaded
Open this post in threaded view
|

Is it possible to change "synchronous" flag of ZIPVFS pager?

Tomasz.Maj
Hi,


According to my observations, "PRAGMA synchronous=...;" query affects only the standard sqlite pager. But for zipped databases sqlite uses additional ZIPVFS pager layer which actually decide whether and when to sync content of files associated with a database. It looks like "PRAGMA synchronous=...;" query doesn't have any effect on zipped databases. Are my observations right? If so, is it possible to manipulate "synchronous" flag of ZIPVFS pager?


Cheers,

Tomasz M

_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to change "synchronous" flag of ZIPVFS pager?

Dan Kennedy-4
On 03/16/2017 01:40 PM, Tomasz Maj wrote:
> Hi,
>
>
> According to my observations, "PRAGMA synchronous=...;" query affects only the standard sqlite pager. But for zipped databases sqlite uses additional ZIPVFS pager layer which actually decide whether and when to sync content of files associated with a database. It looks like "PRAGMA synchronous=...;" query doesn't have any effect on zipped databases. Are my observations right? If so, is it possible to manipulate "synchronous" flag of ZIPVFS pager?

It looks like there is not. Is it something you need to do?

Dan.

_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to change "synchronous" flag of ZIPVFS pager?

Tomek Maj
2017-03-16 9:51 GMT+01:00 Dan Kennedy <[hidden email]>:

> On 03/16/2017 01:40 PM, Tomasz Maj wrote:
>
>> Hi,
>>
>>
>> According to my observations, "PRAGMA synchronous=...;" query affects
>> only the standard sqlite pager. But for zipped databases sqlite uses
>> additional ZIPVFS pager layer which actually decide whether and when to
>> sync content of files associated with a database. It looks like "PRAGMA
>> synchronous=...;" query doesn't have any effect on zipped databases. Are my
>> observations right? If so, is it possible to manipulate "synchronous" flag
>> of ZIPVFS pager?
>>
>
> It looks like there is not. Is it something you need to do?
>
> Dan.
>
>
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to change "synchronous" flag of ZIPVFS pager?

Tomasz.Maj
I have an application which work with many zipped databases in WAL journal mode and the databases are stored on a media of low write bandwidth. There is a problem that that fsync operations invoked from one thread are blocking other fsync operations invoked from another thread. I think that changing the synchronous mode to NORMAL on some connections may make the blocking issue less painful. May the possibility of changing ZIPVFS synchronous mode be implemented in sqlite?


Cheers,

Tomasz M

________________________________
From: sqlite-users <[hidden email]> on behalf of Tomek Maj <[hidden email]>
Sent: Thursday, March 16, 2017 10:50:37 AM
To: SQLite mailing list
Subject: Re: [sqlite] Is it possible to change "synchronous" flag of ZIPVFS pager?

2017-03-16 9:51 GMT+01:00 Dan Kennedy <[hidden email]>:

> On 03/16/2017 01:40 PM, Tomasz Maj wrote:
>
>> Hi,
>>
>>
>> According to my observations, "PRAGMA synchronous=...;" query affects
>> only the standard sqlite pager. But for zipped databases sqlite uses
>> additional ZIPVFS pager layer which actually decide whether and when to
>> sync content of files associated with a database. It looks like "PRAGMA
>> synchronous=...;" query doesn't have any effect on zipped databases. Are my
>> observations right? If so, is it possible to manipulate "synchronous" flag
>> of ZIPVFS pager?
>>
>
> It looks like there is not. Is it something you need to do?
>
> Dan.
>
>
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to change "synchronous" flag of ZIPVFS pager?

Dan Kennedy-4
On 03/16/2017 05:00 PM, Tomasz Maj wrote:
> I have an application which work with many zipped databases in WAL journal mode and the databases are stored on a media of low write bandwidth. There is a problem that that fsync operations invoked from one thread are blocking other fsync operations invoked from another thread. I think that changing the synchronous mode to NORMAL on some connections may make the blocking issue less painful. May the possibility of changing ZIPVFS synchronous mode be implemented in sqlite?

Latest commit in the zipvfs project fixes things so that "PRAGMA
synchronous" works with zipvfs databases.

Dan.



>
>
> Cheers,
>
> Tomasz M
>
> ________________________________
> From: sqlite-users <[hidden email]> on behalf of Tomek Maj <[hidden email]>
> Sent: Thursday, March 16, 2017 10:50:37 AM
> To: SQLite mailing list
> Subject: Re: [sqlite] Is it possible to change "synchronous" flag of ZIPVFS pager?
>
> 2017-03-16 9:51 GMT+01:00 Dan Kennedy <[hidden email]>:
>
>> On 03/16/2017 01:40 PM, Tomasz Maj wrote:
>>
>>> Hi,
>>>
>>>
>>> According to my observations, "PRAGMA synchronous=...;" query affects
>>> only the standard sqlite pager. But for zipped databases sqlite uses
>>> additional ZIPVFS pager layer which actually decide whether and when to
>>> sync content of files associated with a database. It looks like "PRAGMA
>>> synchronous=...;" query doesn't have any effect on zipped databases. Are my
>>> observations right? If so, is it possible to manipulate "synchronous" flag
>>> of ZIPVFS pager?
>>>
>> It looks like there is not. Is it something you need to do?
>>
>> Dan.
>>
>>
>> _______________________________________________
>> sqlite-users mailing list
>> [hidden email]
>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>>
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to change "synchronous" flag of ZIPVFS pager?

Tomasz.Maj
Thank you very much. You're super quick guys :)


Cheers,

Tomasz M

________________________________
From: sqlite-users <[hidden email]> on behalf of Dan Kennedy <[hidden email]>
Sent: Thursday, March 16, 2017 8:34:30 PM
To: [hidden email]
Subject: Re: [sqlite] Is it possible to change "synchronous" flag of ZIPVFS pager?

On 03/16/2017 05:00 PM, Tomasz Maj wrote:
> I have an application which work with many zipped databases in WAL journal mode and the databases are stored on a media of low write bandwidth. There is a problem that that fsync operations invoked from one thread are blocking other fsync operations invoked from another thread. I think that changing the synchronous mode to NORMAL on some connections may make the blocking issue less painful. May the possibility of changing ZIPVFS synchronous mode be implemented in sqlite?

Latest commit in the zipvfs project fixes things so that "PRAGMA
synchronous" works with zipvfs databases.

Dan.



>
>
> Cheers,
>
> Tomasz M
>
> ________________________________
> From: sqlite-users <[hidden email]> on behalf of Tomek Maj <[hidden email]>
> Sent: Thursday, March 16, 2017 10:50:37 AM
> To: SQLite mailing list
> Subject: Re: [sqlite] Is it possible to change "synchronous" flag of ZIPVFS pager?
>
> 2017-03-16 9:51 GMT+01:00 Dan Kennedy <[hidden email]>:
>
>> On 03/16/2017 01:40 PM, Tomasz Maj wrote:
>>
>>> Hi,
>>>
>>>
>>> According to my observations, "PRAGMA synchronous=...;" query affects
>>> only the standard sqlite pager. But for zipped databases sqlite uses
>>> additional ZIPVFS pager layer which actually decide whether and when to
>>> sync content of files associated with a database. It looks like "PRAGMA
>>> synchronous=...;" query doesn't have any effect on zipped databases. Are my
>>> observations right? If so, is it possible to manipulate "synchronous" flag
>>> of ZIPVFS pager?
>>>
>> It looks like there is not. Is it something you need to do?
>>
>> Dan.
>>
>>
>> _______________________________________________
>> sqlite-users mailing list
>> [hidden email]
>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>>
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users