compression

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

compression

Martin Pfeifle
Hello,
does anybody know whether it is possible to compress
sqlite data on the page level. If I compress the
sqlite database file with zlib I get very high
compression rates due to the character of the stored
data.
I think this problem is related to the problem of
using encrypted databases. Perheps it is possible just
to exchange the encryption function call by a zlib
compression call.
Integrating such a call simply into the read and write
functions in the file os_win.c does not work.
Can anybody help me, or give me a hint?
Ciao Martin


               
___________________________________________________________
Was denken Sie ?ber E-Mail? Wir h?ren auf Ihre Meinung: http://surveylink.yahoo.com/wix/p0379378.aspx
Reply | Threaded
Open this post in threaded view
|

Xtudio C/C++ IDE - free IDE is ready for download

Eran Ifrah
Hi All,

What is Xtudio? Xtudio is a C/C++ code editor for windows programmers, you can learn more about it here:
http://www.eistware.com


The Alpha version has just released downloaded FREE
from http://www.eistware.com/Download.html

Xtudio IDE is for Windows users (Tested on XP SP1 / 2) that uses C/C++
It is totaly free (no trials, no 30 days, no commitments etc.)

Xtudio demonstrates the power of Scintilla as an editing
component and using SQLite engine for internal DB.

Xtudio introduces some unique features for editing code and compiling it.

Visit us at:
http:://www.eistware.com

Downloads:
http://www.eistware.com/Download.html

Forum:
http://www.eistware.com/forum/index.php

For mails and responses:
[hidden email]

Enjoy,
mailto:[hidden email] <http://webmail.livedns.co.il/email.compose.php?vwebmailsession=&to=admin@...>


Reply | Threaded
Open this post in threaded view
|

Re: compression

Jay Sprenkle
In reply to this post by Martin Pfeifle
If you're on Linux read about the Reiser 4 file system.
They found they could compress the entire file system on the fly and achieve
higher performance as well. Most CPU's can compress and move data faster
because they make up the difference on the slow I/O channels to hard disks.
Might be a much easier solution


On 9/24/05, Martin Pfeifle <[hidden email]> wrote:

>
> Hello,
> does anybody know whether it is possible to compress
> sqlite data on the page level. If I compress the
> sqlite database file with zlib I get very high
> compression rates due to the character of the stored
> data.
> I think this problem is related to the problem of
> using encrypted databases. Perheps it is possible just
> to exchange the encryption function call by a zlib
> compression call.
> Integrating such a call simply into the read and write
> functions in the file os_win.c does not work.
> Can anybody help me, or give me a hint?
> Ciao Martin
>
>
>
> ___________________________________________________________
> Was denken Sie über E-Mail? Wir hören auf Ihre Meinung:
> http://surveylink.yahoo.com/wix/p0379378.aspx
>



--
---
The Castles of Dereth Calendar: a tour of the art and architecture of
Asheron's Call
http://www.lulu.com/content/77264
Reply | Threaded
Open this post in threaded view
|

Re: compression

Sid Liu
Is there a possibility that this Reiser 4 be used on a
file, rather than a file system? Hopefully on Windows?

--- Jay Sprenkle <[hidden email]> wrote:

> If you're on Linux read about the Reiser 4 file
> system.
> They found they could compress the entire file
> system on the fly and achieve
> higher performance as well. Most CPU's can compress
> and move data faster
> because they make up the difference on the slow I/O
> channels to hard disks.
> Might be a much easier solution
>
>
> On 9/24/05, Martin Pfeifle <[hidden email]>
> wrote:
> >
> > Hello,
> > does anybody know whether it is possible to
> compress
> > sqlite data on the page level. If I compress the
> > sqlite database file with zlib I get very high
> > compression rates due to the character of the
> stored
> > data.
> > I think this problem is related to the problem of
> > using encrypted databases. Perheps it is possible
> just
> > to exchange the encryption function call by a zlib
> > compression call.
> > Integrating such a call simply into the read and
> write
> > functions in the file os_win.c does not work.
> > Can anybody help me, or give me a hint?
> > Ciao Martin
> >
> >
> >
> >
>
___________________________________________________________

> > Was denken Sie ?ber E-Mail? Wir h?ren auf Ihre
> Meinung:
> > http://surveylink.yahoo.com/wix/p0379378.aspx
> >
>
>
>
> --
> ---
> The Castles of Dereth Calendar: a tour of the art
> and architecture of
> Asheron's Call
> http://www.lulu.com/content/77264
>

Reply | Threaded
Open this post in threaded view
|

Re: compression

Christian Smith
On Wed, 28 Sep 2005, Sid Liu wrote:

>Is there a possibility that this Reiser 4 be used on a
>file, rather than a file system? Hopefully on Windows?


Reiser FS is a filesystem. It manages files. So it cannot be used on a
file.

Windows NTFS already has compressed files. Right click a file or directory
in exporer, select propeties, then advanced attributes. You can turn on
compression there. Don't know how to do it from the command line, though.


>
>--- Jay Sprenkle <[hidden email]> wrote:
>
>> If you're on Linux read about the Reiser 4 file
>> system.
>> They found they could compress the entire file
>> system on the fly and achieve
>> higher performance as well. Most CPU's can compress
>> and move data faster
>> because they make up the difference on the slow I/O
>> channels to hard disks.
>> Might be a much easier solution
>>
>>
>> On 9/24/05, Martin Pfeifle <[hidden email]>
>> wrote:
>> >
>> > Hello,
>> > does anybody know whether it is possible to
>> compress
>> > sqlite data on the page level. If I compress the
>> > sqlite database file with zlib I get very high
>> > compression rates due to the character of the
>> stored
>> > data.
>> > I think this problem is related to the problem of
>> > using encrypted databases. Perheps it is possible
>> just
>> > to exchange the encryption function call by a zlib
>> > compression call.
>> > Integrating such a call simply into the read and
>> write
>> > functions in the file os_win.c does not work.
>> > Can anybody help me, or give me a hint?
>> > Ciao Martin
>> >
>> >
>> >
>> >
>>
>___________________________________________________________
>> > Was denken Sie über E-Mail? Wir hören auf Ihre
>> Meinung:
>> > http://surveylink.yahoo.com/wix/p0379378.aspx
>> >
>>
>>
>>
>> --
>> ---
>> The Castles of Dereth Calendar: a tour of the art
>> and architecture of
>> Asheron's Call
>> http://www.lulu.com/content/77264
>>
>
>

--
    /"\
    \ /    ASCII RIBBON CAMPAIGN - AGAINST HTML MAIL
     X                           - AGAINST MS ATTACHMENTS
    / \
Reply | Threaded
Open this post in threaded view
|

Re: compression

Dennis Jenkins
Christian Smith wrote:

>On Wed, 28 Sep 2005, Sid Liu wrote:
>
>  
>
>>Is there a possibility that this Reiser 4 be used on a
>>file, rather than a file system? Hopefully on Windows?
>>    
>>
>
>
>Reiser FS is a filesystem. It manages files. So it cannot be used on a
>file.
>
>  
>
Your third statement is not true.  On Linux (and FreeBSD, but FreeBSD
does not have Reiser as far as I know) you can treat a regular file as
if it were a filesystem and mount that fiel system via the "loop back"
device.  You can mount an ISO image file as an actual CD, for instance.  
Years ago I imaged all of my old DOS floppies.  I access them via the
loop back device now.  In theory, you can do that with any file system
that can use a block device (ntfs, iso9660, ext3, etc...) but not with
nfs, smbfs, proc, etc...

dd if=/dev/fd0 of=floppy_file.img
# Eject floppy, don't need it anymore.

losetup /dev/loop0 floppy_file.img
mount -t vfat /dev/loop0 /mnt/floppy
ls -l /mnt/floppy


>Windows NTFS already has compressed files. Right click a file or directory
>in exporer, select propeties, then advanced attributes. You can turn on
>compression there. Don't know how to do it from the command line, though.
>
>  
>

Reply | Threaded
Open this post in threaded view
|

Re: compression

Dennis Jenkins
Jay Sprenkle wrote:

>On 9/28/05, Dennis Jenkins <[hidden email]> wrote:
>  
>
>>Your third statement is not true. On Linux (and FreeBSD, but FreeBSD
>>does not have Reiser as far as I know) you can treat a regular file as
>>if it were a filesystem and mount that fiel system via the "loop back"
>>device. You can mount an ISO image file as an actual CD, for instance.
>>
>>    
>>
>
>Cool! Thanks for letting us know :)
>
>  
>
I should have mentioned the obvious though... the file must be an image
of a valid file system.

for example, the following will fail:

dd if=/dev/zero of=file bs=4096 count=1024
losetup /dev/loop0 file
mount /dev/loop0 /mnt/xxx

However, the following should work:
dd if=/dev/zero of=file bs=4096 count=1024
losetup /dev/loop0 file
mke2fs /dev/loop0
mount /dev/loop0 /mnt/xxx


You can even encrypt the entire filesystem over loop back:
dd if=/dev/zero of=blob bs=1M count=1024
losetup -e AES256 /dev/loop0 blob
mke3fs /dev/loop0 blob
mount /dev/loop0 /mnt/crypto

as usual, do a "man" on "losetup".

This is from my Gentoo 2005.1 Linux system (home) with whatever packages
it installed a few days ago.
At work I use FreeBSD and I've not used loopback devices there, but the
FreeBSD Handbook (online for free) shows how to do it.


Reply | Threaded
Open this post in threaded view
|

Re: compression

Jay Sprenkle
On 9/28/05, Dennis Jenkins <[hidden email]> wrote:
>
>
> This is from my Gentoo 2005.1 Linux system (home) with whatever packages
> it installed a few days ago.
> At work I use FreeBSD and I've not used loopback devices there, but the
> FreeBSD Handbook (online for free) shows how to do it.
>
>
> I'm using gentoo at home too so that won't be problem :)



--
---
The Castles of Dereth Calendar: a tour of the art and architecture of
Asheron's Call
http://www.lulu.com/content/77264