Is it necessary to encode() for file names in sqlar format?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Is it necessary to encode() for file names in sqlar format?

Peng Yu
I use the following python3 code to create sqlar file. Is it necessary
to sys.argv[2].encode('utf-8') in the line of execute()? In other
word, does the native sqlar tools inteprete the name column as an
encoded value or a non-encode value? Thanks.

import sqlite3
conn=sqlite3.connect(sys.argv[1])
c=conn.cursor()
c.execute('''
CREATE TABLE IF NOT EXISTS sqlar(
name TEXT PRIMARY KEY
, mode INT
, mtime INT
, sz INT
, data BLOB)
''')

import zlib
data = sys.stdin.buffer.read()
c.execute('REPLACE INTO sqlar VALUES(?, ?, ?, ?, ?)', [sys.argv[2], 0,
0, len(data), zlib.compress(data)])
conn.commit()

--
Regards,
Peng
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Is it necessary to encode() for file names in sqlar format?

Keith Medcalf

sys.argv is a list of unicode text strings.  There is no need to specifically encode or decode it so long as sys.getdefaultencoding() returns 'utf-8'.  If your version of Python is so old that it returns something else then you need to modify site.py and have it set the default encoding to 'utf-8' otherwise you may end up with MBCS or some other invalid text encoding in your database text fields.

--
The fact that there's a Highway to Hell but only a Stairway to Heaven says a lot about anticipated traffic volume.

>-----Original Message-----
>From: sqlite-users <[hidden email]> On
>Behalf Of Peng Yu
>Sent: Wednesday, 29 January, 2020 22:57
>To: SQLite mailing list <[hidden email]>
>Subject: [sqlite] Is it necessary to encode() for file names in sqlar
>format?
>
>I use the following python3 code to create sqlar file. Is it necessary
>to sys.argv[2].encode('utf-8') in the line of execute()? In other
>word, does the native sqlar tools inteprete the name column as an
>encoded value or a non-encode value? Thanks.
>
>import sqlite3
>conn=sqlite3.connect(sys.argv[1])
>c=conn.cursor()
>c.execute('''
>CREATE TABLE IF NOT EXISTS sqlar(
>name TEXT PRIMARY KEY
>, mode INT
>, mtime INT
>, sz INT
>, data BLOB)
>''')
>
>import zlib
>data = sys.stdin.buffer.read()
>c.execute('REPLACE INTO sqlar VALUES(?, ?, ?, ?, ?)', [sys.argv[2], 0,
>0, len(data), zlib.compress(data)])
>conn.commit()
>
>--
>Regards,
>Peng
>_______________________________________________
>sqlite-users mailing list
>[hidden email]
>http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users



_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Is it necessary to encode() for file names in sqlar format?

Peng Yu
So to confirm. In python 3, the str type should be used for name? Thanks.

On Thu, Jan 30, 2020 at 12:58 AM Keith Medcalf <[hidden email]> wrote:

>
> sys.argv is a list of unicode text strings.  There is no need to
> specifically encode or decode it so long as sys.getdefaultencoding()
> returns 'utf-8'.  If your version of Python is so old that it returns
> something else then you need to modify site.py and have it set the default
> encoding to 'utf-8' otherwise you may end up with MBCS or some other
> invalid text encoding in your database text fields.
>
> --
> The fact that there's a Highway to Hell but only a Stairway to Heaven says
> a lot about anticipated traffic volume.
>
> >-----Original Message-----
> >From: sqlite-users <[hidden email]> On
> >Behalf Of Peng Yu
> >Sent: Wednesday, 29 January, 2020 22:57
> >To: SQLite mailing list <[hidden email]>
> >Subject: [sqlite] Is it necessary to encode() for file names in sqlar
> >format?
> >
> >I use the following python3 code to create sqlar file. Is it necessary
> >to sys.argv[2].encode('utf-8') in the line of execute()? In other
> >word, does the native sqlar tools inteprete the name column as an
> >encoded value or a non-encode value? Thanks.
> >
> >import sqlite3
> >conn=sqlite3.connect(sys.argv[1])
> >c=conn.cursor()
> >c.execute('''
> >CREATE TABLE IF NOT EXISTS sqlar(
> >name TEXT PRIMARY KEY
> >, mode INT
> >, mtime INT
> >, sz INT
> >, data BLOB)
> >''')
> >
> >import zlib
> >data = sys.stdin.buffer.read()
> >c.execute('REPLACE INTO sqlar VALUES(?, ?, ?, ?, ?)', [sys.argv[2], 0,
> >0, len(data), zlib.compress(data)])
> >conn.commit()
> >
> >--
> >Regards,
> >Peng
> >_______________________________________________
> >sqlite-users mailing list
> >[hidden email]
> >http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
>
>
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
--
Regards,
Peng
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Is it necessary to encode() for file names in sqlar format?

Keith Medcalf

Yes.  If it is bytes type then the data stored by the database will be a BLOB, not TEXT ...

--
The fact that there's a Highway to Hell but only a Stairway to Heaven says a lot about anticipated traffic volume.

>-----Original Message-----
>From: sqlite-users <[hidden email]> On
>Behalf Of Peng Yu
>Sent: Thursday, 30 January, 2020 02:24
>To: SQLite mailing list <[hidden email]>
>Subject: Re: [sqlite] Is it necessary to encode() for file names in sqlar
>format?
>
>So to confirm. In python 3, the str type should be used for name? Thanks.
>
>On Thu, Jan 30, 2020 at 12:58 AM Keith Medcalf <[hidden email]>
>wrote:
>
>>
>> sys.argv is a list of unicode text strings.  There is no need to
>> specifically encode or decode it so long as sys.getdefaultencoding()
>> returns 'utf-8'.  If your version of Python is so old that it returns
>> something else then you need to modify site.py and have it set the
>default
>> encoding to 'utf-8' otherwise you may end up with MBCS or some other
>> invalid text encoding in your database text fields.
>>
>> --
>> The fact that there's a Highway to Hell but only a Stairway to Heaven
>says
>> a lot about anticipated traffic volume.
>>
>> >-----Original Message-----
>> >From: sqlite-users <[hidden email]> On
>> >Behalf Of Peng Yu
>> >Sent: Wednesday, 29 January, 2020 22:57
>> >To: SQLite mailing list <[hidden email]>
>> >Subject: [sqlite] Is it necessary to encode() for file names in sqlar
>> >format?
>> >
>> >I use the following python3 code to create sqlar file. Is it necessary
>> >to sys.argv[2].encode('utf-8') in the line of execute()? In other
>> >word, does the native sqlar tools inteprete the name column as an
>> >encoded value or a non-encode value? Thanks.
>> >
>> >import sqlite3
>> >conn=sqlite3.connect(sys.argv[1])
>> >c=conn.cursor()
>> >c.execute('''
>> >CREATE TABLE IF NOT EXISTS sqlar(
>> >name TEXT PRIMARY KEY
>> >, mode INT
>> >, mtime INT
>> >, sz INT
>> >, data BLOB)
>> >''')
>> >
>> >import zlib
>> >data = sys.stdin.buffer.read()
>> >c.execute('REPLACE INTO sqlar VALUES(?, ?, ?, ?, ?)', [sys.argv[2], 0,
>> >0, len(data), zlib.compress(data)])
>> >conn.commit()
>> >
>> >--
>> >Regards,
>> >Peng
>> >_______________________________________________
>> >sqlite-users mailing list
>> >[hidden email]
>> >http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>>
>>
>>
>> _______________________________________________
>> sqlite-users mailing list
>> [hidden email]
>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>>
>--
>Regards,
>Peng
>_______________________________________________
>sqlite-users mailing list
>[hidden email]
>http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users



_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users