Re: Why is Sqlite mediatype not registered at iana

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Why is Sqlite mediatype not registered at iana

Paul van Genuchten-2
Hi Clemens, thank you for your reply. I’d be happy to volunteer for that exercise, however not sure if i’m a good fit, since i’m currently not a member of the sqlite consortium.

Related to your comment to register sqlite as a structured syntax suffix (RFC 6839). I’d say why not both. For my use-case (detecting the mediatype of a file from its iso19115- or dcat-metadata) it is relevant to have it as a media-type preferably at Iana.

Seems the currently widely used application/x-sqlite3 will probably not be accepted by iana, we need a more formal type, such as application/vnd.sqlite

Hope to hear from you,
Regards, Paul.
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Proposed registration for application/vnd.sqlite3 and +sqlite3 (was: Why is Sqlite mediatype not registered at iana)

Clemens Ladisch
Paul van Genuchten wrote:
> i’m currently not a member of the sqlite consortium.

RFC 6838 § 3.2 says:
| A registration may be placed in the vendor tree by anyone who needs
| to interchange files associated with some product or set of products.
| However, the registration properly belongs to the vendor or
| organization producing the software that employs the type being
| registered, and that vendor or organization can at any time elect to
| assert ownership of a registration done by a third party in order to
| correct or update it.


Okay, did I overlook anything (especially in the security or
interoperability considerations sections)?


========================================================================


Type name:

  application

Subtype name:

  vnd.sqlite3

Required parameters:

  none

Optional parameters:

  none

Encoding considerations:

  binary

Security considerations:

  Database files contain complex data structures, so parsers must take
  care to prevent buffer overflows, stack overruns, and other unexpected
  behaviour caused by malicious content.

  Views and triggers can contain arbitrary SQL expressions (including
  recursion), which can result in arbitrarily large amounts of
  processing time, memory, and disk space required when attempting to
  access data.  Applications should use mechanisms like
  sqlite3_interrupt() or sqlite3_progress_handler() to allow long
  computations to be aborted, and an alternative memory allocator to
  limit the amount of memory used.

  The SQLite library itself, as distributed, does not allow SQL
  statements to access resources or data outside the database.  However,
  if applications add extension modules or functions, they should not do
  so in the database connection used to access untrusted content, or
  they must ensure that these modules/functions are safe to use even
  when called from malicious SQL code.

  The database may leave part of deleted or updated data in the database
  file.  Applications that do not want ot leave traces of old data must
  enable PRAGMA secure_delete before doing any modifications, or run
  VACUUM before transmitting the database file.

  Databases can use indexes to cache data in a format that is faster to
  access for certain queries.  It is possible to construct database
  files with inconsistent data in indexes so that some queries return
  data different from what is actually stored in a table.  To avoid
  this, applications should run REINDEX before accessing a database
  received from a potentially malicious source.

  This format provides no cryptographic integrity protection of any
  kind.

  Databases can be used to store blobs containing data to be handled by
  other applications or libraries; any security considerations of those
  must also be taken into account.

Interoperability considerations:

  At publication of this document, there exists only a single
  implementation, the SQLite library.

  Database files written with recent versions of the library can be read
  and modified by any version back to at least 3.7.0 (released
  2010-07-21).  However, there is no backwards compatibility if SQL
  features introduced in a newer version are actually used.  To ensure
  interoperability with other applications that use an older version of
  the library, applications SHOULD avoid using features that are not
  supported in the version that other applications are known or
  suspected to use.  At publication of this document, features
  introduced in newer versions are:

  3.20.0: deterministic date/time functions;
  3.18.0: printf() thousands marks;
  3.16.0: PRAGMA functions;
  3.15.0: row values; deterministic SQL functions in partial indexes;
  3.9.0: expression indexes;
  3.8.8: more than 500 rows in a VALUES clause;
  3.8.6: hexadecimal integer literals; likely();
  3.8.3: common table expressions (WITH); printf();
  3.8.2: clustered indexes (WITHOUT ROWID tables);
  3.8.1: unlikely(); likelihood();
  3.8.0: partial indexes;
  3.7.16: unicode(); char();
  3.7.15: instr();
  3.7.11: multiple rows in a VALUES clause; bare columns in aggregate queries.

  Some runtime settings (e.g., PRAGMA case_sensitive_like) or
  compilation options can change the semantics of SQL statements.
  Applications SHOULD use the default settings and options; however,
  some settings (e.g., PRAGMA foreign_keys) are disabled by default only
  for backwards compatibility and are commonly enabled.

  When a transaction that changes the database has not yet committed,
  the database file might be in an inconsistent state and require data
  from the rollback journal to get back to a consistent state.
  Therefore, when it is possible that other processes or threads change
  a database, an application that wishes to transmit a database file
  SHOULD prevent concurrent changes by executing BEGIN IMMEDIATE before
  reading/copying the file, or use the backup API to create a consistent
  copy of the database.

  A database in WAL mode can have part of its data in the WAL file.
  Therefore, an application that wishes to transmit a database file in
  WAL mode SHOULD initiate a full checkpoint before reading/copying the
  file, or use the backup API to create a copy of the database.

  The unregistered media type "application/x-sqlite3" MUST NOT be used,
  except where required for backwards compatibility.

Published specification:

  http://www.sqlite.org/fileformat2.html
  http://www.sqlite.org/lang.html

Applications that use this media type:

  Applications that want to store or interchange relational data.

Fragment identifier considerations:

  none

Deprecated alias names for this type:

  application/x-sqlite3

Magic number:

  53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00
  (zero-terminated ASCII "SQLite format 3") at offset 0

File extensions:

  .db, .sqlite, .sqlite3
  (".db" does not uniquely identify SQLite database files.
  Other extensions are commonly used.)

Macintosh file type code:

  none

Contact:

  SQLite mailing list
  <[hidden email]>

Intended usage:

  COMMON

Restrictions on usage:

  none

Author/Change controller:

  Clemens Ladisch
  <[hidden email]>

Provisional registration? (standards tree only):

  N/A


========================================================================


Name

  SQLite3 database

+suffix

  +sqlite3

References

  Same as for "application/vnd.sqlite3".

Encoding considerations

  binary

Interoperability considerations

  Same as for "application/vnd.sqlite3".

  To allow identification of files when the media type name is not
  available, each individual "xxx/yyy+sqlite3" registration SHOULD
  specify an appliction ID value to be set with PRAGMA application_id
  (http://www.sqlite.org/pragma.html#pragma_application_id), and SHOULD
  specifiy it as a second magic number (file offset 68, see
  http://www.sqlite.org/fileformat2.html#application_id) in addition to
  the header string at offset 0.  This value should also be added to the
  magic.txt file in the SQLite repository
  (http://www.sqlite.org/src/artifact?ci=trunk&filename=magic.txt) by
  submitting a patch to <[hidden email]>.

Fragment identifier considerations

  The syntax and semantics of fragment identifiers specified for
  +sqlite3 SHOULD be as specified for "application/vnd.sqlite3".
  (At publication of this document, there is no fragment identification
  syntax defined for "application/vnd.sqlite3".)

  The syntax and semantics of fragment identifiers for a specific
  "xxx/yyy+sqlite3" SHOULD be processed as follows:

  For cases defined in +sqlite3, where the fragment identifier resolves
  per the +sqlite3 rules, then as specified in +sqlite3.

  For cases defined in +sqlite3, where the fragment identifier does not
  resolve per the +sqlite3 rules, then as specified in "xxx/yyy+sqlite3".

  For cases not defined in +sqlite3, then as specified in "xxx/yyy+sqlite3".

Security considerations

  Same as for "application/vnd.sqlite3".

  Each individual media type registered with a +sqlite3 suffix can have
  additional security considerations.  For example, if a specific
  registration requires that certain extension functions are available,
  or that blob fields contain data to be processed by other libraries or
  external tools, or if only a single implementation exists to handle
  a specific registered media type, then this increases the known attack
  surface available to an attacker.

Contact

  SQLite mailing list
  <[hidden email]>

Author/Change controller.

  Clemens Ladisch
  <[hidden email]>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Proposed registration for application/vnd.sqlite3 and +sqlite3 (was: Why is Sqlite mediatype not registered at iana)

Keith Medcalf

I should that the Security Implications are NONE.  

There are no security implications in setting a MIME type for "magic number" containing SQLite3 databases.

I should think that the "Interoperability Considerations" are that the contained data cannot be processed as a stream and it must be saved to a filesystem before it can be used by any application incorporating the SQLite3 database engine.

---
The fact that there's a Highway to Hell but only a Stairway to Heaven says a lot about anticipated traffic volume.

>-----Original Message-----
>From: sqlite-users [mailto:sqlite-users-
>[hidden email]] On Behalf Of Clemens Ladisch
>Sent: Sunday, 1 October, 2017 08:33
>To: [hidden email]
>Subject: Re: [sqlite] Proposed registration for
>application/vnd.sqlite3 and +sqlite3 (was: Why is Sqlite mediatype
>not registered at iana)
>
>Paul van Genuchten wrote:
>> i’m currently not a member of the sqlite consortium.
>
>RFC 6838 § 3.2 says:
>| A registration may be placed in the vendor tree by anyone who needs
>| to interchange files associated with some product or set of
>products.
>| However, the registration properly belongs to the vendor or
>| organization producing the software that employs the type being
>| registered, and that vendor or organization can at any time elect
>to
>| assert ownership of a registration done by a third party in order
>to
>| correct or update it.
>
>
>Okay, did I overlook anything (especially in the security or
>interoperability considerations sections)?
>
>
>=====================================================================
>===
>
>
>Type name:
>
>  application
>
>Subtype name:
>
>  vnd.sqlite3
>
>Required parameters:
>
>  none
>
>Optional parameters:
>
>  none
>
>Encoding considerations:
>
>  binary
>
>Security considerations:
>
>  Database files contain complex data structures, so parsers must
>take
>  care to prevent buffer overflows, stack overruns, and other
>unexpected
>  behaviour caused by malicious content.
>
>  Views and triggers can contain arbitrary SQL expressions (including
>  recursion), which can result in arbitrarily large amounts of
>  processing time, memory, and disk space required when attempting to
>  access data.  Applications should use mechanisms like
>  sqlite3_interrupt() or sqlite3_progress_handler() to allow long
>  computations to be aborted, and an alternative memory allocator to
>  limit the amount of memory used.
>
>  The SQLite library itself, as distributed, does not allow SQL
>  statements to access resources or data outside the database.
>However,
>  if applications add extension modules or functions, they should not
>do
>  so in the database connection used to access untrusted content, or
>  they must ensure that these modules/functions are safe to use even
>  when called from malicious SQL code.
>
>  The database may leave part of deleted or updated data in the
>database
>  file.  Applications that do not want ot leave traces of old data
>must
>  enable PRAGMA secure_delete before doing any modifications, or run
>  VACUUM before transmitting the database file.
>
>  Databases can use indexes to cache data in a format that is faster
>to
>  access for certain queries.  It is possible to construct database
>  files with inconsistent data in indexes so that some queries return
>  data different from what is actually stored in a table.  To avoid
>  this, applications should run REINDEX before accessing a database
>  received from a potentially malicious source.
>
>  This format provides no cryptographic integrity protection of any
>  kind.
>
>  Databases can be used to store blobs containing data to be handled
>by
>  other applications or libraries; any security considerations of
>those
>  must also be taken into account.
>
>Interoperability considerations:
>
>  At publication of this document, there exists only a single
>  implementation, the SQLite library.
>
>  Database files written with recent versions of the library can be
>read
>  and modified by any version back to at least 3.7.0 (released
>  2010-07-21).  However, there is no backwards compatibility if SQL
>  features introduced in a newer version are actually used.  To
>ensure
>  interoperability with other applications that use an older version
>of
>  the library, applications SHOULD avoid using features that are not
>  supported in the version that other applications are known or
>  suspected to use.  At publication of this document, features
>  introduced in newer versions are:
>
>  3.20.0: deterministic date/time functions;
>  3.18.0: printf() thousands marks;
>  3.16.0: PRAGMA functions;
>  3.15.0: row values; deterministic SQL functions in partial indexes;
>  3.9.0: expression indexes;
>  3.8.8: more than 500 rows in a VALUES clause;
>  3.8.6: hexadecimal integer literals; likely();
>  3.8.3: common table expressions (WITH); printf();
>  3.8.2: clustered indexes (WITHOUT ROWID tables);
>  3.8.1: unlikely(); likelihood();
>  3.8.0: partial indexes;
>  3.7.16: unicode(); char();
>  3.7.15: instr();
>  3.7.11: multiple rows in a VALUES clause; bare columns in aggregate
>queries.
>
>  Some runtime settings (e.g., PRAGMA case_sensitive_like) or
>  compilation options can change the semantics of SQL statements.
>  Applications SHOULD use the default settings and options; however,
>  some settings (e.g., PRAGMA foreign_keys) are disabled by default
>only
>  for backwards compatibility and are commonly enabled.
>
>  When a transaction that changes the database has not yet committed,
>  the database file might be in an inconsistent state and require
>data
>  from the rollback journal to get back to a consistent state.
>  Therefore, when it is possible that other processes or threads
>change
>  a database, an application that wishes to transmit a database file
>  SHOULD prevent concurrent changes by executing BEGIN IMMEDIATE
>before
>  reading/copying the file, or use the backup API to create a
>consistent
>  copy of the database.
>
>  A database in WAL mode can have part of its data in the WAL file.
>  Therefore, an application that wishes to transmit a database file
>in
>  WAL mode SHOULD initiate a full checkpoint before reading/copying
>the
>  file, or use the backup API to create a copy of the database.
>
>  The unregistered media type "application/x-sqlite3" MUST NOT be
>used,
>  except where required for backwards compatibility.
>
>Published specification:
>
>  http://www.sqlite.org/fileformat2.html
>  http://www.sqlite.org/lang.html
>
>Applications that use this media type:
>
>  Applications that want to store or interchange relational data.
>
>Fragment identifier considerations:
>
>  none
>
>Deprecated alias names for this type:
>
>  application/x-sqlite3
>
>Magic number:
>
>  53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00
>  (zero-terminated ASCII "SQLite format 3") at offset 0
>
>File extensions:
>
>  .db, .sqlite, .sqlite3
>  (".db" does not uniquely identify SQLite database files.
>  Other extensions are commonly used.)
>
>Macintosh file type code:
>
>  none
>
>Contact:
>
>  SQLite mailing list
>  <[hidden email]>
>
>Intended usage:
>
>  COMMON
>
>Restrictions on usage:
>
>  none
>
>Author/Change controller:
>
>  Clemens Ladisch
>  <[hidden email]>
>
>Provisional registration? (standards tree only):
>
>  N/A
>
>
>=====================================================================
>===
>
>
>Name
>
>  SQLite3 database
>
>+suffix
>
>  +sqlite3
>
>References
>
>  Same as for "application/vnd.sqlite3".
>
>Encoding considerations
>
>  binary
>
>Interoperability considerations
>
>  Same as for "application/vnd.sqlite3".
>
>  To allow identification of files when the media type name is not
>  available, each individual "xxx/yyy+sqlite3" registration SHOULD
>  specify an appliction ID value to be set with PRAGMA application_id
>  (http://www.sqlite.org/pragma.html#pragma_application_id), and
>SHOULD
>  specifiy it as a second magic number (file offset 68, see
>  http://www.sqlite.org/fileformat2.html#application_id) in addition
>to
>  the header string at offset 0.  This value should also be added to
>the
>  magic.txt file in the SQLite repository
>  (http://www.sqlite.org/src/artifact?ci=trunk&filename=magic.txt) by
>  submitting a patch to <[hidden email]>.
>
>Fragment identifier considerations
>
>  The syntax and semantics of fragment identifiers specified for
>  +sqlite3 SHOULD be as specified for "application/vnd.sqlite3".
>  (At publication of this document, there is no fragment
>identification
>  syntax defined for "application/vnd.sqlite3".)
>
>  The syntax and semantics of fragment identifiers for a specific
>  "xxx/yyy+sqlite3" SHOULD be processed as follows:
>
>  For cases defined in +sqlite3, where the fragment identifier
>resolves
>  per the +sqlite3 rules, then as specified in +sqlite3.
>
>  For cases defined in +sqlite3, where the fragment identifier does
>not
>  resolve per the +sqlite3 rules, then as specified in
>"xxx/yyy+sqlite3".
>
>  For cases not defined in +sqlite3, then as specified in
>"xxx/yyy+sqlite3".
>
>Security considerations
>
>  Same as for "application/vnd.sqlite3".
>
>  Each individual media type registered with a +sqlite3 suffix can
>have
>  additional security considerations.  For example, if a specific
>  registration requires that certain extension functions are
>available,
>  or that blob fields contain data to be processed by other libraries
>or
>  external tools, or if only a single implementation exists to handle
>  a specific registered media type, then this increases the known
>attack
>  surface available to an attacker.
>
>Contact
>
>  SQLite mailing list
>  <[hidden email]>
>
>Author/Change controller.
>
>  Clemens Ladisch
>  <[hidden email]>
>_______________________________________________
>sqlite-users mailing list
>[hidden email]
>http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users



_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Proposed registration for application/vnd.sqlite3 and +sqlite3 (was: Why is Sqlite mediatype not registered at iana)

Clemens Ladisch
Keith Medcalf wrote:
> I should that the Security Implications are NONE.
>
> There are no security implications in setting a MIME type for "magic
> number" containing SQLite3 databases.

RFC 6838 § 4.6 says that
| the security considerations MUST NOT state that there are "no security
| issues associated with this type".

> I should think that the "Interoperability Considerations" are that the
> contained data cannot be processed as a stream and it must be saved to
> a filesystem before it can be used by any application incorporating the
> SQLite3 database engine.

Then you're using the wrong VFS.  ;-)


Regards,
Clemens
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users