Record serialization infrastructure of SQLITE

Previous Topic Next Topic
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Record serialization infrastructure of SQLITE

Inspired by the json encoding thread.

We have a need for multi-dimensional data (like json), and currently use a record format for that that is derived from the SQLITE format with Serial Type/Size stored in varint Huffman encoding of twos-complement. It is a great fast, compact storage format. We added support for arrays and objects on top of it, but just recursive processing of blobs would allow for the same. However, we had to build our own make/extract/expression extensions from scratch to do this.

It would be very helpful if SQLITE can expose its internal record serialization infrastructure a bit so that there is a way (using the API's) to create an array of sqlite3_value from a blob, and blob from sqlite3_value array. E.g.

blob = sqlite3_column_blob(_stmt, 2);
sqlite3_array** arr = sqlite3_deserialize(blob, &len);
val1 = sqlite3_value_int(arr[0]);
val2 = sqlite3_value_int(arr[1]);

sqlite3_value* arr[2];
arr[0] = sqlite3_make_value_int(42);
arr[1] = sqlite3_make_value_int(43);
sqlite3_value blob = sqlite3_serialize(arr, 2);
sqlite3_bind_value(stmt, 2, blob);

It would probably also be useful if there's some limited expression functions to interact with it, but those are easy to build yourself. The serialization infrastructure however is pretty complex.

- Deon

sqlite-users mailing list
[hidden email]