Question about automatic schema creation from custom data-strucutre for persistence storage

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Question about automatic schema creation from custom data-strucutre for persistence storage

Rajan, Vivek K
Hello-

 

Has anyone had experience with automatically creating schema from a
custom data-structure for persistence storage using SQLite? So, in my
C/C++ program I have the following type of data-structure:

 

struct _trace {

  std::string           name;  

  std::string           prop;

  std:string            object;

  int                      a;

  double                b;

};

 

std::vector <struct _trace> _myPath;


Now, let's say I want to write and read _myPath data-structure
(persistence) in SQLite database.

 

My questions:

  - Is there a mechanism to dynamically create a schema on-the-fly from
the data-structure?

  - Essentially a capability to store any custom data-structure into
SQLite data-base by automatic creation of schema for the data-structure

 

Rajan

 

 

Reply | Threaded
Open this post in threaded view
|

Re: Question about automatic schema creation from custom data-strucutre for persistence storage

Clay Dowling
Rajan, Vivek K wrote:

>  - Is there a mechanism to dynamically create a schema on-the-fly from
>the data-structure?
>
>  - Essentially a capability to store any custom data-structure into
>SQLite data-base by automatic creation of schema for the data-structure
>  
>
I don't know about creating a schema from a data structure, but I have
some experience going in the other direction, including automatic
generation of SQL queries.  I'm using this on a very large point of sale
system at the moment (although sadly neither with SQLite or C).

In going from schema to code the most important thing is that you need a
certain degree of consistency in your code.  For each table that you
want to generate objects for, it's highly recommended that you have a
single unique primary key, and that key is system-assigned. This lends
itself well to Boyce-Codd normal form, where the computer and the user
have a different idea of what the primary key is, and the user's key has
a uniqueness constraint upon it.  I understand that this doesn't fly
well with some DBAs.  I'm not a sophisticated enough designer to
understand their objections, but the design works well enough for my
purposes.

Clay Dowling
Reply | Threaded
Open this post in threaded view
|

Re: Question about automatic schema creation from custom data-strucutre for persistence storage

Dan Kennedy
In reply to this post by Rajan, Vivek K
As far as I know C++ has no introspection capability so the answer
is probably no. Unless you want to parse your header file yourself,
or something like that.
 
With Java or another high-level programming language that supports
introspection it would be possible I think.


"Rajan, Vivek K" <[hidden email]> wrote:
Hello-



Has anyone had experience with automatically creating schema from a
custom data-structure for persistence storage using SQLite? So, in my
C/C++ program I have the following type of data-structure:



struct _trace {

std::string name;

std::string prop;

std:string object;

int a;

double b;

};



std::vector _myPath;


Now, let's say I want to write and read _myPath data-structure
(persistence) in SQLite database.



My questions:

- Is there a mechanism to dynamically create a schema on-the-fly from
the data-structure?

- Essentially a capability to store any custom data-structure into
SQLite data-base by automatic creation of schema for the data-structure



Rajan






               
---------------------------------
 Yahoo! Music Unlimited - Access over 1 million songs. Try it free.
Reply | Threaded
Open this post in threaded view
|

Re: Question about automatic schema creation from custom data-strucutre for persistence storage

Jason Zhang-12
.Net has a tool (xsd.exe) that can convert C# structure to XML schema.
I don't know whether it can help you.

----- Original Message -----
From: "Dan Kennedy" <[hidden email]>
To: <[hidden email]>
Sent: Monday, October 17, 2005 4:50 PM
Subject: Re: [sqlite] Question about automatic schema creation from custom data-strucutre for persistence storage


> As far as I know C++ has no introspection capability so the answer
> is probably no. Unless you want to parse your header file yourself,
> or something like that.
>
> With Java or another high-level programming language that supports
> introspection it would be possible I think.
>
>
> "Rajan, Vivek K" <[hidden email]> wrote:
> Hello-
>
>
>
> Has anyone had experience with automatically creating schema from a
> custom data-structure for persistence storage using SQLite? So, in my
> C/C++ program I have the following type of data-structure:
>
>
>
> struct _trace {
>
> std::string name;
>
> std::string prop;
>
> std:string object;
>
> int a;
>
> double b;
>
> };
>
>
>
> std::vector _myPath;
>
>
> Now, let's say I want to write and read _myPath data-structure
> (persistence) in SQLite database.
>
>
>
> My questions:
>
> - Is there a mechanism to dynamically create a schema on-the-fly from
> the data-structure?
>
> - Essentially a capability to store any custom data-structure into
> SQLite data-base by automatic creation of schema for the data-structure
>
>
>
> Rajan
>
>
>
>
>
>
>
> ---------------------------------
> Yahoo! Music Unlimited - Access over 1 million songs. Try it free.
Reply | Threaded
Open this post in threaded view
|

Re: Question about automatic schema creation from custom data-strucutre for persistence storage

John Stanton-3
In reply to this post by Dan Kennedy
I addressed this some time back in a C program which dynamically
generates XML.  Generating SQL would be an option.

By building a table with a keyname for each data element and a pointer
to a data item the output structure can be assembled.  As a wise person
once told me "There is no problem in CS which cannot be solved by yet
another level of indirection".

In my case I also added a function pointer so that callbacks could be
used.  With XML the table is actually a transformation of the DTD so
that the result is guaranteed to be "well formed".  If the table were
compiled from the data structure then a similar integrity could be assured.

This method would require a few lines of code, but you would only need
to do it once as opposed to writing code for each data structure.
JS

Dan Kennedy wrote:

> As far as I know C++ has no introspection capability so the answer
> is probably no. Unless you want to parse your header file yourself,
> or something like that.
>  
> With Java or another high-level programming language that supports
> introspection it would be possible I think.
>
>
> "Rajan, Vivek K" <[hidden email]> wrote:
> Hello-
>
>
>
> Has anyone had experience with automatically creating schema from a
> custom data-structure for persistence storage using SQLite? So, in my
> C/C++ program I have the following type of data-structure:
>
>
>
> struct _trace {
>
> std::string name;
>
> std::string prop;
>
> std:string object;
>
> int a;
>
> double b;
>
> };
>
>
>
> std::vector _myPath;
>
>
> Now, let's say I want to write and read _myPath data-structure
> (persistence) in SQLite database.
>
>
>
> My questions:
>
> - Is there a mechanism to dynamically create a schema on-the-fly from
> the data-structure?
>
> - Essentially a capability to store any custom data-structure into
> SQLite data-base by automatic creation of schema for the data-structure
>
>
>
> Rajan
>
>
>
>
>
>
>
> ---------------------------------
>  Yahoo! Music Unlimited - Access over 1 million songs. Try it free.

Reply | Threaded
Open this post in threaded view
|

RE: Question about automatic schema creation from custom data-strucutre for persistence storage

nedbatchelder
I my experience, the best approach is to create a description of your data
in a form that is good for being parsed and feeding into a code generator.
C structures are not good for this, they are good for being compiled into
executable code.

I would create a data description, and use it to generate the SQL and the C
structures.  The data description could be whatever you find convenient: I
use XML, but anything will do so long as it is expressive enough for you and
you can parse it.

I use a code generation tool called Cog (http://nedbatchelder.com/code/cog)
to generate the code itself.

--Ned.
http://nedbatchelder.com
 

-----Original Message-----
From: John Stanton [mailto:[hidden email]]
Sent: Monday, 17 October, 2005 10:40 AM
To: [hidden email]
Subject: Re: [sqlite] Question about automatic schema creation from custom
data-strucutre for persistence storage

I addressed this some time back in a C program which dynamically
generates XML.  Generating SQL would be an option.

By building a table with a keyname for each data element and a pointer
to a data item the output structure can be assembled.  As a wise person
once told me "There is no problem in CS which cannot be solved by yet
another level of indirection".

In my case I also added a function pointer so that callbacks could be
used.  With XML the table is actually a transformation of the DTD so
that the result is guaranteed to be "well formed".  If the table were
compiled from the data structure then a similar integrity could be assured.

This method would require a few lines of code, but you would only need
to do it once as opposed to writing code for each data structure.
JS

Dan Kennedy wrote:

> As far as I know C++ has no introspection capability so the answer
> is probably no. Unless you want to parse your header file yourself,
> or something like that.
>  
> With Java or another high-level programming language that supports
> introspection it would be possible I think.
>
>
> "Rajan, Vivek K" <[hidden email]> wrote:
> Hello-
>
>
>
> Has anyone had experience with automatically creating schema from a
> custom data-structure for persistence storage using SQLite? So, in my
> C/C++ program I have the following type of data-structure:
>
>
>
> struct _trace {
>
> std::string name;
>
> std::string prop;
>
> std:string object;
>
> int a;
>
> double b;
>
> };
>
>
>
> std::vector _myPath;
>
>
> Now, let's say I want to write and read _myPath data-structure
> (persistence) in SQLite database.
>
>
>
> My questions:
>
> - Is there a mechanism to dynamically create a schema on-the-fly from
> the data-structure?
>
> - Essentially a capability to store any custom data-structure into
> SQLite data-base by automatic creation of schema for the data-structure
>
>
>
> Rajan
>
>
>
>
>
>
>
> ---------------------------------
>  Yahoo! Music Unlimited - Access over 1 million songs. Try it free.


Reply | Threaded
Open this post in threaded view
|

RE: Question about automatic schema creation from custom data-strucutre for persistence storage

Rajan, Vivek K
In reply to this post by Rajan, Vivek K
Hello John-

Thanks for your comments. Could you please provide some details on how
you have generated XML from custom data-structure and how you propose to
do this for SQL schema as well.

Any code examples would really help.

Rajan

>-----Original Message-----
>From: John Stanton [mailto:[hidden email]]
>Sent: Monday, October 17, 2005 7:40 AM
>To: [hidden email]
>Subject: Re: [sqlite] Question about automatic schema creation from
custom

>data-strucutre for persistence storage
>
>I addressed this some time back in a C program which dynamically
>generates XML.  Generating SQL would be an option.
>
>By building a table with a keyname for each data element and a pointer
>to a data item the output structure can be assembled.  As a wise person
>once told me "There is no problem in CS which cannot be solved by yet
>another level of indirection".
>
>In my case I also added a function pointer so that callbacks could be
>used.  With XML the table is actually a transformation of the DTD so
>that the result is guaranteed to be "well formed".  If the table were
>compiled from the data structure then a similar integrity could be
assured.

>
>This method would require a few lines of code, but you would only need
>to do it once as opposed to writing code for each data structure.
>JS
>
>Dan Kennedy wrote:
>> As far as I know C++ has no introspection capability so the answer
>> is probably no. Unless you want to parse your header file yourself,
>> or something like that.
>>
>> With Java or another high-level programming language that supports
>> introspection it would be possible I think.
>>
>>
>> "Rajan, Vivek K" <[hidden email]> wrote:
>> Hello-
>>
>>
>>
>> Has anyone had experience with automatically creating schema from a
>> custom data-structure for persistence storage using SQLite? So, in my
>> C/C++ program I have the following type of data-structure:
>>
>>
>>
>> struct _trace {
>>
>> std::string name;
>>
>> std::string prop;
>>
>> std:string object;
>>
>> int a;
>>
>> double b;
>>
>> };
>>
>>
>>
>> std::vector _myPath;
>>
>>
>> Now, let's say I want to write and read _myPath data-structure
>> (persistence) in SQLite database.
>>
>>
>>
>> My questions:
>>
>> - Is there a mechanism to dynamically create a schema on-the-fly from
>> the data-structure?
>>
>> - Essentially a capability to store any custom data-structure into
>> SQLite data-base by automatic creation of schema for the
data-structure

>>
>>
>>
>> Rajan
>>
>>
>>
>>
>>
>>
>>
>> ---------------------------------
>>  Yahoo! Music Unlimited - Access over 1 million songs. Try it free.

Reply | Threaded
Open this post in threaded view
|

Re: Question about automatic schema creation from custom data-strucutre for persistence storage

John Stanton-3
In reply to this post by nedbatchelder
As Ned points out starting with something like XML is neater than
working back to it.  C structures parse very nicely, but XML parsers are
everywhere.  All you need to build is a code generator to optionally
compile SQL or C/C++ from the XML data definition, a simple task.
JS

Ned Batchelder wrote:

> I my experience, the best approach is to create a description of your data
> in a form that is good for being parsed and feeding into a code generator.
> C structures are not good for this, they are good for being compiled into
> executable code.
>
> I would create a data description, and use it to generate the SQL and the C
> structures.  The data description could be whatever you find convenient: I
> use XML, but anything will do so long as it is expressive enough for you and
> you can parse it.
>
> I use a code generation tool called Cog (http://nedbatchelder.com/code/cog)
> to generate the code itself.
>
> --Ned.
> http://nedbatchelder.com
>  
>
> -----Original Message-----
> From: John Stanton [mailto:[hidden email]]
> Sent: Monday, 17 October, 2005 10:40 AM
> To: [hidden email]
> Subject: Re: [sqlite] Question about automatic schema creation from custom
> data-strucutre for persistence storage
>
> I addressed this some time back in a C program which dynamically
> generates XML.  Generating SQL would be an option.
>
> By building a table with a keyname for each data element and a pointer
> to a data item the output structure can be assembled.  As a wise person
> once told me "There is no problem in CS which cannot be solved by yet
> another level of indirection".
>
> In my case I also added a function pointer so that callbacks could be
> used.  With XML the table is actually a transformation of the DTD so
> that the result is guaranteed to be "well formed".  If the table were
> compiled from the data structure then a similar integrity could be assured.
>
> This method would require a few lines of code, but you would only need
> to do it once as opposed to writing code for each data structure.
> JS
>
> Dan Kennedy wrote:
>
>>As far as I know C++ has no introspection capability so the answer
>>is probably no. Unless you want to parse your header file yourself,
>>or something like that.
>>
>>With Java or another high-level programming language that supports
>>introspection it would be possible I think.
>>
>>
>>"Rajan, Vivek K" <[hidden email]> wrote:
>>Hello-
>>
>>
>>
>>Has anyone had experience with automatically creating schema from a
>>custom data-structure for persistence storage using SQLite? So, in my
>>C/C++ program I have the following type of data-structure:
>>
>>
>>
>>struct _trace {
>>
>>std::string name;
>>
>>std::string prop;
>>
>>std:string object;
>>
>>int a;
>>
>>double b;
>>
>>};
>>
>>
>>
>>std::vector _myPath;
>>
>>
>>Now, let's say I want to write and read _myPath data-structure
>>(persistence) in SQLite database.
>>
>>
>>
>>My questions:
>>
>>- Is there a mechanism to dynamically create a schema on-the-fly from
>>the data-structure?
>>
>>- Essentially a capability to store any custom data-structure into
>>SQLite data-base by automatic creation of schema for the data-structure
>>
>>
>>
>>Rajan
>>
>>
>>
>>
>>
>>
>>
>>---------------------------------
>> Yahoo! Music Unlimited - Access over 1 million songs. Try it free.
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Question about automatic schema creation from custom data-strucutre for persistence storage

Jay Sprenkle
Personally I would start from the database and generate code to match the
schema. Like others pointed out it's fairly difficult to go the other way.