Organizing large database into multiple files

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Organizing large database into multiple files

Rajan, Vivek K
Hello-

 

I have a need to store large volumes of data (~5-10G) in SQLite
database. The data which I am storing is organized hierarchically. The
schema for my database has foreign-key constraints, the tables are
interrelated. My questions:

  - How can I organize the entire database into multiple files?

  - How do manage such huge databases with SQLite

 

I am looking for suggestions/advice on how to organize large volumes of
data. Please help.

 

Vivek

Reply | Threaded
Open this post in threaded view
|

Re: Organizing large database into multiple files

Jay Sprenkle
> I have a need to store large volumes of data (~5-10G) in SQLite
> database. The data which I am storing is organized hierarchically. The
> schema for my database has foreign-key constraints, the tables are
> interrelated. My questions:
>
>   - How can I organize the entire database into multiple files?
>
>   - How do manage such huge databases with SQLite
>
>
>
> I am looking for suggestions/advice on how to organize large volumes of
> data. Please help.

We usually break the data into subsets and place them into different tables
or databases. One per fiscal year, geographic region, etc.
We seldom needed reporting over the full data set but it was still
possible to do that way.
Reply | Threaded
Open this post in threaded view
|

RE: Organizing large database into multiple files

roger-15
In reply to this post by Rajan, Vivek K


> -------- Original Message --------
> Subject: [sqlite] Organizing large database into multiple files
> From: "Rajan, Vivek K" <[hidden email]>
> Date: Sat, November 12, 2005 5:09 am
> To: <[hidden email]>
>
> Hello-
>
>  
>
> I have a need to store large volumes of data (~5-10G) in SQLite
> database. The data which I am storing is organized hierarchically. The
> schema for my database has foreign-key constraints, the tables are
> interrelated. My questions:
>
>   - How can I organize the entire database into multiple files?
>
>   - How do manage such huge databases with SQLite
>
>  
>
> I am looking for suggestions/advice on how to organize large volumes
> of
> data. Please help.
>
>  
>
> Vivek
>
>

If your tables are spread over databases in several files you can use
the 'ATTACH DATABASE' command to make them appear as a single database.
If the schema don't conflict you can merge them. Otherwise you can use
the db.table syntax, which I think is what you are looking for.

http://www.sqlite.org/lang_attach.html