CLI feature request: .import option -h

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

CLI feature request: .import option -h

Christof Böckler
Hi all,

I'm sorry if I'm wrong on this list, but I have a small feature request
for the CLI.

I regularly import CSV data with a first line of headers into a SQLite
database by using the .import command. On the first run the destination
table gets automatically created. Later I get more data from the same
source and want to add these rows to the same table by using the same
command (.import).  Unfortunately the (first) header line will this time
be interpreted as data instead of as meta data.

Thus I request an option for the .import command, that will always regard
the first line of input as header information. I suggest we call it -h.

If the destination table does not exist yet, then the option is a no op
compared to the current implementation.

If the destination table already exists, then simply start importing
data on line 2.

I know that this can be achieved by other means outside SQLite,
but I think it should be possible with sqlite3 alone. Otherwise data
preprocessing has to be done based on a condition inside the SQLite
database, which seems pretty complicated compared to ignoring one line
of input.

Greetings
Christof
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: CLI feature request: .import option -h

Shawn Wagner
A more flexible approach might be to add a way to have .import run a
command and use its standard output as the data source, instead of a file.

sqlite> .import "!tail +2 data.csv" mytable

for this example. Lets you do basic ETL stuff for any import format without
having to use temporary files or leave the shell. .once and .output could
be extended the same way, piping output lines to a process.

Should be easy to implement by using popen() instead of fopen() when
requested.

On Sun, Sep 30, 2018 at 3:05 AM Christof Böckler <[hidden email]> wrote:

> Hi all,
>
> I'm sorry if I'm wrong on this list, but I have a small feature request
> for the CLI.
>
> I regularly import CSV data with a first line of headers into a SQLite
> database by using the .import command. On the first run the destination
> table gets automatically created. Later I get more data from the same
> source and want to add these rows to the same table by using the same
> command (.import).  Unfortunately the (first) header line will this time
> be interpreted as data instead of as meta data.
>
> Thus I request an option for the .import command, that will always regard
> the first line of input as header information. I suggest we call it -h.
>
> If the destination table does not exist yet, then the option is a no op
> compared to the current implementation.
>
> If the destination table already exists, then simply start importing
> data on line 2.
>
> I know that this can be achieved by other means outside SQLite,
> but I think it should be possible with sqlite3 alone. Otherwise data
> preprocessing has to be done based on a condition inside the SQLite
> database, which seems pretty complicated compared to ignoring one line
> of input.
>
> Greetings
> Christof
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users