Importing Text to Create a Table

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Importing Text to Create a Table

Lawrence Murphy
I am supporting a website which aims to protect a forest from development.
The website produces an email for supporters to mail out. A copy of the
email is sent to our Gmail address and we wish to capture the supporters
return email address. Google provides an archive of our Gmail account which
is 458Mbs in size and contains a lot of superfluous data.

I have tried making a table with one column, text 255c in size and doing
the import but it takes more than overnight and is still running. Is there
a quicker way to import the data?

Warm Regards,
Lawrence

Mb: 0408 403 324
PO Box 263 Cherrybrook NSW 2126

Lose Weight, Gain Health & Prevent Disease
Find out how here....
http://lwghpd.blogspot.com.au/
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Importing Text to Create a Table

Luuk
Can you give a (short) example of some lines,
and how you import them?

Creating a table with 1 field text 255 seems not the right way to do
this if you are only interested in emailaddresses.



On 22-12-17 01:22, Lawrence Murphy wrote:

> I am supporting a website which aims to protect a forest from development.
> The website produces an email for supporters to mail out. A copy of the
> email is sent to our Gmail address and we wish to capture the supporters
> return email address. Google provides an archive of our Gmail account which
> is 458Mbs in size and contains a lot of superfluous data.
>
> I have tried making a table with one column, text 255c in size and doing
> the import but it takes more than overnight and is still running. Is there
> a quicker way to import the data?
>
> Warm Regards,
> Lawrence
>
>

_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Importing Text to Create a Table

Chris Locke-3
In reply to this post by Lawrence Murphy
What operating system are you using, and what software are you using to do
the import?  What specificially are you trying to import?
If you perform a sequence of 'insert' statements, then that can be time
consuming - its better to incorporate them into one 'transaction' - sqlite
bundles the operations into one big operation.  This is better IO wise - it
performs far less writes to the disk, etc.  But understanding the methods
you're using so far would be helpful.

On Fri, Dec 22, 2017 at 12:22 AM, Lawrence Murphy <[hidden email]> wrote:

> I am supporting a website which aims to protect a forest from development.
> The website produces an email for supporters to mail out. A copy of the
> email is sent to our Gmail address and we wish to capture the supporters
> return email address. Google provides an archive of our Gmail account which
> is 458Mbs in size and contains a lot of superfluous data.
>
> I have tried making a table with one column, text 255c in size and doing
> the import but it takes more than overnight and is still running. Is there
> a quicker way to import the data?
>
> Warm Regards,
> Lawrence
>
> Mb: 0408 403 324
> PO Box 263 Cherrybrook NSW 2126
>
> Lose Weight, Gain Health & Prevent Disease
> Find out how here....
> http://lwghpd.blogspot.com.au/
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Importing Text to Create a Table

Peter da Silva
In reply to this post by Lawrence Murphy
I suspect you would be best advised to do more processing of the data to extract just the email addresses rather than treating it like an unstructured text blob.
 

_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users