Bad database manager. No biscuit.

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Bad database manager. No biscuit.

Simon Slavin-3
tl;dr: CSV is a useful standard which, like most things in computing, has a couple of 'gotchas'.  Not all '.CSV' files are CSV files.

Can't give details but I just identified a problem which had caused the SQL database owner £100,000 == US$130,000 == €117,000 of damage, mostly because end-of-day crashed three days in a row.  Plus my company's fees.

The cause was some or all of the following, depending on your philosophy.

A) SQL schema had constraints defined for columns, but the person who did it didn't think of all the possible things that might go wrong, or didn't think they were all worth guarding against.
B) A CSV file was imported by someone who didn't understand the concepts of escaping quote characters and how missing values are handled.
C) Excel's CSV exporter does not conform to RFC 4180, the standard for CSV files.
D) An organisational culture which allows one geek to do a special one-off operation, outside his specialist area, without another geek casting a skeptical eye over the procedure or results.
E) Software which sanitises the data it put into the database very carefully, but didn't expect anything else to mess with the database.
F) Original writers of the software no longer work for the company.  New maintainers have never had anything serious go wrong before and didn't know how to debug serious problems.  Good enough to do their regular job, not good enough to deal with emergencies.

Situation was complicated by happening just after the Christmas holiday, so some people thought that Christmas disruption or an end-of-year run was part of the problem.
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users