[nycphp-talk] OT: Standard Data Import Tool?
Matt Juszczak
matt at atopia.net
Tue Dec 29 12:28:27 EST 2009
Hi all,
I need to write a tool that somehow takes multiple data feeds (some xls
files, some csv files, etc.), defines each format,
verifies/validates/sanitizes the data, and imports it into standardized
MySQL tables.
As an example, one feed might look like this:
First Name, Last Name, DOB
And another like this:
Name, DOB
And another:
DOB, Last Name, First Name, Middle Initial
I need to somehow be able to say "For this source, this column needs to be
split into two, and this other column is the date of birth column". Once
the data is parsed, it would need to be imported into standard MySQL
tables:
first_name, last_name, birth_date, etc.
This is a simple example - the files and tables actually have 20-30
columns each, so using mysqlimport or a simple shell script probably isn't
the best option.
Are there open source tools out there that do this already? If not,
should I just build something?
Thanks,
-Matt
More information about the talk
mailing list