Solo Technology What do you need done today?

Data Conversion / ETL Project

Spent the last 2.5 months of evenings working on project for several (inter-related) machineLOGIC clients. They're all moving to an online web application from Zywave and they all have legacy data in a variety of formats. That legacy data came from their two existing applications: CDS and Goldmine.

To keep it interesting, the implementations of CDS and Goldmine were rather customized and very flexible. That definitely caused some extra challenges when doing the conversion.

Zywave helps make it easy for conversions though. They provide a set of spreadsheet templates and if you somehow get your data into those templates they can get it loaded. Thus, the challenge was getting the data out of the current systems and then massaged into a useful dataset that could be dumped into spreadsheets.

Enter MS Access 2007. Access makes linking to external data a breeze and the (reasonably) standard SQL implementation makes it a powerful tool in a low-buck ETL project. And that's exactly how it worked for me on this one. I'd pull the data out of the existing systems into CSV and then link those CSV files into Access to look like  tables. Then I'd run my queries  and generate data suitable for exporting to the templates.

This project went several months, mostly due to the amount of data cleansing that had to be done before we could send it to Zywave.