|
|
|
|
|
|
||||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||||||||||||||
|
Civil Rights Litigation Clearinghouse
Clearinghouse Backend
Data projects come in two flavors; those that start from scratch and those that use pre-existing sources. This project was of the latter type. The kernel of the collection had been growing for over ten years so before any development could begin, a thorough examination was necessary. After this initial assessment, the principal investigator, Professor Margo Schlanger ( site), and the technicians defined the task at hand. The following objective emerged:
Step 1: Discovery The first step required an assessment of the existing data. This involved evaluating a collection of nearly 10,000 files, analyzing multiple databases, spreadsheets, rich-text documents, and simple text files. Through close collaboration with the principal investigator an information model was created. From this base, we additionally projected areas where enhancements were needed. Step 2: Organization Next we had to create a path from our current state to our desired state. An important facet here was to account for holes in pre-existing data. Add to that the intelligent grafting of newly identified data points. These were time consuming examinations because a mis-step here could have expensive implications further downstream. Step 3: Data Scrubbing Once our 'blue-sky' model was decided upon, we had to massage the prior data into an amenable format. This phase of work is not only arduous but also precarious given that any alignment or formatting errors risked corrupting the source pool of data. And, this was the place to make wholesale changes because once we entered the next phase, adjusting previously handled data would often require re-processing adjacent data sets. Step 4: Conversion With the data now clean, we were ready to port it into the new system environment. To do so required lots of data audits. When done with one, we did more. The moment we thought we had twisted and pulled the data in every conceivable manner, we did it again. The bottom-line was this: errors here could have wreaked havoc with a young data system. Step 5: Secure and Control Now that the data were scrubbed, organized, and coded to the pre-defined set of rules, it was of paramount importance for us to work to keep the data pristine. The most obvious way to achieve this was by developing a highly structured interface that polices future input to the point where it is impossible for the data to become corrupted. To offer the broadest possible availability of access (anyone, anywhere) we chose to use the internally built, web-based platform depicted below.  Administrator Document Locator  Administrator Document Settings |
|||||||||||||||||
|
|
||||||||||||||||
|
|
|||||||||||||||||
|
Washington University / School of Law / Campus Box 1120 / St. Louis MO 63130 cerl@law.wustl.edu |
