eDiscovery Daily Blog

• December 2, 2011

 

OK, I admit I stole that line from an old Ford commercial… 😉

Yesterday, we talked about addressing parameters of production up front to ensure that those requirements make sense and avoid foreseeable production problems well before the production step.  Today, we will talk about quality control (QC) mechanisms to make sure that the production is complete and accurate.

Quality Control Checks

There are a number of checks that can and should be performed on the production set, prior to producing it to the requesting party.  Here are some examples:

• File Counts: The most obvious check you can perform is to ensure that the count of files matches the count of documents or pages you have identified to be produced.  However, depending on the production, there may be multiple file counts to check:
  • Image Files: If you have agreed with opposing counsel to produce images for all documents, then there will be a count of images to confirm.  If you’re producing multi-page image files (typically, PDF or TIFF), the count of images should match the count of documents being produced.  If you’re producing single-page image files (usually TIFF), then the count should match the number of pages being produced.
  • Text Files: When producing image files, you may also be producing searchable text files.  Again, the count should match either the documents (multi-page text files) or pages (single-page text files) with one possible exception.  If a document or page has no searchable text, are you still producing an empty file for those?  If not, you will need to be aware of how many of those instances there are and adjust the count accordingly to verify for QC purposes.
  • Native Files: Native files (if produced) are typically at the document level, so you would want to confirm that one exists for each document being produced.
  • Subset Counts: If the documents are being produced in a certain organized manner (e.g., a folder for each custodian), it’s a good idea to identify subset counts at those levels and verify those counts as well.  Not only does this provide an extra level of count verification, but it helps to find the problem more quickly if the overall count is off.
  • Verify Counts on Final Production Media: If you’re verifying counts of the production set before copying it to the media (which is common when burning files to CD or DVD), you will need to verify those counts again after copying to ensure that all files made it to the final media.

• Sampling of Results: Unless the production is relatively small, it may be impractical to open every last file to be produced to confirm that it is correct.  If so, employ accepted statistical sampling procedures (such as those described here and here for searching) to identify an appropriate sample size and randomly select that sample to open and confirm that the correct files were selected, HASH values of produced native files match the original source versions of those files, images are clear and text files contain the correct text.
• Redacted Files: If any redacted files are being produced, each of these (not just a sample subset) should be reviewed to confirm that redactions of privileged or confidential information made it to the produced file.  Many review platforms overlay redactions which have to be burned into the images at production time, so it’s easy for mistakes in the process to cause those redactions to be left out or burned in at the wrong location.
• Inclusion of Logs: Depending on agreed upon parameters, the production may include log files such as:
  • Production Log: Listing of all files being produced, with an agreed upon list of metadata fields to identify those files.
  • Privilege Log: Listing of responsive files not being produced because of privilege (and possibly confidentiality as well).  This listing often identifies the privilege being asserted for each file in the privilege log.
  • Exception Log: Listing of files that could not be produced because of a problem with the file.  Examples of types of exception files are included here.

Each production will have different parameters, so the QC requirements will differ, so there are examples, but not necessarily a comprehensive list of all potential QC checks to perform.

So, what do you think?  Can you think of other appropriate QC checks to perform on production sets?  If so, please share them!  As well as any comments you might have or if you’d like to know more about a particular topic.