Managing Litigation for In-House Counsel Breakfast Discussion Group Predictive Coding for E-Discovery: Using Computer Intelligence to Facilitate Document Production Steven Schoenfeld, Esq. May 15, 2012 Trial Partner, Dorsey & Whitney LLP 51 West 52nd Street New York, NY 10019-6119 (212) 415-9341 schoenfeld.steven@dorsey.com www.dorsey.com Call in toll free at 800-536-9136 and use 7-Digit Access Code 415-9341 DISCLAIMER. This presentation is intended for in-house lawyers and is for general information and educational purposes only. Nothing in this presentation either in writing or in any audio or video recording should be relied upon or used as the basis for making decisions without advice of counsel or legal representation. 1
Document Production In Discovery: Obligations and Objectives In A Nutshell Production of relevant (broadly defined) documents. Fed. R. Civ. P. 26(b)(1) Certification of compliance. Fed. R. Civ. P. 26(g) Tempered by the proportionality rule. Fed. R. Civ. P 26(b)(2)(c) Producer objectives reasonable, defensible methodology in compliance with the rules minimize costs Requestor objectives get all of the responsive documents usable format 2
Document Production In Discovery: No Method Is Foolproof Don t let the perfect be the enemy of the good (or good enough). Voltaire There is no perfect way. Documents will be missed Perfection is not required Will the case be about the merits or about the discovery process? How to use technology, and what technology to use, may depend on the circumstances of the particular case. 3
The Present: Linear Human Review With Computer Assistance Computer assistance to narrow and prioritize the data set e.g., key words or boolean (combined words or phrases) Human beings review each document in the narrowed data set one after the other to determine responsiveness and coding. Issues Key words miss documents Dilemma of selecting and using key words too much (e.g., company name and product name are the same) too little (e.g., missing company specific terminology or coded words) Human error and inconsistency Cost, especially as data grows 4
The Potential Future: Computer Intelligence For Document Review 5
The Potential Future: Computer Intelligence For Document Review 6
The Dawn Of The Future: Predictive Coding or Automated Computer Document Review A method of computer-assisted review Use of machine learning technology computer software is trained by people to code and select the relevant/responsive documents without manual review of all documents. 7
First Federal Court Blesses The Use Of Predictive Coding Gender discrimination employment case U.S.M.J. Andrew Peck in the Southern District of New York approves a protocol for the defendant s use of predictive coding. Da Silva Moore v. Publicis Groupe et. al, 2012 WL 607412 (S.D.N.Y. Feb. 24, 2012). U.S.D.J. Andrew Carter approves Judge Peck s decision and rejects plaintiff s objections. 8
Predictive Coding Process Basic Steps A random sampling of documents known as a seed set is identified from the universe of documents (using key words etc.). Senior lawyers with in-depth knowledge of the case review and code the seed set. The seed set educates the computer on what types of documents are relevant or irrelevant (responsive or not to document requests). The computer then takes this knowledge and uses it to predict similar documents in the universe of documents. 9
Predictive Coding Process Basic Steps (cont d) After the computer completes a round of coding, the lawyers review and code a random sample subset of the computer coded documents, including correcting coding errors by the computer. The computer then analyzes and learns from this next set, which will reinforce the computer s sorting of correctly coded documents and revise the computer s sorting of incorrectly coded documents. 10
Predictive Coding Process Basic Steps (cont d) The process repeats with senior lawyer input. After the last round a quality check is done of a random sample subset of the documents coded by the computer as non-responsive to make sure the computer error rate, if any, is in a reasonably acceptable range. 11
Predictive Coding Process Basic Steps (cont d) Does not use linear, human review of all of the documents. But the smaller final production goes through a final human review, particularly for privilege. 12
Predictive Coding Process Basic Steps (cont d) The Da Silva Moore protocol provides for extensive transparency. Plaintiffs were to be given the seed set and other document samplings and defendant s coding determinations, except for privileged documents Plaintiffs had the right to give input in each step of the process including coding. The Da Silva Moore protocol is attached as an exhibit to Judge Peck s opinion. 13
Da Silva Moore - Judge Peck s Reasoning The vast amount of ESI to be reviewed (3 million plus documents from custodians) The need for cost effectiveness and proportionality under Rule 26(b)(2)(C) The transparency of the proposed process The superiority of computer-assisted review compared to the available alternatives (i.e., linear manual review or keyword searches) costs speed accuracy 14
Da Silva Moore - Judge Peck s Reasoning (cont d) The agreement of the parties to the use of predictive coding But it appears that the plaintiffs disagreed, or at least disagreed with the specifics of defendant s proposed protocol 15
Da Silva Moore - Judge Peck s Comments on Plaintiffs Objections Plaintiffs objected to Judge Peck s decision on the basis, in part, that predictive coding would violate: Rule 26(g) by effectively relieving defense counsel of a duty to certify that their client s document production is complete FRE 702 and the Daubert standard for admission of expert evidence (i.e., e-discovery experts) Reliability 16
Da Silva Moore - Judge Peck s Comments on Plaintiffs Objections (cont d) Judge Peck s written opinion is issued two days after the Plaintiffs Rule 72(a) objections to Judge Peck s order are filed with the District Court Judge Peck made some observations about the Plaintiffs objections: Rule 26(g) only required certification that initial disclosures are complete and not that all document production is complete Rule 26(g) incorporates Rule 26(b)(2)(C) s proportionality principle FRE 702 and the Daubert standard were for admissibility of evidence at trial and [were] not applicable to how documents are searched for and found in discovery. Reliability concerns premature 17
Other Predictive Coding Cases: Global Aerospace Inc. v. Landow Aviation in Virginia State Court Global Aerospace Inc., et. al v. Landow Aviation, L.P. dba Dulles Jet Ctr., et. al, Case. No. CL 61040 (Va. Cir. Ct. Apr. 23, 2012) Claims relating to collapse of airport hangers Defendants move for a protective order requesting court approval to use predictive coding to select responsive documents Plaintiffs oppose Court approves 18
Other Predictive Coding Cases: Kleen Prods., v. Packaging Corp. of Am. (cont d) Kleen Prods., LLC, et. al. v. Packaging Corp. of Am. et. al., Case No. 10 C 5711 (N.D. Ill. 2012) Antitrust class action The parties did not agree to use predictive coding. Plaintiffs argued that boolean search string methodology used by defendants would be deficient. Plaintiffs asked the Court to require the defendants to use predictive coding. U.S.M.J. Nolan has not yet issued a decision. 19
Considerations for the Use of Predictive Coding What is the volume of electronically stored information (ESI)? Are other available alternative discovery tools equivalent or superior to predictive coding in terms of cost, speed, accuracy etc.? Is the producing party willing to be transparent about the process being used? Will both parties counsel agree to use predictive coding? 20
About Us Steven R. Schoenfeld, Esq. Steve is a Trial Partner in the New York Office of Dorsey & Whitney LLP. He has been practicing for more than 20 years, and handles a wide variety of civil litigation for his clients, including commercial, intellectual property and bankruptcy litigation. Steve regularly speaks to in-house counsel and bar association groups regarding managing litigation. His full biography is on the Dorsey web site, www.dorsey.com. Dorsey & Whitney LLP is committed to our clients businesses, to client service, and to our clients success. Dorsey is a go-to law firm for businesses large and small. Our partnership began in 1912 in order to meet the needs of a prominent Minneapolis business, and we have become an international firm since then. But our focus on client service hasn't changed. Each day, from 19 offices in the United States, Canada, Europe and Asia, more than 600 Dorsey attorneys and 700 support staff, including paralegals, information technology professionals, case-management specialists and more, team with clients to create solutions built on service, collaboration and a determination to succeed. To learn more, visit www.dorsey.com. Contact: Steven Schoenfeld, Esq. Trial Partner, Dorsey & Whitney LLP 51 West 52nd Street New York, NY 10019-6119 (212) 415-9341 schoenfeld.steven@dorsey.com 21