Utilizing Search Expertise to Reduce the Costs and Risks of Document Review

Practical Guidance for Legal Teams

Faced with ever-growing volumes of data subject to review in litigation and investigations, effective keyword search strategies are becoming increasingly critical to manage cost, timelines and risk in e-discovery. A particularly good example is in the culling phase, during which legal teams strive to eliminate as much irrelevant data as possible without sacrificing relevant data.

Legal teams now have access to a wide array of document analysis technologies that enable them to execute advanced culling strategies based on custodians, dates, file types and keyword criteria. All of these strategies are designed to reduce hosting and review costs as much as possible by winnowing the population to just those potentially relevant documents that actually merit review.

Unfortunately, the keyword search component of culling remains a source of unnecessary cost and unforeseen risk. Untested search terms frequently lead to simultaneous over-inclusiveness and under-inclusiveness, resulting in more time and money spent reviewing off-topic data and a heightened risk of missing critical information.

Errors in the construction of keyword searches are common. It is challenging to obtain a solid understanding of the linguistic content of a review population during the earliest stages of a case, and attorneys are almost invariably working with tight deadlines that leave them with little time to invest in keyword development. And yet, it is at this critical moment in the process when investing intelligently in the creation of effective keywords may be most impactful.

Evolving Best Practices in Keyword Search

Courts, aware of the risks involved in keyword search, have grown increasingly vigilant regarding the use of search terms in e-discovery. A number of well-known cases have brought these issues to the fore, including Judge Facciola’s United States v. O’Keefe,  Judge Grimm’s Victor Stanley, Inc. v. Creative Pipe, Inc., Judge Peck’s William A. Gross Construction Assocs., Inc. v. American Manufacturers Insurance Co. and, most recently, Judge Scheindlin’s Pension Comm. of the U. of Montreal Pension Plan v. Bank of America Sec.^[1] These hallmark opinions call for careful thought, quality control, testing and use of qualified professionals in designing search terms.

Guidance from the courts and pressure from corporations to control e-discovery costs are driving legal teams to adopt new best practices for search. Increasingly, attorneys are bringing specialized technical experts, including linguists and statisticians, into the process early on to develop, rigorously test and thoroughly document their search practices.

Case teams are learning that by introducing statistical sampling and testing along with iteration and linguistic analysis into the keyword development process – and by investing a small amount of time upfront in a targeted analysis of keyword performance – the risk of missing important information can be reduced significantly. More effective culling also lowers hosting costs and minimizes the time spent by attorneys reviewing irrelevant data. Just as importantly, statistical data and first-hand knowledge of the quality of a document collection can lead to a better outcome in Rule 26(f) negotiations, and enable a more expedient and informative path to production.

Key Questions to Ask Your Search Experts

While some law firms and corporations have opted to train junior attorneys and litigation support personnel in search, others are partnering with specialized e-discovery experts to augment their teams’ skill sets. Regardless of the approach, below are some key questions to consider when leveraging specialized experts for search design:

1. What is the experience of the experts? Taking a crash-course in statistics or linguistics, or having a history of running searches, may not qualify someone as an expert. Look for a proven track record in applying linguistics and statistics to e-discovery projects; these experts may be called upon to testify on the search development process and the validity of the results.

2. Is the process principled, iterative and well-articulated? Experts should be able to design comprehensive search terms, test and refine them based on sample review, and measure and report on final results. The process should incorporate robust mechanisms for course-correction based on feedback from attorney-reviewed samples.

3. How are search term results validated? Statistically valid measurements should be utilized to demonstrate the reliability and quality of the final results of the search effort. These quantitative measures, along with attorneys’ qualitative assessments of sampled results, provide insights that enhance the defensibility of the culling process and strengthen attorneys’ ability to negotiate search terms with opposing counsel.

4. How robust is the audit trail? Experts should provide detailed reporting and documentation for the search term culling process, including a comprehensive enumeration of which search terms were used and why, recording all key inputs, decisions and results.

5. Are legal teams free to focus on case strategy? While seemingly obvious, experts should drive, manage and document a process that fits a legal team’s workflow and timeline, freeing attorneys to concentrate on case strategy rather than keyword logistics.

Combining legal and technical expertise with sound scientific search methodology promises to yield more cost-effective and defensible results, in culling and other phases of the e-discovery process. While it may require litigants to engage in more advanced planning and project design early in the process, adoption of search term best practices will bring tangible long-term improvements to the quality, efficiency and cost-effectiveness of document review.

_{[1] United States v. O’Keefe, 537 F. Supp. 2d 14 (D.D.C. 2008); Victor Stanley, Inc. v. Creative Pipe, Inc., 2008 WL 2221841(D. Md. May 29, 2008); William A. Gross Construction Assocs., Inc. v. American Manufacturers Mutual Insurance Co, F.R.D. 134, 135 (S.D.N.Y. 2009); The Pension Committee of the University of Montreal Pension Plan, et al. v. Banc of America Securities LLC, et al., No. 05 Civ. 9016 (SAS), 2010 WL 184312 (S.D.N.Y. Jan. 15, 2010.}