Collect, process, de-duplicate, and index documents

Apply filters (date, custodian, file types) to narrow the universe

Lawyers code an initial sample (relevant, non-relevant, privileged)

Ensure representative coverage of issues and document types

Algorithm learns from coded examples

Ranks remaining documents by likely relevance

Review high-scoring documents first

Feed additional coded decisions back into the model to improve performance

Ongoing training as reviewers code documents

Typically yields faster stabilization and better early recall

Well-suited to large and evolving datasets

Train in discrete rounds (train → score → review → retrain)

Easier to explain procedurally but may require more tuning

Can be slower to reach strong performance on complex matters

Discuss under FRCP Rule 26(f) (or relevant local rules)

Define scope: responsiveness criteria, custodians, date ranges, sources

Training approach (CAL vs batch), ranking thresholds, stopping rules

Treatment of privilege and confidentiality

Handling of “Not Relevant” and “Uncertain” categories

Inform opposing counsel and/or the court when appropriate

Describe high-level approach and tools used

Explain how training samples were selected

Address disputes without necessarily disclosing privileged work product

Precision: proportion of reviewed-as-relevant that are truly relevant

Recall: proportion of all relevant documents successfully identified

Random sample of unreviewed/low-scored documents to estimate missed relevant docs

Statistical confidence levels and error margins for defensibility

Spot checks, reviewer consistency checks, and model performance monitoring

Da Silva Moore v. Publicis Groupe

Rio Tinto v. Vale

Reasonableness, proportionality, cooperation, and defensible validation

Reduced reviewer hours and faster time-to-production

Focus on high-value documents first

Better scalability for large data volumes

Upfront planning and expertise required

Need for clear protocols and statistical validation

Relativity

Everlaw

Logikcull

TAR workflow support (CAL, analytics, reporting)

Sampling/validation tools and audit trails

Security, privacy, and compliance features

Integration with processing, hosting, and production workflows

Cost model, support, and expert services availability

ABA Model Rule 1.1 (technology competence) and relevant state analogs

Reasonable oversight of vendors and reviewers

Protect confidentiality (Rule 1.6) and privilege safeguards

Maintain documentation of protocol decisions, training steps, and validation results