4 Steps to Designing Effective Survey Samples for Trial

The views expressed in this article are the sole responsibility of the authors and cannot be attributed to Compass Lexecon or any other parties.

Introduction

The Federal Trade Commission's move in August to exclude a defense expert's survey in FTC v. Amazon.com Inc. on the basis of flaws in the survey sample design highlights a critical lesson: Survey evidence in litigation requires more than designing and fielding a questionnaire — it also requires effective survey sample design.[1] 

The FTC argued that the survey's heavy reliance on tech-savvy and attentive respondents failed to capture the diversity of the target population — i.e., Amazon Prime members in the U.S. — undermining its representativeness.[2] 

Although the U.S. District Court for the Western District of Washington did not exclude the survey, it concluded that the "FTC's contrary arguments … go[] to the weight afforded to [the] survey."[3] 

As courts continue to rely on surveys to gain insights into topics involving consumer awareness, attitudes and behavior, flaws in survey sample design can result in the survey's exclusion from admissible evidence or being ascribed less weight. 

This article provides a road map with case examples for four critical aspects of effective survey sample design: first, identifying a relevant target population; second, designing an appropriate sampling strategy; third, ensuring the sample is representative of the target population; and fourth, reducing or eliminating nonresponse bias.

This article was originally published by Law360 here.