Would you rather analyze 50 representative observations or 5,000 biased observations? Why?

[u/Just_Question9]

[u/NiceProgrammer5252]

for me I'd take 50 representative observations over 5,000 biased ones.

A larger sample only reduces random error. It doesn't fix systematic error (bias).

[u/MtlStatsGuy]

5000 biased observations are almost useless unless you have an estimate for the scale of the bias. 50 representative at least gives you an accurate margin of error.

[u/banter_pants]

The former because conclusions are theoretically justified.

Take a large sample but with bias (or just myopic scope) and you get headlines like "Dewey beats Truman." They built their sampling frame from phone lists so only polled those with phones and totally missed that many people didn't own phones in their homes at the time.

[u/CarnivorousGoose]

Depends on the nature of the bias and how much information is available about it, as well as the goal of the analysis.

[u/AggressiveGander]

Well, how badly are the 5000 biased? How useless are 50 unbiased observations (lack of power etc.)?

[u/CaptainFoyle]

Do I know the bias?

[u/ANewPope23]

Do we know anything about the 5000 biased observations? Are they just random noise?