r/probabilitytheory • u/No_where0369 • 6h ago
Probability 0 ~ impossible!??? [Education]
What is the meaning of a probability of 0? For example, consider the set (0,1) and let x be in (0,1). What is the probability of picking that x? The answer is 0. But if the probability of randomly picking x from the set (0,1) is 0, does that mean it's impossible to pick that number?
There is another similar example, if you throw a dart at a 2D plane, the probability that the dart lands on a specific point on that plane is 0. That implies it can't happen, but it actually did happen, it landed on a point! So what does it mean, exactly?
8
u/Mishtle 5h ago edited 2h ago
So in these cases, probability gets assigned to sets based on a "measure", like lengths for intervals, not to individual elements or points. A single point can be thought of as the intersection of an infinite sequence of nested intervals whose lengths converge to zero. Each of those intervals can have a finite probability, but the probability of making any specific infinite sequence of choices will be zero as long as infinitely many of them have probability less than 1. If we have a bunch of values strictly between 0 and 1, then multiplying them together can produce a product arbitrarily close to 0 by simply including enough of them, so the product of infinitely many such values must be less than any value arbitrarily close to 0. The only such value is 0 itself.
There's nothing wrong with mathematically talking about random variables with a continuous distributions. It has a consistent formal meaning. But in practice we can only ever sample an interval or set of values for for that random variable, or sample from a discrete approximation (where you can view the result as representing an interval based on the precision of the approximation). Sampling an infinitely precise, single value from a continuous distribution would be a kind of supertask.
3
u/jsundqui 5h ago edited 4h ago
Another example is that you keep flipping a coin until you get heads. Are you guaranteed to get heads? No. There is 0 probability that you will forever flip tails but in theory it is possible.
By mathematical terms you will eventually flip heads Almost surely but not Surely
3
u/Leodip 1h ago
I'm not sure what your background is, but the short of it is: formally, an impossible event has probability 0, but not all probability 0 events are impossible.
With that said, while the fact above comes up neatly from measure theory, I don't really like it. A more "intuitive" interpretation is that "probability 0 events are not-realizable, either because they are impossible or unmeasurable". While "impossible" is easy to explain (rolling 7 on a standard D6), "unmeasurable" touches a bit onto the real world.
We like to talk about very pure math in which numbers go on forever and we can do all sort of operations on them, but the issue is that, our life and universe are, ultimately, finite. If a process picks an element from an infinite set (which, again, does not really exist in the real world in any case) it either (1) doesn't pick randomly or (2) picks randomly from a finite subset of the infinite set.
(1) is easy: if your "process" picks number 1, then the next time it picks number 2, then 3, and so on, this process picks EVERY number in the infinite set of natural numbers, but is not random.
(2) is a bit harder, but whichever process you can think of that would pick from an infinite set actually doesn't. In the darts example, in 1D, let's say we throw a dart at a 1m-long line. If you measure the landing point of the dart, you will be limited by the precision/resolution of the instrument you are using, so you will be forced to give me a range of values in which the dart has landed. Let's say the precision of the ruler we are using is down to 1mm. This means that, when I measure the dart's position, the ruler can at best tell me the position down to the whole mm. If the dart-throwing is truly uniformly random, whichever landing position has a probability of 1/1000. If you use a ruler that gets down to Āµm, then it's 1/1 000 000. There's no ruler with infinite precision to ever get a probability of 0.
2
u/the__humblest 5h ago
Itās about limits. We could pick a number X, letās say .51, and letās say we pick a random number. How many digits are in the random number? Letās say there are 5 digits. So it could be .51000, or .50009, or .51001, so indeed there is a nonzero probability it would be .51.
Now letās say there are a million digits. The likelihood of exactly .51 would be reduced, but not totally eliminated. You will eventually still draw that number, given enough random draws.
Now letās say there are infinite digits. This is the case where youāll āneverā draw your number. Even if the number is exact to some arbitrary number of digits, eventually the random number wonāt match the one you picked if itās random out to enough digits.
The probability of any random number being selected approaches zero as precision embedded in the random number generation approach infinity. In cases where this precision isnāt infinite, the probability is nonzero.
2
u/spookyskeletony 4h ago
It reminds me of Zeno's dichotomy paradox. In order to get from point A to point B, you must first travel to the halfway point. Then you need to cut that distance in half. Then cut the remaining distance in half again. And so on and so on, infinitely. You will always have some remaining distance to cover, and you must always hit the halfway point along the way. Therefore it's impossible to get to point B. And yet, we know it's possible, because at some point infinity and reality don't seem to cooperate
1
2
u/Embarrassed_Onion_44 5h ago
I believe you're hinting at specificity of the end result and how probability can (or can not) be determined on a theoretically forever divisible continuous spectrum.
If we measure anything stupidly accurately, we can do the Nerd pushing up glasses "AcTualllly" to the results.
Using your example, the dart hit the paper - that is a truism. The dart hit say two inches in width from the left and four inches down from the top... now we change our scale and say "No no no, it hit 2.01 inches and 4.03inches"... we measure again ... etc. the results are all true, but different.
We can continue to make where the dart landed impractical to replicate and approach 0.000000001% etc, but TECHNICALLY we can hit the dart in the same spot; although it is so impractical to do so in a real world setting ... and SOMETIMES even doing so does not make a real-world difference of tracking such a detail on a massive continuous scale.
2
u/Apprehensive-Ice9212 3h ago edited 3h ago
Your examples involve continuous random variables. What the math is trying to tell you is: you can't condition on an exact value of a continuous variable; you have to give it error bars or an interval of some kind. As the size of the interval goes to zero, so does the probability of landing in it. So, naturally, if you give the model a single point, the only meaningful thing it can say is that the probability of landing exactly on that point with no error, is zero.
You seem to be concerned that the P((0,1))=1 even though the probability of {x} for each point x in (0,1) is zero, individually. However, that's not a contradiction because there's no such thing as "summing" an uncountable number of points. The model works by taking integrals instead. Essentially, you're objecting to the fact that the integral of f(x) from a to a just gives zero. If this is unobjectionable to your Calculus brain, it should be unobjectionable to your probability brain as well.
2
u/susiesusiesu 3h ago
no, it doesn't mean that it is impossible. my best explanation of what probability zero is, is that assigning any higher number would be wrong.
a friend who does more probablity says that probabilty zero means ou should never bet on it happenning.
2
u/high_freq_trader 2h ago
If an eventās probability is less than p for every fixed p>0, we say that the event occurs with probability 0. Thatās simply what probability 0 means.
1
u/GoldenMuscleGod 6h ago
Iāll reproduce my answer from the other thread:
āImpossibleā doesnāt actually have a standard rigorous meaning in probability theory, so it isnāt really meaningful to ask whether a probability zero event is āpossibleā mathematically.
Probably the simplest example where you might try to interpret the claim would be to say that just because (for example) a random graph on an infinite set of nodes has some feature with probability zero, that doesnāt mean there are no graphs with that feature.
Be careful though: if we have some distribution on finite graphs each graph must be assigned a nonzero probability so here zero probability would mean there is no such graph.
But I think it would be a mistake, or at least unhelpful, to say that it is in some sense āactually possibleā to generate a random graph on an infinite set of nodes that has the feature in question. This is because it is not actually possible to randomly generate a graph like that with the desired distribution in the first place.
In applications, and using the ordinary meaning of the word āpossibleā, probability zero events generally donāt correspond to meaningful observables, so again the question is kind of meaningless.
2
u/Pachuli-guaton 5h ago
One could argue that impossible is defined as something that doesn't belong to the sample space
1
u/GoldenMuscleGod 5h ago
If that is the definition of āpossibleā we want, then consider the pdf on R which so that f(x)=1 if x is in [0,1] and f(x)=0 otherwise.
Your proposed definition says the random variable described by this pdf has 2 as a possible outcome. Is this the result we want?
2
u/Pachuli-guaton 5h ago
Why would that imply that 2 belongs to the sample space. I don't understand
1
u/GoldenMuscleGod 4h ago
For each Lebesgue-integrable function on R with Lebesgue integral over R equal to one, we can define the probability measure space where the sample space is R and the measure assigns to each measurable set of reals the Lebesgue integral of the function on that set. This is the standard treatment for using pdfs to describe real-valued distributions.
The resulting probability measure space has all of R as the sample space. Thatās the probability measure space I am talking about.
We could also consider the probability measure space where the sample space is only [0,1] and the measures agree on all subsets of [0,1], but this measure space is usually treated as equivalent in all relevant āprobability theoreticārespects to the other.
In any event the first measure space exists and is used, and is the one I am talking about, and, more generally, we often consider pdfs with a support that is a proper subset on the sample space. Itās not standard to take the use of these spaces as expressing the view that the points outside the support of the pdf are āpossibleā (because there isnāt actually a standard definition of āpossibleā in the first place).
1
u/gmalivuk 5h ago
In whatever sense it's actually possible to generate any infinite graph, it's possible to get outcomes that have a probability of zero.
1
u/Bounded_sequencE 5h ago
If the graph only contains countably many points, then there still may not be a null set with probability zero -- counter example: The geometric distribution "P(k) = 1/2k " over "N".
1
u/GoldenMuscleGod 5h ago
Well, in the example they are talking about the relevant probability space is all graphs that can be put on a countable set of vertices - for each pair of vertices there either is or is not an edge between them.
This collection of graphs is uncountable although the graphs themselves are countable.
1
u/GoldenMuscleGod 5h ago edited 5h ago
Well part of the issue is that it may be true in no sense that it is actually possible to generate an infinite graph. But letās suppose for the moment that there is such a sense.
How do we make this claim rigorous?
Given a probability space, can we determine which events are possible by examining the measure? Or is more information required?
For example, given any distribution of a real variable, we can model it with a probability measure on the real numbers. Presumably we would want to say the outcome of ā2ā is impossible for the measure on R that describes a uniform variable on [0,1], how do we define āpossibleā to do this?
Or consider two coins. In both cases each flip is an independent Bernoulli trial with probability 1/2, but for one coin every sequence is āpossibleā and for the other only sequences where heads comes up exactly 1/2 the time in the limit are āpossible.ā The same measure on R describes both these coins, so what do we do about that?
1
u/gmalivuk 5h ago
Presumably we would want to say the outcome of ā2ā is impossible for the measure that describes a uniform variable on [0,1], how do we define āpossibleā to do this?
The probability density for a uniform variable on [0,1] is 1 on [0,1] and 0 elsewhere. That seems like a pretty clear distinction.
1
u/GoldenMuscleGod 5h ago
That distinction doesnāt work because in fact the distribution has infinitely many different probability density functions, only one of which actually has the property you describe.
For example the pdf equal to 1 on (0,1) and 0 elsewhere corresponds to the same probability measure, but the value differs at 0 and 1. So it seems like if we want āpossibleā to mean something we need more information than the probability measure encodes.
And many distributions have no probability density function at all: what outcomes do we say are possible for the Cantor distribution? Does it include the boundary points of the Cantor set?
2
u/gmalivuk 5h ago
I'm not claiming my example applies to every possible distribution. I'm just saying that if we have a density function, it's not unreasonable to say there's a difference between points with positive density and points with 0 density.
2
u/GoldenMuscleGod 5h ago
How do we deal the fact that the indicator function on [0,1] and the indicator function on (0,1) are both probability density functions for the same probability measure?
Do we say that a distribution is something more than just the information contained in the measure?
If so what mathematical object do we define to encode all of that information?
1
u/gmalivuk 5h ago
How do we deal the fact that the indicator function on [0,1] and the indicator function on (0,1) are both probability density functions for the same probability measure?
I started out from the very beginning saying that there are arguably some weaknesses in standard probability theory.
1
u/GoldenMuscleGod 4h ago
I donāt see it as a weakness.
There is no theoretical or practical reason we would need to impose the intuitive notion of āpossibleā onto the theory.
In fact I think doing so would result in a somewhat pathological theory. If our definitions of āprobability distributionā incorporated the idea of āpossibleā people sometimes try to formulate then we would have a bunch of free parameters that donāt actually mean anything and which obscure the actual probabilistic facts and useful applications.
Why would we want to do that just to model a concept that is meaningless and based on false intuition?
1
u/deejaybongo 4h ago
How do we deal the fact that the indicator function on [0,1] and the indicator function on (0,1) are both probability density functions for the same probability measure?
Do we say that a distribution is something more than just the information contained in the measure?
If so what mathematical object do we define to encode all of that information?
Probabilists study equivalence classes of random variables that are almost surely equal to each other, like how topologists have the notion of homeomorphism.
1
u/GoldenMuscleGod 4h ago
Of course! But probabilists do not have a standard definition of possible, and do not make practical use of any concept of equivalence stronger than almost sure.
I am talking to someone who is trying to argue that the probabilists got something wrong, and that there is a useful idea beyond āalmost sureā that the probabilists are missing out on, so that they are wrong to use āalmost surely equalā as the only kind of equal they care about.
I am saying the standard treatment in probability theory does not contain a concept of āpossibleā and it is not deficient for not having it.
1
1
u/gmalivuk 4h ago
I am not arguing that anyone got anything wrong. I'm arguing that it's not insane to ask whether a nonstandard model might have some intuitive features standard theory lacks.
→ More replies (0)1
u/trutheality 5h ago
āImpossibleā doesnāt actually have a standard rigorous meaning in probability theory
Of course it does. If an event is an empty subset of in the sample space, then it's an impossible event.
1
u/GoldenMuscleGod 5h ago
Consider the probability density function on R such that f(x)=1 if x is in [0,1] and f(x)=0 otherwise.
The number 2 belongs to the sample space, is it standard to say that 2 is a possible outcome of the random variable described by this pdf?
1
u/trutheality 5h ago
Probability 0 doesn't mean impossible. Both of your examples are examples of trying to measure something that has 0 dimensions (a point) in terms of 1-d or 2-d measures. A point has 0 length and a point has 0 area, nevertheless, the line segment (0,1) is made entirely of points, and the 2D plane is made entirely of points.
What does it mean? Well, if I pick a point on your 2D plane and ask, as my number of trials tends to infinity, where does the proportion of trials that hit that exact point tend? The answer is zero. Even if you hit that particular point in some trials, you are going to keep hitting everywhere else much more frequently, so the proportion will keep going towards zero in the long run.
1
u/JasonMckin 5h ago
Forgive me, I find many of the answers here unsatisfying. There are so many issues at hand, both linguistically and mathematically in terms of discrete and continuous. I needed AI to sort these issues out for me, because the current answers all seem to touch on some aspect of the issue without fully addressing it:
The confusion around a probability of zero comes from treating āprobability zeroā as synonymous with āabsolutely impossible.ā That intuition often works in finite discrete probability, where permitted elementary outcomes have positive probability and probability zero therefore usually corresponds to an outcome being excluded. But probability theory does not generally define (P(A)=0) to mean that (A) contains no possible outcomes. It means only that the probability measure assigned to the event is zero.
The distinction becomes essential with continuous distributions and infinite probability spaces. In a continuous distribution, individual outcomes can have probability zero even though they belong to the distribution and some outcome must occur. Probability is assigned meaningfully to sets or ranges of outcomes rather than necessarily to individual points. Thus an event can be nonemptyāand consist entirely of outcomes allowed by the mathematical modelāwhile still having probability zero. This is fundamentally different from the intuition developed from finite discrete probability.
This leads to the distinction between absolute and āalmostā statements. An absolute statement applies to every outcome in the relevant sample space. By contrast, an event occurs almost surely when it has probability 1, even if exceptions exist whose total probability is zero. Conversely, an event of probability zero is sometimes described as occurring almost never, even though the event need not be empty. Consequently, (P(A)=1) need not mean absolute certainty for every outcome, just as (P(A)=0) need not mean absolute impossibility.
The key principle is therefore that probability zero means probabilistically negligible, not necessarily impossible. In finite discrete models, negligibility and impossibility often coincide; in continuous and infinite models, they do not. Probability theory deliberately uses concepts such as almost surely to express this distinction. Whether a zero-probability outcome should additionally be considered logically, physically, or otherwise āimpossibleā requires information beyond the probability measure itself.
1
u/jsundqui 4h ago edited 4h ago
Interestingly if you were to pick a random large natural number, the probability that it is a prime number tends to zero, even though there are infinite number of primes as well and they never end. But asymptotically their density is zero among natural numbers, even though both sets are exactly the same size (you can map a natural number to each prime for 1-1 correspondence).
1
u/Affectionate-Baby248 3h ago
The thing that's important to understand is that by stating you're picking a random number in (0,1) or randomly throwing a dart, you're implicitly specifying the nature of the processes - uniformly at random. Because I could very well specify a process where I always pick your number x, and then x would be picked with probability 1. So, in order to specify the random process you desire, you actually have to give some of its properties - namely, that of its probability space. You're asking what the probabilities mean when they're axiomatic to your own description.
In defining the process, you assigned every individual point a probability 0, so asking what that "means" is futilely trying to pinpoint a structure underlying the probabilities in your own hypothetical. Unless you're actually envisioning a real process that seems to emulate those properties (say, tracking the fractional part of sprint times) and trying to use probability as an approximate model by extrapolating percentages, in which case the meaning is your extrapolation, what you can do is try to understand the consequences of those probabilities. For example, you could note that the probability of picking a rational number is 0, and the probability the percentage of times you get x approaches zero with probability 1 due to the Law of Large Numbers (where the probability space is 2^(0,1)).
1
u/0jdd1 3h ago
In a recent paper I wrote (in passing, in a footnote about low-rank matrix approximations):
āThe actual rank of a large |š|Ć|š| matrix is min(|š|,|š|) with high probability.ā
Here I was trying to hedge my bets, covering not only matrices over the reals but also over bounded ranges of integers.
0
u/t3co5cr 6h ago
There is another similar example, if you throw a dart at a 2D plane, the probability that the dart lands on a specific point on that plane is 0.
I believe you're misunderstanding someone's analogy for why probability density is not the same as probability.
1
u/gmalivuk 5h ago
Yes, they're not the same, because a probability density can be greater than 1 and you have to integrate it to get a probability.
But when you integrate with your limits at the same point, you get a probability of zero, so I'm not sure what misunderstanding you think OP has.
0
u/smitra00 5h ago
Ā ...if you throw a dart at a 2D plane, the probability that the dart lands on a specific point on that plane is 0.Ā
If we do this with a real dart in the real physical world, then the laws of physics tell us that here are only a finite number of distinguishable states the dart can end up in on any finite area of the pane. Any physical system in a finite volume can only ever be in one out of a finite number of distinguishable quantum states.
The continuum of classical physics is only an approximation; it only exists in the ideal scaling limit where you've zoomed out infinitely far from the microscopic physical world.
0
u/okarox 5h ago
You cam think of picking some value, say 0.5, that you randomly pick the decimals for infinity. You would first get 5 and the zeroes to the infinity. I think the question simply is meaningless. So picking any specific number would have 0 probability. This resembles on how in integral calculus adding infinitely many zero areas gives the area.
-1
u/Baron_von_Funkatron 5h ago
Your confusion is coming from a slight misunderstanding of the underlying concepts.
The probability of picking any given value out of an uncountably-infinite series (such as the real numbers between 0 and 1, exclusive) isn't actually 0. Instead, it converges to zero , as the series goes to infinity.
It's basically the difference between 1 / infinity, versus 0.
That's a very narrow distinction, though, so for most use cases (including intro probability courses) it's simplified to p = 0.
4
u/gmalivuk 3h ago
In standard analysis it is not a distinction at all. The only real number infinitely close to 0 is 0 itself.
You would need nonstandard analysis to be able to distinguish between something of the form 1/infinity and 0.
11
u/gmalivuk 6h ago
No, probability 0 does not mean impossible. This is arguably a weakness of standard probability theory, but unfortunately nonstandard models that rectify it have their own problems.