PROBABILITY THEORY:PROBABILITY AND PROBABILITY DISTRIBUTIONS

PROBABILITY AND PROBABILITY DISTRIBUTIONS

Returning to the example of the demand for a product during a month, note that the actual demand is not a constant; instead, it can be expected to exhibit some “variation.” In particular, this variation can be described by introducing the concept of probability defined over events in the sample space. For example, let E be the event {w = 0, w = 1, w = 2, . . . , w = 10}. Then intuitively one can speak of P{E}, where P{E} is referred to as the probability of having a demand of 10 or less units. Note that P{E} can be thought of as a numerical value associated with the event E. If P{E} is known for all events E in the sample space, then some “information” is available about the demand that can be ex- pected to occur. Usually these numerical values are difficult to obtain, but nevertheless their existence can be postulated. To define the concept of probability rigorously is be- yond the scope of this text. However, for most purposes it is sufficient to postulate the ex- istence of numerical values P{E} associated with events E in the sample space. The value 1A countably infinite set of values is a set whose elements can be put into one-to-one correspondence with the set of positive integers. The set of odd integers is countably infinite. The 1 can be paired with 1, 3 with 2, 5 with 3, . . . , 2n - 1 with n. The set of all real numbers between 0 and 1/2 is not countably infinite because there are too many numbers in the interval to pair with the integers.

P{E} is called the probability of the occurrence of the event E. Furthermore, it will be assumed that P{E} satisfies the following reasonable properties:

1. 0 ::: P{E} ::: 1. This implies that the probability of an event is always nonnegative and can never exceed 1.

2. If E0 is an event that cannot occur (a null event) in the sample space, then P{E0} = 0.

Let E0 denote the event of obtaining a demand of -7 units. Then P{E0} = 0.

3. P{n} = 1. If the event consists of obtaining a demand between 0 and N, that is, the entire sample space, the probability of having some demand between 0 and N is certain.

4. If E1 and E2 are disjoint(mutually exclusive) events in n, then P{E1 U E2} = P{E1} + P{E2}. Thus, if E1 is the event of 0 or 1, and E2 is the event of a demand of 4 or 5, then the probability of having a demand of 0, 1, 4, or 5, that is, {E1 U E2}, is given by P{E1} + P{E2}.

Although these properties are rather formal, they do conform to one’s intuitive notion about probability. Nevertheless, these properties cannot be used to obtain values for P{E}. Occasionally the determination of exact values, or at least approximate values, is desirable. Approximate values, together with an interpretation of probability, can be obtained through a frequency interpretation of probability. This may be stated precisely as follows. Denote by n the number of times an experiment is performed and by m the number of successful occurrences of the event E in the n trials. Then P{E} can be interpreted as assuming the limit exists for such a phenomenon. The ratio m/n can be used to approximate P{E}. Furthermore, m/n satisfies the properties required of probabilities; that is,

1. 0 ::: m/n ::: 1.

2. 0/n = 0. (If the event E cannot occur, then m = 0.)

3. n/n = 1. (If the event E must occur every time the experiment is performed, then m = n.)

4. (m1 + m2)/n = m1/n + m2/n if E1 and E2 are disjoint events. (If the event E1 occurs

m1 times in the n trials and the event E2 occurs m2 times in the n trials, and E1 and E2 are disjoint, then the total number of successful occurrences of the event E1 or E2 is just m1 + m2.)

Since these properties are true for a finite n, it is reasonable to expect them to be true for The trouble with the frequency interpretation as a definition of probability is that it is not possible to actually determine the probability of an event E because the question “How large must n be?” cannot be answered. Furthermore, such a definition does not permit a logical development of the theory of probability. However, a rigorous definition of prob- ability, or finding methods for determining exact probabilities of events, is not of prime importance here.

The existence of probabilities, defined over events E in the sample space, has been described, and the concept of a random variable has been introduced. Finding the relation between probabilities associated with events in the sample space and “probabilities” as- sociated with random variables is a topic of considerable interest.

Associated with every random variable is a cumulative distribution function (CDF). To define a CDF it is necessary to introduce some additional notation. Define the symbol Eb = {w|X(w) ::: b} (or equivalently, {X ::: b}) as the set of outcomes w in the sample space forming the event Eb such that the random variable X takes on values less than or

equal to b.† Then P{EX} is just the probability of this event. Note that this probability is well defined because EX is an event in the sample space, and this event depends upon both the random variable that is of interest and the value of b chosen. For example, suppose the ex- periment that measures the demand for a product during a month is performed. Let N = 99, and assume that the events {0}, {1}, {2}, . . . , {99} each has probability equal to 1/100;

Another random variable associated with this experiment is X1, the time of the arrival of the first customer on the first day. Thus, FX (b) = P{X1 ::: b}, which can be obtained simply if probabilities of events over the sample space are given.

There is a simple frequency interpretation for the cumulative distribution function of a random variable. Suppose an experiment is repeated n times, and the random variable X is observed each time. Denote by x1, x2, . . . , xn the outcomes of these n trials. Order these outcomes, letting x(1) be the smallest observation, x(2) the second smallest, . . . , x(n) the largest. Plot the following step function Fn(x):

In most problems encountered in practice, one is not concerned with events in the sample space and their associated probabilities. Instead, interest is focused on random variables and their associated cumulative distribution functions. Generally, a random variable (or random variables) is chosen, and some assumption is made about the form of the CDF or about the random variable. For example, the random variable X1, the time of the first arrival on the first day, may be of interest, and an assumption may be made that the form of its CDF is exponential. Similarly, the same assumption about X2, the time of the first arrival on the second day, may also be made. If these assumptions are valid, then the CDF of the random variable X = (X1 + X2)/2 can be derived. Of course, these assumptions about the form of the CDF are not arbitrary and really imply assumptions about probabilities associated with events in the sample space. Hopefully, they can be substantiated by either empirical evidence or theoretical considerations.