MARKOV CHAINS:STOCHASTIC PROCESSES

By -

STOCHASTIC PROCESSES

A stochastic process is defined as an indexed collection of random variables {Xt}, where the index t runs through a given set T. Often T is taken to be the set of non- negative integers, and Xt represents a measurable characteristic of interest at time t. For example, Xt might represent the inventory level of a particular product at the end of week t.

Stochastic processes are of interest for describing the behavior of a system operating over some period of time. A stochastic process often has the following structure.

The current status of the system can fall into any one of M + 1 mutually exclusive categories called states. For notational convenience, these states are labeled 0, 1, . . . , M. The random variable Xt represents the state of the system at time t, so its only possible values are 0, 1, . . . , M. The system is observed at particular points of time, labeled t = 0, 1, 2, . . . . Thus, the stochastic process {Xt} = {X0, X1, X2, . . .} provides a mathematical representation of how the status of the physical system evolves over time.

This kind of process is referred to as being a discrete time stochastic process with a finite state space. Except for Sec. 29.8, this will be the only kind of stochastic process con- sidered in this chapter. (Section 29.8 describes a certain continuous time stochastic process.)

A Weather Example

The weather in the town of Centerville can change rather quickly from day to day. However, the chances of being dry (no rain) tomorrow are somewhat larger if it is dry today than if it rains today. In particular, the probability of being dry tomorrow is 0.8 if it is dry today, but is only 0.6 if it rains today. These probabilities do not change if information about the weather before today is also taken into account.

The evolution of the weather from day to day in Centerville is a stochastic process. Starting on some initial day (labeled as day 0), the weather is observed on each day t, for t = 0, 1, 2, . . . . The state of the system on day t can be either

INTRODUCTION TO OPERATIONS RESEARCH-0706

An Inventory Example

Dave’s Photography Store has the following inventory problem. The store stocks a particular model camera that can be ordered weekly. Let D1, D2, . . . represent the demand for this camera (the number of units that would be sold if the inventory is not depleted) during the first week, second week, . . . , respectively, so the random variable Dt (for t = 1, 2, . . .) is Dt = number of cameras that would be sold in week t if the inventory is not depleted. (This number includes lost sales when the inventory is depleted.)

It is assumed that the Dt are independent and identically distributed random variables hav- ing a Poisson distribution with a mean of 1. Let X0 represent the number of cameras on hand at the outset, X1 the number of cameras on hand at the end of week 1, X2 the number of cameras on hand at the end of week 2, and so on, so the random variable Xt (for t = 0, 1, 2, . . .) is

Xt = number of cameras on hand at the end of week t.

Assume that X0 = 3, so that week 1 begins with three cameras on hand.

{Xt} = {X0, X1, X2, . . .}

is a stochastic process where the random variable Xt represents the state of the system at time t, namely,

State at time t = number of cameras on hand at the end of week t.

As the owner of the store, Dave would like to learn more about how the status of this sto- chastic process evolves over time while using the current ordering policy described below.

At the end of each week t (Saturday night), the store places an order that is delivered in time for the next opening of the store on Monday. The store uses the following order policy:

If Xt = 0, order 3 cameras.

If Xt > 0, do not order any cameras.

Thus, the inventory level fluctuates between a minimum of zero cameras and a maximum of three cameras, so the possible states of the system at time t (the end of week t) are Possible states = 0, 1, 2, or 3 cameras on hand.

Since each random variable Xt (t = 0, 1, 2, . . .) represents the state of the system at the end of week t, its only possible values are 0, 1, 2, or 3. The random variables Xt are dependent and may be evaluated iteratively by the expression

These examples are used for illustrative purposes throughout many of the following sections. Section 29.2 further defines the particular type of stochastic process considered in this chapter.

Comments

Post a Comment

Popular posts from this blog

DUALITY THEORY:THE ESSENCE OF DUALITY THEORY

By -
Image
THE ESSENCE OF DUALITY THEORY Given our standard form for the primal problem at the left (perhaps after conversion from another form), its dual problem has the form shown to the right. Thus, with the primal problem in maximizatio n form, the dual problem is in minimization form instead. Furthermore, the dual problem uses exactly the same pa r amete r s as the primal problem, but in different locations, as summarized below. 1. The coefficients in the objective function of the primal problem are the right-hand sides of the functional constraints in the dual problem. 2. The right-hand sides of the functional constraints in the primal problem are the coef- ficients in the objective function of the dual problem. 3. The coefficients of a variable in the functional constraints of the primal problem are the coefficients in a functional constraint of the dual problem. To highlight the comparison, now look at these same two problems in matrix notation (as introduced at the beginn...
Read more

NETWORK OPTIMIZATION MODELS:THE MINIMUM SPANNING TREE PROBLEM

By -
Image
THE MINIMUM SPANNING TREE PROBLEM The minimum spanning tree problem bears some similarities to the main version of the shortest-path problem presented in the preceding section. In both cases, an undi r ecte d and connected network is being considered, where the given information includes some mea- sure of the positive length (distance, cost, time, etc.) associated with each link. Both problems also involve choosing a set of links that have the shortest total length among all sets of links that satisfy a certain property. For the shortest-path problem, this property is that the chosen links must provide a path between the origin and the destination. For the minimum spanning tree problem, the required property is that the chosen links must provide a path between ea c h pair of nodes. The minimum spanning tree problem can be summarized as follows: 1. You are given the nodes of a network but not the links. Instead, you are given the po- tential links and the positive length for each ...
Read more

INTEGER PROGRAMMING:THE BRANCH-AND-CUT APPROACH TO SOLVING BIP PROBLEMS

By -
TH E BRANCH-AND-CUT APPROACH TO SOLVING BIP PROBLEMS Integer programming has been an especially exciting area of OR since the mid-1980s be- cause of the dramatic progress being made in its solution methodology. Background To place this progress into perspective, consider the historical background. One big break- through had come in the 1960s and early 1970s with the development and refinement of the branch-and-bound approach. But then the state of the art seemed to hit a plateau. Rel- atively small problems (well under 100 variables) could be solved very efficiently, but even a modest increase in problem size might cause an explosion in computation time be- yond feasible limits. Little progress was being made in overcoming this exponential growth in computation time as the problem size was increased. Many important problems arising in practice could not be solved. Then came the next breakthrough in the mid-1980s, with the introduction of the b r an c h- and-cu t approach to sol...
Read more