Applied Probability Models for Decision Making

Welcome to the course MATH S350 Applied Probability Models for Decision Making.

This is a higher level course about the application of probability to modelling real-life situations. This course and the course MATH S346 Linear Statistical Modelling are formed as the compulsory subject in the degree of statistics and decision science programmes, and optional in the degree of the mathematical studies programme.

The course follows the middle level course MATH S280 Statistical Methods for Decision Analysisand develops the ideas about probability and random processes that are introduced there.

The approach adopted is to develop a probability model for a practical situation and then to investigate the properties of the model. The emphasis is on modelling and problem-solving and, for the most part, no attempt is made to test statistically whether a model for a given situation is a good one; this course is not concerned with that aspect of the subject. Neither is the course concerned with the fine details of the analysis of the models. For example, if the proof of a result is long and mathematically detailed, it is usually omitted. However, such proofs may be found in many undergraduate texts if you are interested; some suggestions for further reading are given later.

A number of results from mathematical analysis are used in the course without checking that the assumptions needed for the results to be valid are satisfied; where such a result is used, you may assume that its application is justified. The same approach is used for some of the statistical techniques that are used to investigate the models; in this course we are primarily concerned with the application of a technique to problem-solving, and not necessarily with why the method works.

Along with Books 1-5 (Genetics and Diffusion Processes are optional) study units, you will need to use the Statistics Tables(by H. R. Neave).

2.1 Preparatory work

MATH S350 is written for students who have studied MATH S280 (or its predecessors MATH S245, MATH S246 and MATH S248). If you have the units for these courses, then it would be a good idea to remind yourself of some of their contents, particularly the sections on probability, random variables and expectation, and random processes. You will also need to be familiar with the normal distribution, and you should be able to apply the Central Limit Theorem and use tables for the normal distribution to calculate probabilities. Much of this material is reviewed in Unit 1 of MATH S350, as it is important that you are familiar with it. However, if you have not studied MATH S280 (or any one of its predecessors),or if it is some years since you studied it, you are strongly advised to revise this material before beginning the course.

If you do not have the relevant materials, then you could use the following reference book to revise the topics mentioned above (or any similar text would do).

Reference books:

• Ajit C. Tamhane and Dorothy D Dunlop, Statistics and data analysis from elementary to intermediate (Upper Saddle River, N.J. : Prentice Hall).

2.2 Calculator

You will need a calculator for some of the examples and questions in the units. You will be allowed to use your calculator in the final examination. However, the University has strict rules about which types of calculators are permitted. You should check with the list of approved calculators provided by Registry. If you studied MATH S280, the calculator you used then will be adequate. It should possess basic mathematical functions such as ex, log x, sin x, x^y ,and at least one memory. You will not need to use statistical functions in this course.

2.3 Computer

You will need access to a computer connecting to the Internet. If you have problems accessing a computer, then you can come to one of the University's PC labs. The University provides the Online Learning Environment (OLE) for all courses. You should check the MATH S350 OLE site regularly for updates or news as well as useful and interesting postings on the discussion board. For information about the OLE, please access the online OLE User Guide at http://ole.ouhk.edu.hk/help.html.

3.1 Study texts

The topics studied in M350 are covered in five books. The associated software based activities are contained in a separate computer book.

1. Book 1: Introduction of basic concept of probability and random variables, probability generating functions, specific distributions that are going to be applied in the other four books and their simulations.
2. Book 2: Introduction to point process in time and space, various Poisson Processes in real line, 2 Dimensional Poisson Processes and Detecting Non-random Patterns.
3. Book 3: Introduction to Branching Processes, the Galton Watson model, Extinction probabilities, random walks and Markov Chains.
4. Book 4: Introduction to Birth and Death Processes, Queuing Models and General and Stochastic Epidemic Models.
5. Unit 12: Introduction to decision analysis, decision making using game theory, statistical decision theory and decision models.
6. Book 5: Special Topics include Genetics, Renewal processes and Diffusion processes.

3.2 M350 Software

The M350 software consists of a set of programs for exploring the behaviour of various models. Each program is associated with a section (or subsection) of one of the books. The first program, for example, is linked to Subsection 6.1 of Book 2. Instructions for using the software are given in the computer book.

3.3 The Handbook and regulation

You should become as familiar as you can with the contents and layout of the Handbook during the course of the year. It contains standard results which you will need to use but may not wish to memorize. Please carefully read the handbook regulation of this presentation as given below:

3.4 M350 OLE

You can access the M350 website from your Student Home page (OLE). The website provides information about the module and electronic copies of the main teaching texts.

The assignment and the specimen examination paper and its solutions are available electronically through the website.

It is important that you check the website frequently, as details as any errata will be posted there.

The M350 website will have a discussion board, which you can use to communicate with other students.

3.5 Audio CDs

There are two audio CDs in this course. Each consists of several audio tracks. Each track is linked to one unit of the course, the first being associated with Unit 5. The audio tracks form an integral part of the units and are used primarily to provide you with practice in techniques. (A margin symbol of a cassette tape is used to indicate the position of the track in the written text.)

3.6 Continuous assessment

The continuous assessment for this course consists of four assignments, as shown below.

The best three assignment scores are used to calculate the overall continuous assessment score.

3.7 The examination

There is a three-hour examination at the end of the course which is based on the whole course. You will be sent a specimen examination paper some time in advance, for which sample solutions will also be provided. You will be allowed to take into the examination your calculator only and no other course materials will be allowed.

3.8 Supplementary reading

This is a course about applications of probability, so the details of many proofs have been omitted; however, proofs are contained in many undergraduate texts. If you are interested, you can find them in some of the following reference books. By dipping into any one of these books you will also find out more about the various topics that have been introduced in this course. Different authors approach topics in different ways, and you will find some of the language unfamiliar.

1. N. T. J. Bailey, The Elements of Stochastic Processes with Applications to the Natural Sciences (John Wiley, New York, 1964).
2. D. R. Cox and H. D. Miller, The Theory of Stochastic Processes (Chapman and Hall, London, 1965).
3. W. Feller, An Introduction to Probability Theory and Its Applications, vol I (John Wiley, New York, 3rd edn, 1968).
4. G. R. Grimmett and D. R. Stirzaker, Probability and Random Processes (Clarendon Press, Oxford, 1982).
5. S. Karlin and H. M. Taylor, A First Course in Stochastic Processes (Academic Press, London, 2nd edn, 1975).
6. J. H. Pollard, Mathematical Models for the Growth of Human Populations (Cambridge University Press, 1973).
7. S. Ross, Introduction to Probability Models (Academic Press, New York, 2nd edn, 1980).

Book1: Probability and random variables

1. Probability

1.1 Basic ideas of probability
1.2 Conditional probability
1.3 The Theorem of Total Probability

2. Discrete random variables

2.1 Random variables
2.2 Specific probability distributions
2.3 The mean of a discrete random variable
2.4 Variance

3. Probability generating functions

3.1 The probability generating function
3.2 The p.g.f.s of standard distributions
3.3 Calculating means and variances
3.4 Sums of independent random variables
3.5 The p.f.g. of a constant multiple

4. Continuous random variables

4.1 Cumulative distribution functions and probability density functions
4.2 Expectation

5. Specific continuous distributions

5.1 The uniform distribution
5.2 The exponential distribution
5.3 The gamma distribution
5.4 The normal distribution
5.5 The χ² distribution

6. Simulation

6.1 Simulation for continuous distributions
6.2 Simulation for discrete distributions

7. Related Variables

7.1 Discrete bivariate distributions
7.2 Continuous bivariate distributions
7.3 The Theorem of Total Probability

8. Useful mathematical results and techniques

8.1 Powers, logarithms and exponential functions
8.2 Geometric series
8.3 Differentiation and integration
8.4 Ordinary differential equations

9. Further exercises on Book 1

Book 2: Modelling events in time and space

1. What is a random processes

1.1 Basic ideas
1.2 The Bernoulli process

2. Further examples

3. The Poisson process

3.1 Basic ideas and results
3.2 Notation for continuous-time processes
3.3 Simulation

4. A more formal approach to the Poisson process

5. The multivariate Poisson process

6. The non-homogeneous Poisson process

6.1 The model
6.2 Basic results
6.3 Simulation

7. The compound Poisson process

8. Point processes

8.1 The index of dispersion
8.2 Types of point process

9. Spatial Patterns

10. Random Patterns in space

10.1 The two-dimensional Bernoulli process
10.2 The two-dimensional Poisson process
10.3 Simulation for Poisson processes

11. Non-random spatial patterns

11.1 Patterns with regularity
11.2 Patterns with clustering

12. Counts and distances

12.1 Counts of objects and their properties
12.2 Object-to-object and point-to-object distances

13. Detecting departures from randomness

13.1 A test based on counts
13.2 A test based on distances
13.3 Significance testing
13.4 Postscript

14. Exercises on Book 2

Book 3: Discrete time random processes

1. Introducing the Galton-Watson model

1.1 The model
1.2 Exploring the model

2. The size of the n-th generation

2.1 Probability generating functions
2.2 The sum of a random number of random variables
2.3 The p.g.f. of Zn
2.4 The mean and variance of Z

3. Extinction

3.1 Extinction after n generations
3.2 The probability of eventual extinction

4. Other models for branching processes

4.1 Extensions to the Galton-Watson model
4.2 Other models

5. What is a random walk?

5.1 Simple random walks
5.2 Random walks with barriers
5.3 Random walks and Markov chains

6. The gambler's ruin

6.1 The gambler's ruin as a random walk
6.2 The probability of ruin
6.3 The duration of the match
6.4 Gambling against a casino

7. Unrestricted random walks

7.1 Return to the origin
7.2 Exploring simple random walks
7.3 The position of the particle

8. Return probabilities and generating functions

8.1 Return probabilities
8.2 The recurrence of a random walk
8.3 First return probabilities for simple random walks

9. Other types of random walks

9.1 Random walks with reflecting barriers
9.2 The limiting distribution
9.3 More random walks

10. Basic Ideas

10.1 What is a Markov Chain?
10.2 Transition Matrices
10.3 Calculating probabilities

11. Conditional and unconditional probabilities

11.1 Transition probabilities
11.2 Absolute probabilities

12. The long-run behaviour of Markov Chains

12.1 Limiting distributions
12.2 Markov chains with an infinite state space
12.3 Stationary distributions

13. Properties of Markov Chains

13.1 Communicating classes
13.2 Recurrence and transience
13.3 Periodicity
13.4 Classifying the states of a finite Markov chain
13.5 Limiting distributions and stationary distributions

14. Recurrence

14.1 Return probabilities
14.2 Return times for recurrent states
14.3 Postscript

15. Exercises on Book 3

Book 4: Random processes in continuous time

1. Arrivals, births and deaths

1.1 The Poisson process
1.2 The simple birth process
1.3 The pure birth process
1.4 Markov processes

2. The distribution of X(t)

2.1 Differential-difference equations
2.2 Functions of two variables
2.3 Probability generating function approach
2.4 Lagrange's equation

3. Solving Lagrange's equation

3.1 The general solution
3.2 Particular solutions
3.3 Lagrange's method

4. Two growing populations

4.1 The simple birth process
4.2 The immigration-birth process

5. Birth and death processes

5.1 Markov processes
5.2 Birth and death processes
5.3 The immigration-birth-death process

6. The simple birth-death process

6.1 The size of the population
6.2 Extinction
6.3 The embedded random walk

7. The immigration-death process

7.1 The size of the population
7.2 What happens in the long run?

8. Deterministic Models

9. Models and notation

10. The simple queue

10.1 Queue size
10.2 The equilibrium queue size
10.3 Queuing time
10.4 Idle periods and busy periods

11. Markov queues

11.1 The M/M/n queue
11.2 Other Markov queues

12. General Service times

12.1 The M/G/1 queue
12.2 More than one server

13. Modelling the spread of an infectious disease

14. The simple epidemic

14.1 The model
14.2 The deterministic model
14.3 The stochastic model: waiting times
14.4 The stochastic model: distribution of Y(t)

15. The general epidemic

15.1 The stochastic model
15.2 The deterministic model
15.3 A variation on the model

16. The stochastic general epidemic model

17. The threshold phenomenon

17.1 Exploring the stochastic model
17.2 The threshold phenomenon for the stochastic general epidemic model

18. Other models for epidemics

19. Modelling lifetimes

20. Life tables

21. Stationary populations

22. Stable populations

22.1 The growth of a stable population
22.2 Stable populations and stationary populations

23. Population pyramids

24. Exercises on Book 4

Unit 12: Decision Analysis

1. What is decision analysis

1.1 The decision-making process
1.2 Construct a payoff table
1.3 The decision tree

2. Decision under uncertainty

2.1 Decision-making criterion

3. Decision-making using game theory

3.1 History of game theory
3.2 Game decision models
3.3 Two-person zero-sum game

4. Statistical decision under risk

4.1 Introduction to statistical decision theory
4.2 The statistical decision model with distributions

Book 5: Further applications

1. Biological background

1.1 Chromosomes, genes and alleles
1.2 Cell division

2. Mendel's laws of genetics

2.1 Mendel's first law
2.2 Mendel's second law

3. Population genetics

3.1 The Hardy-Weinberg law
3.2 Examples of population genetics
3.3 Genes with more than two alleles

4. Linkage

4.1 Sex linkage
4.2 Hardy-Weinberg equilibrium for sex-linked genes
4.3 Chromosome linkage

5. Evolution

6. Discrete time renewal processes

6.1 Some examples
6.2 Outcomes of the event E
6.3 Waiting times

7. Continuous inter-event times

7.1 The waiting times between events
7.2 Characterising components
7.3 The hazard function

8. Renewal processes

8.1 The ordinary renewal process
8.2 The total lifetime of the component in use on arrival
8.3 The residual lifetime of the component in use on arrival
8.4 The equilibrium renewal process
8.5 Variations

9. The limit of a random walk

9.1 Random walks
9.2 A continuous model
9.3 What does Brownian motion look like?

10. Ordinary Brownian motion

10.1 The distribution of X(t)
10.2 Using the Markov property
10.3 Conditional distributions

11. Waiting times

11.1 Exploring waiting times
11.2 The distribution of Wa

12. Brownian motion with drift

13. Geometric motion with drift

14. Simulation

14.1 Simulation using tables
14.2 Simulation using the software

15. The definition of a diffusion process

16. Exercises on Book 5