Dy whe wen Fob) Trpie ah te Fe sy te tat erate PAG) = (= 1AFRal = MAPA), EL (oe can sve thes eantins recurve, but oe he competations are Ge ey we wil a hem end spy state that te stn Pal (te pot too dia o sf tat Pa) defn a ove dos ay thee eesti) he ve for Pe) iv ali ct, frat, Xe bi 8 £0 ‘Rone eseibution with parsmetarp-=<~™. Tos allow Gs imspaciately {S caapute the expected population eat inet, Eno Haye, nb, B(x) ‘yjooul drive hint sel ners wap Lat #0) = B(XG). Thon yo Siemans maro reo = Snare = M0. ‘Thee, sts th toed eguton for exponential saws ad the inital contin J(0) =? immediately ge the slaion (0) = 0 "Theres oon other may we an lk tthe Yule procs, Cone the tne "Fe when the population Set raceme Yq= inte: X=). ‘Thea Ye = Tt ---+ Tay whee 7; mosses th tine between the Stl ofthe ih and (Ue iid The sandom vsabno Ty ae ‘Bdcpeadent at bon expeneatiadtbuion with parameter In partsuar BCR) = 1/0) aad VertR) = 1/(0) Theres, ‘Neo Vari¥a) © SIE, (A) < so. Hoses, Yo equals lnn/A up tw sal fandom sor ouaded mgt large If taste lan 12 ech population of adidas thea i tine ¢ wo would expect individual "Now conser the Kt grog popslaton mods, Beample 5, with 2h, Aga let ws cola the te un he whined eters the Dopulton. Ia Us cae aa Seretng phenomenon over. Conder me ao i 070) Sats) =F cw tn pry wth tat «tT mg tt a ate oe pee ae one rte Ode opeaneren lea tab case EatenSupper bv oti (pe nas) sae ace 6 and an Seracedage ar eulibeeuctdsepenbos sty Seon wane c= Deenew Tew can the exec arm da adh ate eo tsi tnshocnesnes inane Mato dish ha aoe Par LXse 8) = a(e)at te Se uw) hw ha te i of ee em cn dca SCAU T eit Lr stam ip tno toe cpa nies na of te fgg puntoe nd Seton 3 Pw tre ein be a eS aca dewwt afens l (Castes watt tt xh oh owl nce par los) =P(Svn¥ Xena) = Pike my X22) ‘ao iv» isa unt fenton bein eae usa) “= aeavsdnadtya) + Drlesednadee) leat ~ a(t +0) + Daevellteadae (00) fsa Spe ae +e, we can combine the lt err tam a tha Seen a a then wean ode hah ain pobre of ee ey ore 9+ Zeternens hoe the daca e wi espost to tine, These ae sometimes called {he forward ution fo he ein Ta moet cas of ater ning ll ‘axa in the fs thn sections, (114) can be sid, Tha te ‘tampa, bonever, whe tbe lorwrd equations ate al sete ‘Theres auothe et of equations called the Sache sunton wich slays Boi Fr the backward sqatons we ite mates) = Traders) = Zhe satan) $A ofe)at + o4f8)ova0) ‘The er ter depends only on. Wit a Sle monk ane can tow that one can always tala the st a At gp oO aa pet Hite (esa) + Date ante 1s the cas of faite state spe with instil pntrtor Ay the ‘uehwardequstins fr the rset matic, becomes ei ae Hal dnean, hich can be compare bo the formar equation (28): Both quis (te Intl contin #5 = 1) have the nation 8.1 Suppose tat he numberof cll per Bou aig a xeetng ‘service follows 8 Poisson process with tind Was erty ta er a cle coe oh (Suppo hat ae ale aie in hit oe. What te proabity ‘tat tot toc wl area crn eat (e) Tm pon sven he nsw wn poo eal ave sete bebe ging ach What the opera seu a ie that {he pon wi walt (€) Spe i own that ect et clad is he Ett sot Whe th tay at xB nme 2 (0) Suppose snows ta rate rvo in eit fou bows ‘What the peotin tht sat of thse in th is bar 22 Lat Xan ¥ be wo ndepesdent Pinon proce ith rt pra= ster 3p end Jp, oper, ig te mar af tenes ce intra tan’ repecy(9 hath pnb tat ener ain se br any {b) Wont i the probity tht i he Sst hour, a otal of exactly four cosiomecs hive sive the wo stores? {Gen tat exaly four coromers have ecu at the two sores shat is the probity that all four meta sae (@) tT donut the aoe of aval ofthe St estome tore Then “xy ie sumer of customers in nor 1 te tine of the fa cstomer ‘Slat store Pind the pobablity distribution of Xp (Le for ea fed Ptr = 4). 18.2 Suppom ond Y re independont Piso process wth pastes 2nd y ropenively, mewring tbe uber of cals ring st ‘igre ponse Let 2. = 4 Yi (c) Show lint 2y iv Potoo proces. What eth at prance fr () What tho probability shat the fst ell comes onthe ist phone? {Shue denote toes tne that teas oe ell as Some fo each tie to poor, Fad the deny op astabution fantion of the random ‘vibe 34 Let A bo the inital generator fr a Ines, faite, ceatincowte Mathv chain. Then the rows ofA nddup to O and the ondiagonleaneats of ae aosnepive {G] Ueto some pstve suber greater thn ll te entre of A. Lat Pies Show that P ste reson mate fra scree, Iredale apeiadi Marto ca, "by Unis to conlae: A nea uxige let egoovetor with eparaoe Shas oa pebublty vector andl tho other egenalas ofA be rel pasts ha 3.5 Let: bea Macio cain with stat space {2,2} and rte a2) = (2) = Bed ‘36 Let Xe bean ida, continuousine Markov cai, Show tht fovea nj and every #> 0, Pasi ten > 7 Consider he contiouou-ie Mayen with stato pace 1,24) bd Snel georatar 24 eee (6) Find the suit dtbaton 42) Seppo the chan sats in sate, What ee expected amount of| pe uae change st or he it ine? 6) Arai sare techni sara in state 1. What tbe expected anbust cof ime antl the ci sin tate 4 26 Let 2 be cotinourtme ita proce with birth rate de = Ts {ijfn-+ 1) and doth eg ~ 1. Te tis poo pie ‘eourent; ml cuter ots? What fy == (23)? 3.0 Consider the population mode (Example 5 Sexson 33) Fr hich selice of and 1 extinction oataln, Ley when ithe proba of Featiog sat O equa 017 8.10 Consider the populin sda wit immigration (Brame 4, Sc Son.) For wich values of ye ete chal postive secret, tal ecurtet, rasan? {8.11 Conde etb-and death roca with y= 10-1) Shey ta em pt es and ete nay Ba 1.12 Sop on has demi poplin wl he ‘ndtv propre tsp of ane ope tik wl te Yondon wie te fed oon cur, fern stn <> 8 Anne) = St ie osa Gy apn af eat al Seve! pes ae ee 3.18 Consider a Wrt-and-death prooss with beth rate and death ‘Sa Wht eth enn rth an in pao?CHAPTER « Optimal Stopping 41 Optimal Stopping of Maskoy Chains Imagine the follwing simple game. A plays rls nde. he plage rolls 1 the player wine so money Other the ple nay eter sat the {amo aod win dlls where te rol of he deo ay sol agent the player rol ga, tho game cnn ti eter 6 soled oe player quits Tae sta pay forthe gan a always & dace, where K {he vale ofthe at rol (ele the voll 8 a wich cae the pay (0). Wha Is the optinal stateay fr the layer? in order to determine th opti satay 3 noms to doce what sould be optinired. For example plaer only wast 2 gurantee ‘hat tho pap le postive, then the gare should be eloped alter the rt rollator the player bas avant (a6 old) or te layer sa guurenae a postive pao! by stopping Hower ie emcee fo ‘nade: what happasif the ple deco manish eptdpyol ‘et usanalye ths probe and teu show bow ti appro more goer] Macior cate problons Wie fst et (b) denote the ayo anoristed with ech ol Inthe example (4) = 4 fk <5 and (0) ~0, We le eh) be the cxpctod ‘ang: ofthe pyr gies tet te frosting et he player lates the optimal stay. AL this moment we sy not koow wht the opal srtagy ie at sll abe see to nse w-We wil infact ‘te dowa su eguaon tat stices and se thst deere vad Ue optimal strategy. Wo ft note tat o8) =O and (6) = 6. The ler Is tee cet cll dow oot py trol aa thet rly 0 tho ‘ptnlstetegy sto stop aed pic up 45 Ts ot so cee what oa) "Now lt a(t § be the amount of payathat ie expected I the ger dus nc top afer alg abbot fom thea o plas acsering 0 ‘he opanal wae. ths persciar example, wh) te actually the se ovale] Then tony to oe that aa) Teele det + dot + La. (+ ju) + E00,‘Wo ao ean wrt the optimal stztagy In terms of w(t) 76) > w(t Be Nec al aap and take the mavey; H(A) < w(t, te paver ‘ould rl agai. In ker words, () = max{ (0.200) In paisa, o(t) 2 7(). Tie fet lie that wf) > (= Tis 5[2 We now kom mare about te optimal steiegy— the ist Sots Lara 2 the plyer shoud cll gin. Hee, a) = oa) + # (O/B fo) +--+ OO + 918, (2) =f) +--+ 9/8 +870. Suppose the ft oli, Suppose that the opti strategy ware to conte lying, Teen clnly tht nou ao be te opal strategy i SEER tie 3 Une ths tate, the pume would conti wl = Soc 6's rolled and each of thane ending rls woold be equal Bly ‘Tis wuld grea expected pa of (5+ 0/2 = 8/2, wich ethan Teles anne be ie opine rabgy stating wih a. The Dag, ‘hasan shuld meop with 2 aad v4) = (@) = 4. We fly const ‘PES apes te fit ss #3. Sappocte lye 0 ga wheaeves Seon up ond we he optimal avarey otherwise Lets ete expected wong oti ne, Thea = Pfell =) forall. Thea 2) = BCP) | Xo = =) 2 UCT) Xo = 2) = ol) eno evry superharmonic anton that age in fers than equal 12 he vale function. Azo we not (5 Haare 48) tat if {ull} Seay oflection of parses fneions then ote) =i) ‘sal supecarmonic. We ehefore have te folowing: # i the arnt epeearmnie function Wilh pect FP Wet grater Uae eel fi ‘slaty, of) = tela, ‘share te timum soar il superharmoni factions «wth w(z) > (2). ‘he characiavzation lens oan algo for deter v. Start with the feos w(s) tat eqns (etfs an asain state and oterwise tus te maximum vale off. Ths pes a pero fnction tat Spear thn f Let ‘a(e) = max(Paule. S10) Sine oi wuperinronic and vs > fn(e) < mC) Ado, Pane) ¢ Pale) < vale Hence, ie superiacoal function rete tha fConsoung, we deine mar{Pog-a(2 FC) aul sau we to that uy Ise supechamonie function geste than f but ss than vant: We ean then check (ste od of thi sation) that. We) = a, val Example 1. If wo coder the game that we aleady anstaed end stared with he fanetion = [56,6 0) ben a 10 ertons Wo would ee tno .002,500, 3.002460 Bxample 2. Suppown 2 ea simple random wale (p 1/2) wih 8b sobs es 0 (0.2.4.4) the ol nn fe Grey = [02,45,8,30] (we wie tb pal faction ws «vsti ‘ra Wola anne openl eta Cha eve spe ate {Cod one bas stop tO aod 6 rom sate thre ea obit 12 of pling and 1/2of gong to 6 the expecta pay ven that we ute ‘Sat lost 0/2 > #(9)=3, 20 om bre comlaue fone sat asa then oe can ge an expected puyol o (4+8)/2 = 1 by taking one top nd then stopping. ince hiss roster tan (8) = 5, stat be optimal to ‘onlnue fom sat 3 and o(S) © 18/2 Not tha on sate 2 oe cn gt by playing an expected pp ft laa [F1)40(9]/2 > 17/4 > FB) = ace, no continue on ate? nad» (2) > 17/4 Slay, oe onitaves ‘om slate Iwo can obtain an expcid pyo of (2)/2.2 17/8 f() =, 10 the optimal strategy iso sotaue Thesfoe he rapping m2 hs esos = (046). The vl intion ca be ened oy M2) = BUC | Xo 2) = APEXe = | Ko = 2h. The corresponding probaity bas het copued ble fe (1.16) wad == 0,9/4,9/2.21/4.0,9/20. 1 this example had arid withthe feo vs = [1,9 8,90,0) ad performed the algo sbave we wosld ave gion wit fe {ual val of» i abat 0 Heratione In ving the optimal topping problem we snatancoaly compat he ‘ale function vad the opti stopping tata. Supper that we ae the talgy that we would choose, Le, me lithe sat space to two fete Sand Ss a0 that we cortinoe oa 8 and sop Sy Laws) be ie ‘pected pay ng ths sea, hes psc ule) =e), #5 4) wG)= Pale), #08 as) ‘This aa diacrete analogue of boundary vie problem stn alld {he Distt problem. Te Doanary ithe et 3 where peed alee sre vn. Oath "interor™ patsy, some difreoe equation lAsw hve son the probate farm ofthe sluin ofthis eater an be trent ste) = BL (%)L¥o=2), : re mis()20:%)€ 54 Tore aie tate Mario cnn, th sition ca be own dwt beaare (a) and (2) ieee nse eqoton ink wlnowas where (deiner of pein ad he unksowas ae wa) = © 8 ‘Wes ras tthe apt does oovere to he vale fein ate.) yn (Sino the Ged nk o opecarmonic Fee lS reprtarmane (Kore 48) Abo wa) > 7) foal Hens by te charateation of we et wie) 20. «6 et the sopping set abe dened by Sua fiat =f) sum {es0te)> eh ‘on Sy Pa(e)= we) (i Pea) < of), he fa some m Pons) < (8) ss(Pan(eh sah © ot) mie ips (ote) |X 2) ‘nb i he seoegy anita ith the ets Sx, Sa. lace) the Tara capvted valu vm all coins of oping ss, ote) sole, an ‘Combining (6) and (7) wos tha (e) = fal 142 Optimal Stopping with Cost (Conse the rat example end spo that chix charg of foreach aioe! tl, as on each roll we ea ter take the pagal svoated SAN ca or py 61 and sal agl Tn gence, we nay ene that ‘tei cost oe) mocaed with sch tate hus mat e ai ocontane thechae Agia weammme w Bae a pai Finction/ an Wt ws) be {32 Speted eae ofthe pay mins fhe cot rnin a stoping ‘chown that mnie ths expected vas, We ean wate wt ap nucee)~ Seat) x0= 9 severe agin Hh cupremam is ovr al egal stopping tines 7. Then ws) nice sx{J(), Pate) — al) oe te, the expected py minor corti thecal is cotined i Poe) — fe), Apsa ween vide nto Sand Sy whore ‘od the opts stopping rl eo stop when the chain eaters tae in "ung sine segue mf Sesion, te ae faetion vf i camp can be characte! ate eae anton greta han J thal ‘ete (a) 2 Pale) ~ ate I ater words we) = int), rhe he nis is overall ising u(s) > 0) and (=) > Pale)— (fe). To Bathe vals Fanci, we may sea lg sia hat a oon 1 We dot so beth funtion that ans fom all bob Fate aad equals he mein me off evexyhre os. We then define sale) = axa) Perf) a2, nd thes oe) = in el) ‘Example 1, Suppose we consider the de game wih f= [128,40] sail ye fisi Ici trl Ee ct faction lest Ts aly cht wold testo sel gst that we shold stop if we got aor a5, Tina, we soa stp fe got 3 dace we wre diferent lore with {his ol and costo rll again IF we get , the by oling aga we BS fits expert poo of lat 8/2 with eot of Hence we cau ‘Syet oot pin of at na 3/2. Thcfore we shoud ply If we 8 ‘uppse we ol again menor we sta Ter 2 and op otherwise Let se) cet tsi i ts Fs) = 20) =» os tue) Bao 7 5 ne thee than f2) 2, at be Solving or + sae Carrot to op a8 Hono he sopping ts S Sle fn Is 2,8,4,8,0) and the == 8/6.28,45,0) Tews started wit tho inital wy — [6,555.0] and performed the siguttn decribed ava fer only ew etios we would have 9 = 1.62,54.5.0)Bxampl 2. Consider the oter example of the previous section where Xun a cape random walk with abnrbing boundary on (lee 0) and £7 (0.2,4.5.008,0, Suppose we impos a esto 3 to move oa sate 01,2 and cs of | to move fom 3,4.5,6, 0 cast fancton #5 (5,5,5.2,443) 1 we stars with nial wg = [0,989,980 hen in oly sx eatons we w= 0.2 455,0,85,0), wale ie th value ors In this eso she stopiag sat 6S: = 01.248). Example 3, With » at function tis pote to ave a nonsvial robe even fhe Mack cal ie educile. Suppose we ply the fa lowing ge: ol fro dice the player tay scp st ny ine ad tals the elles the ce or the player ay pay 2 uta he rol i ne han 8 and ‘unit ito roll spat than of equal to 6 and voll aga Ts this case ‘he tte spacnie (23,412), =I IQ 9= 2.2213} ie start with th nil guess uy = 12,1232] hen within 20 ere slo we converge tothe alo fusion, 10,113). "The sapping ste Sy = (78,12) 4.3 Optimal Stopping with Diccounting 1s ofca appropriate hv ieling financial matters to asume cat the value of money doweasr with tine. Let sve hata donot factor tr < ls Bien. By ths wo mean thes 1 dalle recede oe tne wat {te the nme a a dla oeive nthe presen Ag sigpoes we hare 1 Maskov ine %y with Wain mate P and pyelRencion fT is now th galt epimio the expected vale of th puyol, sling iaxo tousierton the detesing vale ofthe puyol I we wep afer sep {hen the prsat yl ofthe pad in steps fa Hine the acta pga "hn tis ae the vat function I ea by fe) = mp B(0" (Xx) 1X0= 2), where aga the supremum i ve al lag stopping rubs To obtlnthe ‘ne fneio, we characte the salt acon w tying we) 20), u(e) 2 oPute) ‘We may obtais» with similar lgrto a bre Start with a alta function oy equal tof a all aborblng ets end opal vo te xm salooof Fetal other states, Tava dale wo recusey oy aCe) = max{s(e).0P 45-12) Theo ot Binal Bxample 2. Consider the de gas agin Asuna incountng factor ‘a= Site scouting cas only sale see fy fo elop iis cy tose that on shoul stop ifthe et rolls to 1 the es os 1,ope can et an expected pyo of at -(142434+4+5)/ = 2 by roll gain, 28 best orl aga. Sappoee wean be sly fort ‘ain with a2 and to stop otberwise and et» be Ue expected lange (Goes that one toll again: hn so Ble /6 44/0. 9/6 5 4/645/0, Sovig fru we get w= 24/11 > 290 mtb opti a ol gan with 2. Theatre $3 = (45,8) aod fay, 24711,5.45,0) Example 2. Consldr the example ofa simple rendam nak with ab sorbing bosndsis on {01,6} and f = 0,2,68,8,0, Supper at {ere 20 cost function, bot the value of money i dante ot rte 8 we star wth y = [.9,90,0,00 then in seven rains we fonverge the rane = [,24,588,0,405,0, ‘This stoppage (01,248) Tis poale to incude both scant fnctlon and desountng face Suppose in adton tothe otber assumptions fn this sion, we hove 9 ‘oat fnetong() thst intr the cot af tang tp gees at the tain Instat 2. Then the alse nein ie alot Tanto ‘sisting wt) 2 10), s(s) 2 aPale)~ fe), Bxample 8. Coser the random walk with shoching boundaries do vated before wih J = [.2.4,5,5,9,0) ead wth both the cst faction 111) and the dicouat factor 9 =f I we sat wth 0) teat ety tre son we converge to 2 .2.4.5,0,206,0‘The oping mt 10,1,2.3.46) ‘Beample 4. Consider random walk with aborbing boundaris onthe state pure (0-0) Seppe the pyolfincon the wquare of the estopped a bey : 19 1045,,.-.200) Wo asume tat thc «costae cart of 6 and 8 dseountng factor of We thea start th che al gous 100,200,100. 100) ond fer 60 aration wo ge y= 151 445,81, 18,25, 984,648,100 "The saying st (04,5010). 44 Bnorceee 4.1 Consider spa random walk (p= 1/2) with abating boundaries 2 0,,2,-- 10) Suppor he Ellwing poy fn sven 30, Find the opt stopping rule and give th expected pay starting st 14.2 The lowing gs eae: you ral mo die. you ral 7, th shone sve ond you ning. Otrvie You ay and eee a2 ‘Sas equ ote caso tn ve doe Iyou cma, you al nel ‘Pemetatn whey ou ral sf or ware oa op. 900 0) Fete lag? ou seo amon sa um fhe eo ‘Be cetas te Wat keer pad wings» go ag op fe tent ol) Sou pay opine our expel wings? 1o2.4.310,06, 4.3. Comer Bretce 41-Do the problem again astning (a) seta ont of 75 fr each move, (@) daoont fet a= 05, (6) bots ‘Consider Brace 42, Do the problem agin naming: (a) cost function of g = [2.2.32 1 12,11 (i) acdeoone ctr «= 3) (bots 145 Comider a simple random walk onthe following fou-vetex gran Assume tha he pga functions (4) =2,1(B) = 4 (0) = 5,410). 2 Aamo tht thre ano cot snoed with tevng bus thar ‘ont factor a. What is the ape peso value of 0 thet the oping! Hopping eetngy iv stp at erry vere, a0 that Sy = (4,0,0,D)I 146 If (yen) ae ll unctions that are sopesarmocic at = for Pile, Pace) swe), de et be the fncton| M0) = ats) 447 Consider imple "Wheel of Forte” gan A wheal divided into 12 equasisd wedges. eve of the edge reared with the amber 109, 00,100 denoting an smo of money mon fe whoa nde oo {ate atte. The erelts wedge mri! “base” A payer ca is ‘ss maay ine sh rsh wast Hac ine te whe lands ons mumbered ‘wedge, the ples rsev shat ech money whch Se aed to haber Previws winlgs. However, If he whod ee lauds the “buakrap™ ‘wedge, te ple lose al ot his/ber money that bas Doe mon Up to at Pot ‘The player muy qui at en tine, end tle all he money bea se ar won (aauing the “bankrpt* wedge as nat come “Anurag tot the gal into masiss one’ exported waning i kin ns, dove an option strategy for playing thi game ood compo one ‘Speed wianinge You may wah lo ty computer slant,(CHAPTER 5 Martingales 5.1 Conditional Expectation 4tnths section we dees the notin of cantina expectation, We mare with some ener ramos nd ld up toa growrl Glin, Supper” Isa random variable mosrring the satan oma ram expriet. {fone knows voting about te otzome of th expernent then the Dest ‘gms for the vale of Yi E(Y), the expectation, OF eau, i ose hae ‘ample nowledgeof ie oatzome af the expernect, thn one kus the ‘act mlsn of ¥- Toe ation of codons expectation del with making {he bet guns for Y ven sme but not complet Inraton about tne foxtcome Th gv ogeerl defo of conditional expectation one de the famenerk of masa tboaetleprobblty thor. We wil at for a ganeal Gaia, bel iesaod aasictourmives to deg the conditional expectation of teodom vacabe Y wile epet to fale Smber of random vain XX Suppose that aad ¥ are duce random vaiales wit ois probe iy deat Fneion fle) snd macal probability dost funeoae fale) = Dslew) frty) {x=2,¥ 04) Chew ‘To dase he conditional expectation of ¥ ven XE(Y |X) we fn to ‘ive the ost yale of ¥ ray vale f= A ite thought wl aw tat sreshonld deine BO LI) Dery anixes Doric ear anys (Lyre wh txt)‘This wooed i fe) > @ and we do not worry about doing TBO [je fr oer yl of nce wh lus sur with probaly 1 he ca ezampleroppooe tant two iadependet dice ae rolled ond Uthat ac of Se trl ab ¥ donotas the sam ofthe two ral ‘Theo od Stacy f X50 ae rete random variables wih ot pode silty demty fasta Haye rt0s0) =P sa the magia Gest wih expect Lo Xu en By enone) = Do yleaecstma yoke tn¥ hs ‘thos the enailonl expectation of ¥en Xp. en By [Xin Maen) =U Hs ene se abet ‘Tie we die i, sil outcome fr the expecinen, se lan ves) > 0 Agi we hak of B(V [s,.-%y) as bng the Tsk gune fr tho tue of ¥ gen the vl of 2, Xu EE and Ye cntinons random vaio wth Jit deny (6,9) nd maria destes sa [teow thea the cota expectation of ¥ gia X i defined in an esalognus tea 1 ere) =1f vena eu pid 1) > Ty 7 ek ae Pre (fren eae nd oe wos 1 The vale of the sendom variable BOF | Xx) depends only on the vate of Nyon Xuy ty we ea wake BGP | Xiynona) = Eb.) for some fneion$ (fe thee who now measure they, {he nclion mst be a aseuable fen) IF sandr varble Zea ‘pe win aw functon of, it called reowrble with expert Xie 2 Suppose Ai any een tht depends ony 00 X A lei be te eve a Forex, dam agrees nnn emer pant mer) etn ee ce wre ont a on ar vee Tee) [ok [arisen cnet Fen ated LRA Tena = £ L Lae. tera) de dems eder Ba cata et cen mci ot ena ona airs meet na sere ce tegensede gi ompers ne eee ee ea eee Seaied hnere tien Seren eaten at raiteerana er Seceni pnienm yea iy Sac nna ne oot es a aero ee ease ain 3th rae tas Sie pee ne ienrin es Rete Same erin 5), 6Conditions expectation ee Koa operating i, are constants Bevis hr F) = eB IF) +8 F). 63) ‘Ty prove thie, we er cay note that the sight hand side ie mensrable eh epost 0 Xi, Xm and eaiic (61) "Tan wot two properties cab Se derived smal FY He zeny a fancon of Xy-- yy them BY LFA) =¥, 69 Porany ¥,ifm Olek Se the paral uD ‘Then Ala = Sy ~ ns mastngle with fees o Fy he laformation sae yn cea hl cam ely be eed by ing Example of ‘Section 5, ‘(Mya | Fa) = B(Guta ~ (2+ DL Fe) = BUS | Fe) OND (Sate) = (04 a= Me ‘In paiea, p = 0, hen Sy ew mating with pet 10 Fe Bxample 2. Te following version ofthe “mastngl betting sae cay” whieh way to but fe ame. Supper Xa ade. Si random vesinbls with Po = ape ‘Wo as thnk ofthe rade vale: a5 the rests of gms whee 008 [bc cn sed ns goes up bead and Jones 1 ome pte BESS beat te game to ep doubling ou bt wat we eentualy or fe plat me sop Let We dnote the winnings (or lass) tua nfs ofthe on ang tie taragy, Wo = 0, Wheaees we win (ne stop Paying, on our wining atop cangg nd PUWynn = 11M 1} = iow suppost the int n ips ofthe cin have turned up talk, Aes each [ip we bv da or beso we hve et ar 2a4bs0-4 2-1 = 28— 1 ‘Sus and We = -(2" 1), At the te we double our bet again sd Seas 2 onthe net ip, This ie (Wass = 11 Wa=—(2"=D) eWay =~ Tse then ny to wal hat Ways Fe) = Was sd ence Wa a martingale with ope 10 Fy. ‘Example 3 Wecan gveize he provos example. Suppowe Xa, Xa sin mf Beary 2 Spottt onthe nth fp mesake «bet eal a $a ts determining he mont of he et we ny lok he ren of Sd (ex It ips bat camot look beyond that. In ater word, By ‘Pano vviable wenerole wih pet to Fs (ve sume tha Be Jue conat), The wining afer» ip, Wy are ve by Wo = 0 and w= Sas “Assume that (Ba) a) he indicator funtion ofthe eet (> 2) i anurable with respect to (nce we tod aly the nor "anion up tiough line's to deurmie if we hawestoppd ty ine »).lc tt ann vila wich ua My #4 we wae aera Sager = 3h Leto tal the contol expectation wth rept to Fi—iv Be | Fea) = BME = Fs) + Sawottr=n ite Por j £H— 1 Myl(D = 9) ts Fenn earls eos B(GT = 4) | Faca) = MEE = 3) ioce 7 x known to be no mote than, the event (T =X} i the same ore ne {Hei}, The ter ewan is mearorable with pesto Prot Hence, ing (61), Btat(=K)| Fes) = BUMatE > K-19 Fes) 1@> K-Fed HE> KTM ‘Th nt eq foto roma the fact the M38 mating, Tere, pode Fuca) = HPs RDM + Ete =) pitts KB t DMP <3) we work hrogh his aeguntent again, thi tine conditioning wth pest Blate| Fes) = BUEN Fe) Uo > K-3)Meat DME We can contin his proses wat wt Ber |) = Me ‘Thee are many examples flatzost where te stoping tine Tot aE Suppose Pra stappig time with F(T < o) = 1 iy ale carr eee tas one voy evenly (Tbe te aalatd Yo ie at. Rea cog stegy sen tis concn). Whan an we ooaeade sous BU) ‘Tye mla(Pn)- Note thet Mp = Mr, + Mot (0 > 9} ~ Malt >) Br) = B(Mr,) +B(UeI(P > a) B(Mo (E> oD Since 7 sa bounded topping tis, flows fom he above at BM {E(Mg) We woul Mo bo able wo ay that the other wo era do not sthute men + on, The me term is aot much of «problem. Since he preebaty of the event (> guna to O an ~ co, mea tke Ing the expect ofthe random valle Mr etic to sal aod ‘ie One can show (ve Seton 8.) thet IF P(r) < 0 then Biwete >) 0 (ied tenn 'o mor trouble, Consider Example 2 of Scton 8.2 gan Tn eis exon the vet {7"> =) te event thatthe st Sige ‘eta and tne probably 2 1 Gs ove crus the Beitr ba as Sool of dal, ie, My =1~ 2 Hence B(Myl(T> of) = T°0-2), which dost not goto 0 asm co, Thi why the dese est fafa This cre, Home, fq and 7 are ven 9 hak Ba, sOMele(r >) hen we witb ble to consid that (Az) = Bo). We sums this Optional Sampling Theorem. Suppose Mo, Mi... satnge itt tegert to Xo aad Pisa stopping tine etisying PUT < shen ie) <2, 60) wd Ee BUI > =D. ey ‘The, Bile) = BUM). ‘Example 1, Tet Xp beep rand walk (9 = 1/2) 08 (0,30) swt nburbing bouadaee, Soppom Xy = ex Then Xq i © marine Terenas = Dor). ea supping tne, and oer X— Some (£10) aad (611) are aati [note shal (610) apd (2) oie titel i ibe maringae Se Bounded and POT < 00) = IBut in this case S(t) = PU p=). There, Pitre N) =X, ‘This fe nother dation of the gable’ ra el for sponds ‘Example 2. Let be asin Exampl 1 and fet My = X2— a. Then ‘ya martingale wih respect lo qT ee Ui ole Chat by Beane 2, Besion 1, Bs | Fa) = BOUEya~ (042) Fah = AE HA~ (242) = Me Ani, Jt = mings: j= O01}. In thi cate Mi nota bounded ‘masngale ot aot nsdn that (5.10) and (811) hl. However one fa prove (Seer 17) tl thee ext Coo and p <1 sath tt Pinan} sor Since [Mal $+, on6 can then show ta (A) < oo and (MGIEET> 9}) < CoM? + 9) 0. ‘ence the condone of she optional sampling theorem hold and we can tovcinde Br) = Be) = 6 Not hat B(e) = BAY) 8(0) = NPP Xe = W) —BUE) = 08 - BC, Benes, aa) Bxample 3. Let X00 simple random walk (p= 1/2) onthe lange J rit =. Wo bave soe tet the 8 aga } Sine spe Foedom walk rourent, PUP < “Toeeor, he onan ofthe sonal ssn theoren must Dot hol ‘We wil not give tbe del hare bat i can be shows in th cave that P(t'> s} ex forme conan By te nr it torr, the {andon walk ten to goa tence of onder inn stops In ts coe BUX > n) dow bot go 0. ‘54. Uniform Integeability Cndisin (6:11) eaen hard wo verify Fortis razon we wou sto ‘eseribe some conltions that imply (631) that maybe ease to wry. ee, eee ‘se sueuet oy aa f° ee. Sib (41>) he eet on wc ae ty ey ow a (e)=0. BOX|ta) ¢ BUXIELX| > oP. ‘oom this wo can ce he following ea random variable with Bx) < gyi tomy c> Otani 2 > Dh hr (A) thn E(u) 0 tare ei 5 Occhi P(A) <6, then xl) ce x) for each. Tho ly pat ofthe dion ae tit 8 may not depend on ‘sn (612) mat Sole forall anes fm "To davlp 1 vane ofthe defation we wil at by giving an example of random von thas ae ot uaimiySntegrabe.Conlér Baal 2 of Soon 5:2, the martingale ting sregy, and conse oe tendo ails 7, is Way ne let Aye te ewnt (Ea = Hq = a1) then PCa) = 2% sod (Walla) = 2G" ~ 1) = 1. We ‘Abr Canntsaiey te conditions for wif integra fa any «<1 Now supooe tht iff,» wll integrable atingal with sspet to py. -- and Tea sopping Ue wie PT oo} = 1. Then ampere dace by uno intarabiy Be Flare > w= 05 {hati (6.1) holds We can sherfre gve soother sateen of he op- ‘onal ampli Sewea. : Optional Sumpting Theorem. Sapniee MyM. wiry ae ‘egrale matings wt respest to Xq, yaad 7 la stoping tine satnping P(T < co) = 1 a6 Me) c. Then (Br) = 3) ‘Th cndilon of unfon ntapabity can be ito wel, Tare ae ‘number of exer conte that np ually, Wo mention oe ae and give another in he excels (Exc 83), eet, 1 Xa neuen of rnd arable and thee exits 8(© < co much that B(X2) < € foreach n then the sence ssf lasgrble “to pow he fit etc > De given and nt 6= 2/40. Supe P(A) < «Ten (ella) Bibealr(A eben 2 26/€) Te lFA teal < 26/0) SG POVBIDAAPACA NM 220799) “(a0/eP(A al < 20/2} (ornopelx) + QC HAP) <« Example 1. Random Harmonie Serls. 1s wall kaows tht he ass Tah bn diverge sil te aerating barmonie PETE PEG A! Svenpee: Whe the ples and slau are ‘Sota fandba to alti et 1... be Independent rand ‘ible we PO, ye bet Bg 0 a oe mo = PO al ‘By ample 1, Section 2, My i martingale Sine B(M w(t) = Vertu) Lvegu-E 4 =F ), to eT = mim). The Sr m >, the oslo sampling theorem atest Ptr. Brg) 2 B(Maul{® $ ml) = Bete Sm) 2 OPET Sm) Hee, Pur sm} caf Sine thsi tr foram, Piro) Saft “rhs says tht wth probably ofa let 1 (( the proportion of 08 Tis Seer pts hgh 8. Now upon he proportion ord bal does Paper thon: Whot then tbe probaly that the operon ose ‘fown iow o sain? By the same asyumont ape to che propaton of ‘om bo we cna that the proba of dropping blow tae Ty's) By sotnung ths aumen, we canto thay arin eee atliy sung above, ea el © ogi, ten above 8, Shen ‘hw or fotal f tmes, canbe bounded above by My) GO = — hich tends te 0 06m, Hence, the proportion dos nt fuchnte Tbutly often uewceaw end Since a apd 8 ze brary, this some ‘ait mpouie for tbe proportion te Sactuste fly oen between {ny ive suber, oot wer he Ii Mea =i Me exits, Te iit May ran vsinbl 8 not difict w hom (ne ‘Eerie 9) at bassunirm etebatin x 1). ‘Wea ea general rs. Martingale Convergence Theorem. Sppose Mo, My. 8 ma single with respect to Noy), soc Sal there exe 1G <2 with [Bll © forall ‘eu thee ext random vail My such at My Mew‘The proof of hs harem sae othe argumest shor. Wht we wll show it nt fe ewary Den << oo the peobabty thot tie acgale Tucks ifately ten betwen e andi 0. Sine Oe wil be ewe very a <6 Itmust be tho cam tat tbe martini My converge to soe ‘alte Ie ica b We loader the allowing btn stratgy wish i Intent of the martingale being stag. We hn of By maging the ‘cumultive results of spe a gaseand Nig, bag he a te gue st tine n+ 1. Whenever fs (6~ EU) ~ BUM ~ al. Sloe 58a ~2l) 5 ECM) +05 0-40, wo et BW) $= @)*LB(H) +0 +3. Sine this oli for every, the expected numberof upeeniags p 10 ‘ny bounded and bens wth pobaiity 1 the suber of opens Sli Ths rv tere FE folows imately fom Ube marigale property that for evry my (tq) = Big). es not vecasarly tev, however, that (Mon) = ‘B(iig)-Fo a eouatererampie,weretara oto maraagale being tice. Wn = Ji Wa =, sd hance (Wen) f (We) = 0. he mastngl isfy ingle Tes ev that te iting random vaable ha he same expectation (oe Bei 8) MAREGALE CowvaceNce TONER En ‘Pac IM ony iategbie mating wt pet Xe, X ats 20d Bn) = BH), Example 4. Lat X5 be the number favo the nt gosraton of branching ross whe olseogdstibiion as nas sat eae gh Aasame Xp = abd let My = p-*Xn be the eaodated wanton Un S 1, we kaow tha exneion Goss with probelste 1 eat Men, MTS ZOE i ae BUN) # Bile). Seton 84 we soto tit a eatery integrable > Jan hace Me ea sonaiees ‘dor vibe with et X20... be independent reson vais with = aad ee 1 as ood in Seton 6.4 hat My uso ineyabe martingale Hence Ma approaches a random wtih Mee Ths cae thet Cet Darmonie wits converge with roby a zzmole 3. at 6, be the proportion of red bale x Fly un. thio cae, muppoe that al tine m = O theese # ted al nd n peat (so sar m draws there are n+ tm Bal). Since Nos baud sy tose tnt sunny intogrblearsogie tad Me, sparen, ado variate Mas with E (Mee) ™= B(Aq) = (erm), ees beaks sia th otes of Ma ins beta dation wih fteion Yoed ‘my La, hae deny Tem +2 aay Se Oa, cee ‘This rtibtion aris naturally in Bayan satis Suppose one sin (cist in etinatng the probability pth sone event ora ny as 9 information about then ono might inl sy thet p's e ane sible with 8 ualr datibution os (1, Now supper tat sae an forms the tll +m ~ 2 tines and the eck area Ft teas Te ‘ees patrioedebtin (asag Bayes eer) for tbe prbeniy ene {Gen by the eto dsrbution with praaters and m: I ean cay be ‘uted that the expect vel of «bet anda ase en). Example 4 Lt iy be martingale with reapers to Xo Xi. and IT be stopping tne with POP oo} = 1 Le Ty = wlan} ead Ye Mey Then Yu Yo where Yn = Ap. ho we sow i te ponysapling theorist abays the cave that (Fn) = BY), However, {his re He ula ntepable Example 5, Let Xi be at ardacibe Mariow chain on coustbly Indnte tate pace’ S with ansison function lz) A funtion fe cae armenia i Coteansor 1 Chapter 2 we coosidred the problem of determining wits o tthe ‘ain nas recut, We ow prove one of the aston we made there, Seppone sn ne tate in and iat (2) dente te probity staring Srila «thet oe chlo ev teachs state = Ia thor word, renin 20:%)=2), 1 hee ae) = PET < 00 | Xo= 2) ‘As we ooted hen ae) = Land we) i harmanle wt any = # ‘uppume sow Gat we san find some functonv tha esti: w= cy Of) $2, oy we)= Spleab), 2A x9 107 i deel ng above, ad Ty = inf, om ean cnet = ‘sGlq) antiga with expect to Xo,Xys- Sie» Bound My tem ivgrble nad 2s Mn Ma = Mo nt With (Mu) = (6) fhe cin oro hem PT < oo) = 1 and Mag = ve) = 2 once Xo 2-1 = B{he) = v(e). Toa tbe cui seers tere {2 oe noosa slion Yo equations (5.18) ~ (615) Bxample 6, Let X1,%3,--be Independent random vesabos with DEX. = 8/2) = P= 12} 12. Let My = 1 and for > 0, Mt My = Xy-~-Xq: Note ta B(Ma) = TBs) B(x) = abd fe, notes che infomation continod (teen |Fa) = BOL Kees Fd Ha Kan Fad xy XyB Olen) = Me aooe Ma i ® marnanle with reapactto 2,2. Siace Ml lit) =, de edition ofthe martingale convergace theorem bald nd noe for mame random rsisble Mo My wiry nab? Te avers in fact, the iting random marble Max = and ace B(Man) 2 ‘E(u To sce Ws, conser te logaitunof the martingale, a= Sue, ‘ota it um npn eel it asin wt a rox) tadstadce yn a of gms a ~ ~~ ‘aia 2) = 140) 5D = 6/9 ote mond mem norm bund 5.1 Conder the exparineat of voll two dle, Lat X be the valve of ‘he Set roll snd Y sho sum of ha two den Find B[X |), Le, ge he vafoeof BC |7Y) fe ally. 15.2 Suppor that X, is» Poison process wit pacamiter A = 1 Fad BG 1X3) and 5%) 5.3 Let Xs Xa, Xa. bo indopendent ental dst rand ar abies Let (0) = BUG) be the moment generating funtion of fd eae ofeach X) Fal size m(?) ©. Lat Sp = 0 ad for n> 0, eK tt Ke $s, Show that Af 36 a martnge with rape to 54 Let Xe Xi... bo tbe valu ofa branching pres an Chapter 2, Stoton 2.4, eX gs the aber of indvidul o the th generation. Stppose atte sean mater of pring pe naa Show eat Mg #-7M fa martingale with eect 10 Xo, Xs, 5.5 COMPUTER SIMULATION. Consider the Poly ura model, Sia Inte cis model with a eamputer by starting with enero and ane geen ‘ral and consovig ul the somber of alfa he ra 1000. Noto thofiction of re Dal nthe 100 tlle Do thi imation et 2000 tims ab note How many sien the actin of ed basin th sera [eros) lon-3).-[86/). Bom the sinlstion dst aie s cmlesre 1 to wn the into of the acto of rd all ok he {56 Consider bast random walk on the teas with probably p< 1/2 moving tote ight and probbily 1~ po moving othe el Lat ‘Sh be tho vale ot tne mand smme tha So whore 00 <1 "e} Show tht iy =[G—p/p" wa marge (G) LACT be the Bt se thar the rao wal reaches Oo 1, To min(n 5,000 ¥). ‘Use optonl sampling othe martingale My to compete P(S(7) = 0} 7 Let Sy bea a erie 58 (ey Stow the fy = 5q + (2~Zp iso mating. (G) LET beth Be thr the random wal reaches Oo T=minga 5,200), Let = mia} 08 lt be he martingale Zy = Mr Show that thre ois a Cc co wach thal B(Z3) < for alm, You may wt to se (@) Avoly the optional sampling theorem to B(Mn) and ue this wad tho remit fom Era 56 to Sad the expected suber of sap Ua serpioor (8). 5 Let X be the mumber af nivalis the nth generation of branch Ing ross in which each ncvdual produces ospig from a diteouion sith me and arene o?, We have psn provi tat My = Hy Bewertingale oo (a) Lat denote be ffoemation costae a Xgy. sy Xy, Show tat UE | Fa) Hx 0h (©) Suppo» > 1. Show that thr xis a 6 < oo wc hat or all Bad) 02%, 3 and for > 1 et Ta minin> 0: Xn gor,In other words, 7, menezes th amouat of tine etwoen he (— et era sd he fh return to sae in pneal teu to determine the nebo faneba Pfr 7 given te santo mati for tbe das. We eed previo oe (11) tat am)= 5, whew + denotes te via probity meamze fr the chain. I we Tarend atthe cai some stata y = w oa dete ¥ emin(a > 0:4==h Tyamin(a> 0: Xren =a), snd realy, eamlnge> 0: Ayemecenertn ? Bxample 8. Le X, bean treduble, pone recatet,consiouoae sine Mask nin starting inate =. Debe Ry = hne> 0: X89}, Siatatle> Re Xe=e}, sadn goog, Romintle> 0: Xn yamine #eh Se=int{e> 0: Xneninavonne 2h ‘The random varables Rs ae exponential with paramatera(s), the rate at ‘whch the chia hanging fom rate =. Tho datbuin ofthe Sy, A80| ‘once tho Fina oeay to determin. Example 4. ¢/9/1 Queue, We wil imagine tht wo ba « qsoe witha engi serve, Cutters sie sosorng to a Peon proces with {hts J, inthe waling ines beteoon customer arias ee independent ‘expen ndom variables with parsetar We wil sess tht he ‘erie tines customs ao ladeprodent,seutllydstrbuted random ‘iby wih meas. However we wil at same tht th vie es [e expneatal in moet camer of inert one done aot pet at the terre imesh hse the ls of memory” property 0 a espe ‘ltstbution i at appropriate). The @ in MG stance for “genre” (evn dsutbution. Two let, dante the number of peopel the queue, then ¥% i ot 1 Matty procme: However tery 8 natal renal proces 0 Fa ssocate with the que. Suppose Yo = 0, Let Byminffe> 0: =2), B= Iall¢> 0:Yiyoe Oh Tham+s, Sindy, we dee or > 1, Heminile> O:Ypeamiee= Vy Smtale> 0: Fneynsemie =O), T=Rt ‘ote thatthe variables R, are epson wit ate butte debaon the Scan be very complied Nemrthle ner te smsmpien tat BR) < oo, we cans that TT, sai the conditions forsee roc. We ca think ofthe ie ropeeested bythe Ry othe “ie ina” ‘od the te reprseated hy the 3 te “sy ines” Suppose we have 8 renewal poe renewal process coreponing tothe random aries yt. tn gor, Myf at a Mariow roe; certo Pred whes the net oerureace Wl happen we not te now whe tie Jae ccrartnce tok pon For his eso eral const ge anf manemn HSE ‘Tho proses (Na Ay) ca be thought ofa Matloy proces. Th Poison ‘cms a special extmpleof ene proces tate atk prowmay {oe the Pisa proces the probability a an evet acct n teaser (6t-+ di independent of Ay Tae fla eon the Moe ef mama ope ace wh th exon dato ‘The Sn eu or pew proces wil be te alge of th (srg) uw of nrge somber. eal hat the nw of age ambers ate hat probity, jg htt 1m trae ofthe esewal proces Ny this sates tet fo ll ¢ > 0m i lily lege Mons 2% aulvleiy, or alle > 0,1 ¢kefiictly laree,