Lecture6: Lossless Compression Techniques. Conditional Human Codes

Size: px

Start display at page:

Download "Lecture6: Lossless Compression Techniques. Conditional Human Codes"

Agnes McKenzie
5 years ago
Views:

ecture6: ossless Compresso Techques Codtoal uma Codes -Cosder statoary dscrete arov process, = { s, s, s } wth codtoal pmfs P P s s wth,, tates o Ps/so.9.5.

1 ecture6: ossless Compresso Techques Codtoal uma Codes -Cosder statoary dscrete arov process, = { s, s, s } wth codtoal pmfs P P s s wth,, tates o Ps/so Ps/s Ps/s The margal probabltes ca be derved form these codtoal oes: Ps The etropy of the source for the gve margal pmf o codto P log P =.6444xlog xlog xlog. =.575 bts/symbol Ps =.575 /3/6 ECTURE

2 ecture6: ossless Compresso Techques Codtoal uma Codes -The uffma codes for margal pmf o codto ca be obtaed as: o x.444x.x It s clear that t s bouded by P max Ps = /3/6 ECTURE

3 ecture6: ossless Compresso Techques Codtoal uma Codes -The codtoal etropy of gve evet - =s.e., the etropy for each state aloe / s P s log P s o P s Ths defto cocerg that the prevous value of the source codto s sure evet s o or s or s wth probablty o log P s =-.9log.9- x.5log.5=.569 bts/symbol -mlarly =.884 bts/symbol, ad =.357 bts/symbol o s Etropy Ps/s o /so=.569 bts/symbol Ps/s /s=.884 bts/symbol Ps/s /s=.357 bts/symbol Ps =.575 bts/symbol /3/6 ECTURE 3

4 ecture6: ossless Compresso Techques Codtoal uma Codes -The codtoal etropy of gve evet - =s.e., the etropy for each state aloe / s P s log P s Ths defto cocerg that the prevous value of the source codto s sure evet s o or s or s wth probablty o P s o log o Ps/so Ps/s Ps/s P s =-.9log.9- x.5log.5=.569 bts/symbol -mlarly =.884 bts/symbol, ad =.357 bts/symbol /3/6 ECTURE 4

5 ecture6: ossless Compresso Techques Codtoal uma Codes -The uffma codes for each codto aloe ca be obtaed as: For - =s o For - =s For - =s o o o o -The average code legth of each state aloe s s gve by o.9x.5xx..5x.8x.5x..5x.5x.6x. 4 -It s clear that they are bouded by /3/6 ECTURE 5

6 ecture6: ossless Compresso Techques Codtoal uma Codes -The average value of the etropy at each state of arov source s Average arov P s arov =.6444x x.884+.x.357=.733 bts/symbol o s Etropy Ps/s o X C /so=.569 bts/symbol Ps/s X C /s=.884 bts/symbol Ps/s X C /s=.357 bts/symbol Ps =.575 bts/symbol + + Average arov =.733Bts/symbol -Wth resultg average code legth of arov source Averagearov P s Average arov.6444x..444x..x It s clear that they are bouded by Averagearov Averagearov Average arov ECTURE /3/6 6

7 ecture6: ossless Compresso Techques Codtoal uma Codes Cocluso o s Etropy Ps/s o C /so=.569 bts/symbol Ps/s C /s=.884 bts/symbol Ps/s C /s=.357 bts/symbol Ps =.575 bts/symbol Average arov =.733 Bts/symbol No codtoal = Codtoal average arov =.733 arov.578 /3/6 ECTURE 7

8 ecture6: uffma Code Exteded uffma Code I applcatos where the alphabet sze s large; P max s geerally qute small, ad the amout of devato of the etropy from the average code legth or terms of a percetage of the rate s qute small owever, cases where the alphabet s small ad the probablty of occurrece of the dfferet letters s sewed, the value of P max ca be qute large ad the uffma code ca become rather effcet whe compared to the etropy Ex: Fd the uffma code for the followg source gve the correspodg probabltes o.8..8 o.8.8. ECTURE /3/6 8..8log. log.8log x.3.x x bts ymbol.8..8x.x.8x. bts ymbol There s a bg dfferece betwee the average code legth ad the etropy Code Redudacy,ρ=.384 bts/symbol

9 We ca reduce the rate average legth by groupg blocgsymbols together whch s called Exteded Bloc uffma Code -Cosder a source that emts a sequece of letters [s, s, s 3, m ] -Each elemet of the sequece s geerated depedetly -The etropy for ths source s gve by N b P P log -Oe ca geerate a uffma code for ths source wth rate R I data compresso t s bts per symbol le the average legth ot bts per secod as commucato such that R -Ecode the sequece by geeratg oe codeword for every symbols, hece there are m combatos of symbols -Deote the rate for the ew source as R, hece R Exteded uffma Code ecture6: uffma Code 9 -R s the umber of bts requred to code symbols together. Therefore, the umber of bts requred per symbol, R, s gve by R R / R -Oe ca prove that R

10 ecture6: uffma Code Exteded uffma Code ece, oe ca see that by ecodg the output of the source loger blocs of symbols ca guaratee a closer rate to the etropy But as we bloc more ad more symbols together, the sze of the alphabet grows expoetally, ad the uffma codg scheme becomes mpractcal ECTURE /3/6

11 ecture6: uffma Code Exteded uffma Code Ex: Fd the exteded uffma code for the followg source gve the correspodg probabltes by blocg every two symbols m=3, =, the umber of possble symbol pars or the exteded symbols are 3 =9 -The probablty of each exteded symbol s calculated by multplyg the probabltes of the orgal sgle tems together -Follow the steps of gettg uffma code, you get the code that table Exteded ymbol probablty model uffma Code s s.64 s s.4 s s 3.44 s s.6 s s.4 s s 3.36 s 3 s.44 s 3 s.36 s 3 s 3.34 ECTURE /3/6

12 ecture6: uffma Code Exteded uffma Code.8log. log.8log x.3.x x bts ymbol ext.78 bts / symbol Each symbol the exteded alphabet correspods to two symbols from the orgal alphabet.78 /.864bts / symbol Redudacy =.45 bts/symbol There s o bg dfferece betwee the average code legth ad the etropy, ECTURE /3/6

13 ecture6: uffma Code Nobary uffma Code Nobary ary uffma Code We ca obta a obary uffma code almost exactly the same way. The umber of symbols to be combed the frst step s determed accordg to N modulo - N s the umber of symbols of the source s the ary requred code If N modulo - = - module - we start to combe - symbols Ex: Fd the terary uffma code for the followg source gve the correspodg pmf symbol Code x..5 o..5 o x x If you dertere code ad calculate of the ary t s 3. /3/6 ECTURE 3

K-Map 1. In contrast, Karnaugh map (K-map) method provides a straightforward procedure for simplifying Boolean functions.

K-Map 1. In contrast, Karnaugh map (K-map) method provides a straightforward procedure for simplifying Boolean functions. K-Map Lesso Objectves: Eve though Boolea expressos ca be smplfed by algebrac mapulato, such a approach lacks clear regular rules for each succeedg step ad t s dffcult to determe whether the smplest expresso