Index Compression. Prior to V9, DB2 only supported compression for tablespaces In V9, we introduce Index Compression

Transcription

1 Compression Prior V9, DB2 only supported compression for tblespces In V9, we troduce Compression 14

2 Compression L CREATE UNIQUE WHERE NOT NULL ASC tble-nme ( column-nme DESC RANDOM expression COMPRESS NO BUFFERPOOL bp-nme COMPRESS YES, INDEX dex-nme ON ) You enble compression by CREATg or ALTERg dex COMPRESS YES Compression is not supported some cses: For versioned dexes YET. For dexes 4K bufferpools You my lso ALTER n dex chnge COMPRESS option -676 SQLCODE for non-db2 mnged dtsets with n comptible CISIZE -676 SQLCODE when lterg COMPRESS YES with 4K bufferpool You cn lter bufferpool nd COMPRESS option sme sttement 15

3 Compression How it works re is NO COMPRESSION DICTIONARY Compression is done on fly es will when itilly populted A Hybrid pproch is used In bufferpool, pges re physiclly expnded pge size bufferpool 8K, K, or 32K On disk, pges re lwys 4K

4 Compression How it works compression you cn chieve is limited by bufferpool you use For 8K, up 51% compression For K, up 76% compression For 32K, up 88% compression DB2 s tht bufferpool pges will on pges on disk. If dt doesn t compress well, nd you use lrge pge sizes, you my wste bufferpool spce It is VERY IMPORTANT choose n pproprite pge size Wht or fcrs ffect choice pge size for dexes??? Assume Keys compress 4:1 BP s 8K K 32K Disk s 4K 4K 4K 17

5 Compression Wht consider when choosg pge size compressbility dt DSN1COMP helps estimte compression rtios Worklod/Usge Ptterns on dex s re de on red nd on write Synchronous I/Os for Rndom Accesses my cur synchronous CPU penlties Lrger pges tke longer decompress Prefetch I/O for dex scnners cn decompress synchronously Still uses CPU, but doesn t ffect elpsed time In fct, compression my improve elpsed time for I/O bound queries 18

6 Compression Considertions pool considertions Only s re Only 4K non-lef will used s non-lef bufferpool spce don t pply for dexes Compressed dexes will much like non- dexes with 4K pges non-lef dt typiclly occupies smll percentge dex spce Hit Rtio bufferpool If you get high buffer hit rtios, compression overhed is mimized As long s pge rems bufferpool, it stys un Rules Thumb Consider usg compression for wrehouse pplictions Be wre CPU overhed for OLTP pplictions st wy understnd impct is try it out! 19

7 DSN1COMP DSN1COMP provides n estimte how well n dex will compress Estimtes how bufferpool will for given pge sizes i.e. will buffer spce wsted for dex DSN1COMP DSN1COMP JOB JOB 'USER=$$USER','<USERNAME:JOBNAME>', 'USER=$$USER','<USERNAME:JOBNAME>', MSGCLASS=A,MSGLEVEL=(1,1), MSGCLASS=A,MSGLEVEL=(1,1), CLASS=A,USER=ADMF001,PASSWORD=GOODSTUF, CLASS=A,USER=ADMF001,PASSWORD=GOODSTUF, REGION=4096K REGION=4096K JOBLIB JOBLIB DSN=USER.TESTLIB,DISP=SHR DSN=USER.TESTLIB,DISP=SHR DSN=DB2A.SDSNLOAD,DISP=SHR DSN=DB2A.SDSNLOAD,DISP=SHR /*ROUTE /*ROUTE PRINT PRINT STLVM14.BLYLE STLVM14.BLYLE ************************************************************ ************************************************************ * * STEP1 STEP1 - - Run Run DSN1COMP DSN1COMP on on dex dex ************************************************************ ************************************************************ STEP1 EXEC PGM=DSN1COMP,PARM='LEAFLIM(1000) SYSPRINT SYSPRINT SYSDUMP SYSDUMP SYSABEND SYSABEND SYSUT1 SYSUT1 DSN=TESTCAT.DSNDBD.DB1.I1.I0001.A001,DISP=SHR DSN=TESTCAT.DSNDBD.DB1.I1.I0001.A001,DISP=SHR SYSUT2 SYSUT2 LEAFLIM option limits how mny lef pges re exmed If not specified, ll lef pges dtset will used 20

8 DSN1COMP Exmple Tble Smple Tble for Compression Tests Rndomly nd sequentilly Generted Dt Lrge Tblespce 1 Million Rows 7 columns RegionId Integer 10 distct vlues SreId Integer 50 distct vlues DeptId Integer 50 distct vlues BigSeqBigInt +1/row Unique Seq. Stte Chr() 50 distct vlues City Chr() 50 distct vlues Dte Dte +1 dy/1000 rows on vg CREATE CREATE TABLE TABLE T1( T1( regionid regionid INTEGER, INTEGER, sreid sreid INTEGER, INTEGER, deptid deptid INTEGER, INTEGER, bigseqno bigseqnobigint, stte stte VARCHAR(32), VARCHAR(32), City City VARCHAR(32), VARCHAR(32), dte dte DATE DATE )) ITION ITION BY BY (regionid) (regionid) ( ( 1 1 (100), (100), 2 2 (200), (200), 3 3 (300), (300), 4 4 (400) (400) ) ) IN IN DB1.TS1; DB1.TS1; 21

9 DSN1COMP Exmple Dt INSERT INTO T1 INSERT INTO T1 with dt(regionid, sreid, deptid, bigseqno, with dt(regionid, sreid, deptid, bigseqno, stte, City, dte, rstte, rcity, r1000 ) s stte, City, dte, rstte, rcity, r1000 ) s ( SELECT t(rnd(10)*10), t(rnd(11)*50), t(rnd(12)*50), 1, ( SELECT t(rnd(10)*10), t(rnd(11)*50), t(rnd(12)*50), 1, 'Cliforni', 'Americn Cnyon', current dte, 'Cliforni', 'Americn Cnyon', current dte, t(rnd(15)*50), t(rnd()*50), t(rnd(17)*1000) t(rnd(15)*50), t(rnd()*50), t(rnd(17)*1000) FROM SYSIBM.SYSDUMMY1 FROM SYSIBM.SYSDUMMY1 UNION ALL UNION ALL SELECT t(rnd(1)*10), t(rnd(2)*50), t(rnd(3)*50), bigseqno+1, SELECT t(rnd(1)*10), t(rnd(2)*50), t(rnd(3)*50), bigseqno+1, cse rstte cse rstte when 0 n 'Albm' when 1 n 'Alsk' when 0 n 'Albm' when 1 n 'Alsk' when 2 n 'Arizon' when 3 n 'Arknss' when 2 n 'Arizon' when 3 n 'Arknss' when 4 n 'Cliforni' when 5 n 'Colordo' when 4 n 'Cliforni' when 5 n 'Colordo' when 6 n 'Conneticut' when 7 n 'Delwre' when 6 n 'Conneticut' when 7 n 'Delwre' when 8 n 'Florid' when 9 n 'Georgi' when 8 n 'Florid' when 9 n 'Georgi' when 10 n 'Hwii' when 11 n 'Idho' when 10 n 'Hwii' when 11 n 'Idho' when 12 n 'Illois' when 13 n 'Indin' when 12 n 'Illois' when 13 n 'Indin' when 14 n 'Iow' when 15 n 'Knss' when 14 n 'Iow' when 15 n 'Knss' when n 'Kentucky' when 17 n 'Louisin' when n 'Kentucky' when 17 n 'Louisin' when 18 n 'Me' when 19 n 'Mrylnd' when 18 n 'Me' when 19 n 'Mrylnd' when 20 n 'Msschusetts' when 21 n 'Michign' when 20 n 'Msschusetts' when 21 n 'Michign' when 22 n 'Mnesot' when 23 n 'Mississippi' when 22 n 'Mnesot' when 23 n 'Mississippi' when 24 n 'Missouri' when 25 n 'Montn' when 24 n 'Missouri' when 25 n 'Montn' when 26 n 'Nebrsk' when 27 n 'Nevd' when 26 n 'Nebrsk' when 27 n 'Nevd' when 12 n 'New Hmpshire' when 29 n 'New Jersey' when 12 n 'New Hmpshire' when 29 n 'New Jersey' when 30 n 'New Mexico' when 31 n 'New York' when 30 n 'New Mexico' when 31 n 'New York' when 32 n 'rth Crol' when 33 n 'rth Dkot when 32 n 'rth Crol' when 33 n 'rth Dkot when 34 n 'Ohio' when 35 n 'Oklhom' when 34 n 'Ohio' when 35 n 'Oklhom' when 36 n 'Oregon' when 37 n 'Pennsylvni' when 36 n 'Oregon' when 37 n 'Pennsylvni' when 38 n 'Rhode Islnd' when 39 n 'South Crol' when 38 n 'Rhode Islnd' when 39 n 'South Crol' when 40 n 'South Dkot' when 41 n 'Tennessee' when 40 n 'South Dkot' when 41 n 'Tennessee' when 42 n 'Texs' when 43 n 'Uth' when 42 n 'Texs' when 43 n 'Uth' when 44 n 'Vermont' when 45 n 'Virgi' when 44 n 'Vermont' when 45 n 'Virgi' when 46 n 'Wshgn' when 47 n 'West Virgi' when 46 n 'Wshgn' when 47 n 'West Virgi' when 48 n 'Wiscons' when 49 n 'Wyomg' when 48 n 'Wiscons' when 49 n 'Wyomg' else 'Unknown' else 'Unknown' end, end, Insert Sttement for tble cse rcity cse rcity when 0 n 'Adeln' when 1 n 'Agour Hills' when 0 n 'Adeln' when 1 n 'Agour Hills' when 2 n 'Almed' when 3 n 'Albny' when 2 n 'Almed' when 3 n 'Albny' when 4 n 'Alhmbr' when 5 n 'Aliso Viejo' when 4 n 'Alhmbr' when 5 n 'Aliso Viejo' when 6 n 'Altur' when 7 n 'Amdor' when 6 n 'Altur' when 7 n 'Amdor' when 8 n 'Americn Cnyon' when 9 n 'Anheim' when 8 n 'Americn Cnyon' when 9 n 'Anheim' when 10 n 'Anderson' when 11 n 'Angels Cmp' when 10 n 'Anderson' when 11 n 'Angels Cmp' when 12 n 'Antioch' when 13 n 'Apple Vlley' when 12 n 'Antioch' when 13 n 'Apple Vlley' when 14 n 'Arcdi' when 15 n 'Arct' when 14 n 'Arcdi' when 15 n 'Arct' when n 'Arroyo Grnde' when 17 n 'Artesi' when n 'Arroyo Grnde' when 17 n 'Artesi' when 18 n 'Arv' when 19 n 'Atscdero' when 18 n 'Arv' when 19 n 'Atscdero' when 20 n 'Arn' when 21 n 'Atwter' when 20 n 'Arn' when 21 n 'Atwter' when 22 n 'Auburn' when 23 n 'Avlon' when 22 n 'Auburn' when 23 n 'Avlon' when 24 n 'Avenl' when 25 n 'Azus' when 24 n 'Avenl' when 25 n 'Azus' when 26 n 'Bkersfield' when 27 n 'Bldw Prk' when 26 n 'Bkersfield' when 27 n 'Bldw Prk' when 12 n 'Bnng' when 29 n 'Brsw when 12 n 'Bnng' when 29 n 'Brsw when 30 n 'Beumont' when 31 n 'Bell' when 30 n 'Beumont' when 31 n 'Bell' when 32 n 'Bell Grdens' when 33 n 'Bellflower' when 32 n 'Bell Grdens' when 33 n 'Bellflower' when 34 n 'Belmont' when 35 n 'Belvedere' when 34 n 'Belmont' when 35 n 'Belvedere' when 36 n 'Benici' when 37 n 'Berkeley' when 36 n 'Benici' when 37 n 'Berkeley' when 38 n 'Beverly Hills' when 39 n 'Big Ber Lke' when 38 n 'Beverly Hills' when 39 n 'Big Ber Lke' when 40 n 'Biggs' when 41 n 'Bishop' when 40 n 'Biggs' when 41 n 'Bishop' when 42 n 'Blue Lke' when 43 n 'Bly' when 42 n 'Blue Lke' when 43 n 'Bly' when 44 n 'Brdbury' when 45 n 'Brwley' when 44 n 'Brdbury' when 45 n 'Brwley' when 46 n 'Bre' when 47 n 'Brentwood' when 46 n 'Bre' when 47 n 'Brentwood' when 48 n 'Brisbne' when 49 n 'Buelln' when 48 n 'Brisbne' when 49 n 'Buelln' else 'Unknown' else 'Unknown' end, end, dte + cse r1000 when 0 n else end, dte + cse r1000 when 0 n else end, t(rnd(21)*50), t(rnd(22)*50), t(rnd(23)*1000) t(rnd(21)*50), t(rnd(22)*50), t(rnd(23)*1000) FROM dt FROM dt WHERE bigseqno < WHERE bigseqno < ) ) select regionid, sreid, deptid, bigseqno, select regionid, sreid, deptid, bigseqno, stte, city, dte stte, city, dte from dt; from dt; 22

10 DSN1COMP Output Prt 1 CREATE CREATE UNIQUE UNIQUE INDEX INDEX I1 I1 ON ON T1 T1 (bigseqno) (bigseqno) PCTFREE PCTFREE 0; 0; Lrge Lrge Tblespce Tblespce 1 Million Million Rows Rows 1 Key Key Column Column BigSeq BigSeq BIGINT BIGINT +1/row +1/row Unique Unique Seq. Seq. DSN1944I DSN1944I DSN1COMP DSN1COMP INPUT INPUT PARAMETERS PARAMETERS PROCESSING PARMS FOR INDEX DATASET: PROCESSING PARMS FOR INDEX DATASET: NO NO LEAFLIM LEAFLIM WAS WAS REQUESTED REQUESTED DSN1940I DSN1940I DSN1COMP DSN1COMP COMPRESSION COMPRESSION REPORT REPORT 4,220 4,220 s s Processed 1,000,000 Keys Keys Processed 1,000,000 Rids Rids Processed,602,602 KB KB Key Key Dt Dt Processed 7,837 7,837 KB KB Compressed Keys Keys Produced 7,837 /,602 = 47.2% Doesn t count non-lef or pge heder overhed This gives n ide how well compress More detiled nlysis follows. 23

11 DSN1COMP Output Prt 2 CREATE CREATE UNIQUE UNIQUE INDEX INDEX I1 I1 ON ON T1 T1 (bigseqno) (bigseqno) BigSeq BigSeq BIGINT BIGINT +1/row +1/row Unique Unique Seq. Seq. te se re pproximtions My few percent f Depends on how re lid out on pges Doesn t consider non-lefs Don t expect exct numrs For this dex, 8K might good choice EVALUATION OF COMPRESSION WITH DIFFERENT INDEX PAGE SIZES: EVALUATION OF COMPRESSION WITH DIFFERENT INDEX PAGE SIZES: 8 8 K pproximtely pproximtely pool pool K pproximtely pproximtely pool pool K pproximtely pproximtely pool pool 24

12 DSN1COMP Exmple 2 CREATE CREATE INDEX INDEX I2 I2 ON ON T1 T1 (( regionid, regionid, sreid, sreid, deptid, deptid, bigseqno bigseqno )) ; ; RegionId RegionId Integer Integer distct distct vlues vlues SreId SreId Integer Integer 50 distct vlues 50 distct vlues DeptId DeptId Integer Integer distct distct vlues vlues BigSeq BigSeq BigInt BigInt +1/row Unique Seq. +1/row Unique Seq. For this dex, K good choice EVALUATION OF COMPRESSION WITH DIFFERENT INDEX PAGE SIZES: EVALUATION OF COMPRESSION WITH DIFFERENT INDEX PAGE SIZES: 8 8 K pproximtely pproximtely pool pool K pproximtely pproximtely pool pool K pproximtely pproximtely pool pool 25

13 DSN1COMP Exmple 3 CREATE INDEX INDEX I5 I5 ON ON T1 T1 (regionid, stte, stte, city, city, bigseqno) RegionId RegionIdInteger Integer distct distct vlues vlues Stte Stte Chr() Chr() distct distct vlues vlues City City Chr() Chr() Distct Distct vlues vlues BigSeq BigSeq BigInt BigInt +1/row Unique Seq. +1/row Unique Seq. K nd 32K re both options Wht s more Importnt you? Disk spce or pool spce EVALUATION OF COMPRESSION WITH DIFFERENT INDEX PAGE SIZES: EVALUATION OF COMPRESSION WITH DIFFERENT INDEX PAGE SIZES: 8 8 K pproximtely pproximtely pool pool K pproximtely pproximtely pool pool K pproximtely pproximtely pool pool 26

14 Compression Summry Compression Dictionry is used choice bufferpool is very importnt Too smll nd you limit your compression rtio Too lrge nd you wste bufferpool spce DSN1COMP helps you choose Effects on system performnce vry Compress/Decompress is done t I/O time My synchronous or synchronous For -memory dexes, overhed is miml Once pges re de, y rem de bufferpool 27