Codes AL-FEC pour le canal à effacements : codes LDPC-Staircase et Raptor

Transcription

1 Codes AL-FEC pour le canal à effacements : codes LDPC-Staircase et Raptor Vincent Roca (Inria, France) 4MMCSR Codage et sécurité des réseaux 12 février Copyright Inria 2016 license Work distributed under Creative Commons (CC) - Attribution-Noncommercial-No Derivative Works

2 Outline 1. the erasure channel 2. AL-FEC codes 3. what is a good AL-FEC code? The performance metrics 4. example: LDPC-Staircase 5. example: Raptor(Q) 3 The erasure channel! erasure channel definition: a symbol either arrives to the destination, without any error or is erased and never received 0 0 erased! 1 1 BSC (binary symmetric) and AWGN channels the integrity assumption is a strong hypothesis a received symbol is 100% guaranteed error free largely simplifies decoding: belief propagation decoding iterative decoding maximum likelihood decoding Gaussian elimination more details to come 4

3 The erasure channel (cont )! where do we find erasure channels? the Internet is intrinsically an erasure channel an IP datagram is either received or erased usually caused by a router congestion, or a routing error because of Ethernet CRC or TCP/UDP/IP checksum verifications when the underlying layers failed to recover/identify errors, the packet is eventually discarded by higher layers checks certain MAC layers can ask for retransmissions, but it's usually not the case 5 The erasure channel (cont )! where do we find erasure channels (cont) due to an intermittent connection model caused by the environment (obstacle in wireless comm.) a mobile terminals receives a subset of the packets sent caused by the application (e.g., if it crashes) packets sent during the off-period are lost these situations are easily recovered with point-to-point connections TCP will do the job if the disconnection is short if a new connection is needed, it s sufficient to remember where the download stopped and ask the sender to continue from that point but it s quite different if the content is broadcast/multicast the same content is sent to 100,000s of receivers! 6

4 The erasure channel (cont )! where do we find erasure channels (cont) with distributed storage of a content (e.g., a file), when a storage point fails RAID disks network-wide distributed storage, where a client selects a subset of the storage node, each of them having a chunk of the content (source or repair data) 7 Quite different from PHY-layer FEC! completely different assumptions! information bits coming from the upper layers, once encoded, form a codeword information bits (e.g. 8 bits) <1,0,0,1,1,0,1,1> FEC encoding PHY-level FEC codeword (e.g. 12 bits) <1,0,0,1,1,0,1,1 0,1,0,0> information bits modified bits coded bits PHY channel introduces errors information bits <1,0,0,1,1,0,1,1> FEC decoding PHY-level FEC erroneous codeword <1,0,0,0,1,1,1,1,0,1,0,0> 8

5 Outline 1. the erasure channel 2. AL-FEC codes 3. what is a good AL-FEC code? The performance metrics 4. example: LDPC-Staircase 5. example: Raptor(Q) 9 AL-FEC code definitions! AL-FEC are codes for the erasure channel only! we always assume there s no error in what we receive codes and codecs are not the same Be careful: codes codecs specifies the way repair symbols are generated implementation of the encoder and decoder 10

6 AL-FEC code definitions (cont )! the symbol is the data unit manipulated by the AL- FEC codec usually a symbol is carried in a UDP/IP datagram either received or erased NB: sometimes there are several symbols per UDP/IP datagram goal is to artificially increase the # of symbols in an object useful since LDPC/Raptor codes perform better when there are more than a few 1,000s of symbols let's keep it simple from now on one symbol per packet initially k source symbols, n encoding symbols after FEC encoding 11 AL-FEC code definitions (cont ) example: source object, of size 5 kb, segmented into 5 source symbols of size 1 kb each, to which FEC encoding adds 4 repair symbols, also of size 1 kb. Source symbols are part of the 5+4 encoding symbols (systematic FEC code). k=5 source symbols source object FEC ENCODING UDP/IP datagram UDP/IP datagram n-k=4 repair symbols n=9 encoding symbols 12

7 AL-FEC code definitions (cont )! the code rate is a key parameter code _ rate = k n = before _encoding after _ encoding close to 1 close to 0 little/no redundancy high amount of redundancy examples: code_rate = 0.5 means that there are as many redundancy symbols as source symbols code_rate = 2/3 means that n=1.5*k, i.e. we add 50% of redundancy 13 AL-FEC code definitions (cont )! systematic codes codes for which the k source symbols are part of the n encoding symbols see previous example preferred to non-systematic codes because: without loss, no decoding is needed save processing a receiver that does not implement the FEC code can still use source symbols (backward compatibility) of critical importance 14

8 AL-FEC code definitions (cont )! ideal or MDS (maximum distance separation) codes a code for which decoding is always possible after receiving any set of k symbols among n possible example: Reed-Solomon codes over GF(2 m ), with m=4, 8 or 16 for instance it s an optimum code in terms of erasure recovery one cannot find something better which does not mean it s necessarily the best possible code for a given use case there are constraints example: RS over GF(2 8 ) have strict limits k n An example of use! trivial example with a (9; 5) code n=9 encoding symbols erasure channel k=5 source symbols source object ENCODING DECODING decoded object n-k=4 repair symbols UDP/IP datagrams 16

9 An example of use (cont )! more complex example (larger object) total of 15 source symbols step 1: step 2: segment into partition into source blocks, given symbols (size the maximum 1000 bytes) k the AL-FEC step 3: block 1: k=8 src sym. ENCODING n=12 encoding symbols n=11 enc. symb. source object (size= bytes) encoder supports (e.g. k max =8) block 2: k=7 src ENCODING 17 An example of use (cont )! broadcast of a digital library to vehicles, using FLUTE/ALC in carousel mode, over a long period multimedia content FEC encoding FLUTE/UDP carrousel broadcast transmission to a multicast group FEC decoding reception multimedia content 18

10 AL-FEC differ from PHY-layer FEC! AL-FEC codes usually implemented in higher communication layers (rather than in the PHY layer), e.g.: within the application within the transport system (between RTP and UDP for streaming flows, in FLUTE/ALC for filecasting applications) within the MAC layer (e.g., in DVB-H/MPE-FEC, or in DVB-SH/ MPE-IFEC) hence their name Application Level-FEC (AL-FEC) also called Upper Layer-FEC (UL-FEC) by those who work at the PHY layer 19 Outline 1. the erasure channel 2. AL-FEC codes 3. what is a good AL-FEC code? The performance metrics 4. example: LDPC-Staircase 5. example: Raptor(Q) 33

11 Performance metrics! how can we define good AL-FEC codes? define performance metrics 34 Performance metrics! metric 1: decoding overhead as a measure of the erasure recovery capabilities major performance criteria since many AL-FEC are not MDS measured as the overhead ratio: #_ of _ symbols _ required _ for _ decoding decoding _ overhead _ ratio = k e.g. overhead=0.63% means that 0.63% of symbols in addition to k are needed for decoding to succeed or as the raw overhead (number of extra symbols): 1 derivatives: average overhead, 99% margin above (resp. below) the average, etc. decoding _ overhead = #_ of _ symbols _ required _ for _ decoding k 35

12 Performance metrics (cont )! metric 3: decoding failure probability = f(number of symbols received) similar to metric 2 but as a function of the number of symbols actually received the curve is an average over a large number of experiments, for a given {code rate, loss rate} decoding failure probability 1 decoding limit, depending on block size fall should happen as soon as possible slope should be as vertical as possible decoding impossible (too few symbols ) k decoding OK with high proba number of symbols received 37 Performance metrics (cont )! metric 5: encoding and decoding speed to appreciate the algorithmic complexity more reliable than theoretical algorithmic complexity analyzes that ignore major aspects (e.g., memory access/cache behavior/implementation details/ ) there is an appropriate balance to find between erasure recovery and complexity some algorithms are faster but lead to lower erasure recovery capabilities, and vice-versa sometimes decoding complexity is the key e.g., lightweight portable device sometimes encoding complexity is the key e.g., deep space (autonomous) probe 39

13 Performance metrics (cont )! metric 6: required memory during encoding and decoding even if RAM is used (rather than costly chipset memory), it should be used with caution e.g., if data can fit in the CPU L2 cache, it s great high impact of the decoding algorithm 40 Performance metrics (cont )! performances depend on many parameters: block size impacts the decoding overhead some codes (e.g., LDPC and Raptor) are good asymptotically but sub-optimal with small blocks symbol size impacts speed and memory requirements decoding algorithm e.g., iterative decoding is fast, but leads to sub-optimal overhead results good AL-FEC = good code + good codec ( + good lobbying!) 41

14 Outline 1. the erasure channel 2. AL-FEC codes 3. what is a good AL-FEC code? The performance metrics 4. example: LDPC-Staircase 5. example: Raptor(Q) 42 LDPC codes! in short Low Density Parity Check (LDPC) linear block codes discovered by Gallager in the 60 s re-discovered in mid-90s original codes were extremely costly to encode generator matrix (G) creation requires a matrix inversion but we use high performance variants in the remaining we only consider binary codes 44

15 LDPC-Staircase codes! LDPC-staircase codes (RFC 5170) a particular class of binary LDPC codes A.K.A. Repeat Accumulate codes a simple structure that defines a set of linear equations IETF RFC parity check matrix (H) source symbols parity symbols S 1 S 2 S 3 S 4 S 5 P 1 P 2 P 3 P 4 P N 1 1 s per column (e.g. 3) linear system S 3 S 4 P 1 = 0 S 1 S 4 S 5 P 1 P 2 = 0 S 1 S 2 S 3 P 2 P 3 = 0 S 2 S 4 S 5 P 3 P 4 = 0 S 1 S 2 S 3 S 5 P 4 P 5 = 0 45 LDPC-Staircase codes (cont )! encoding it s trivial " produce repair symbols in sequence: P 1, then P 2, then P 3 guaranties a high encoding speed example: CR=2/3, N1=7, symbol size=1024 bytes Samsung Galaxy S2 smartphone, ARM Cortex A9, 1.2GHz 1000 Samsung Galaxy SII 800 average bitrate (Mbps) Mbps " block size (k) 46

16 LDPC-Staircase codes (cont )! decoding (much more complex) solve a system of binary linear equations scheme 1: ITerative decoding (IT) scheme 2: Structured Gaussian Elimination (SGE) SGE is an old optimized way of using GE over sparse systems, midway between IT and regular GE start with (1), finish with (2) if in bad reception conditions [16] B. A. LaMacchia and A. M. Odlyzko, Solving large sparse linear systems over finite fields, in Advances in Cryptology (Crypto 90), LNCS 537, Springer-Verlag, [17] C. Pomerance and J. W. Smith, Reduction of huge, sparse matrices over finite fields via created catastrophes, Experimental Mathematics, Vol. 1, No. 2, [18] E. Paolini, B. Matuz, G. Liva, and M. Chiani, Pivoting algorithms for 47 LDPC-Staircase codes (cont )! decoding (cont ) CR=2/3, N1=7, symbol size=1024 bytes Samsung Galaxy S2 smartphone, ARM Cortex A9, 1.2GHz 1000 IT decoding (5% loss rate) ML decoding (33% loss rate) 800 average bitrate (Mbps) IT only decoding (good rx conditions) > 600 Mbps " 200 IT+SGE decoding (bad rx conditions) block size (k) Mbps " 48

17 LDPC-Staircase codes (cont )! erasure recovery performance are close to that of Raptor for medium to large blocks it s good enough, no need to go any further! Pr = 1: cannot decode 1 LDPC-Staircase N1=7 CR=2/3 k=1000 Raptor CR=2/3 k=1000 Decoding failure probabilitiy Pr << 1: high decoding probability 1e Number of received symbols 49 Conclusions: LDPC-Staircase vs. Raptor! achievements an IETF standard, RFC 5170 enables interoperable, independent implementations LDPC-Staircase codes are included in the ISDB-Tmm Japanese standard for mobile multimedia with a good codec they achieve: recovery performances close to ideal high speed encoding and decoding, even on smartphones and high flexibility adjust the recovery capabilities / processing load trade-off 50

18 Concl.: LDPC-Staircase vs. Raptor (cont )! during the 3GPP/eMBMS contest we showed that in representative use-cases RS+LDPC-Staircase outperform Raptor codes " have performance that approach RaptorQ codes "! morality: in spite of their simplicity, LDPC-staircase codes are high-performance codes illustrates the need for high potential codes high quality codecs both aspects are critical in practice 51 Outline 1. the erasure channel 2. AL-FEC codes 3. what is a good AL-FEC code? The performance metrics 4. example: LDPC-Staircase 5. example: Raptor(Q) 52

19 Raptor codes! history at the beginning were Tornado codes (Luby et al., 1997) the great idea is the discovery that irregular LDGM codes largely improve the performance of iterative decoding Tornado = cascade of irregular LDGM + many patents our LDPC codes perform as well and are patent free " then LT (Luby Transform) codes (1999) it s just a simple evolution of irregular LDGM codes, so as to enable small code rates + many patents but the erasure recovery capabilities are not good enough finally, with the help of Amin Shokrollahi, were Raptor codes (2001-now) a two stage encoding, taking advantage of LT codes plus optimized decoding techniques + many patents 53 Raptor codes (cont )! the theory A. Shokrollahi, Raptor Codes, IEEE Trans. on Information Theory, Volume 52, Number 6, 2006.! the practice all the details of the codes found in 3GPP and DVB systems are in the IETF RFC5053 (Raptor) and RFC6330 (RaptorQ) significantly different from the theory! some refinements (e.g., predefined source block sizes) can be found in other standardization docs! the patents search any patent database site with Luby, Shokrollahi, or Raptor keywords E.g., in: 54

20 Raptor encoding! two steps encoding precoding: generate Intermediate Symbols (IS) LT encoding: produce encoding symbols from IS! this is a systematic encoding source are part of encoding symbols thanks to a specific way of producing IS 55 LT encoder! discovered by Michael Luby who very humbly gave his name: Luby Transform (LT) theory rooted on work on the Soliton distribution, optimized for IT decoding! enables the generation of a potentially infinite number of encoding symbols unlimited only in theory! each repair symbol is the XOR of a different number of source symbols irregular scheme: the number is not constant a small number is the XOR of a high number of source symbols while most of them are the XOR of a very small number (2) 56

21 LT encoder as per RFC5053! example of LT encoding: each column corresponds to an IS (input) each row corresponds to an encoding symbol (output) each row contains a. for each IS considered during the encoding of a specific encoding symbol it s a sparse matrix 57 Precoding! LT is fine, but it s not systematic systematic codes are required in practice for situations where backward compatibility and/or low encoding overhead are needed hence a trick : do a pre-coding in such a way that the following LT encoding produces source symbols in the set of encoding symbols Idea: LT LT same k src symbols k src symbols L (>k) intermediate symbols add. repair symbols 58

22 Precoding (cont )! LT decoding creates k equations between source symbols and IS! in order to have an invertible system, since there are L>k IS, we add L-k additional equations L = k + S + H S symbols are generated from an LDPC encoding of K first IS H half symbols are generated from the remaining k+s other IS in practice L is only slightly higher than k: K" S" H" L" 200" 23" 10" 233" 500" 41" 12" 553" 1000" 59" 13" 1072" 59 Precoding (cont )! But these L equations do not directly enable the generation of the L IS let s put that in a matrix form A = matrix composed of the L equations D = (0.. 0; source vector) D is a vector composed of L-k null symbols plus k source symbols C = (intermediate symbols) then: D = A*C or: C = A -1 * D encoding requires computing A -1 Raptor specifications are such that for any k in {4; 8192}, A -1 exists (this is not the case by default!) 60

23 A and A -1 matrices 0 + k source symbols A matrix L intermediate symbols S+H 0 S H G LDPC T G Half T Id 0 Id N D = K G LT T C L 61 A and A -1 matrices (cont ) A (theory) A (example) A -1 (example) G LDPC I S 0 G Half,,I H G LT 62

24 Raptor encoding (final step)! when the L IS are generated, a simple LT encoding suffices to produce the encoding symbols of course, there s no need to recompute the source symbols! 63 Raptor decoding! final goal rebuild the missing source symbols (from C [0] to C [k-1]) from the set of N available encoding symbols (D [0] to D [N-1]) N available encoding symbols (received) source symbols D [0],,D [N-1] Raptor decoding C [0],,C [k-1]! a two step decoding LT decoding to reconstruct the full set of IS (hard) LT encoding to reconstruct missing source symb. (trivial) 64

25 Raptor decoding (cont )! step 1: the hard part rebuild L intermediate symbols from the N encoding symbols available taking advantage of the L-k extra relationships (null symbols) otherwise the decoding overhead would be poor as N L > k N available encoding symbols (received) L intermediate symbols D [0],,D [N-1] LT decoding C[0],,C[L-1] it s a matter of solving a linear system Ax = B, which can be done in many different ways 65 Raptor decoding (cont )! step 1 (cont ): Gaussian Elimination (GE) is fine but costly ITerative decoding (IT) is fast, but sub-optimum there s a midway solution: Structured Gaussian Elimination that s the key to the problem, and this is not Raptor specific, it can be applied to any sparse linear system, e.g. LDPC- Staircase codes B. A. LaMacchia and A. M. Odlyzko. Solving Large Sparse Linear Systems over Finite Fields, Advances in Cryptology (Crypto 90), C. Pomerance and J. W. Smith. Reduction of huge, sparse matrices over finite fields via created catastrophes, Experimental Mathematics, Vol. 1, No. 2,