Effect of Buffer Placement on Performance When Communicating Over a Rate-Variable Channel

Transcription

1 29 Fourth International Conference on Systems and Networks Communications Effect of Buffer Placement on Performance When Communicating Over a Rate-Variable Channel Ajmal Muhammad, Peter Johansson, Robert Forchheimer Division of Information Coding, Electrical Engineering Department, Linköping University, SE Linköping, Sweden muhaj25@student.liu.se, {pejoh, robert}@isy.liu.se 1 Abstract We compare the performance of two real- muldia communication systems for quality versus end-to-end schedule by creating look-ahead delay for the streaming of of a smoother, which will pre-fetch some of the data before delay. We develop an analytical framework for comparison when pre-recorded video. The authors of [11] have used the concept the systems use a deterministic -varying channel. Moreover, we assess their performance for the Gilbert-Elliott channel model of rate shaper which shapes the video i.e. reduces the bit rate which alternates between a good and a bad state with by dropping part of the precoded video, according to current durations that are exponentially distributed. network conditions. The same authors have extended the The goal of the paper is to select the best system with low average concept of rate shaper by adopting MPEG-4 fine granularity distortion while obeying a real- constraint. scalability (FGS for source coding and the same erasure codes Index Terms Buffers, Interactive systems, Source coding, as in [11] for forward error correction (FEC coding in [12]. Time-varying channel. These last three papers have used the idea to buffer before the rate-control decision device but for pre-encoded videos. None of the authors have used pre-fetching for interactive I. INTRODUCTION communication, neither have we found a comparison of the INTERACTIVE communication is becoming common on performance of the two systems for same end-to-end delay. present-day networks. In most of the present communication networks/channels the available rate or throughput are muldia communication system models namely the input In this paper, we compare the performance of two real- highly dynamic, such as in Internet or IP based networks in buffer system and the transmit buffer system. Each of them which delays are neither bounded nor predictable and data is consists of first in, first out (FIFO buffers, encoder, decoder prone to packets losses due to congestion. Similarly in wireless and a varying channel. Performance is measured as SNR channel due to multipath propagation, radio interference and (signal-to-noise ratio value versus end-to-end delay. The endto-end delay is measured from the when the data enters fading the throughput varies severely with. Media Signals used in point-to-point interactive communication is often represented using lossy compression techniques we will describe in detail both the system models, formulate the system until the same data leaves the system. First of all, where the output rate and the accuracy of the representation at the framework for their comparison when using an optimal the decoder are chosen by the encoder. This provides a tradeoff between channel utilization and signal quality. Moreover, nel. Secondly, we will suggest a model for stochastic - rate-control technique over a deterministic -varying chan- it is common to use a transmit buffer to temporary store varying channel and will evaluate the performance of the the encoded data for the input signal when the channel rate optimal rate-control scheme over this channel model, both is limited and does not match with the encoded rate. As analytically and through simulation. interactive communication imposes strict constraints on overall The paper is organized as follows. In Section II, we describe delay, this translates to a limited size of the transmit buffer. the two communication systems in detail along with the framework for comparison. In Section III, we formulate a model Ultimately the limited transmit buffer puts constraint on the encoding rate i.e. the signal needs to be encoded at the rate for stochastic -varying channel. While in Section IV we which guarantees the overall delay constraints. To cope with introduce the rate-control strategy for communication over this challenge various potential solutions have been proposed the formulated channel, and analyzed this strategy in detail based on source-rate control methods performed at frame level for both the systems in Section V. Section VI provides the [1]-[6] and at macro block level [7]-[9]. These methods are theoretical and simulation performance results. Conclusions applicable to interactive and pre-encoded video and rely on are drawn in Section VII. the information about the state of the channel, transmitter buffer size and it fullness, and the amount of bits used to encode previous frames. Moreover, these authors have used II. COMMUNICATION SYSTEM MODELS the concept of transmit/encoder buffer and receive/decoder buffer for smoothing the variation in bit rates produced by In this Section we will describe in detail the two systems and the encoder as well as channel delay variations to keep the will formulate an analytical framework for their comparison end-to-end /9 delay constant. $ Similarly, IEEE [1] has used the concept which will be used in further sections. DOI 1.119/ICSNC.29.95

2 2 Fig. 1. Input buffer communication system model A. Input Buffer System In this system the transmitter consists of an input buffer and an instantaneous lossy encoder as shown in Figure 1. We assume a memory-less Gaussian source with unit variance and use the mean squared error distortion measure. Gaussian sources are the most difficult to compress and represent the worst case scenario [13]. The Distortion-rate function for a Gaussian source with unit variance and squared distortion measure is given by D(R =4 R/2W (1 Where R is the instantaneous rate in bits/s and W is the bandwidth of the analog input signal. In the rest of this paper we will assume that (1 holds also for the operational distortion-rate function of the used encoders. The channel is assumed to be error free, due to an ideal error control scheme, such as coding or ARQ (automatic repeat-query, but vary its rate over periodically between two states with transmission delay equal to zero 1. In the bad state, with duration t 1, the channel transmits R 1 bits/s and in the good state, with duration t 2, the rate is R 2 bits/s. Moreover, R 1 and R 2 indicate the throughput of the channel during these states and R 1 is assumed to be less then that of R 2 e.g. due to addition of more redundancy bits for error control. The receiver unit consists of an instantaneous decoder followed by an output (play-out buffer of the same size as the input buffer to compensate for the jitter produced by the transmitter. The total delay t d for this system is given by the capacity of the input buffer, which needs to be measured in seconds as the signal is still in its non-digitized or uncompressed form. Initially we assume that the allowed delay is always shorter than the duration t 1 of the bad state. If we encode the input signal at the channel rate then the input buffer will remain empty with t d =seconds. while if we encode the input signal at higher rate than that of the channel rate, after some we will have filled the input buffer. This makes the delay in the transmitter equal to the signal stored in the input buffer. At that instant we can either switch to the channel rate or encode at a slower rate than the channel rate to empty the input buffer. The rate-control strategy that we adopt for this deterministic varying channel is constant rate per state which uses different constant encoding rates for each state within the constraint imposed by the input buffer, and is discussed in the Appendix. The optimal constant rate for the bad state is to encode at a rate R s1 that fills the input buffer at the end of that state. We use t to denote the index for the input signal. At t = seconds both s (The channel t and the input signal t will be zero, but at the end of the bad state t will lag from channel by t d seconds t 1 R 1 = t R s1 =(t 1 t d R s1 (2 Similarly in the good state, we adopt a constant encoding rate R s2 that empties the input buffer at the end of that state and so t will catch up with the channel. Consequently, we have t 2 R 2 = t R s2 =(t 2 + t d R s2 (3 Now to geometrically interpret the constant rate per state we take the curve of cumulative number of bits transmitted over the channel and shift it to the left by t d seconds as shown in Figure 2. The accumulated number of bits produced at the encoder will be a curve in the corridor of Figure 2. The average distortion for these encoding rates will be given as D = ( t 1 t d 4 R s1 /2W + ( t 2 + t d 4 R s2 /2W (4 In the bad state, (t 1 t d seconds of input signal is transmitted. The number of bits available are R 1 t 1. Thus it is seen that R s1 = R 1t 1 (5 t 1 t d Similarly for the good state (t 2 + t d seconds of input signal is sent. While the available numbers of bits are R 2 t 2. This requires that R s2 = R 2t 2 (6 t 2 + t d It should be noted that these choices of rates are only valid as long as R s1 <R s2. At the point where they become equal due to a sufficiently large input buffer, a common fixed rate will be optimal. It can be seen that this happens when the allowed delay is t d = t 1t 2 (R 2 R 1 (7 t 1 R 1 + t 2 R 2 Corresponding to the cross-over rate, R s1 = R s2 = (t 1R 1 + t 2 R 2 (8 At higher allowed delays, the optimal rate depends on the relation between the input buffer size and the total transmission Accumulate number of bits Input buffer capacity Any other delay can be accounted for by shifting the final results Fig. 2. Allowable corridor for the source coding rate for the input buffer accordingly system model

3 3. If we consider the total transmission to be infinite, the above rate is the best that can be used. Any further allowed delay will not improve the performance for the constant rate per state strategy. As explained earlier the receiver unit consists of an instantaneous decoder followed by an output (play-out buffer to overcome the jitter produced by the transmitter. For jitter-free play-out we need to avoid both overflow and underflow. To avoid output buffer underflow the initial play-out delay δ has to be selected such that at any instant t we have at least (t δ seconds of the input signal available for output. i.e., δɛr : t (t δ, t T session (9 The left-hand side of (9 defines the index for the channel signal for t seconds received by the instantaneous decoder while the right-hand side defines the output (play-out signal index. Moreover, (9 should hold for the whole transmission session i.e. T session. The value of δ is chosen by rightshifting the encoding rate curve by t d seconds shown in Figure 3 which is equal to the capacity of the input buffer. This means that the total delay for the input buffer system is equal to the capacity of the input buffer which we have already stated. Moreover, to avoid output buffer overflow, the size of the output buffer must be equal to max{t (t δ}. The maximum difference occurs when the input buffer becomes empty (and the output buffer fills fully. So this means that the size of the output buffer must be equal to the input buffer size. B. Transmit Buffer System In the transmit buffer system the transmitter unit consists of an instantaneous coder followed by an output buffer of size B t bits shown in Figure 4. The receiver unit consists of a receive buffer of size B r bits to cope with jitter produced by the transmitter followed by an instantaneous decoder. We use the same source coder and channel models as we assumed for the input buffer system. The transmit buffer is used for storing encoded data when the encoding rate is higher than that of the channel rate. For the transmit buffer system, the constant rate per state strategy will encode the input signal at R s1 rate and Fig. 4. Transmit buffer communication system model will fully fill the transmit buffer at the end of the bad state. Thus, t 1 R s1 = t 1 R 1 + B t (1 Similarly, for the good state the encoding rate R s2 would be chosen such that to empty the transmit buffer at the end of the good state. t 2 R s2 = t 2 R 2 B t (11 So, R s1 R s2 = B t t 1 + R 1 (12 = R 2 B t t 1 (13 To geometrically interpret the constant rate per state strategy for the transmit buffer system we take the curve of cumulative number of bits transmitted over the channel and shift it upwards by B t bits as shown in Figure 5. The accumulated numbers of bits produced at the encoder will generate a curve in the corridor of Figure 5. While the average distortion will be equal to D = ( t1 4 Rs 1 /2W + ( t2 4 Rs 2 /2W (14 The system delay for the transmit buffer system can be deduced as follows. Consider the case when the transmit buffer is full. As we have already stated this happens when the channel has been in the bad state and the transmit buffer is filled with the input signal coded at R s1 bits/s. This means that the transmit buffer contains B t /R s1 seconds of the input signal. At that instant the receive buffer is empty (in steady state and the receiver will not add any additional delay. Thus 4 35 Cumulative received bits curve Cumulative play out curve Cumulative decoded bits without delay 45 4 Accumulate number of bits Output buffer capacity Accumulate number of bits Transmit buffer capacity 5 5 td Fig. 3. Output buffer size for un-interrupted play-out Fig. 5. Allowable corridor for the source rate for the transmit buffer system

4 4 t d = B t R s1 (15 By replacing B t above in the expressions for R s1 and R s2, we get R s1 = R 1t 1 t 1 t d (16 R s2 = R 2 t dt 1 R 1 (17 t 2 (t 1 t d It should be noted again that these choices of rates are only valid as long as R s1 <R s2. At the point where they become equal due to a sufficiently large transmit buffer, a common fixed rate will be optimal. It can be seen that this happens for the transmit buffer size Bt = t 1t 2 (R 2 R 1 (18 Corresponding to the rate R s1 = R s2 = t 1R 1 + t 2 R 2 (19 This is the same cross-over rate as for the input buffer case. Inserting this rate yields the cross-over delay t d = B t ( = t 1t 2 (R 2 R 1 (2 R 1 t 1 + R 2 t 2 R 1 t 1 + R 2 t 2 Comparing with (7 shows that both systems achieve the same performance for this value of the delay. It should be noted that the receive buffer will not have the same size as the transmit buffer. Let us consider the situation at the end of the good state. The transmit buffer is now empty while the receive buffer is full. The latter will now hold a signal segment coded at R s2 bits/s. This means that the signal stored in the receive buffer corresponds to B t /R s2 seconds. Since the system cannot allow different delays at different s we find that, t d = B t R s1 = B r R s2 (21 From (21 it is clear that the size of the receiver buffer needs to be larger than the transmit buffer in order to hold the larger amount of data produced in the good channel state for the same duration of the input signal. E[D/Bad state] = III. STOCHASTIC TIME-VARYING CHANNEL In the previous Section, we considered a deterministic varying channel where we had full knowledge of the channel Similarly for the good state exact duration and rate in each state. Many practical communication channels behave randomly and their durations in any E[D/Good state] = state varies over. To take account of the random nature of the channel, we now assume a Gilbert-Elliott model [14] that alternates between two states, the bad state with rate R 1 bits/s and the good state, with rate R 2 bits/s. Moreover, R 1 and R 2 indicate the throughput of the channel in these states. distortion The spent in each state is exponentially distributed, with parameter λ 1 and λ 2 respectively. Thus 1 E[D] = 1 t P (T i t = λ i e λit λ λ 2 dt ( The average in the bad state is thus 1/λ 1 seconds and for the good state it is 1/λ 2 seconds. IV. RATE CONTROL STRATEGY FOR STOCHASTIC TIME-VARYING CHANNEL In this Section we discuss the rate control strategy for the two systems when using the above channel model. The scheme which we used namely one rate per state for stochastic varying channel is similar to the constant rate per state strategy i.e. when the channel switches state the algorithm will make a choice of rate during that state and will pursue that rate within the constraint imposed by the input buffer or transmit buffer. The algorithm will guess a length τ i for the that the channel will stay in the current state i. For the transmit buffer system at the beginning of the bad state interval, the chosen coding rate will be R s1 = B t τ 1 + R 1. If the channel duration in the bad state exceeds τ 1 (guess duration for the bad state the coding rate will switch to that of the channel rate R 1 to avoid overloading the buffer. It is also possible that the channel switches to the good state before τ 1, thus without having filled the transmit buffer fully. Similarly, in the good = R 2 B t τ 2. Several state the chosen coding rate will be R s2 other cases may occur depending on the duration of the channel in the good state. For the input buffer system we will also have the same possibilities with coding rate R 1 t 1 /(τ 1 t d in the bad state and R 2 t 2 /(τ 2 + t d in the good state. The coding rate will switch to R 1 in the bad state upon filling the input buffer and will follow R 2 when it empties the input buffer in the good state. V. THEORETICAL ANALYSIS OF THE ONE RATE PER STATE A. Transmit Buffer System We will analyze the one rate per state strategy by assuming that the buffer is empty (b =at the start of the bad state and is full (b = B t at the beginning of the good state. The -average of the expected distortion for this algorithm is computed by first conditioning on the interval T being longer than τ 1, and then conditioning on the interval being at most τ 1. So the expected distortion for the bad state will be e λ 1τ 1 D(R 1 +(1 e λ 1τ 1 D(R 1 + B t τ 1 (23 e λ2τ2 D(R 2 +(1 e λ2τ2 D(R 2 B t τ 2 (24 we use (15 for the relation between B t and t d. Now bringing the results together gives the -average of the expected ( E[D/Bad state] + λ 1 E[D/Good state] λ 2 (25

5 5 B. Input Buffer System For the input buffer system we take the same assumption about the input buffer state at the beginning of the bad and good state. Thus the expected distortion for the bad state is given as E[D/Bad state] = e λ1τ1 D(R 1 + (1 e λ1τ1 D( R 1τ 1 (26 τ 1 t d While for the good state we have E[D/Good state] = e λ2τ2 D(R 2 +(1 e λ2τ2 D( R 2τ 2 (27 τ 2 + t d Combining these two results and taking the duration of each state, we will get E[D] = (τ 1 t d (τ 1 + τ 2 E[D/Bad state]+ (τ 2 + t d E[D/Good state] (τ 1 + τ 2 (28 From (28 we see that as the capacity of the input buffer increases the distortion due to the bad state will decrease accordingly, but this is valid only when τ 1 >t d. VI. EXPERIMENTAL RESULTS To evaluate the performance of the proposed rate-control strategies, we plot in Figure 6 the performance results using the expected distortion derived in sections II-A, II-B, V-A, V- B and the corresponding simulation results. The parameters used are λ 1 =1, λ 2 =1corresponding to average s of.1 seconds in bad state and 1 seconds in the good state. We used these durations as the guess s of the channel in the two states respectively. The rates R 1 and R 2 are 64 Kbit/s and 192 Kbit/s respectively and signal bandwidth is W = 12.5 KHz, typical for speech or lower quality audio signal. For the cross-over delay we find that t d =.645 seconds at the rate R s1 = R s2 = Kbit/s. We further find that the cross-over buffer size for the transmit buffer system becomes Bt = B r =11.6 Kbits. For simulation we used Matlab Programming, where for each value of system SNR(dB one rate per state simulated for transmit buffer system constant rate per state for transmit buffer system constant rate per state for input buffer system one rate per state theoretical for transmit buffer system one rate per state simulated for input buffer system one rate per state theoretical for input buffer system delay(s delay the simulation experiment run for 8 cycles (A cycle consists of one bad state and the ensuing good state. We varied the values of channel duration in bad and good state for each cycle randomly, such that for 8 cycles their mean values are approximately equal to.1 seconds and 1 seconds respectively. Finally we take the average of the whole session which give the average distortion value for that value of system delay. It can be seen from the figure that the input buffer system performs up to 1.75 db better than the transmit buffer system, when communicating over the deterministic -varying channel. The reason for the better performance of the input buffer system is that the lower rate R s1 is used for a shorter duration (t 1 t d compared to the transmit buffer system (t 1. For higher channel rates this performance difference would increase further. Now if we look at the simulation results of the one rate per state strategy for the two systems, it is clear that the input buffer system outperforms the transmit buffer system with up to 1 db. We have not taken into account the situation when the channel rate all of a sudden becomes worse and the transmit buffer will be full, so all the data in the buffer will be discarded which will further aggravate the performance. For the input buffer system in such a situation the encoding rate will switch to the channel rate without discarding any data. For the crossover delay the simulation performance of the two systems become the same. Moreover, the simulation result of the one rate per state strategy displays better performance then the corresponding analytical result. The reason for this difference is the approximation made that the input buffer will be full at the end of an interval when the channel is in the bad state, and similarly it will be empty at the end of the good state. This will only happen if the intervals has length larger then τ i. But here we take τ i =1/λ i i.e. τ 1 =.1 seconds corresponding to λ 1 =1, and τ 2 =1seconds corresponding to λ 2 =1.As a result, the probability that the above assumption is correct is only.368. Furthermore, unlike the deterministic varying channel which gives the same performance for both the systems at cross-over delay, the theoretical performance of the one rate per state for the transmit buffer system is approximately 4 db lower than the input buffer system. The reason for this better performance of the input buffer system is due to the fact that for the theoretical case we have two encoding rates for the bad state, namely R s1 and R 1 having ratio of.632 and.367 respectively. Same is the case for the good state. Although at cross-over delay R s1 becomes equal to R s2, however the expected distortion in the bad state will not be equal to that for the good state. Furthermore, the input buffer system uses the bad state expected distortion for a duration equal to (t 1 t d seconds in the total expected distortion shown in (28, as compared to the transmit buffer system which is using it for fixed t 1 seconds shown in (25. So, as the value of t d increases, the performance difference will increase accordingly. Finally from these results, we observe that the input buffer system performs better for deterministic and stochastic -varying channels, both theoretically and through simulation results. Fig. 6. Performance of different rate-control strategies for the two systems

6 6 VII. CONCLUSION In this paper, we evaluate the performance of two real- muldia communication systems for quality in terms of SNR versus delay. We develop an analytical framework for their comparison when the systems use a deterministic varying channel. Moreover, we show analytically that the buffers of the input buffer system will be symmetric to each other at variable bit encoding, while for the transmit buffer system the buffers will not be the same size due to storing of differently encoded signal of the same duration. We look at the optimal rate-control scheme when the communication channel switches randomly between a good state and a bad state. We capture this randomness of the channel by taking the Gilbert-Elliott channel model, and apply the rate-control strategy to both the systems. The performance of the input buffer system is better than the transmit buffer system for both channel models. The systems become equal in term of performance for a specific end-to-end delay value, when the encoding rates can be chosen the same for both states. At that point the buffers of the transmit buffer system become symmetric to each other. Using these rate-control strategies beyond that end-to-end delay will not improve the performance any further. Instead further research is needed to look for other strategies beyond that specific end-to-end delay. where T is the duration for which R s1 encoding rate will be sustained in the good state. while R 2 is the channel rate in the good state, which will be followed upon emptying the buffer in the good state. For the constant rate per state strategy the distortion for the good state is given by D =4 R s 2 /2W (3 From (13 we have R s2 = R 2 Bt t 2, so (3 becomes as D =4 (R2 B t t 1 2 2W =4 R2/2W.4 B t/2t 2 W (31 Now if we subtract (31 from (29 we get the difference equal to 4 R 2/2W ( t 2 4 B t/2t 2 W + T ( 4 R s 1 /2W 4 R s 2 /2W (32 For practical parameters values, the value of t 2 4 (B t/2t 2 W will be some positive value, similarly 4 (Rs 1 /2W 4 (Rs 2 /2W (as R s1 <R s2 will give some positive difference value, therefore the net distortion difference will be a positive value. This means that the maximum constant rate inflict more distortion as compared to the constant rate per state, and from which follows that the optimal rate-control strategy for our assumed channel is the constant rate per state. APPENDIX REFERENCES OPTIMAL SOURCE CODING RATE FOR PERIODIC CHANNEL [1] S. Zhou, J. Li, J. Fei and Y. Zhang, Improvement on rate-distortion performance of H.264 rate control in low bit rate, IEEE Trans. on We have assumed that the optimal strategy is to keep a circuits and systems for video tech, vol. 17, No.8, pages , Aug. constant rate until the buffer is full or empty, then change 27. [2] H. Wang and A. Ortega, Rate-distortion optimized schelduling for rate. So if we use this assumption for the channel which redundant video representations, IEEE Trans. on Image processing, varies rate periodically between two states, then the maximum vol.18, No.2, pages , Feb. 29. constant rate for R s (t will be to touch the upper corridor [3] Z. Zhang, G. Liu, H. Li and Y. Li, A novel PDE-based rate-distortion model for rate control, IEEE Trans. on circuits and systems for video wall (bounding curve at the end of the bad state. For the tech, vol. 15, No.11, pages , Nov. 25. good state there are two possibilities either to continue that [4] H. Lee, T. Chiang, and Y. Zhang, Scalable rate control for MPEG-4 constant rate (maximum constant rate strategy until we touch video, IEEE Trans. On circuits and systems for video Tech, vol.1, No.6 pages , Sept. 2. the lower boundary, or to switch to another constant rate [5] H. Song and J. Kuo, Rate control for low-bit-rate video via variableencoding frame rates, IEEE Trans. On Circuits and Systems for video for that state (constant rate per state shown in the Figure 7. To sort out which constant rate strategy among these is Tech, vol.11, No.4, pages , Apr. 21. [6] C. Yuan, A. Ortega and M. Khansari, Rate control for robust video giving better performance we calculate the distortion for these transmission over burst-error wireless channels, IEEE Trans. On selected strategies in the good state only (both will have same distortion areas in communications Tech, vol. 17, No.5, pages , May in the bad state. For the maximum constant rate strategy [7] H. Wang and S. Kwong, Rate-distortion optimization of rate control for H.264 with adaptive initial quantization parameter determination, IEEE D = T 4 Rs 1 /2W +(t 2 T 4 R2/2W Trans. On circuits and systems for video Tech, vol.18, No.1, pages 14- (29 144, Jan. 28. [8] C. An and T. Q. Nguyen, Iterative rate-distortion optimization of H.264 with constant bit rate constraint, IEEE Trans. on Image processing, vol.17, No.9, pages , Sept [9] J. R. Corbera and S. Lei, Rate control in DCT video coding for low 3 delay communications, IEEE Trans. On circuits and systems for video Constant rate per state Tech, pages , Feb [1] P. Thiran, J. Boudec and F. Worm, Network calculus applied to optimal muldia smoothing, in Proc. INFOCOM 1, vol.3, pages , Apr [11] T. P. Chen and T. Chen, Adoptive joint source-channel coding using Maximum constant rate rate shaping, in Proc. ICASSP 2, vol.2, pages , May [12] T. P. Chen and T. Chen, Fine-grained rate shaping for video streaming 5 over wireless networks, in Proc. ICASSP 3, vol.5, pages , Apr [13] M. Quirk, Diversity coding for communication systems Bell labs. Engineer s notes, Dec [14] E. N. Gilbert, Capacity of burst noise channel, Bell sys. Tech. Fig. 7. Different constant rate-control strategies J., vol.39, No.9, pages , Sept Cumulative number of bits