Macroblock Level Rate Control for Low Delay H.264/AVC based Video Communication
•Download as PPTX, PDF•
1 like•986 views
We propose a macro-block (MB) level rate control algorithm for low delay H.264/AVC video communication which is based on the rho domain rate model. In our approach, an exponential model is used to characterize the relation between rho and the quantization step (Qstep) at the MB level, from which the quantization parameter (QP) for a MB can be obtained. Furthermore, a switched QP calculation scheme is introduced which prevents large deviations of the actual frame size from the target bit budget. Compared with the original rho domain rate control algorithm, the proposed method can achieve better video quality and improved bit-rate accuracy. Additionally, the computational complexity is also significantly reduced.
Recommended
Recommended
More Related Content
What's hot
What's hot (13)
Similar to Macroblock Level Rate Control for Low Delay H.264/AVC based Video Communication
Similar to Macroblock Level Rate Control for Low Delay H.264/AVC based Video Communication (20)
Recently uploaded
Recently uploaded (20)
Macroblock Level Rate Control for Low Delay H.264/AVC based Video Communication
- 1. Macroblock Level Rate Control for Low Delay H.264/AVC based Video Communication Burak Cizmeci burak.cizmeci@tum.de Eckehard Steinbach eckehard.steinbach@tum.de Michael Eiler michael.eiler@tum.de Min Gao mgao@hit.edu.cn Debin Zhao dbzhao@hit.edu.cn Wen Gao wgao@jdl.ac.cn Chair of Media Technology Technische Universität München IEEE The 21st International Packet Video Workshop 2nd June 2015, Cairns, Australia
- 2. Technische Universität MünchenChair of Media Technology Outline 2 • Introduction • ρ-domain rate control • The proposed MB level rate control • Experiments & Conclusion 2 June 2015 Burak Cizmeci: MB level rate control for low delay H.264/AVC based Video Communication
- 3. Technische Universität MünchenChair of Media Technology Introduction: End-to-end Live Video Communication 32 June 2015 Burak Cizmeci: MB level rate control for low delay H.264/AVC based Video Communication Fig. 1: End-to-end video communication □ Application areas: □ Live streaming □ Cloud gaming □ Telepresence □ UAV control □ Constant bitrate (CBR) is preferred for live streaming
- 4. Technische Universität MünchenChair of Media Technology Introduction: MB level rate control 42 June 2015 Burak Cizmeci: MB level rate control for low delay H.264/AVC based Video Communication Fig. 2: General Macroblock based encoding scheme for a given frame [1] Chicken-egg dilemma [1] Z.G. Li et. al. “Adaptive rate control for H.264”, Elsevier, Journal of Visual Communications, Image R., 2006
- 5. Technische Universität MünchenChair of Media Technology ρ-domain rate control □ ρ-domain rate model [2,3]: □ Estimation of header bits for ρ-domain rate control [4]. 5 [2] Z. He, and S.K. Mitra “A linear source model and a unified rate control algorithm for DCT video coding,” IEEE Trans. Circuits Syst. Video Technol., 2002. [3] Z. He, and D. O. Wu “Linear rate control and optimum statistical multiplexing for H.264 video broadcast,” IEEE Trans. Multimedia, 2008. [4] F. Zhang and E. Steinbach “Improved rho domain rate control with accurate header size estimation,” in Proc. IEEE Int. Conf. (ICASSP) , 2011 (1 )R , ρ is the percentage of zero coefficients. 2 June 2015 Burak Cizmeci: MB level rate control for low delay H.264/AVC based Video Communication Fig. 3: Relation between R and 1-ρ on Foreman@CIF sequence. bits Percentage of non-zero coefficients
- 6. Technische Universität MünchenChair of Media Technology ρ-domain rate control 62 June 2015 Burak Cizmeci: MB level rate control for low delay H.264/AVC based Video Communication Fig. 4: Macroblock based encoding scheme with ρ-domain rate control [3] [3] Z. He, and D. O. Wu “Linear rate control and optimum statistical multiplexing for H.264 video broadcast,” IEEE Trans. Multimedia, 2008.. )(XP
- 7. Technische Universität MünchenChair of Media Technology ρ-domain rate control Loop 2: Determine QP for each MB □ Compute the remaining bit-budget using: □ Compute needed fraction of zeros using: □ Find corresponding QP using: □ Apply compression using the QP □ Update model parameter using: □ Update for the next iteration 2 June 2015 Burak Cizmeci: MB level rate control for low delay H.264/AVC based Video Communication 7 )(384 m mt NM RR R R 1 zerom m NN R .384 ' x xP S QP )( 1 )( )(XP
- 8. Technische Universität MünchenChair of Media Technology Changes on loop 1 Rate distortion optimization for stage 1 We removed the statistics collection at stage 1 Instead we □ calculate the average QP from the previous frame □ for each MB □ perform motion estimation and compensation □ perform mode decision using the average QP □ record MVs, prediction difference and MAD for the best mode 82 June 2015 Burak Cizmeci: MB level rate control for low delay H.264/AVC based Video Communication
- 9. Technische Universität MünchenChair of Media Technology Contribution: speed up with ρ-QP model Relationship between ρ and QP: □ Exponential model 9 1 b Qstep a e Fig. 5: Relation between 1-ρ and Qstep for Foreman@CIF sequence. 2 June 2015 Burak Cizmeci: MB level rate control for low delay H.264/AVC based Video Communication 2 6 4 QP stepQ
- 10. Technische Universität MünchenChair of Media Technology Contribution: speed up with ρ-QP model Estimation of parameters in the proposed model: □ After performing RDO, two QPs are required to get two ρ. □ Solve the above equations for and □ Obtain the model function for Loop 2. 10 1 1 1 b Qstep a e 2 2 1 b Qstep a e 2 June 2015 Burak Cizmeci: MB level rate control for low delay H.264/AVC based Video Communication a b 1 b Qstep a e Table 1: Correlation between the actual values and the estimated ones. Sequences BR (kbps) Correlation coefficient Football 400 0.917 1000 0.983 Foreman 400 0.970 1000 0.951 Mobile 400 0.969 1000 0.981
- 11. Technische Universität MünchenChair of Media Technology Enhancement: efficient bit allocation 112 June 2015 Burak Cizmeci: MB level rate control for low delay H.264/AVC based Video Communication 1 2( ) lefti iT MB i left MB F R MADR R S N N MAD □ MB level bit allocation adapted from [5]: [5] M. Jiang and N. Ling, “Low-delay rate control for real-time H.264/AVC video coding”, IEEE Trans. Multimedia, 2006 is the remaining bits budget; is the number of uncoded MBs; is the total number of MBs in a frame; is MAD of ith MB in a frame. is MAD of the current frame; is a scaling factor. leftR MBN FMAD leftN iMAD iS 0 1i MB i S N
- 12. Technische Universität MünchenChair of Media Technology Enhancement: efficient bit allocation 122 June 2015 Burak Cizmeci: MB level rate control for low delay H.264/AVC based Video Communication □ Compute □ From the ρ-QP model, compute QPi □ Limiting QP to provide quality smoothness adapted from [5]: □ Apply compression using QPi □ Update model parameter using: [5] M. Jiang and N. Ling, “Low-delay rate control for real-time H.264/AVC video coding”, IEEE Trans. Multimedia, 2006 i hdr i MB i RR 1 1 1{ , { , }}i i i iQP min QP QP max QP QP QP 1, 25 2, jif QP QP otherwise zerom m NN R .384 '
- 13. Technische Universität MünchenChair of Media Technology Experiments Experimental conditions: □ x264, baseline profile, CAVLC Testing sequences: □ Bus, Container, Football, Foreman and Mobile (CIF) □ Frame rate: 25 fps □ Total number of frames: 250 (IPPP …PP) Comparison: □ Original ρ-domain rate control [3] 132 June 2015 Burak Cizmeci: MB level rate control for low delay H.264/AVC based Video Communication [3] Z. He, and D. O. Wu “Linear rate control and optimum statistical multiplexing for H.264 video broadcast,” IEEE Trans. Multimedia, 2008.
- 14. Technische Universität MünchenChair of Media Technology Experiment 1: encoding delay reduction 14 100%Org Pro C Org C C C 2 June 2015 Burak Cizmeci: MB level rate control for low delay H.264/AVC based Video Communication Table 2 : Encoding time reduction Target BR (kbps) Avg. Encoding time reduction (%) 300 52.67 500 50.45 1000 46.77 2000 44.72
- 15. Technische Universität MünchenChair of Media Technology Experiment 2: Video quality in PSNR 152 June 2015 Burak Cizmeci: MB level rate control for low delay H.264/AVC based Video Communication Table 3: Video quality and achieved bitrate comparison Target BR (kbps) Original (Avg.) Proposed (Avg.) BR (kbps) PSNR (dB) BR (kbps) PSNR (dB) PSNR Gain (dB) 300 235.17 28.16 297.26 28.57 0.41 500 459.70 30.28 496.90 30.77 0.49 1000 965.06 32.33 993.21 33.58 1.25 2000 1972.33 34.05 1989.37 36.77 2.72
- 16. Technische Universität MünchenChair of Media Technology Experiment 3: bitrate accuracy tests 162 June 2015 Burak Cizmeci: MB level rate control for low delay H.264/AVC based Video Communication Fig. 6: Bitrate for individual frames for Football@CIF sequence at 500 kbps. 0 500 1000 1500 2000 2500 3000 3500 1 8 15 22 29 36 43 50 57 64 71 78 85 92 99 106 113 120 127 134 141 148 155 162 169 176 183 190 197 204 211 218 225 232 239 246 Target Proposed Original [3] [3] Z. He, and D. O. Wu “Linear rate control and optimum statistical multiplexing for H.264 video broadcast,” IEEE Trans. Multimedia, 2008. frame id bytes
- 17. Technische Universität MünchenChair of Media Technology Experiment 3: bitrate accuracy tests 172 June 2015 Burak Cizmeci: MB level rate control for low delay H.264/AVC based Video Communication Fig. 7: Bitrate for individual frames for Football@CIF sequence at 1000 kbps. Target Proposed Original [3] [3] Z. He, and D. O. Wu “Linear rate control and optimum statistical multiplexing for H.264 video broadcast,” IEEE Trans. Multimedia, 2008. 0 1000 2000 3000 4000 5000 6000 1 8 15 22 29 36 43 50 57 64 71 78 85 92 99 106 113 120 127 134 141 148 155 162 169 176 183 190 197 204 211 218 225 232 239 246 bytes frame id
- 18. Technische Universität MünchenChair of Media Technology Conclusion & Future Work A MB level rate control algorithm for low-delay video communication. Rate control is based on ρ-domain rate model. We made two contributions: □ Contribution: acceleration of ρ-QP model derivation 50% reduction on encoding time □ Enhancement: efficient MB level bit allocation [5] improved bitrate accuracy smooth spatial quality Future Work: Solutions to reduce the two stage processing into one stage. 2 June 2015 Burak Cizmeci: MB level rate control for low delay H.264/AVC based Video Communication 18 [5] M. Jiang and N. Ling, “Low-delay rate control for real-time H.264/AVC video coding”, IEEE Trans. Multimedia, 2006
- 19. Technische Universität MünchenChair of Media Technology Thanks! 192 June 2015 Burak Cizmeci: MB level rate control for low delay H.264/AVC based Video Communication Min Gao mgao@hit.edu.cn Burak Cizmeci burak.cizmeci@tum.de
Editor's Notes
- Mention Min Gao, and refer him as the main implementer of the project
- -CBR is preferred because it doesn’t allow data spikes Can be adapted to the channel changes with a joint channel-source coding scheme
- Chicken egg dilemma: Which QP and mode will lead the target rate?
- In addition to R-Q model, it was found in [2] that the bit-rate shows a linear relationship with rho, and rho is defined as the percentage of zero transform coefficients after quantization. Based on this model, a rho-domain rate control was proposed in [3] for H.264/AVC. An improved rho-domain rate control was proposed in [4] with accurate header bits estimation. The work in this paper is also based on the rho-domain rate model, so let us first review the original rho-domain rate control for H.264/AVC.
- Loop 1: Transform coefficients are not quantized.
- Rt: Total bitbudget per frame Rm: bits have been already used to encode Nm MBs R‘m: bits have been already used to encode MBs including the current one M: Total number of MBs in a frame Nm: Number of coded MBs Nzero: Number of zero coefficients S: total number of transform coefficients Separate P(X) for inter and intra predictions
- To obtain QP from rho, we propose a rho-QP model. From our experiments, we find the exponential function can accurately model the relationship between rho and Qstep.
- So we can directly calculate QP from rho with the exponential model. To do that, the model parameters a and b should be estimated. Here two QPs are enough to estimate the parameters.
- So we can directly calculate QP from rho with the exponential model. To do that, the model parameters a and b should be estimated. Here two QPs are enough to estimate the parameters.
- After getting the assigned bits, the texture bits equal to the difference between the assigned bits and estimated header bits. Here we use the average number of header bits generated by previously coded MBs as its estimated header bits So the rho can be calculated using the rho domain rate model, and the QP can also be calculated using the proposed exponential model. To guarantee the spatial smoothness of a frame, the QP for the current MB should be limited within a range.
- To measure the coding performance, we use the x264 as the testing platform, and the baseline configuration is used. The test sequences include five CIF video sequences. They are Bus, Container, Football, Foreman and Mobile. Their frame rate is 25 f/s. For each sequence, 250 frames are encoded. The original rho domain rate control is used as the anchor. In this method, all possible QPs are checked to find the optimal QP for a given rho.
- This is the experimental results in computational complexity. Here we use the reduction of encoding time to measure the computational complexity. From this table, we can see that the proposed method can significantly reduce the encoding time.
- This is the experimental results in video quality. From this table, we can see that the proposed method can achieve better video quality than the original method for most cases. This is because the proposed method adopts the MB level bit allocation. We can also see that the proposed method has worse video quality than the original method on Football sequence at low bit rate. This is because the spatial and temporal content in Football sequence is very complex. So the effect of MB level bit allocation is reduced due to the limited bit budget.
- This is the experimental results in bit accuracy. Here we use the average deviation of the frame size from the target bit budget to measure the bit accuracy. From this table, we can see that the proposed method can achieve better bit accuracy.
- This is the experimental results in bit accuracy. Here we use the average deviation of the frame size from the target bit budget to measure the bit accuracy. From this table, we can see that the proposed method can achieve better bit accuracy.