[book club issue 13] + Chapter III coding format of video files

Today's main learning content knowledge points Review ：

• FFmpeg Support many video coding formats , such as MPEG4、H263、H264、H265、VP8、VP9 etc. . 
• Today's focus H264 Video encoding format （H264 Rate control 、H264 Of I The frame type 、H264 Of P The frame type 、H264 Of B The frame type 、H264 Frame reference ）
• Video compression , Each frame represents a still image . In actual compression , Various algorithms will be adopted to reduce the capacity of data , among IPB（I>P>B ） Is the most common .
• I Frames are keyframes , It belongs to intra compression . That's right. AVI The compression is the same . P It's a forward search It means .B It's a two-way search . They are all based on I Frame to compress data .
• H264 The core algorithm is intra frame compression and inter frame compression , Intra compression is the generation of I Frame algorithm , Inter frame compression is the generation of B The frame and P Frame algorithm .

1.H264 Rate control

• VBR：Variable BitRate, Dynamic bit rate mode , The bit rate can vary with the complexity of the image , Therefore, its coding efficiency is relatively high . The quality is guaranteed , And bandwidth constraints . This algorithm gives priority to image quality
• CBR：Constant BitRate, Constant bit rate mode , Because the code rate is constant , When there are dramatic changes in the picture ,QP The parameter will increase , The image quality will get worse , When the scene is still , And waste bandwidth . The overall image quality of this mode is unstable . This algorithm gives priority to bit rate ( bandwidth ), Count the average bit rate over a period of time , It would be more reasonable .
• ABR：Average BitRate, Average bit rate mode , yes VBR A derivative form of , Within the specified file size , Static or near static image parts use relatively low traffic , High traffic is used in the complex picture , It can be regarded as VBR and CBR A compromise of . It can be understood as Reach the set bit rate within a certain time range , But the local bit rate peak can exceed the set bit rate , Constant average bit rate .

2.H264 Of I The frame type

• One GOP The first image in the sequence is called IDR Images （ Refresh the image now ）,IDR The images are all I Frame image , but I Not all frames are IDR frame , Only GOP The first of the sequence 1 individual I Frames are IDR frame （I Frames are frame groups GOP The base frame of ( First frame ), Only one in a group I frame .）.
• I frame , Also called intra reference frame , Keep a complete picture .
• Use only when decoding I Frame data can be reconstructed into a complete image .
• I Frame is generated without reference to other pictures .
• I Frames are P The frame and B Frame reference frame .
• I Frames don't need to consider motion vectors .
• I The frame occupies the largest amount of data .
• It's a full frame compressed encoded frame . It takes the whole frame of image information JPEG Compression coding and transmission .
• I Frames represent keyframes , You can understand it as a complete preservation of this frame ; Decoding only needs the frame data to complete （ Because it contains the whole picture ）.

3.H264 Of P The frame type

• P frame , Also called forward reference frame , It represents this frame and the previous key frame （ or P frame ） The difference between , In decoding, the difference defined in this frame needs to be superimposed on the previous cached picture , Generate the final picture ,P Frame has no full picture data , Only data different from the picture of the previous frame .
• P Frames are I The frame is separated by 1~2 Frame encoding frame .
• P The frame is transmitted with motion compensation I or P Frame difference and motion vector ( Prediction error ).
• P The frame belongs to the inter frame coding of forward prediction . It only refers to the one closest to it in front of it I The frame or P frame .
• P The frame can be behind it P Frame reference frame , It can also be before and after B Frame reference frame .
• P Frames are reference frames , It may cause the spread of decoding errors .
• P The compression ratio of frames is between I The frame and B Between frames .
• P The frame represents this frame and the previous keyframe （ or P frame ） The difference between , In decoding, the difference defined in this frame needs to be superimposed on the previous cached picture , Generate the final picture .（ That's the difference frame ,P Frame has no full picture data , Only data different from the previous frame ）.
• P Frame prediction and reconstruction :P The frame is based on I Frames are reference frames , stay I Find... In the frame P frame “ A point ” The predicted values and motion vectors of , Take the prediction difference and transmit it with the motion vector . At the receiving end, according to the motion vector from I Find... In the frame P frame “ A point ” And add the predicted value to the difference value P frame “ A point ” Sample value , So we can get complete P frame .

4.H264 Of B The frame type

• B frame , Also called bidirectional reference frame , That is to say B A frame records the difference between this frame and the preceding and following frames , To decode B frame , Not only to get the previous cache screen , You have to decode the image , Through the superposition of the front and back pictures and the data of this frame, the final picture is obtained .
• B The compression rate of frames is the highest , But when decoding CPU The utilization rate will be relatively high .
• B The frame is from the front I or P The frame and the following P Frame to predict .
• B The frame transmits it with the preceding I or P The frame and the following P Prediction errors between frames and motion vectors .
• B Frame is not a reference frame , It doesn't cause the spread of decoding errors .
• B Frames are two-way differential frames ,B High frame compression rate , But when decoding CPU I will be tired .
• B Frame prediction and reconstruction ：B Frame before I or P The frame and the following P Frames are reference frames ,“ find ”B frame “ A point ” And two motion vectors , And take the prediction difference and the motion vector transmission . According to the motion vector in two frames “ find ( Work out )” Predict the value and sum it with the difference , obtain B frame “ A point ” Sample value , So we can get complete B frame .

5.H264 Frame reference

I、B、P Each frame is based on the needs of the compression algorithm , It's artificially defined , They are real physical frames . Generally speaking ,I The frame compression rate is 7（ Follow JPG almost ）,P Frames are 20,B Frames can reach 50. Visible use B Frames can save a lot of space , The space saved can be used to save more I frame , So at the same bit rate , Can provide better picture quality .