Speculation on the way to realize the smooth drag preview of video editing software

Discussion by situation ：

One 、 Quick rough drag ：

        1、 With 10ms Is a unit of time , If you exceed a certain threshold while dragging the timeline （ For example, over 1 individual GOP）, Only GOP The key frame of （I frame ）, And first try to find data from the mapping table , If it cannot be found, decode it again . Because even if you render B、P frame , The human eye can't see , It's better to show I The frame is practical . And put what has been rendered I Frame put <pts, frame > In the mapping table of .

        1.1、 if GOP The length exceeds a certain time threshold , Then you still need to render the image at the corresponding position in the threshold time point , In case of fast dragging for a long distance , When there should be image change, there is no image change .

        1.2、 To prevent 1.1 and 1 The process shows that the thread needs to wait for the decoding process, which leads to the Caton problem , Near the current drag position GOP Will be made by （CPU The core number - 1） Number of sub threads to decode , And on the <pts, frame > In the mapping table .

Two 、 Slow precise drag ：

        1、 At this point, the user is likely to be in a GOP Drag frame by frame , if B、P The frame should also carry GOP Multiple frames are decoded together , It will lead to many times of useless work . In order to avoid time-consuming and useless work , It should point to a pts when , Just put the current operation GOP And nearby GOP The actual composite picture rendering of each frame in the , And put it in the mapping table , At present GOP When dragging in , It can be decoded only once , You can know from the mapping table that every PTS What should the corresponding picture look like .

3、 ... and 、 Technical scheme for saving memory ：

         The mapping table may grow as the user drags the timeline in more directions , It may eventually lead to insufficient physical memory . here , have access to MMAP Methods such as , Map a part of external memory to memory address , Use LRU principle , The tree structure of the mapping table is kept in memory , But the contents are used according to the number of times , Below threshold MMAP In the map , And replace the memory pointer in the mapping table with MMAP The pointer , Those above the threshold are put into memory , Delete MMAP Value , Realize the dynamic balance between the use of memory and external memory , In the way of sacrificing space for time , In order to ensure the drag efficiency at the same time , It is unlikely that the program cannot continue due to insufficient memory .