Gradual Frame Rate Reduction in Video Playback on the Response to Resource Reduction

During video playback, for various reasons, available resources, such as CPU and memory, might be reduced. Video player might have to use less resources and still keep audio and video in synchronization.

One way is to reduce the frame rate of video playback. However, not all video contents can have gradual frame rate reduction during decoding process. For example,

1.      If the GOP structure is IPBBPBB..IPBBPBB… with 30 frames per GOP, original frame rate is 30 fps, and B frame is not used for reference but I and P frames are used for reference, the frame rates available are 30 fps, ~20 fps, ~10fps and 1 fps for artifacts free playback. With those intermediate artifacts-free frame rates, in general enough resources can be freed. 

2.      However, if the GOP structure is IPPPP..IPPP… with 30 frames per GOP, original frame rate is 30 fps, and each frame is used for reference, the only frame rates available are 30 fps and 1 fps for artifacts free playback. The frame rate reduction is too drastic.

For video contents in case 1, video decoder can drop non-reference frames and reduce frame rate gradually to free enough resources. It can free resources for decoding of compressed pictures, processing and rendering of uncompressed pictures. However, for video contents in case 2, video decoder itself can’t achieve gradual frame rate reduction. Instead, it might have to reduce frame rate by dropping some decoded frames, which won’t reduce resources for decoding of compressed picture but only processing and rendering of uncompressed pictures.

To achieve optimized frame rate reduction, decoder need to detect whether the content is in case 1 or case 2. That is, decoder need to accumulate some statistics about the percentage of non-reference pictures in the video sequence, and application or system pipeline can make optimized decision based on the video content properties.