Region-based reference frame management
地域：
Markham
摘要
Disclosed herein is a region-based reference management system using in video frame encoding. Source content, such as video game streaming or remote desktop sharing, that includes scene changes or significant instantaneous changes in a region from one frame to the next can present encoding challenges. Techniques disclosed herein use hints about changes in regional frame content, dissect frame content into regions, and associate the dissected regions with stored reference frame data using the hints and information about the regions to more efficiently encode frames.
說明書
A popular use-case for video encoding involves a user playing a video game while the screen content is being encoded and streamed to other users watching in real-time. Another popular use-case for video encoding is screen-sharing wherein a user's screen is encoded and streamed to other viewers. Yet another popular use case for video encoding is remote/virtual desktop environments where computer-generated images are rendered in the cloud and the content is encoded and streamed to a remote user.
A common occurrence for all of the above scenarios is the instantaneous change of the source content. For example, in the video game streaming scenario, a frame instantaneously shows an “inventory list” for a game character, or the frame instantaneously shows a top-view of the “game world”.
Instantaneous changes from one frame to the next frame result in large differences between successive frames being encoded. A conventional encoder encodes a current frame using the previous frame as a reference and the difference between the current frame and the reference frame. Thus, the conventional encoder needs to encode a large difference when these instantaneous changes occur, which results in poor compression performance due to the low inter-frame correlation. The low inter-frame correlation encountered by a conventional encoder results in compression challenges such as poor quality, high latency, and/or bandwidth challenges when encoding in real-time.
權(quán)利要求
What is claimed is:1. A method comprising:receiving a first frame and a first indication associated with the first frame; storing a reconstructed version of the first frame and the first indication; receiving a second frame, wherein the second frame is received after the first frame; storing a reconstructed version of the second frame; receiving a third frame and another indication associated with the third frame, wherein the third frame is received after the second frame; determining that the first indication matches the another indication; and encoding the third frame, wherein the encoding includes:encoding a portion of the third frame using the reconstructed version of the first frame based on the first indication and the another indication; and encoding at least some of a remaining portion of the third frame using the reconstructed version of the second frame. 2. The method of claim 1 , wherein the encoding further includes encoding the portion of the third frame using a corresponding portion of the reconstructed version of the first frame. 3. The method of claim 2 further comprising:analyzing the third frame; and identifying the corresponding portion of the reconstructed version of the first frame. 4. The method of claim 1 , wherein the first indication identifies a window in the first frame and the another indication identifies the window in the third frame. 5. The method of claim 1 , further comprising:dividing the first frame into multiple regions; and dividing the third frame into multiple regions. 6. The method of claim 5 , wherein the dividing the first frame and the dividing the third frame is based on an analysis of a content of the first frame and the third frame. 7. The method of claim 1 , further comprising:encoding the first frame; encoding the second frame; and streaming the encoded first frame, the encoded second frame, and the encoded third frame. 8. The method of claim 1 , further comprising:tracking a cost of encoding the third frame using the first indication and the another indication; and adjusting a choice of a future reference frame based on the tracking. 9. The method of claim 1 , further comprising:analyzing the reconstructed version of the first frame and the third frame based on the first indication and the another indication; identifying a region in both the reconstructed version of the first frame and the third frame that contains same content; and updating location information associated with the region and storing the location information along with a reconstructed version of the third frame. 10. A non-transitory computer readable storage medium included in an apparatus, the non-transitory computer readable storage medium comprising executable instructions that when executed by at least one processor, cause the apparatus to:receive a first frame and a first indication associated with the first frame; store a reconstructed version of the first frame and the first indication; receive a second frame, wherein the second frame is received after the first frame; store a reconstructed version of the second frame; receive a third frame and another indication associated with the third frame, wherein the third frame is received after the second frame; determine that the first indication matches the another indication; and encode the third frame, wherein:a portion of the third frame is encoded using the reconstructed version of the first frame based on the first indication and the another indication; and at least some of a remaining portion of the third frame is encoded using the reconstructed version of the second frame. 11. The non-transitory computer readable storage medium of claim 10 , wherein the executable instructions, when executed by the at least one processor, further cause the apparatus to:encode the portion of the third frame using a corresponding portion of the reconstructed version of the first frame. 12. The non-transitory computer readable storage medium of claim 11 , wherein the executable instructions, when executed by the at least one processor, further cause apparatus to:analyze the third frame; and identify the corresponding portion of the reconstructed version of the first frame. 13. The non-transitory computer readable storage medium of claim 10 , wherein the first indication identifies a window in the first frame and the another indication identifies the window in the third frame. 14. The non-transitory computer readable storage medium of claim 10 , wherein the executable instructions, when executed by the at least one processor, further cause the apparatus to:divide the first frame into multiple regions; and divide the third frame into multiple regions. 15. The non-transitory computer readable storage medium of claim 14 , wherein the first frame and the third frame are divided based on an analysis of a content of the first frame and the third frame. 16. The non-transitory computer readable storage medium of claim 10 , wherein the executable instructions, when executed by the at least one processor, further cause the apparatus to:encode the first frame; encode the second frame; and stream the encoded first frame, the encoded second frame, and the encoded third frame. 17. The non-transitory computer readable storage medium of claim 10 , wherein the executable instructions, when executed by the at least one processor, further cause the apparatus to:track a cost of encoding the third frame using the first indication and the another indication; and adjust a choice of a future reference frame based on the tracking. 18. The non-transitory computer readable storage medium of claim 10 , wherein the executable instructions, when executed by the at least one processor, further cause the apparatus to:analyze the first frame and the third frame based on the first indication and the another indication; identify a region in both the first frame and the third frame that contains same content; and update location information associated with the region and storing the location information along with the third frame. 19. A computing system comprising:a memory storing operational instructions thereon; and a processor operationally coupled to the memory, wherein the processor, when executing the operational instructions, is configured to: receive a first frame and a first indication associated with the first frame; store a reconstructed version of the first frame and the first indication; receive a second frame, wherein the second frame is received after the first frame; store a reconstructed version of the second frame; receive a third frame and another indication associated with the third frame, wherein the third frame is received after the second frame; determine that the first indication matches the another indication; encode the third frame, wherein:a portion of the third frame is encoded using the reconstructed version of the first frame based on the first indication and the another indication; and at least some of a remaining portion of the third frame is encoded using the reconstructed version of the second frame; track a cost of encoding the third frame using the first indication and the another indication; and adjust a choice of a future reference frame based on the tracking. 20. The computer system of claim 19 , wherein the processor is further configured to:determine that the first indication matches the another indication. 21. The non-transitory computer readable storage medium of claim 10 , wherein the executable instructions, when executed by the at least one processor, further cause the apparatus to:track a cost of encoding the third frame using the first indication and the another indication; and adjust a choice of a future reference frame based on the tracking.
意見反饋
使用該功能遇到問題
該功能需要專業(yè)版企業(yè)版VIP權(quán)限，您可以：
您也可以聯(lián)系官方QQ: 2157717237 電話: 13264338900