Methods and apparatus to facilitate enhancing the quality of video
地域：
CA
CA San Diego
摘要
The present disclosure relates to methods and devices for facilitating enhancing the quality of video. An example method disclosed herein includes estimating an optical flow between a first noisy frame and a second noisy frame, the second noisy frame following the first noisy frame. The example method also includes warping a first enhanced frame to align with the second noisy frame, the warping being based on the estimation of the optical flow between the first noisy frame and the second noisy frame, the first enhanced frame being an enhanced frame of the first noisy frame. The example method also includes generating a second enhanced frame based on the warped first enhanced frame and the second noisy frame, and outputting the second enhanced frame.
說明書
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
This application claims the benefit of U.S. Provisional Application Ser. No. 62/822,599, entitled “METHODS AND APPARATUS TO FACILITATE ENHANCING THE QUALITY OF VIDEO” and filed on Mar. 22, 2019, which is expressly incorporated by reference herein in its entirety.
The present disclosure relates generally to processing systems and, more particularly, to one or more techniques for facilitating enhancing the quality of video.
Digital video capabilities can be incorporated into a wide range of devices, including digital televisions, digital direct broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-book readers, digital cameras, digital recording devices, digital media players, video gaming devices, video game consoles, cellular or satellite radio telephones, so-called “smart phones,” video teleconferencing devices, video streaming devices, and the like. Digital video devices implement video coding techniques, such as those described in the standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264/MPEG-4, Part 10, Advanced Video Coding (AVC), ITU-T H.265/High Efficiency Video Coding (HEVC), and extensions of such standards. The video devices may transmit, receive, encode, decode, and/or store digital video information more efficiently by implementing such video coding techniques.
權(quán)利要求
What is claimed is:1. A method of video processing, comprising:estimating an optical flow between a first noisy frame (Xt?1) and a second noisy frame (Xt), the second noisy frame (Xt) following the first noisy frame (Xt?1); warping a first enhanced frame (Yt?1) to align with the second noisy frame (Xt), the warping being based on the estimation of the optical flow between the first noisy frame (Xt?1) and the second noisy frame (Xt), the first enhanced frame (Yt?1) being an enhanced frame of the first noisy frame (Xt?1); generating a second enhanced frame (Yt) based on the warped first enhanced frame (Yt?1) and the second noisy frame (Xt); and outputting the second enhanced frame (Yt). 2. The method of claim 1 , further comprising:estimating an optical flow between a third noisy frame (Xt?2) and the first noisy frame (Xt?1), the third noisy frame (Xt?2) preceding the first noisy frame (Xt?1); warping a third enhanced frame (Yt?2) to align with the first noisy frame (Xt?1), the warping being based on the estimation of the optical flow between the third noisy frame (Xt?2) and the first noisy frame (Xt?1), the third enhanced frame (Yt?2) being an enhanced frame of the third noisy frame (Xt?2); and generating the first enhanced frame (Yt?1) based on the warped third enhanced frame (Yt?2) and the first noisy frame (Xt?1). 3. The method of claim 2 , further comprising outputting the first enhanced frame (Yt?1) before outputting the second enhanced frame (Yt). 4. The method of claim 1 , wherein the first noisy frame (Xt?1) and the second noisy frame (Xt) are decoded frames of a compressed video. 5. The method of claim 1 , further comprising decompressing a compressed video to generate the first noisy frame (Xt?1) and the second noisy frame (Xt). 6. The method of claim 1 , wherein the estimating of the optical flow is based on parameters determined during training. 7. The method of claim 6 , wherein determining the parameters during training comprises estimating an optical flow between a first training frame and a second training frame, the first training frame and the second training frame being consecutive frames, warping the first training frame based on the estimated optical flow, and utilizing a difference between the warped first training frame and the second training frame as a loss to train the parameters. 8. The method of claim 1 , wherein the generating of the second enhanced frame is based on parameters determined during training. 9. The method of claim 8 , wherein the parameters are determined during training based on a dataset comprising original training content and modified training content, the modified training content being a compressed and decompressed version of the original training content. 10. The method of claim 1 , wherein the optical flow between the first noisy frame (Xt?1) and the second noisy frame (Xt) identifies a movement of a pixel from the first noisy frame (Xt?1) to the second noisy frame (Xt). 11. The method of claim 10 , wherein warping the first enhanced frame (Yt?1) to align with the second noisy frame (Xt) is applying the movement identified in the optical flow to the pixel in the first enhanced frame (Yt?1). 12. An apparatus for video processing, comprising:a memory; and at least one processor coupled to the memory and configured to:estimate an optical flow between a first noisy frame (Xt?1) and a second noisy frame (Xt), the second noisy frame (Xt) following the first noisy frame (Xt?1); warp a first enhanced frame (Yt?1) to align with the second noisy frame (Xt), the warping being based on the estimation of the optical flow between the first noisy frame (Xt?1) and the second noisy frame (Xt), the first enhanced frame (Yt?1) being an enhanced frame of the first noisy frame (Xt?1); generate a second enhanced frame (Yt) based on the warped first enhanced frame (Yt?1) and the second noisy frame (Xt); and output the second enhanced frame (Yt). 13. The apparatus of claim 12 , wherein the at least one processor is further configured to:estimate an optical flow between a third noisy frame (Xt?2) and the first noisy frame (Xt?1), the third noisy frame (Xt?2) preceding the first noisy frame (Xt?1); warp a third enhanced frame (Yt?2) to align with the first noisy frame (Xt?1), the warping being based on the estimation of the optical flow between the third noisy frame (Xt?2) and the first noisy frame (Xt?1), the third enhanced frame (Yt?2) being an enhanced frame of the third noisy frame (Xt?2); and generate the first enhanced frame (Yt?1) based on the warped third enhanced frame (Yt?2) and the first noisy frame (Xt?1). 14. The apparatus of claim 13 , further comprising outputting the first enhanced frame (Yt?1). 15. The apparatus of claim 12 , wherein the first noisy frame (Xt?1) and the second noisy frame (Xt) are decoded frames of a compressed video. 16. The apparatus of claim 12 , further comprising decompressing a compressed video to generate the first noisy frame (Xt?1) and the second noisy frame (Xt). 17. The apparatus of claim 12 , wherein the at least one processor is further configured to estimate the optical flow based on parameters determined during training. 18. The apparatus of claim 17 , wherein determining the parameters during training comprises estimating an optical flow between a first training frame and a second training frame, the first training frame and the second training frame being consecutive frames, warping the first training frame based on the estimated optical flow, and utilizing a difference between the warped first training frame and the second training frame as a loss to train the parameters. 19. The apparatus of claim 12 , wherein the at least one processor is further configured to generate the second enhanced frame based on parameters determined during training. 20. The apparatus of claim 19 , wherein the parameters are determined during training based on a dataset comprising original training content and modified training content, the modified training content being a compressed and decompressed version of the original training content. 21. The apparatus of claim 12 , wherein the optical flow between the first noisy frame (Xt?1) and the second noisy frame (Xt) identifies a movement of a pixel from the first noisy frame (Xt?1) to the second noisy frame (Xt). 22. The apparatus of claim 21 , wherein warping the first enhanced frame (Yt?1) to align with the second noisy frame (Xt) is applying the movement identified in the optical flow to the pixel in the first enhanced frame (Yt?1). 23. A computer-readable medium storing computer executable code for video processing, comprising code to:estimate an optical flow between a first noisy frame (Xt?1) and a second noisy frame (Xt), the second noisy frame (Xt) following the first noisy frame (Xt?1); warp a first enhanced frame (Yt?1) to align with the second noisy frame (Xt), the warping being based on the estimation of the optical flow between the first noisy frame (Xt?1) and the second noisy frame (Xt), the first enhanced frame (Yt?1) being an enhanced frame of the first noisy frame (Xt?1); generate a second enhanced frame (Yt) based on the warped first enhanced frame (Yt?1) and the second noisy frame (Xt); and output the second enhanced frame (Yt). 24. The computer-readable medium of claim 23 , further comprising code to:estimate an optical flow between a third noisy frame (Xt?2) and the first noisy frame (Xt?1), the third noisy frame (Xt?2) preceding the first noisy frame (Xt?1); warp a third enhanced frame (Yt?2) to align with the first noisy frame (Xt?1), the warping being based on the estimation of the optical flow between the third noisy frame (Xt?2) and the first noisy frame (Xt?1), the third enhanced frame (Yt?2) being an enhanced frame of the third noisy frame (Xt?2); and generate the first enhanced frame (Yt?1) based on the warped third enhanced frame (Yt?2) and the first noisy frame (Xt?1). 25. The computer-readable medium of claim 23 , wherein the first noisy frame (Xt?1) and the second noisy frame (Xt) are decoded frames of a compressed video. 26. The computer-readable medium of claim 23 , further comprising code to decompress a compressed video to generate the first noisy frame (Xt?1) and the second noisy frame (Xt). 27. The computer-readable medium of claim 23 , wherein the estimating of the optical flow is based on parameters determined during training, and wherein determining the parameters during training comprises estimating an optical flow between a first training frame and a second training frame, the first training frame and the second training frame being consecutive frames, warping the first training frame based on the estimated optical flow, and utilizing a difference between the warped first training frame and the second training frame as a loss to train the parameters. 28. The computer-readable medium of claim 23 , wherein the generating of the second enhanced frame is based on parameters determined during training, and wherein the parameters are determined during training based on a dataset comprising original training content and modified training content, the modified training content being a compressed and decompressed version of the original training content. 29. The computer-readable medium of claim 23 , wherein the optical flow between the first noisy frame (Xt?1) and the second noisy frame (Xt) identifies a movement of a pixel from the first noisy frame (Xt?1) to the second noisy frame (Xt). 30. The computer-readable medium of claim 29 , wherein warping the first enhanced frame (Yt?1) to align with the second noisy frame (Xt) is applying the movement identified in the optical flow to the pixel in the first enhanced frame (Yt?1).
意見反饋
使用該功能遇到問題