Immersive media streaming prioritized by frequency of asset reuse
地域：
CA
CA Palo Alto
摘要
Packaging media for optimizing immersive media distribution of a media steam performed by at least one processor, is provided, including: receiving immersive media data for an immersive presentation; acquiring asset information associated with media assets corresponding to a set of scenes included in the immersive media data for the immersive presentation; analyzing characteristics of the media assets based on the asset information, the characteristics comprising an asset type associated with a respective media asset and a frequency that indicates a number of times the respective media asset is used among the set of scenes included in the immersive presentation; ordering the media assets in a sequence based on the asset type and the frequency associated with each of the media assets; and streaming the immersive media data for the immersive presentation based on the ordered sequence of the media assets.
說明書
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
This application is based on and claims priority to U.S. Provisional Patent Application No. 63/276,536, filed on Nov. 5, 2021, the disclosure of which is incorporated by reference herein in its entirety.
The present disclosure describes embodiments generally related to architectures, structures and components for systems and networks that distribute media, including video, audio, geometric (3D) objects, haptics, associated metadata, or other content for a client presentation device. Some embodiments are directed to systems, structures, and architectures for distribution of media content to heterogeneous immersive and interactive client presentation devices.
Immersive Media generally refers to media that stimulates any or all human sensory systems (e.g., visual, auditory, somatosensory, olfactory, and possibly gustatory senses) to create or enhance the perception of the user being physically present in the experience of the media, i.e., beyond what is distributed over existing (e.g., “l(fā)egacy”) commercial networks for timed two-dimensional (2D) video and corresponding audio; such timed media also being known as “l(fā)egacy media” Immersive media may also be defined as media that attempts to create, or imitate the physical world through digital simulation of kinetics and laws of physics, thereby stimulating any or all human sensory systems so as to create the perception by the user of being physically present inside a scene that depicts a real or virtual world.
權利要求
What is claimed is:1. A method for packaging media for optimizing immersive media distribution, performed by at least one processor, the method comprising:receiving immersive media data for an immersive presentation; acquiring asset information associated with media assets included in a set of scenes included in the immersive media data for the immersive presentation, wherein the media assets are three dimensional (3D) objects having multiple layers; analyzing characteristics of the media assets based on the asset information, the characteristics comprising an asset type associated with a respective media asset and a frequency associated with the respective media asset that indicates a number of times the respective media asset is used among the set of scenes included in the immersive presentation, wherein the frequency associated with the respective media asset is indicated in a base layer among the multiple layers associated with the respective media asset; and ordering the media assets in a sequence based on the asset type and the frequency associated with each of the media assets. 2. The method according to claim 1 , wherein the sequence of the media assets are first ordered by the asset type, then ordered by increasing or decreasing frequency. 3. The method according to claim 1 , wherein the immersive media data includes one or more scenes and the one or more scenes are timed, untimed, or a combination of timed and untimed scenes. 4. The method according to claim 1 , wherein the asset information includes a base representation of the respective media asset and a set of asset enhancement layers, the set of asset enhancement layers including attribute information corresponding to the characteristics of the media assets, andwherein when the set of asset enhancement layers are applied to the base representation of the media asset, the base representation of the respective media asset is augmented to include features that are not supported in the base layer containing the base representation of the media asset. 5. The method according to claim 1 , further comprising:separating the sequence of the media assets into individual packets for representation and streaming on a network; and streaming the immersive media data for the immersive presentation based on the ordered sequence of the media assets. 6. The method according to claim 1 , further comprising determining if a format of the immersive media data corresponding to scenes in the set of scenes is to be transformed from a first format to a second format before immersive media distribution, based on a complexity of the scenes; anddetermining, based on a determination that the immersive media data corresponding to the scenes is to be transformed, if a source of the immersive media data or a client of the immersive media data is to perform a transformation from the first format to the second format. 7. The method according to claim 1 , further comprising:determining if the respective media asset has previously been streamed; and if it is determined that the respective media asset has previously been streamed, creating a proxy to substitute the respective media asset in the sequence of the media assets. 8. A device for packaging media for optimizing immersive media distribution, the device comprising:at least one memory configured to store computer program code; and at least one processor configured to read the computer program code and operate as instructed by the computer program code, the computer program code including:receiving code configured to cause the at least one processor to receive immersive media data for an immersive presentation; acquiring code configured to cause the at least one processor to acquire asset information associated with media assets included in a set of scenes included in the immersive media data for the immersive presentation, wherein the media assets are three dimensional (3D) objects having multiple layers; analyzing code configured to cause the at least one processor to analyze characteristics of the media assets based on the asset information, the characteristics comprising an asset type associated with a respective media asset and a frequency associated with the respective media asset that indicates a number of times the respective media asset is used among the set of scenes included in the immersive presentation, wherein the frequency associated with the respective media asset is indicated in a base layer among the multiple layers associated with the respective media asset; and sequencing code configured to cause the at least one processor to order the media assets in a sequence based on the asset type and the frequency associated with each of the media assets. 9. The device of claim 8 , wherein the ordered sequence of the media assets are first ordered by the asset type, then ordered by increasing or decreasing frequency. 10. The device of claim 8 , wherein the immersive media data includes one or more scenes and the one or more scenes are timed, untimed, or a combination of timed and untimed scenes. 11. The device of claim 8 , wherein the asset information includes a base representation of the respective media asset and a set of asset enhancement layers, the set of asset enhancement layers including attribute information corresponding to the characteristics of the media assets, andwherein when the set of asset enhancement layers are applied to the base representation of the media asset, the base representation of the respective media asset is augmented to include features that are not supported in the base layer containing the base representation of the media asset. 12. The device of claim 8 , the computer program code further including:separating code configured to cause the at least one processor to separate the ordered sequence of the media assets into individual packets for representation and streaming on a network; and streaming code configured to cause the at least one processor to stream the immersive media data for the immersive presentation based on the ordered sequence of the media assets. 13. The device of claim 8 , the computer program code further including:format determining code configured to cause the at least one processor to determine if a format of the immersive media data corresponding to scenes in the set of scenes is to be transformed from a first format to a second format before immersive media distribution, based on a complexity of the scenes; and transformation determining code configured to cause the at least one processor to determine, based on a determination that the immersive media data corresponding to the scenes is to be transformed, if a source of the immersive media data or a client of the immersive media data is to perform a transformation from the first format to the second format. 14. The device of claim 8 , the computer program code further including:determining code configured to cause the at least one processor to determine if the respective media asset has previously been streamed; and proxy creating code configured to cause the at least one processor to create a proxy to substitute the respective media asset in the ordered sequence of the media assets, if it is determined that the respective media asset has previously been streamed. 15. A non-transitory computer-readable medium storing instructions that, when executed by at least one processor of a device for packaging media for optimizing immersive media distribution, cause the at least one processor to:receive immersive media data for an immersive presentation; acquire asset information associated with media assets included in a set of scenes included in the immersive media data for the immersive presentation, wherein the media assets are three dimensional (3D) objects having multiple layers; analyze characteristics of the media assets based on the asset information, the characteristics comprising an asset type associated with a respective media asset and a frequency associated with the respective media asset that indicates a number of times the respective media asset is used among the set of scenes included in the immersive presentation, wherein the frequency associated with the respective media asset is indicated in a base layer among the multiple layers associated with the respective media asset; and order the media assets in a sequence based on the asset type and the frequency associated with each of the media assets. 16. The non-transitory computer-readable medium of claim 15 , wherein the ordered sequence of the media assets are first ordered by the asset type, then ordered by increasing or decreasing frequency. 17. The non-transitory computer-readable medium of claim 15 , wherein the instructions further cause the at least one processor to:separate the ordered sequence of the media assets into individual packets for representation and streaming on a network; and stream the immersive media data for the immersive presentation based on the ordered sequence of the media assets. 18. The non-transitory computer-readable medium of claim 15 , wherein the asset information includes a base representation of the respective media asset and a set of asset enhancement layers, the set of asset enhancement layers including attribute information corresponding to the characteristics of the media assets, andwherein when the set of asset enhancement layers are applied to the base representation of the media asset, the base representation of the respective media asset is augmented to include features that are not supported in the base layer containing the base representation of the media asset. 19. The non-transitory computer-readable medium of claim 15 , wherein the instructions further cause the at least one processor to:determine if a format of the immersive media data corresponding to scenes in the set of scenes is to be transformed from a first format to a second format before immersive media distribution, based on a complexity of the scenes; and determine, based on a determination that the immersive media data corresponding to the scenes is to be transformed, if a source of the immersive media data or a client of the immersive media data is to perform a transformation from the first format to the second format. 20. The non-transitory computer-readable medium of claim 15 , wherein the instructions further cause the at least one processor to:determine if the respective media asset has previously been streamed; and create a proxy to substitute the respective media asset in the ordered sequence of the media assets, if it is determined that the respective media asset has previously been streamed.
意見反饋