Multimodal multimedia communication with link parameters optimisation

Abstract:
Recently, we introduced a problem, in which a given user can receive a description of certain event in several “modes” (say, audio and video, only audio or only text), and a “mode” may itself be available at various levels of “perceptual quality” (e.g., video can be useful at multiple levels of spatial, temporal and SNR (pixel “depth”) resolution). The “quality” for a given mode depends on the “encoding rate” at the receiver (information bits per second of media), which depends on the resources utilised. If resources are costly, or if many users must share limited resources, both the mode and the specific level of “quality” should be jointly optimised. Previously, we provided an analytical solution for the multiuser multimodal/multimedia resource allocation problem, based on the Karush-Kuhn-Tucker conditions. However, for simplicity, we assumed that we could directly allocate “encoding rate” (or bandwidth), which is not always realistic. Presently, we jointly optimise the mode together with link-layer variables such as power, symbol rate, and modulation (which determine the perceptual quality). That is, to each user we optimally assign a mode (video, audio, text, etc.) together with a vector of link-layer parameters.