With the surging demand for video content at any time and anywhere, mobile network operators are looking for the latest solutions to manage the burden on infrastructure resources while optimizing the consumer experience.
The introduction of Versatile Video Coding (VVC), the latest coding standard to emerge from JVET, has been developed to achieve sizable compression advantages over previous codecs. We caught up with Yan Ye, with Alibaba’s DAMO Academy, and Aytac Biber, MC-IF board member and Qualcomm Standards and Industry Alliances, to discuss the latest trends and performance data surrounding VVC. Here is what they had to say:
Q:What do we mean when we talk about the edge and, and where is it that you are seeing VVC begin to play a role there? What are the kinds of things and experiences and applications that take place at the edge where VVC may have an important role to play?
Biber: Certainly, when talking about the edge, the meaning will vary from person to person often depending on the audience. But when I speak about devices on the edge, I think of smartphones, tablets, IoT devices like embedded systems where bandwidth, power and compute resources are often constrained.
With the advanced compression VVC offers, there are several benefits for these types of devices. At the top level, with VVC providing up to 40% compression gain, those benefits are reduced data usage and reduced streaming cost when it comes to video playback. Regarding consumers, the benefits are minimized buffering and improved video start times that collectively improve the video quality of experience.
Q: Perhaps we can dive into which edge devices and video services you’d see as standing to benefit the most from this latest compression technology that is embodied in VVC?
Ye: I would definitely agree that a lot of devices and services can benefit. Let’s start with discussing what VVC provides to mobile devices and live streaming services.
Although not as prevalent in the U.S., we know that in many countries people take public transportation to commute to and from work. While commuting, they often watch a lot of videos on their phones during this time.
This behavior tends to create a lot of peak bandwidth usage for the mobile network operators and content distribution network (CDN) providers. On demand streaming of live content is even more demanding. The challenge is that there are few other ways to offload the traffic.
Aytac mentioned how VVC can offer up to 40% gains through compression. Even a reduction of 20% or 30% in peak bandwidth usage, creates significant benefits for service providers. So VVC could certainly reduce significant cost to the service providers, and benefit users as well by reducing network congestion.
The higher coding efficiency from VVC enables networks to be less congested which means faster startup times, fewer failed deliveries and fewer video stalls for end users.
Q: With so much effort these days pouring into creating those optimal conditions for content delivery, what do you see VVC doing, especially in that last mile? How is it that you’re seeing it specifically manifest and deliver an enhanced experience, especially when many people are on the go and there’s so much demand?
Ye: Absolutely. That is the key phrase in your question, on the go. When people are “on the go,” and sometimes even when they are not, the last mile in video delivery is where difficulties arise for most network traffic. That last mile is usually the most bandwidth constrained or the least reliable or a combination of the two.
The example of peak usage during commute time highlights the issues that stem from congestion on mobile networks. In crowded areas, bandwidth tends to go down and latency becomes longer, which could also apply to satellite communication as well. Greater efficiency in video delivery is the most critical in the last mile because it can manifest in faster startup times for users, less dropped calls, and significantly improve the user experience overall.
Biber: Perhaps I can give a kind of different perspective as it relates to the last mile and broadcast.
Much of our discussion thus far has been about unicast because it’s become the major form of streaming entertainment or video calls. But broadcast is also important. Maybe I can provide additional context based on my experience in the work I’m doing with Qualcomm on the LTE based 5G broadcast.
Like unicast networks, broadcast operates within resource constraints, specifically from the limited spectrum that is allocated to each broadcaster. From a broadcaster’s perspective, the adoption of a more efficient video compression such as VVC, would directly translate into 20% or 30 % more services such as live TV or radio channels within the bounds of allocated spectrum.
As a network topology, broadcast remains the most spectrum efficient method for delivering live media to large audiences. The summer games last year, as an example, required delivery of bits to millions of people with the smallest latency.
Just as VVC benefits unicast delivery of video streaming and video calls, I want to emphasize that VVC can significantly enhance broadcast services as well.
Q: Is there a competitive advantage for content distributors implementing applications that leverage VVC? What new things can be done?
Ye: Yes, I think competitive advantage is very important for new initiatives and innovative services. When VVC was being designed the versatility was one of the more important elements, keeping the coding of 360 degree video top of mind as an example.
Recently we were able to provide a demonstration to a major 360 degree video service provider and device manufacturer, showing the clear visual quality benefits using VVC compared to using HEVC. With 360 degree video there are a lot of seams because multiple camera feeds are stitched together to create the spherical effect. We were able to illustrate that VVC offers much better quality. Overall VVC offers significant rate savings, which is even more important for these types of applications because they need larger resolutions like 8K and above. Rate savings of 30-40% of a very large resolution video directly translates to significant reduction in cloud storage for their users for this device manufacturer.
The benefits don’t stop at new applications like 360 degree video. Even for the normal 2D content that we consume a lot these days, upgrade cycles on mobile service have evolved from 540p to 720p, and those that were already serving 720p moved up to 1080p.
VVC’s higher efficiency provides a very natural advantage over other codecs for these types of service modernizations because it reduces or eliminates the additional bandwidth cost for service providers to upgrade to higher quality video delivery.
Q: Do you see it as an opportunity for service providers to separate themselves from the pack?
Biber: Certainly, these two things here go hand in hand. Based on my observations of the mobile device category, as the mobile displays have evolved with higher pixel densities, high dynamic range capabilities and faster refresh rates, we have elevated consumer expectations.
Consumer expectations for video quality have risen accordingly. While 540p was good enough in recent years, it has been replaced with expectations for high resolution — at 720p or 1080p — as the baseline.
In fact, we crossed the bar where most people have more pixel density on their phones than they do on their living room TVs.
With the proliferation of 5G and advanced compression technologies, high resolution streaming has become more accessible than ever. Users now equate high resolution with quality immersion value, and this is precisely where a new format like VVC can make a meaningful impact by enabling this elevated experience.
The service companies are pushing the bar in response to mobile display capabilities rising. It’s going hand in hand, and improvements on both ends make sense.
Q: At what point do you see this competitive advantage becoming a competitive requirement?
Biber: Let’s say that while having a single universal video format would be the ideal scenario for everyone involved, it’s not a practical reality. Content providers like Alibaba and similar companies must continue supporting legacy devices that have been in people’s hands for the last 10 years or so. As a result, content providers need to come up with a plan to service all the legacy devices as well.
VVC is an important part of that plan.
Currently, VVC is being used alongside AV1, HEVC, AVC, some of the earlier codecs. My anticipation is that VVC will continue to grow over this year and in the coming years. It will coexist among other codecs, but it will continue to take a share from other formats in the foreseeable future. This process has begun and will accelerate.
Q: Can VVC be implemented without disrupting current distribution infrastructures, avoiding a “rip and replace” scenario?
Ye: That’s a good question. I see VVC as an additional service that will be provided alongside HEVC and AVC deployments to serve those legacy devices.
For certain types of content or for some applications, a software-based VVC solution can already be deployed onto existing mobile devices or other types of devices. And you only need to make a relatively small amount of changes to your operation to enable this.
I see a smooth transition from HEVC and AVC to VVC.
Q: Traditionally, a software-based solution has been seen as inefficient, energy intensive and not ideal. But VVC has actually proven itself to be quite amenable to its software version. Can you shed a little bit of light on why that is?
Biber: It’s all about focus, and seeing the desire. We have been able to demonstrate well optimized VVC software implementations with our partner companies, Alibaba and Tencent, running on Qualcomm devices at exceptionally high performance points such as 1080p and 4K. We have also demonstrated 1080p and even 4K on mobile platforms.
We achieved 1080p at a high refresh rate on laptops, emphasizing the role of software based VVC implementations to support game streaming, for example.
So, we have seen very impressive VVC software implementations running on currently available and deployed devices. We don’t need to wait to glean the benefits of this codec; it’s within reach right now.
Q: As we move forward, what are some of the investments that need to be made to begin mainstreaming VVC into our environment?
Ye: Proper upgrade cycles to the software in apps and services are necessary to ensure robustness. However, we don’t see investment costs to be very high, because it’s running on existing infrastructure. There are also hardware upgrade cycles. Some consumers upgrade their mobile phones at a faster pace than other devices, which means higher compute capability getting into more people’s hands. That provides a positive feedback loop to put more software-based VVC solutions into more devices.
As VVC continues to ramp up and co-exist alongside HEVC and AVC, we’ll also see hardware based encoder and decoder come onto the market for integration onto chip sets for mobile devices, PCs and automobiles.
Meanwhile, hardware capabilities for VVC will reach people as part of the hardware upgrade cycle. The investment will be incremental rather than disruptive or prohibitively expensive.
Let’s dig into Aytac’s points about optimized software for VVC encoder and decoder. We have established that 1080p represents the baseline for mobile services today, so we can explore an example of 1080p software encoding and decoding on a device that is running on a Snapdragon that has been in the mainstream market for two years.
We can see that software decoding by a single thread can achieve more than 120 frames per second with no problems. Memory usage is less than 100 megabytes which is very reasonable.
Although software encoding on the device is more intensive, even at four threads – running on the same phone that’s two years old – we can reach 30 frames per second and achieve 50% rate reduction compared to HEVC on that same device. Those are just some data points that we see in the field on devices today.
Q: Any other observations on the economics and how they shift as we introduce VVC and it matures into the infrastructure and into our environment?
Biber: I appreciate Yan sharing those data points which should be encouraging for some of the industry skepticism that we see around VVC in software.
This is something our content platforms partners will see from their operational view. Each of them has their own internal model to define at what point it becomes financially viable to adopt a new video compression format like VVC. Adding a new codec doesn’t mean they can take out an old codec. It is additive.
The break-even point will not be similar from company A to company B, and the factors in consideration will vary. For example, long form versus short form platforms will have different calculations.
The number of devices that can support the new format within each platform’s network also plays a role in this calculation. The type of content, whether it is anime, drama or user generated content, have different characteristics and different implications on compression efficiencies.
That said, I believe we have reached a tipping point for most major content platforms where incorporating VVC makes economic sense, especially given the growing pace of devices that can be served with the software decoder. The momentum is there and it’s only going to accelerate as more companies come on board and start taking advantage of VVC.
Q: Any other sort of perspectives on the return on investment picture as folks evaluate their options here in the near, mid and long term?
Ye: Yeah, definitely. For most of the content providers and service providers, there is a threshold. Each has a different financial situation and formula, whether its compute cost versus bandwidth cost versus the size of their user base. There is a threshold to decide whether it makes financial sense to add VVC.
In our experience, the threshold is not very high. A long form content provider or a short form content provider can benefit from adding VVC to their content. It depends on whether the service provider has a sufficient base of popular content. The more people are viewing the highly popular content, which most service providers have, then adding VVC makes financial sense. That’s our experience.
Service providers that have a lot of viral content see the connection with rising popular content and the return on investment of implementing VVC. I can say with near certainty that service providers with viral content will benefit significantly from adding VVC to their service.
Click here to view the full vidcast interview.
###