Consumer video freezes when setPreferredLayers is executed continuously #990
Comments
|
It's hell to address issues reported based on mediasoup-demo behavior. We have verbose logs in the worker to explain what's going on regarding simulcast, layer switching, retransmissions and everything. Without those logs we cannot do anything. If you can replicate the issue locally and indicate what is indeed wrong as per printed logs, then we may investigate it. I'm afraid I have to close this issue otherwise. BTW simulcast in screensharing is experimental in libwebrtc so we don't know if your Chrome is properly reacting to PLIs received by mediasoup (there are also logs about that in mediasoup worker if properly enabled). In addition, BWE in mediasoup is being modernized in an ongoing PR. |
|
I got it. Thanks. The reproduction procedure is easy, so I will try to reproduce it locally tomorrow and investigate. |
|
My guess is that it appears to occur when setPreferredLayers is run again after the following logs and before the keyframe is received. I'll check the code when I have time. |
|
@ibc Should I still keep this closed? Still need something more? |
This is a legitimate warning, not a bug. Why it happens in this scenario? I don't know, but we don't know yet about any clear bug here. BTW there is an old PR that for which we never got time to review, not sure if related, probably not: #500 |
I understand that. Even with this log, if I run setPreferredLayers after receiving the keyframe, it appears to work correctly. By the way, can't we open an issue here if we don't understand the cause of the bug? |
|
Imagine this is a legitimate behavior (Chrome fails to encode upon receipt of too many PLIs). Is that a mediasoup bug? It would be open here forever. Let's please reopen this issue once there are some evidences that it's a bug in mediasoup. |
After the last setPreferredLayers was performed at this time, several keyframes were received in When it is working properly, the following log is output in response to RTC::SimulcastConsumer receiving a keyframe, but it is not output when consumer video is freezes |
|
@ibc After executing setPreferredLayers(spatial:0 temporal:2), spatialLayerToSync is set to 0 and is waiting to receive keyframes. The currentSpatialLayer remains unchanged at 2. If a keyframe with spatial:0 is received in that state, it is droped here. And |
|
Issue reopened |
In that case the consumer video was frozen.
In that case the consumer video was frozen.
In that case the consumer video was frozen.
In that case the consumer video was frozen.
This does not seem to be possible. |
|
Definitely I don't see the problem. I'm trying to draw the flow you described @satoren.
Where's the issue @satoren? |
|
Same feelings here. I exposed some scenarios in the PR: |
SimulcastConsumer continues to request key frames at short intervals. It does not appear that packets are being forwarded. It may be more problematic that I will share the code so that everyone can reproduce the problem relatively easily. (It would be best if I could write a test code that reproduces 100% of the problem, but I can't.) apply this patch to rust example # in rust/examples
RUST_LOG=debug cargo run --example svc-simulcast
# in rust/examples-frontend/svc-simulcast
yarn
yarn startPressing the up button repeatedly will freeze the video on the receiving end. Reproduce with chrome. I could not reproduce it in Safari because not out logs as |
|
I'll try it when I get some time. |
|
@satoren can we confirm (if possible) that the issue happens when the "cannot switch to this stream because high RTP extra offset" log is shown? That's indeed weak code and logic and probably we are not resetting some state/flags when that happens. |
|
Ok, let's imagine this scenario:
It will enter here: // Check whether this is the packet we are waiting for in order to update
// the current spatial layer.
if (this->currentSpatialLayer != this->targetSpatialLayer && spatialLayer == this->targetSpatialLayer)
{
// Ignore if not a key frame.
if (!packet->IsKeyFrame())
return;
shouldSwitchCurrentSpatialLayer = true;
// Need to resync the stream.
this->syncRequired = true;
this->spatialLayerToSync = spatialLayer;
}Here have set Then imagine that "cannot switch stream due to too high RTP timestamp extra offset needed" happens: else if (tsExtraOffset > maxTsExtraOffset)
{
MS_WARN_TAG(
simulcast,
"cannot switch stream due to too high RTP timestamp extra offset needed (%" PRIu32
"), requesting keyframe",
tsExtraOffset);
RequestKeyFrameForTargetSpatialLayer();
this->keyFrameForTsOffsetRequested = true;
return;
}Here we return but we haven't reset neither This means that we will drop all new packets that are no key frames: // If we need to sync and this is not a key frame, ignore the packet.
if (this->syncRequired && !packet->IsKeyFrame())
return;Not sure about the implications of not resetting |
…d due to too high RTP timestamp extra offset needed - It may fix #990. - Rationale here: #990 (comment). - Basically we should reset this->syncRequired` nor `this->spatialLayerToSync` when the keyframe that is supposed to switch to another spatial layer is discarded due to too high RTP timestamp extra offset needed.
|
PR created: #999 |
|
Very good news :) |
…d due to too high RTP timestamp extra offset needed (#999) - Fixes #990. - Rationale here: #990 (comment). - Basically we should reset this->syncRequired` nor `this->spatialLayerToSync` when the keyframe that is supposed to switch to another spatial layer is discarded due to too high RTP timestamp extra offset needed.
…d due to too high RTP timestamp extra offset needed (versatica#999) - Fixes versatica#990. - Rationale here: versatica#990 (comment). - Basically we should reset this->syncRequired` nor `this->spatialLayerToSync` when the keyframe that is supposed to switch to another spatial layer is discarded due to too high RTP timestamp extra offset needed.
Bug Report
When using simulcast and implementing switching with setPreferredLayers depending on the size to be displayed, we found a problem with video freezing in rare cases.
I have confirmed that it can be reproduced on the demo site, please see the video of the reproduction.
Execute setPreferredLayers again and the consumer video will start playing again.
Your environment
Issue description
Occurs on demo page too. https://v3demo.mediasoup.org/
videofreeze2.mov
It can be confirmed that a large amount of packet loss is occurring at the timing of the stop.
Logs reproduced locally
The text was updated successfully, but these errors were encountered: