closed [6985] nvdec stream corruption when decoding multiple videos at once #6985

Closed
opened 2026-01-02 20:03:40 -03:00 by Accord · 0 comments
Owner

Branch: __refs_pull_6985_head

Labels: nvdec

willox opened issue at 2021-09-08 13:06:

In Hatsune Miku Project DIVA Mega Mix, the song "Gaikotsu Gakudan to Riria" attempts to decode two h264 videos at once. Yuzu appears to be putting packets from both of these videos into a single avcodec stream.
The two videos are of the same resolution and length. One is the background of the scene while the other contains lighting information which I'm guessing exists to be applied over the 3d models, but it might have some other purpose.
Dumping the stream before it's sent to avcodec and passing it into ffprobe yields a bunch of errors. It's not a very happy stream:

[h264 @ 0x560c6921e280] reference picture missing during reorder
[h264 @ 0x560c6921e280] Missing reference picture, default is 65562
[h264 @ 0x560c6921e280] mmco: unref short failure
Last message repeated 1 times
[h264 @ 0x560c6921e280] illegal short term buffer state detected
Last message repeated 4 times
[h264 @ 0x560c6921e280] co located POCs unavailable
[h264 @ 0x560c6921e280] illegal short term buffer state detected
Last message repeated 13 times
[h264 @ 0x560c6921e280] co located POCs unavailable
[h264 @ 0x560c6921e280] illegal short term buffer state detected
(one million lines omitted)

Here's a video of what it looks like. The coloured circles that appear are from an entirely separate video file on the rom than the rest of the video.
https://user-images.githubusercontent.com/1499676/132513838-9ca6d444-7cfb-4cc4-968c-8db816b31760.mp4
And what it should look like: https://youtu.be/4v45irelP1w
The warnings in the attached logs from the stubbed methods in core\hle\service\mm\mm_u.cpp seem like they could be relevant. I modified that system a little to dish out unique identifiers (instead of 1 every time) and the game is certainly trying to call SetAndWait and Get on two different identifiers during this song.
I also thought maybe some register in NvdecRegisters could be used to signal that a different stream is being decoded but I couldn't spot anything like that being set.
yuzu_log_741.txt
yuzu_log_ea_2040.txt

Squall-Leonhart commented at 2023-11-08 13:39:

This occurs because all decoding shares the same context, and DTS order isn't respected, FFMPEG decoder is using PTS order where DTS is expected.

G-Spawn commented at 2023-12-05 17:59:

https://streamable.com/horb6x
Seemingly this was fixed a while back. Please feel free to open this again @willox if the issue persists on newer yuzu 🙏

[Export of Github issue for yuzu-emu/yuzu. Generated on 2024.02.28 at 10:04:32.]

**Branch:** [__refs_pull_6985_head](https://accord.dravee.dev/Accord/yuzu-mirror/src/branch/__refs_pull_6985_head) > **Labels**: `nvdec` #### <div><img width="50" height="50" src="https://avatars.githubusercontent.com/u/1499676?u=39c028bbec304564b2542cb9c7da8669cc57a118&v=4" width="50"></div>[willox](https://github.com/willox) opened issue at [2021-09-08 13:06](https://github.com/yuzu-emu/yuzu/issues/6985): > <!--- > Please keep in mind yuzu is EXPERIMENTAL SOFTWARE. > Please read the FAQ: > https://yuzu-emu.org/wiki/faq/ > THIS IS NOT A SUPPORT FORUM, FOR SUPPORT GO TO: > https://community.citra-emu.org/ > If the FAQ does not answer your question, please go to: > https://community.citra-emu.org/ > When submitting an issue, please check the following: > - You have read the above. > - You have provided the version (commit hash) of yuzu you are using. > - You have provided sufficient detail for the issue to be reproduced. > - You have provided system specs (if relevant). > - Please also provide: > - For any issues, a log file > - For crashes, a backtrace. > - For graphical issues, comparison screenshots with real hardware. > - For emulation inaccuracies, a test-case (if able). > --> > In Hatsune Miku Project DIVA Mega Mix, the song "Gaikotsu Gakudan to Riria" attempts to decode two h264 videos at once. Yuzu appears to be putting packets from both of these videos into a single avcodec stream. > The two videos are of the same resolution and length. One is the background of the scene while the other contains lighting information which I'm guessing exists to be applied over the 3d models, but it might have some other purpose. > Dumping the stream before it's sent to avcodec and passing it into `ffprobe` yields a bunch of errors. It's not a very happy stream: > ``` > [h264 @ 0x560c6921e280] reference picture missing during reorder > [h264 @ 0x560c6921e280] Missing reference picture, default is 65562 > [h264 @ 0x560c6921e280] mmco: unref short failure > Last message repeated 1 times > [h264 @ 0x560c6921e280] illegal short term buffer state detected > Last message repeated 4 times > [h264 @ 0x560c6921e280] co located POCs unavailable > [h264 @ 0x560c6921e280] illegal short term buffer state detected > Last message repeated 13 times > [h264 @ 0x560c6921e280] co located POCs unavailable > [h264 @ 0x560c6921e280] illegal short term buffer state detected > (one million lines omitted) > ``` > Here's a video of what it looks like. The coloured circles that appear are from an entirely separate video file on the rom than the rest of the video. > https://user-images.githubusercontent.com/1499676/132513838-9ca6d444-7cfb-4cc4-968c-8db816b31760.mp4 > And what it should look like: https://youtu.be/4v45irelP1w > The warnings in the attached logs from the stubbed methods in `core\hle\service\mm\mm_u.cpp` seem like they could be relevant. I modified that system a little to dish out unique identifiers (instead of 1 every time) and the game is certainly trying to call SetAndWait and Get on two different identifiers during this song. > I also thought maybe some register in `NvdecRegisters` could be used to signal that a different stream is being decoded but I couldn't spot anything like that being set. > [yuzu_log_741.txt](https://github.com/yuzu-emu/yuzu/files/7128933/yuzu_log_741.txt) > [yuzu_log_ea_2040.txt](https://github.com/yuzu-emu/yuzu/files/7128934/yuzu_log_ea_2040.txt) #### <div><img width="50" height="50" src="https://avatars.githubusercontent.com/u/1098176?v=4" width="50"></div>[Squall-Leonhart](https://github.com/Squall-Leonhart) commented at [2023-11-08 13:39](https://github.com/yuzu-emu/yuzu/issues/6985#issuecomment-1801910272): > This occurs because all decoding shares the same context, and DTS order isn't respected, FFMPEG decoder is using PTS order where DTS is expected. #### <div><img width="50" height="50" src="https://avatars.githubusercontent.com/u/23653025?u=7c6e73d3e257f4d10b2b8f9416c3db35faef85c1&v=4" width="50"></div>[G-Spawn](https://github.com/G-Spawn) commented at [2023-12-05 17:59](https://github.com/yuzu-emu/yuzu/issues/6985#issuecomment-1841328745): > https://streamable.com/horb6x > Seemingly this was fixed a while back. Please feel free to open this again @willox if the issue persists on newer yuzu 🙏 > ------------------------------------------------------------------------------- > [Export of Github issue for [yuzu-emu/yuzu](https://github.com/yuzu-emu/yuzu). Generated on 2024.02.28 at 10:04:32.]
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Accord/yuzu-mirror#6985
No description provided.