mirror of
https://github.com/nyanmisaka/ffmpeg-rockchip.git
synced 2026-01-24 07:31:22 +01:00
error when use h264/256_rkmpp codec write SEI information #203
Labels
No labels
bug
enhancement
help wanted
invalid
pull-request
question
upstream
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: starred/ffmpeg-rockchip#203
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @wxd9199 on GitHub (Aug 15, 2025).
Description
When using h264_rkmpp encoder with udu_sei=1 and prefix_mode=1, SEI data attached to AVFrame via AV_FRAME_DATA_SEI_UNREGISTERED is not written into the output bitstream.
The same code works with libx264 encoder.
This makes it impossible to embed custom SEI (user data unregistered) in H.264/H.265 streams when using the Rockchip MPP hardware encoder through FFmpeg.
Environment
SoC: RK3588
OS: Debian (arm64)
Encoder: h264_rkmpp (also tested with hevc_rkmpp)
[test_example]
Steps to reproduce
Create a decoded frame.
Call:
AVFrameSideData* sd = av_frame_new_side_data(frame, AV_FRAME_DATA_SEI_UNREGISTERED, uuid_size + payload_size);
memcpy(sd->data, uuid, 16);
memcpy(sd->data + 16, payload, payload_size);
Enable SEI in encoder:
av_opt_set_int(encCtx->priv_data, "udu_sei", 1, 0);
av_opt_set_int(encCtx->priv_data, "prefix_mode", 1, 0);
Encode with h264_rkmpp.
Inspect output stream — no matching SEI NAL units are found.
Expected behavior
When udu_sei=1, prefix_mode=1, and AV_FRAME_DATA_SEI_UNREGISTERED is present in the frame, h264_rkmpp should pass this data to MPP via MppMeta (e.g., KEY_USER_DATA_UNREGISTERED) and the output bitstream should contain a corresponding SEI NAL unit.
Actual behavior
No SEI NAL units are present in the output, even though the frame side_data contains correct UUID and payload.
Additional context / Possible fix
Looking at the FFmpeg libavcodec/h264_rkmpp.c source, there is no code that extracts AV_FRAME_DATA_SEI_UNREGISTERED from AVFrame and passes it to the MPP API.
A possible fix would be:
In rkmpp_encode_frame(), check for AV_FRAME_DATA_SEI_UNREGISTERED side_data.
Call:
mpp_meta_set_ptr(meta, KEY_USER_DATA_UNREGISTERED, side_data->data);
mpp_meta_set_s32(meta, KEY_USER_DATA_UNREGISTERED_SIZE, side_data->size);
Ensure udu_sei and prefix_mode options are honored.
@nyanmisaka commented on GitHub (Aug 15, 2025):
I'm not sure where you came to this conclusion. The source code only contains
libavcodec /rkmppenc.c, notlibavcodec/h264_rkmpp.c. Therefore, I suspect you're not using ffmpeg from this repository.e2bbfe4b31/libavcodec/rkmppenc.c (L643-L700)@wxd9199 commented on GitHub (Aug 15, 2025):
@nyanmisaka Oh, I wrote it wrongly. It should be libavcodec /rkmppenc.c
The repository I use is the latest version and can currently write sei to frame. But I feel that the underlying MPP does not use sei information, so I didn't find where to use it.
testcode
@nyanmisaka commented on GitHub (Aug 15, 2025):
Apply this debug patch to ffmpeg, and then run the following command, you will find that SEI info has been written correctly. This proves that MPP applies SEI information to the bitstream, rather than ignoring it.
@wxd9199 commented on GitHub (Aug 18, 2025):
@nyanmisaka 我发现了问题:UUID设置得不合规范,
随机的UUID不工作:
uint8_t uuid[16] = {
0x41, 0x49, 0x44, 0x45, 0x54, 0x45, 0x43, 0x54, // "AIDETECT"
0x49, 0x4F, 0x4E, 0x52, 0x45, 0x53, 0x55, 0x4C // "RESULT"
};
使用下面这个UUID,可以正常写入:
const uint8_t uuid[16] = {
0x57, 0x68, 0x97, 0x80, 0xe7, 0x0c, 0x4b, 0x65,
0xa9, 0x06, 0xae, 0x29, 0x94, 0x11, 0xcd, 0x9a
};