[hevc_rkmpp] Zerolatency encoding #27

New issue

Closed

opened 2025-12-23 10:30:53 +01:00 by backuprepo · 2 comments

backuprepo commented

2025-12-23 10:30:53 +01:00

Owner

Originally created by @Gredys on GitHub (Mar 13, 2024).

I am using hevc_rkmpp encoder in C++ and trying to figure out how to enable zero-latency mode.
I.e. send frame = receive packet without delays, buffers and etc.

What I am trying to do - send real-time captures of game frames over the LAN, display and save them for later usage.
I was using other implementation of ffmpeg for rkmpp (Orange PI 5 Plus) and summary delay wasn't bigger than 1 frame, but it had issues with bit-rate control.
Now, with your project, I can easily control bit-rate and quality, but summary delay is bigger than 3-4 frames, which is not real-time for me. (its hard to react in action games with such delay)

Current problem with encoder is that I always get AVPacket for AVFrame with 1 frame delay.
I.e. sending and receiving data like this:

send frame     : 0 - 1 - 2 - 3 - 4 - ...
receive packet : X - 0 - 1 - 2 - 3 - ...

Code flow and set options are pretty basic: (I'll skip really basic ones, those are to test zero-latency mode)

 - gop_size = 0
 - max_b_frames = 0
 - refs = 0
 - flags |= AV_CODEC_FLAG_LOW_DELAY
 - flags2 |= AV_CODEC_FLAG2_FAST
 - preset = ultrafast
 - tune = zerolatency

 1. receive frame from internal queue
 2. use `sws_scale` to convert to correct format
 3. `av_rescale_q` to fix pts
 4. `avcodec_send_frame` to send frame to encoder
 5. right after that `avcodec_receive_packet` but there is no data for the first frame
 6. doing steps from 1 to 4 and calling `avcodec_receive_packet` for the second time gives packet for the first frame

Is there is any option to force encoder give exactly one packet per frame without buffering it on its side?

P.S. It may be another issue, but will ask in any way - hevc_rkmpp decoder always crashes after working for a few seconds if using it from C++ code. Is it intended behavior or I need to create another issue?

Originally created by @Gredys on GitHub (Mar 13, 2024). I am using `hevc_rkmpp` encoder in C++ and trying to figure out how to enable zero-latency mode. I.e. `send frame = receive packet` without delays, buffers and etc. What I am trying to do - send real-time captures of game frames over the LAN, display and save them for later usage. I was using other implementation of ffmpeg for rkmpp (Orange PI 5 Plus) and summary delay wasn't bigger than 1 frame, but it had issues with bit-rate control. Now, with your project, I can easily control bit-rate and quality, but summary delay is bigger than 3-4 frames, which is not real-time for me. (its hard to react in action games with such delay) Current problem with encoder is that I always get `AVPacket` for `AVFrame` with 1 frame delay. I.e. sending and receiving data like this: ``` send frame : 0 - 1 - 2 - 3 - 4 - ... receive packet : X - 0 - 1 - 2 - 3 - ... ``` Code flow and set options are pretty basic: (I'll skip really basic ones, those are to test zero-latency mode) ``` - gop_size = 0 - max_b_frames = 0 - refs = 0 - flags |= AV_CODEC_FLAG_LOW_DELAY - flags2 |= AV_CODEC_FLAG2_FAST - preset = ultrafast - tune = zerolatency ``` ``` 1. receive frame from internal queue 2. use `sws_scale` to convert to correct format 3. `av_rescale_q` to fix pts 4. `avcodec_send_frame` to send frame to encoder 5. right after that `avcodec_receive_packet` but there is no data for the first frame 6. doing steps from 1 to 4 and calling `avcodec_receive_packet` for the second time gives packet for the first frame ``` Is there is any option to force encoder give exactly one packet per frame without buffering it on its side? P.S. It may be another issue, but will ask in any way - `hevc_rkmpp` decoder always crashes after working for a few seconds if using it from C++ code. Is it intended behavior or I need to create another issue?

backuprepo

2025-12-23 10:30:53 +01:00

closed this issue
added the
question
label

backuprepo commented

2025-12-23 10:30:55 +01:00

Author

Owner

@nyanmisaka commented on GitHub (Mar 13, 2024):

 - gop_size = 0 //< I-frame only
 - max_b_frames = 0 //< not supported by hw
 - refs = 0 //< not supported by mpp
 - flags |= AV_CODEC_FLAG_LOW_DELAY
 - flags2 |= AV_CODEC_FLAG2_FAST //< not supported
 - preset = ultrafast //< not supported by mpp
 - tune = zerolatency //< not supported

 1. receive frame from internal queue
 2. use `sws_scale` to convert to correct format //< if possible use scale_rkrga to blit in hw
 3. `av_rescale_q` to fix pts
 4. `avcodec_send_frame` to send frame to encoder
 5. right after that `avcodec_receive_packet` but there is no data for the first frame
 6. doing steps from 1 to 4 and calling `avcodec_receive_packet` for the second time gives packet for the first frame

There is no so-called zero-latency mode in MPP. Latency can only be reduced by disabling async encoding, but it's definitely not zero.

I've just push several commits to support setting sync mode with the -flags +low_delay option, which is equal to the avctx->flags & AV_CODEC_FLAG_LOW_DELAY. This mode achieves one-input-one-output at the expense of lower FPS.

P.S. It may be another issue, but will ask in any way - hevc_rkmpp decoder always crashes after working for a few seconds if using it from C++ code. Is it intended behavior or I need to create another issue?

One ticket one issue. The description is too vague.

@nyanmisaka commented on GitHub (Mar 13, 2024): ``` - gop_size = 0 //< I-frame only - max_b_frames = 0 //< not supported by hw - refs = 0 //< not supported by mpp - flags |= AV_CODEC_FLAG_LOW_DELAY - flags2 |= AV_CODEC_FLAG2_FAST //< not supported - preset = ultrafast //< not supported by mpp - tune = zerolatency //< not supported ``` ``` 1. receive frame from internal queue 2. use `sws_scale` to convert to correct format //< if possible use scale_rkrga to blit in hw 3. `av_rescale_q` to fix pts 4. `avcodec_send_frame` to send frame to encoder 5. right after that `avcodec_receive_packet` but there is no data for the first frame 6. doing steps from 1 to 4 and calling `avcodec_receive_packet` for the second time gives packet for the first frame ``` There is no so-called zero-latency mode in MPP. Latency can only be reduced by disabling async encoding, but it's definitely not zero. I've just push several commits to support setting sync mode with the `-flags +low_delay` option, which is equal to the `avctx->flags & AV_CODEC_FLAG_LOW_DELAY`. This mode achieves one-input-one-output at the expense of lower FPS. > P.S. It may be another issue, but will ask in any way - hevc_rkmpp decoder always crashes after working for a few seconds if using it from C++ code. Is it intended behavior or I need to create another issue? One ticket one issue. The description is too vague.

backuprepo commented

2025-12-23 10:30:56 +01:00

Author

Owner

@nyanmisaka commented on GitHub (Mar 25, 2024):

Closing as the requested feat has been implemented.

@nyanmisaka commented on GitHub (Mar 25, 2024): Closing as the requested feat has been implemented.

backuprepo referenced this issue

2025-12-23 10:36:36 +01:00

[PR #27] [CLOSED] rkmppdec: allow user to force drm:prime format when decoding to prevent #242