starred/ffmpeg-rockchip-nyanmisaka
1
0
Fork 0
mirror of https://github.com/nyanmisaka/ffmpeg-rockchip.git synced 2026-01-24 02:20:56 +01:00
Code Issues Projects Packages Wiki Activity Actions

rendering docs

boogie 2024-01-22 11:06:36 +01:00
parent f5d860a85f
commit 51af4c37b1
11 changed files with 775 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

13
Home.md

@ -1,14 +1,17 @@
## [Compilation](https://github.com/nyanmisaka/ffmpeg-rockchip/wiki/Compilation)
## [Compilation](Compilation)
This chapter guides you how to compile the FFmpeg-Rockchip in this project.
## [Decoder](https://github.com/nyanmisaka/ffmpeg-rockchip/wiki/Decoder)
## [Decoder](Decoder)
This chapter briefly introduces you how to use the MPP decoder in FFmpeg CLI.
## [Encoder](https://github.com/nyanmisaka/ffmpeg-rockchip/wiki/Encoder)
## [Encoder](Encoder)
This chapter briefly introduces you how to use the MPP encoder in FFmpeg CLI.
## [Filter](https://github.com/nyanmisaka/ffmpeg-rockchip/wiki/Filter)
## [Filter](Filter)
This chapter briefly introduces you how to use the RGA filter in FFmpeg CLI.
## [Video Transcode](https://github.com/nyanmisaka/ffmpeg-rockchip/wiki/Video-Transcode)
## [Rendering](Rendering)
This chapter introduces you how to use the MPP decoders with existing players and how to render frames efficiently.
## [Video Transcode](Video-Transcode)
This chapter guides you how to quickly transcode your videos using MPP and RGA in FFmpeg CLI.

188
Rendering.md Normal file

@ -0,0 +1,188 @@
# Rendering
## Overview
The frames decoded by rkmpp decoders are DRM_PRIME frames and not all players support those type of frames. To render a DRM_PRIME frame there are two common ways.
### Rendering Through EGL
The EGL interface coming from MESA or a custom GL implementation from a vendor (ie: arm mali blobs) can import those DRM_PRIME frames and render on the screen. However that kind of rendering scheme is not yet implemented in all players. In this case the rendering flow is as follows.
```
Kernel (Rockchip BSP) UserSpace (Linux)
+----------------------+ +------------------------+
| Rockchip HW Decoders | | |
| ↓ | | |
| mpp service ------+------+---------> mpp |
| | | ↓ |
| rgamulti <-----+------+-librga -> ffmpeg |
| | | ↓ (DRMPRIME) |
| | | player |
| | | ↓ |
| GPU Driver <-----+------+-------- mesa(EGL) |
| | | | |
| +----------+------+--> mesa (pan*/lima*) |
| | | ↓ |
| DRM/KMS (vop2) <---+------+-------- DE (gnome/KDE) |
| ↓ | | |
| Display Out | | |
| (hdmi/dp/mipi,lvds) | | |
+----------------------+ +------------------------+
```
### Rendering Through KMS/GBM
This flow is exactly like above but the gpu+DE part is bypassed, and the player directly renders on the kernels DRM interface, since the GPU is bypassed this is to most efficient rendering method but supported by very few players. (Only kodi as of writing)
```
Kernel (Rockchip BSP) UserSpace (Linux)
+----------------------+ +------------------------+
| Rockchip HW Decoders | | |
| ↓ | | |
| mpp service ------+------+---------> mpp |
| | | ↓ |
| rgamulti <-----+------+-librga -> ffmpeg |
| | | ↓ (DRMPRIME) |
| | | player |
| | | | |
| DRM/KMS (vop2) <---+------+-----------+ |
| ↓ | | |
| Display Out | | |
| (hdmi/dp/mipi,lvds) | | |
+----------------------+ +------------------------+
```
### Soft Frames (Not Supported)
When an FFMPeg decoder decodes the picture using CPU (ie: libx64 or libdavid), they are provided to a memory buffer and generally copied around several times. This type of flow is ok may be 1080p or even 4k picture to some point, but those several copies introduces a lot of delay and choppiness in the video. Therefore rkmpp decoders do not support soft Frames as a priority, because this is not the intended use of an hardware decoder.
## Buffer Concerns and AFBC topic
As mentioned above, the 2 known rendering schemes, should never copy frames to another buffer to get the desired performance. And even in the case of 0-copy rendering, generating big frames like 8K in 60fps require a lot of DDR bandwidth and not even possible in generic DDR4 speeds. To tackle this problem, ARM introduced a compression mechanism for the frames called AFBC (Arm Frame Buffer Compression). Rkmpp* decoders support AFBC compressed frames generation, but it is very important to have an AFBC capable mesa and/or AFBC capable DRM driver(vop2) to get really up 8k@60fps speeds smoothly.
In the case AFBC is not supported by the rendering chain, there is still option to decode the frames AFBC compressed, and decompress the with RGA filter in ffmpeg as well. This method will also give 8K@60fps performance levels with a trade of increasing the required memory size (this also depends on your DDR performance.)
## Usage on players
### FFPlay for testing (Not Supported)
FFplay supports DRMPrime Frames only when decoder gets this frames over VAAPI interface, otherwise there is no support in FFplay take advantage of those rkmpp accelerated frames.
So currently there is no proper support for FFPlay.
### MPV
Mpv supports DRMPrime frames through EGL
To get basic support with mpv run mpv with below syntax:
`mpv --profile=fast --hwdec=rkmpp path-to-file`
* Limitations:
- This will bring a speed up to 4k@60 fps rendering. For faster rendering you should activate the AFBC mode due to DDR bandwidth.
- This will most likely not play 10bit files, because MESA and MPV both currently do not support NV15, NV20 10bit plane formats that the rkmpp decoders generate.
To workaround those issues, ffmpeg can use RGA filters to decompress the AFBC compressed frames, and convert the 10bit NV15,NV20 frames to something more accepted in mesa.
Below flags for mpv, will run rkmpp decoders in afbc mode, and pass those to RGA filters. RGA will convert NV15 frames to P010 and NV20 frames to P210 format.
`mpv --profile=fast --hwdec=rkmpp --vd-lavc-o=afbc=on --vf=scale_rkrga=force_yuv=auto path-to-file`
This will get true 10bit decoded rendering (if your display and mesa actually supports it) but due to the fact that P010 and P210 picture formats are not very efficient formats, above usage may still hit memory bandwidth limitations around 8k@55fps.
To improve the performance in that regard, there should be direct AFBC rendering support in both mesa and mpv. I have tried several approaches, but could not find a proper solution get afbc support in mpv, and mesa in its current form. May be someone can take this up and improve.
To improve the performance more, a dynamic 10bit to 8bit conversion can be apllied with rga as below. This works exactly like above, but converts NV15 frames to NV12 and NV20 to NV16.
`mpv --profile=fast --hwdec=rkmpp --vd-lavc-o=afbc=on --vf=scale_rkrga=force_yuv=8bit path-to-file`
* Limitations:
- In both cases, mesa expects 64 byte aligned picture buffers, however mpp currently gives dynamicly aligned frame buffer which may not be 64 byte aligned. If the the picture width is an oddnumber * 64 (720x480), then it is possible that mesa will not accept the provided frames over EGL.
* Tweaking mpv
- `--profile=fast` is only required when your mesa is not fast enough to render the decoded frames. In a faster GL implementation like mali, of hopefully future panthor, you do not have to enable this feature
- `--swapchain-depth=8` might help to increase the delay and reduce to dropped frames due to whatever bottleneck in the rendering path
- `--msg-level=ffmpeg=debug or --msg-level=ffmpeg=trace` can give extra useful information about the ffmpeg decoding process. `trace` option might be overkill, `debug` should be ok
- `--vo=gpu-next` flag allows to use the new gpu backend in mpv which uses libplacebo. This might give slightly better performance
- `--ao=null --ao-null-untimed` disables to sync video from audio. If you are testing from command line and have no proper audio backend, you can prevent frame drops due to lack of audio sync when testing
### Kodi
Kodi provides DRMPrime frames support through both EGL and KMS/GBM.
When the windowing manager is using X, there is no way to support DRMPrime frames, neither through EGL or GBM/KMS
When the windowing system is using Wayland, you can get EGL support.
When there is no windowing manager, and you start Kodi with GBM, you can get both EGL and GBM/KMS support.
#### Kodi under Wayland
You have to do the following configuration to get decoding over EGL.
```
settings->player->videos->render method
Allow using DRM PRIME Decoder=enable
Allow Hardware Acceleation with DRM PRIME=enable
Prime Render Method=EGL
```
* Limitations:
- Same restrictions of mpv withtout any rga usage apply here as well, unfortunately Kodi can not use FFMpeg filters and make use of RGA
- 10bit formats will not work
- AFBC improvements can not be used
- Performance will be limited 4k@60
- 64 byte unaligned frames will not be rendered
#### Kodi under GBM
This type of rendering is the fastest method you can get. To run kodi with gbm support, the active Desktop Environment must be stopped so that Kodi can directly interact with KMS. You need to start kodi with
`FFMPEG_RKMPP_DEC_OPT="afbc=on" kodi --windowing=gbm --audio-backend=alsa`
Note: Audio backend force to alsa is not necesssary if you have a proper pipewire configuration.
Then configure kodi to render directly over KMS planes.
```
settings->player->videos->render method
Allow using DRM PRIME Decoder=enable
Allow Hardware Acceleation with DRM PRIME=enable
Prime Render Method=Direct to Plane
```
As you might notice, the decoder is currently running in AFBC mode, so there is no restriction in this mode in terms of performance, you should be able to get 8k@60 without any DDR bandwidth limitation.
* Limitations:
- if your attached monitor's resolution is <4K, you will not be able to render 8K frames properly, because vop2 activates the 8K rendering and scaling capabilities if the attached monitor is actually an 8k monitor. This is a limitation in rockchip hardware.
- As a general thumb of rule, 8K@60 means only if the hardware actually allows this. In rk3588 this means only HEVC and H264 frames. AV1 and VP9 decoder is limited to 4K performance in rk3588. This depends on your actual hardware if the device in use is not RK3588.
### Moonlight
Moonlight automatically detects which FFMpeg decoder can create a DRM Prime frame with hardware acceleration and detect rkmpp decoders automatically. If not users can still force the rkmpp decoders with below environment variables.
`H264_DECODER_HINT=h264_rkmpp HEVC_DECODER_HINT=hevc_rkmpp AV1_DECODER_HINT=av1_rkmpp moonlight`
But as mentioned it is not necessary to force the codecs. Make sure you have the latest moonlight with the patch fixes regession fro V4L2 codecs listed below.
## Required Patches and Fixes in the toolchain.
* Rockchip Linux
- [0001-rga3_uncompact_fix.patch](patches/rockchip-kernel/0001-rga3_uncompact_fix.patch) : To get P010 & P210 support rockchip kernel must be fixed, if not the rendered picture will display broken
- [0002-vop2_rbga2101010_capability_fix.patch](patches/rockchip-kernel/0002-vop2_rgba2101010_capability_fix.patch) : When using Kodi in GBM Mode, you will get a black screen if this patch is not applied. This is a bug in rockchip kernel.
* Librga
- [0001-normalrga-cpp-add-10b-compact-endian-mode.patch](patches/librga/0001-normalrga-cpp-add-10b-compact-endian-mode.patch) : To get P010 & P210 support librga also must be fixed, if not the rendered picture will display broken
* mpv
- [0000-hwdec_drmprime-add-AV_PIX_FMT_NV16-support.patch](patches/mpv/0000-hwdec_drmprime-add-AV_PIX_FMT_NV16-support.patch)
- [0001-hwdec_drmprime-add-AV_PIX_FMT_P010-support.patch](patches/mpv/0001-hwdec_drmprime-add-AV_PIX_FMT_P010-support.patch)
- [0002-hwdec_drmprime-add-AV_PIX_FMT_P210-support.patch](patches/mpv/0002-hwdec_drmprime-add-AV_PIX_FMT_P210-support.patch) : To get individual formats supported over EGL with mpv, they need to be patched. Some/all of the might already be marged in mainline mpv.
* KODI
- [0001-windowing-gbm-Dynamic-plane-selection.patch](patches/kodi/0001-windowing-gbm-Dynamic-plane-selection.patch) : If not applied, video will be displayed on top of OSD when Kodi used in GBM mode.
- [0002-VideoLayerBridgeDRMPRIME-Use-crop-fields-to-render-t.patch](patches/kodi/0002-VideoLayerBridgeDRMPRIME-Use-crop-fields-to-render-t.patch) : If not applied, the decoded AFBC frames will have several pixels of offset on top when rendered. Both of those fixes has a PR in mainline kodi.
* MOONLIGHT
- [0001-Only-give-pixel_format-nv12-option-to-v4l2m2m-or-v4l.patch](patches/moonlight/0001-Only-give-pixel_format-nv12-option-to-v4l2m2m-or-v4l.patch) : If not applied, moonlight will not able to initalize the rkmpp decoders even if forced.

242
patches/kodi/0001-windowing-gbm-Dynamic-plane-selection.patch Normal file

@ -0,0 +1,242 @@
From e3a833b50fba83f8a55e617d8a1b09b9a5c030dc Mon Sep 17 00:00:00 2001
From: boogie <boogiepop@gmx.com>
Date: Sat, 13 Jan 2024 19:39:52 +0100
Subject: [PATCH 1/2] windowing/gbm: Dynamic plane selection
This commit allows kodi to select gui and video planes according to
format and modifier combination and sorts to gui plane on top of video
plane if zpos is supported and not immutable by the planes.
---
.../HwDecRender/RendererDRMPRIME.cpp | 4 +-
xbmc/windowing/gbm/drm/DRMAtomic.cpp | 43 +++++++++++++++++++
xbmc/windowing/gbm/drm/DRMAtomic.h | 1 +
xbmc/windowing/gbm/drm/DRMObject.cpp | 30 +++++++++++++
xbmc/windowing/gbm/drm/DRMObject.h | 2 +
xbmc/windowing/gbm/drm/DRMUtils.cpp | 21 +++++----
xbmc/windowing/gbm/drm/DRMUtils.h | 4 +-
7 files changed, 93 insertions(+), 12 deletions(-)
diff --git a/xbmc/cores/VideoPlayer/VideoRenderers/HwDecRender/RendererDRMPRIME.cpp b/xbmc/cores/VideoPlayer/VideoRenderers/HwDecRender/RendererDRMPRIME.cpp
index a07b4f7440..04d42c0982 100644
--- a/xbmc/cores/VideoPlayer/VideoRenderers/HwDecRender/RendererDRMPRIME.cpp
+++ b/xbmc/cores/VideoPlayer/VideoRenderers/HwDecRender/RendererDRMPRIME.cpp
@@ -77,9 +77,11 @@ CBaseRenderer* CRendererDRMPRIME::Create(CVideoBuffer* buffer)
if (!plane)
return nullptr;
- if (!plane->SupportsFormatAndModifier(format, modifier))
+ if (!drm->FindPlanes(format, modifier))
return nullptr;
+ drm->SortPlanes();
+
return new CRendererDRMPRIME();
}
diff --git a/xbmc/windowing/gbm/drm/DRMAtomic.cpp b/xbmc/windowing/gbm/drm/DRMAtomic.cpp
index ff7f137d60..de03b8476e 100644
--- a/xbmc/windowing/gbm/drm/DRMAtomic.cpp
+++ b/xbmc/windowing/gbm/drm/DRMAtomic.cpp
@@ -61,6 +61,49 @@ bool CDRMAtomic::SetScalingFilter(CDRMObject* object, const char* name, const ch
return true;
}
+void CDRMAtomic::SortPlanes()
+{
+ bool supports_zpos = m_gui_plane->SupportsProperty("zpos");
+ bool zpos_immutable = supports_zpos && m_gui_plane->IsPropertyImmutable("zpos").value();
+
+ auto crtc_offset = std::distance(
+ m_crtcs.begin(),
+ std::find_if(m_crtcs.begin(), m_crtcs.end(),
+ [this](auto& crtc) { return crtc->GetCrtcId() == m_crtc->GetCrtcId(); }));
+
+ // Disable unused planes after planes are re-selected in the active crtc
+ for (auto& plane : m_planes)
+ {
+ if (!(plane.get()->GetPossibleCrtcs() & (1 << crtc_offset)))
+ continue;
+
+ if ((m_video_plane == nullptr || m_video_plane->GetId() != plane.get()->GetId()) &&
+ (m_gui_plane == nullptr || m_gui_plane->GetId() != plane.get()->GetId()))
+ {
+ CLog::Log(LOGDEBUG, "CDRMUtils::{} - disabled plane {}", __FUNCTION__,
+ plane.get()->GetPlaneId());
+ AddProperty(plane.get(), "FB_ID", 0);
+ AddProperty(plane.get(), "CRTC_ID", 0);
+ }
+ }
+
+ if (!supports_zpos || zpos_immutable)
+ return;
+
+ // re-sort the video and gui planes
+ std::optional<uint64_t*> limits = m_gui_plane->GetRangePropertyLimits("zpos");
+
+ if(!limits)
+ return;
+
+ m_gui_plane->SetProperty("zpos", limits.value()[1]);
+ m_video_plane->SetProperty("zpos", limits.value()[0]);
+ CLog::Log(LOGDEBUG, "CDRMUtils::{} - gui plane id,zpos: {}, {}", __FUNCTION__,
+ m_gui_plane->GetId(), limits.value()[0] + 1);
+ CLog::Log(LOGDEBUG, "CDRMUtils::{} - video plane id,zpos: {}, {}", __FUNCTION__,
+ m_video_plane->GetId(), limits.value()[0]);
+}
+
void CDRMAtomic::DrmAtomicCommit(int fb_id, int flags, bool rendered, bool videoLayer)
{
uint32_t blob_id;
diff --git a/xbmc/windowing/gbm/drm/DRMAtomic.h b/xbmc/windowing/gbm/drm/DRMAtomic.h
index 6b19657587..ee6314ffaf 100644
--- a/xbmc/windowing/gbm/drm/DRMAtomic.h
+++ b/xbmc/windowing/gbm/drm/DRMAtomic.h
@@ -33,6 +33,7 @@ public:
bool InitDrm() override;
void DestroyDrm() override;
bool AddProperty(CDRMObject* object, const char* name, uint64_t value);
+ void SortPlanes();
bool DisplayHardwareScalingEnabled();
diff --git a/xbmc/windowing/gbm/drm/DRMObject.cpp b/xbmc/windowing/gbm/drm/DRMObject.cpp
index 5ffce40fa3..2fafdc7b95 100644
--- a/xbmc/windowing/gbm/drm/DRMObject.cpp
+++ b/xbmc/windowing/gbm/drm/DRMObject.cpp
@@ -105,6 +105,25 @@ std::optional<uint64_t> CDRMObject::GetPropertyValue(std::string_view name,
return {};
}
+std::optional<uint64_t*> CDRMObject::GetRangePropertyLimits(std::string_view name)
+{
+ auto property = std::find_if(m_propsInfo.begin(), m_propsInfo.end(),
+ [&name](const auto& prop) { return prop->name == name; });
+
+ if (property == m_propsInfo.end())
+ return {};
+
+ auto prop = property->get();
+
+ if (!static_cast<bool>(drm_property_type_is(prop, DRM_MODE_PROP_RANGE)))
+ return {};
+
+ if (prop->count_values != 2)
+ return {};
+
+ return prop->values;
+}
+
bool CDRMObject::SetProperty(const std::string& name, uint64_t value)
{
auto property = std::find_if(m_propsInfo.begin(), m_propsInfo.end(),
@@ -130,3 +149,14 @@ bool CDRMObject::SupportsProperty(const std::string& name)
return false;
}
+
+std::optional<bool> CDRMObject::IsPropertyImmutable(const std::string& name)
+{
+ auto property = std::find_if(m_propsInfo.begin(), m_propsInfo.end(),
+ [&name](const auto& prop) { return prop->name == name; });
+
+ if (property == m_propsInfo.end())
+ return {};
+
+ return static_cast<bool>(drm_property_type_is(property->get(), DRM_MODE_PROP_IMMUTABLE));
+}
diff --git a/xbmc/windowing/gbm/drm/DRMObject.h b/xbmc/windowing/gbm/drm/DRMObject.h
index c4200b1a86..bee880de20 100644
--- a/xbmc/windowing/gbm/drm/DRMObject.h
+++ b/xbmc/windowing/gbm/drm/DRMObject.h
@@ -40,6 +40,8 @@ public:
bool SetProperty(const std::string& name, uint64_t value);
bool SupportsProperty(const std::string& name);
+ std::optional<bool> IsPropertyImmutable(const std::string& name);
+ std::optional<uint64_t*> GetRangePropertyLimits(std::string_view name);
protected:
explicit CDRMObject(int fd);
diff --git a/xbmc/windowing/gbm/drm/DRMUtils.cpp b/xbmc/windowing/gbm/drm/DRMUtils.cpp
index 3dd4ee9783..e3aab14d3a 100644
--- a/xbmc/windowing/gbm/drm/DRMUtils.cpp
+++ b/xbmc/windowing/gbm/drm/DRMUtils.cpp
@@ -181,20 +181,23 @@ bool CDRMUtils::FindPreferredMode()
return true;
}
-bool CDRMUtils::FindPlanes()
+bool CDRMUtils::FindPlanes(uint32_t format, uint64_t modifier)
{
for (size_t i = 0; i < m_crtcs.size(); i++)
{
if (!(m_encoder->GetPossibleCrtcs() & (1 << i)))
continue;
- auto videoPlane = std::find_if(m_planes.begin(), m_planes.end(), [&i](auto& plane) {
- if (plane->GetPossibleCrtcs() & (1 << i))
- {
- return plane->SupportsFormat(DRM_FORMAT_NV12);
- }
- return false;
- });
+ auto videoPlane =
+ std::find_if(m_planes.begin(), m_planes.end(),
+ [&i, &format, &modifier](auto& plane)
+ {
+ if (plane->GetPossibleCrtcs() & (1 << i))
+ {
+ return plane->SupportsFormatAndModifier(format, modifier);
+ }
+ return false;
+ });
uint32_t videoPlaneId{0};
@@ -467,7 +470,7 @@ bool CDRMUtils::InitDrm()
if (!FindCrtc())
return false;
- if (!FindPlanes())
+ if (!FindPlanes(DRM_FORMAT_NV12, DRM_FORMAT_MOD_LINEAR))
return false;
if (!FindPreferredMode())
diff --git a/xbmc/windowing/gbm/drm/DRMUtils.h b/xbmc/windowing/gbm/drm/DRMUtils.h
index f92f716fc4..fcf9d772e3 100644
--- a/xbmc/windowing/gbm/drm/DRMUtils.h
+++ b/xbmc/windowing/gbm/drm/DRMUtils.h
@@ -64,6 +64,7 @@ public:
static uint32_t FourCCWithoutAlpha(uint32_t fourcc);
void SetInFenceFd(int fd) { m_inFenceFd = fd; }
+ bool FindPlanes(uint32_t format, uint64_t modifier);
int TakeOutFenceFd()
{
int fd{-1};
@@ -89,13 +90,13 @@ protected:
int m_inFenceFd{-1};
int m_outFenceFd{-1};
+ std::vector<std::unique_ptr<CDRMCrtc>> m_crtcs;
std::vector<std::unique_ptr<CDRMPlane>> m_planes;
private:
bool FindConnector();
bool FindEncoder();
bool FindCrtc();
- bool FindPlanes();
bool FindPreferredMode();
bool RestoreOriginalMode();
RESOLUTION_INFO GetResolutionInfo(drmModeModeInfoPtr mode);
@@ -106,7 +107,6 @@ private:
std::vector<std::unique_ptr<CDRMConnector>> m_connectors;
std::vector<std::unique_ptr<CDRMEncoder>> m_encoders;
- std::vector<std::unique_ptr<CDRMCrtc>> m_crtcs;
};
}
--
2.43.0

114
patches/kodi/0002-VideoLayerBridgeDRMPRIME-Use-crop-fields-to-render-t.patch Normal file

@ -0,0 +1,114 @@
From 870e8cc7572c9ebf2c5311e2726cb96db1336df3 Mon Sep 17 00:00:00 2001
From: boogie <boogiepop@gmx.com>
Date: Sat, 13 Jan 2024 21:48:53 +0100
Subject: [PATCH 2/2] VideoLayerBridgeDRMPRIME: Use crop fields to render the
picture offsets
Hardware decoders when used with AFBC compression, may output picture
with offsets which may be different for each frame. Since this offset is
applied after the decompression is done, only way to represent this in
SRC_X and SRC_Y plane props. This commits utilizes AVFrame crop fields
to pass picture offsets.
---
.../cores/VideoPlayer/Buffers/VideoBufferDRMPRIME.h | 2 ++
.../VideoPlayer/DVDCodecs/Video/DVDVideoCodec.cpp | 2 ++
.../VideoPlayer/DVDCodecs/Video/DVDVideoCodec.h | 2 ++
.../DVDCodecs/Video/DVDVideoCodecDRMPRIME.cpp | 13 +++++++++++++
.../HwDecRender/VideoLayerBridgeDRMPRIME.cpp | 11 ++++++-----
5 files changed, 25 insertions(+), 5 deletions(-)
diff --git a/xbmc/cores/VideoPlayer/Buffers/VideoBufferDRMPRIME.h b/xbmc/cores/VideoPlayer/Buffers/VideoBufferDRMPRIME.h
index b83ee8ca68..495c428909 100644
--- a/xbmc/cores/VideoPlayer/Buffers/VideoBufferDRMPRIME.h
+++ b/xbmc/cores/VideoPlayer/Buffers/VideoBufferDRMPRIME.h
@@ -54,6 +54,8 @@ public:
virtual const VideoPicture& GetPicture() const { return m_picture; }
virtual uint32_t GetWidth() const { return GetPicture().iWidth; }
virtual uint32_t GetHeight() const { return GetPicture().iHeight; }
+ virtual uint32_t GetXOffset() const { return GetPicture().iXOffset; }
+ virtual uint32_t GetYOffset() const { return GetPicture().iYOffset; }
virtual AVDRMFrameDescriptor* GetDescriptor() const = 0;
virtual bool IsValid() const { return true; }
diff --git a/xbmc/cores/VideoPlayer/DVDCodecs/Video/DVDVideoCodec.cpp b/xbmc/cores/VideoPlayer/DVDCodecs/Video/DVDVideoCodec.cpp
index 0d986e9154..abfc937f5a 100644
--- a/xbmc/cores/VideoPlayer/DVDCodecs/Video/DVDVideoCodec.cpp
+++ b/xbmc/cores/VideoPlayer/DVDCodecs/Video/DVDVideoCodec.cpp
@@ -56,6 +56,8 @@ void VideoPicture::Reset()
iWidth = 0;
iHeight = 0;
+ iXOffset = 0;
+ iYOffset = 0;
iDisplayWidth = 0;
iDisplayHeight = 0;
}
diff --git a/xbmc/cores/VideoPlayer/DVDCodecs/Video/DVDVideoCodec.h b/xbmc/cores/VideoPlayer/DVDCodecs/Video/DVDVideoCodec.h
index 3c55a7b06c..7a4b0ef3ec 100644
--- a/xbmc/cores/VideoPlayer/DVDCodecs/Video/DVDVideoCodec.h
+++ b/xbmc/cores/VideoPlayer/DVDCodecs/Video/DVDVideoCodec.h
@@ -73,6 +73,8 @@ public:
unsigned int iWidth;
unsigned int iHeight;
+ unsigned int iXOffset;
+ unsigned int iYOffset;
unsigned int iDisplayWidth; //< width of the picture without black bars
unsigned int iDisplayHeight; //< height of the picture without black bars
diff --git a/xbmc/cores/VideoPlayer/DVDCodecs/Video/DVDVideoCodecDRMPRIME.cpp b/xbmc/cores/VideoPlayer/DVDCodecs/Video/DVDVideoCodecDRMPRIME.cpp
index eb2943bb8c..75ec720346 100644
--- a/xbmc/cores/VideoPlayer/DVDCodecs/Video/DVDVideoCodecDRMPRIME.cpp
+++ b/xbmc/cores/VideoPlayer/DVDCodecs/Video/DVDVideoCodecDRMPRIME.cpp
@@ -506,6 +506,19 @@ void CDVDVideoCodecDRMPRIME::SetPictureParams(VideoPicture* pVideoPicture)
pVideoPicture->iWidth = m_pFrame->width;
pVideoPicture->iHeight = m_pFrame->height;
+ /* according to ffmpeg devs crop fields in AVFrame are not always maintained actively
+ * and may cause under-shot buffers or mis-aligned strides if not implemented correctly
+ * in decoders. drm_prime frames on the other hand has their own memory allocators,
+ * and as long as the plane number is single, there should be no future regression
+ * related to usage of crop fields.
+ */
+ AVDRMFrameDescriptor* desc = reinterpret_cast<AVDRMFrameDescriptor*>(m_pFrame->data[0]);
+ if (m_pFrame->format == AV_PIX_FMT_DRM_PRIME && desc->nb_layers == 1)
+ {
+ pVideoPicture->iXOffset = m_pFrame->crop_left;
+ pVideoPicture->iYOffset = m_pFrame->crop_top;
+ }
+
double aspect_ratio = 0;
AVRational pixel_aspect = m_pFrame->sample_aspect_ratio;
if (pixel_aspect.num)
diff --git a/xbmc/cores/VideoPlayer/VideoRenderers/HwDecRender/VideoLayerBridgeDRMPRIME.cpp b/xbmc/cores/VideoPlayer/VideoRenderers/HwDecRender/VideoLayerBridgeDRMPRIME.cpp
index 34d1ab6235..33db29b140 100644
--- a/xbmc/cores/VideoPlayer/VideoRenderers/HwDecRender/VideoLayerBridgeDRMPRIME.cpp
+++ b/xbmc/cores/VideoPlayer/VideoRenderers/HwDecRender/VideoLayerBridgeDRMPRIME.cpp
@@ -118,9 +118,10 @@ bool CVideoLayerBridgeDRMPRIME::Map(CVideoBufferDRMPRIME* buffer)
flags = DRM_MODE_FB_MODIFIERS;
// add the video frame FB
- ret = drmModeAddFB2WithModifiers(m_DRM->GetFileDescriptor(), buffer->GetWidth(),
- buffer->GetHeight(), layer->format, handles, pitches, offsets,
- modifier, &buffer->m_fb_id, flags);
+ ret = drmModeAddFB2WithModifiers(m_DRM->GetFileDescriptor(),
+ buffer->GetWidth() + buffer->GetXOffset(),
+ buffer->GetHeight() + buffer->GetYOffset(), layer->format,
+ handles, pitches, offsets, modifier, &buffer->m_fb_id, flags);
if (ret < 0)
{
CLog::Log(LOGERROR, "CVideoLayerBridgeDRMPRIME::{} - failed to add fb {}, ret = {}",
@@ -188,8 +189,8 @@ void CVideoLayerBridgeDRMPRIME::SetVideoPlane(CVideoBufferDRMPRIME* buffer, cons
auto plane = m_DRM->GetVideoPlane();
m_DRM->AddProperty(plane, "FB_ID", buffer->m_fb_id);
m_DRM->AddProperty(plane, "CRTC_ID", m_DRM->GetCrtc()->GetCrtcId());
- m_DRM->AddProperty(plane, "SRC_X", 0);
- m_DRM->AddProperty(plane, "SRC_Y", 0);
+ m_DRM->AddProperty(plane, "SRC_X", buffer->GetXOffset() << 16);
+ m_DRM->AddProperty(plane, "SRC_Y", buffer->GetYOffset() << 16);
m_DRM->AddProperty(plane, "SRC_W", buffer->GetWidth() << 16);
m_DRM->AddProperty(plane, "SRC_H", buffer->GetHeight() << 16);
m_DRM->AddProperty(plane, "CRTC_X", static_cast<int32_t>(destRect.x1) & ~1);
--
2.43.0

23
patches/librga/0001-normalrga-cpp-add-10b-compact-endian-mode.patch Normal file

@ -0,0 +1,23 @@
--- a/core/NormalRga.cpp 2023-09-19 18:32:57.977482511 +0800
+++ a/core/NormalRga.cpp 2023-09-16 04:46:35.072791045 +0800
@@ -1413,6 +1413,20 @@
if (src1)
rgaReg.pat.rd_mode = src1->rd_mode ? src1->rd_mode : raster_mode;
+ /* rga3 is_10b_compact and is_10b_endian */
+ if (rgaReg.src.rd_mode == raster_mode) {
+ rgaReg.src.is_10b_compact = !!src->is_10b_compact;
+ rgaReg.src.is_10b_endian = !!src->is_10b_endian;
+ }
+ if (rgaReg.dst.rd_mode == raster_mode || rgaReg.dst.rd_mode == tile_mode) {
+ rgaReg.dst.is_10b_compact = !!dst->is_10b_compact;
+ rgaReg.dst.is_10b_endian = !!dst->is_10b_endian;
+ }
+ if (src1 && rgaReg.pat.rd_mode == raster_mode) {
+ rgaReg.pat.is_10b_compact = !!src1->is_10b_compact;
+ rgaReg.pat.is_10b_endian = !!src1->is_10b_endian;
+ }
+
rgaReg.in_fence_fd = dst->in_fence_fd;
rgaReg.core = dst->core;
rgaReg.priority = dst->priority;

41
patches/moonlight/0001-Only-give-pixel_format-nv12-option-to-v4l2m2m-or-v4l.patch Normal file

@ -0,0 +1,41 @@
From 2e3cf89e1f971eccfe331fb6fa3405e9fd683e5c Mon Sep 17 00:00:00 2001
From: boogie <boogiepop@gmx.com>
Date: Sat, 27 Jan 2024 00:10:44 +0100
Subject: [PATCH] Only give "pixel_format=nv12" option to v4l2m2m or
v4l2requests decoders
pixel_format is a global option which changes avctx->pix_fmt to the
given format, and overrides the AV_PIX_FMT_DRM_PRIME requirement of a
decoder. So v4l2 out of tree patches has done some exception which is
not perfectly nice, therefore limit this option only if the given
decoder is v4l2 decoder. Otherwise rockchip mpp based ffmpeg decoders
can not work properly
---
app/streaming/video/ffmpeg-renderers/drm.cpp | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/app/streaming/video/ffmpeg-renderers/drm.cpp b/app/streaming/video/ffmpeg-renderers/drm.cpp
index 786ac62c..15d575d4 100644
--- a/app/streaming/video/ffmpeg-renderers/drm.cpp
+++ b/app/streaming/video/ffmpeg-renderers/drm.cpp
@@ -4,6 +4,7 @@
#endif
#include "drm.h"
+#include "string.h"
extern "C" {
#include <libavutil/hwcontext_drm.h>
@@ -157,7 +158,8 @@ bool DrmRenderer::prepareDecoderContext(AVCodecContext* context, AVDictionary**
{
// The out-of-tree LibreELEC patches use this option to control the type of the V4L2
// buffers that we get back. We only support NV12 buffers now.
- av_dict_set_int(options, "pixel_format", AV_PIX_FMT_NV12, 0);
+ if(strstr(context->codec->name, "_v4l2") != NULL)
+ av_dict_set_int(options, "pixel_format", AV_PIX_FMT_NV12, 0);
// This option controls the pixel format for the h264_omx and hevc_omx decoders
// used by the JH7110 multimedia stack. This decoder gives us software frames,
--
2.43.0

49
patches/mpv/0000-hwdec_drmprime-add-AV_PIX_FMT_NV16-support.patch Normal file

@ -0,0 +1,49 @@
From e0c202998dfd9eaacbd9afccd75dd75e2b912e5a Mon Sep 17 00:00:00 2001
From: hbiyik <boogiepop@gmx.com>
Date: Thu, 28 Sep 2023 22:25:25 +0200
Subject: [PATCH] hwdec_drmprime: add nv16 support
NV16 is the half subsampled version of NV12 format. Decoders which
support High 4:2:2 of h264 provide the frame in NV16 format to establish
richer colorspace. Similar profiles are also available in HEVC and other
popular codecs. This commit allows NV16 frames to be displayed over
drmprime layers.
Signed-off-by: hbiyik <boogiepop@gmx.com>
---
video/out/hwdec/dmabuf_interop_gl.c | 1 +
video/out/hwdec/hwdec_drmprime.c | 2 ++
2 files changed, 3 insertions(+)
diff --git a/video/out/hwdec/dmabuf_interop_gl.c b/video/out/hwdec/dmabuf_interop_gl.c
index bd334742897e..e7fb1031a06a 100644
--- a/video/out/hwdec/dmabuf_interop_gl.c
+++ b/video/out/hwdec/dmabuf_interop_gl.c
@@ -176,6 +176,7 @@ static bool vaapi_gl_map(struct ra_hwdec_mapper *mapper,
if (p_mapper->desc.layers[i].nb_planes > 1) {
switch (p_mapper->desc.layers[i].format) {
case DRM_FORMAT_NV12:
+ case DRM_FORMAT_NV16:
format[0] = DRM_FORMAT_R8;
format[1] = DRM_FORMAT_GR88;
break;
diff --git a/video/out/hwdec/hwdec_drmprime.c b/video/out/hwdec/hwdec_drmprime.c
index 5051207413e2..290f11c53542 100644
--- a/video/out/hwdec/hwdec_drmprime.c
+++ b/video/out/hwdec/hwdec_drmprime.c
@@ -29,6 +29,7 @@
#include "libmpv/render_gl.h"
#include "options/m_config.h"
+#include "video/fmt-conversion.h"
#include "video/out/drm_common.h"
#include "video/out/gpu/hwdec.h"
#include "video/out/hwdec/dmabuf_interop.h"
@@ -117,6 +118,7 @@ static int init(struct ra_hwdec *hw)
int num_formats = 0;
MP_TARRAY_APPEND(p, p->formats, num_formats, IMGFMT_NV12);
MP_TARRAY_APPEND(p, p->formats, num_formats, IMGFMT_420P);
+ MP_TARRAY_APPEND(p, p->formats, num_formats, pixfmt2imgfmt(AV_PIX_FMT_NV16));
MP_TARRAY_APPEND(p, p->formats, num_formats, 0); // terminate it
p->hwctx.hw_imgfmt = IMGFMT_DRMPRIME;

24
patches/mpv/0001-hwdec_drmprime-add-AV_PIX_FMT_P010-support.patch Normal file

@ -0,0 +1,24 @@
From b6eaafb253f1970eebe05584939c6d04ec9c672c Mon Sep 17 00:00:00 2001
From: boogie <boogiepop@gmx.com>
Date: Sat, 20 Jan 2024 00:08:30 +0100
Subject: [PATCH 1/2] hwdec_drmprime: add AV_PIX_FMT_P010 support
---
video/out/hwdec/hwdec_drmprime.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/video/out/hwdec/hwdec_drmprime.c b/video/out/hwdec/hwdec_drmprime.c
index a5f9c664fb..2d52ebcc4c 100644
--- a/video/out/hwdec/hwdec_drmprime.c
+++ b/video/out/hwdec/hwdec_drmprime.c
@@ -132,6 +132,7 @@ static int init(struct ra_hwdec *hw)
MP_TARRAY_APPEND(p, p->formats, num_formats, IMGFMT_NV12);
MP_TARRAY_APPEND(p, p->formats, num_formats, IMGFMT_420P);
MP_TARRAY_APPEND(p, p->formats, num_formats, pixfmt2imgfmt(AV_PIX_FMT_NV16));
+ MP_TARRAY_APPEND(p, p->formats, num_formats, pixfmt2imgfmt(AV_PIX_FMT_P010));
for (int i = 0; i < MP_ARRAY_SIZE(forked_pix_fmt_names); i++) {
enum AVPixelFormat fmt = av_get_pix_fmt(forked_pix_fmt_names[i]);
--
2.43.0

37
patches/mpv/0002-hwdec_drmprime-add-AV_PIX_FMT_P210-support.patch Normal file

@ -0,0 +1,37 @@
From a606c27dcc6cf73e27ec93696272596c2ca7fd17 Mon Sep 17 00:00:00 2001
From: boogie <boogiepop@gmx.com>
Date: Sat, 20 Jan 2024 00:09:39 +0100
Subject: [PATCH 2/2] hwdec_drmprime: add AV_PIX_FMT_P210 support
---
video/out/hwdec/dmabuf_interop_gl.c | 1 +
video/out/hwdec/hwdec_drmprime.c | 1 +
2 files changed, 2 insertions(+)
diff --git a/video/out/hwdec/dmabuf_interop_gl.c b/video/out/hwdec/dmabuf_interop_gl.c
index e7fb1031a0..a353ade9bb 100644
--- a/video/out/hwdec/dmabuf_interop_gl.c
+++ b/video/out/hwdec/dmabuf_interop_gl.c
@@ -186,6 +186,7 @@ static bool vaapi_gl_map(struct ra_hwdec_mapper *mapper,
format[2] = DRM_FORMAT_R8;
break;
case DRM_FORMAT_P010:
+ case DRM_FORMAT_P210:
#ifdef DRM_FORMAT_P030 /* Format added in a newer libdrm version than minimum */
case DRM_FORMAT_P030:
#endif
diff --git a/video/out/hwdec/hwdec_drmprime.c b/video/out/hwdec/hwdec_drmprime.c
index 2d52ebcc4c..e245f3f75a 100644
--- a/video/out/hwdec/hwdec_drmprime.c
+++ b/video/out/hwdec/hwdec_drmprime.c
@@ -133,6 +133,7 @@ static int init(struct ra_hwdec *hw)
MP_TARRAY_APPEND(p, p->formats, num_formats, IMGFMT_420P);
MP_TARRAY_APPEND(p, p->formats, num_formats, pixfmt2imgfmt(AV_PIX_FMT_NV16));
MP_TARRAY_APPEND(p, p->formats, num_formats, pixfmt2imgfmt(AV_PIX_FMT_P010));
+ MP_TARRAY_APPEND(p, p->formats, num_formats, pixfmt2imgfmt(AV_PIX_FMT_P210));
for (int i = 0; i < MP_ARRAY_SIZE(forked_pix_fmt_names); i++) {
enum AVPixelFormat fmt = av_get_pix_fmt(forked_pix_fmt_names[i]);
--
2.43.0

22
patches/rockchip-kernel/0001-rga3_uncompact_fix.patch Normal file

@ -0,0 +1,22 @@
diff --git a/drivers/video/rockchip/rga3/rga3_reg_info.c b/drivers/video/rockchip/rga3/rga3_reg_info.c
index f6e4345..84388f3 100644
--- a/drivers/video/rockchip/rga3/rga3_reg_info.c
+++ b/drivers/video/rockchip/rga3/rga3_reg_info.c
@@ -331,7 +331,7 @@
(s_RGA3_WIN0_RD_CTRL_SW_WIN0_YUV10B_COMPACT(1)));
/* Only on raster mode, yuv 10bit can change to compact or set endian */
- if (msg->win0.rd_mode == RGA_RASTER_MODE && yuv10 == 1) {
+ if (msg->win0.rd_mode == 0 && yuv10 == 1) {
reg =
((reg & (~m_RGA3_WIN0_RD_CTRL_SW_WIN0_YUV10B_COMPACT)) |
(s_RGA3_WIN0_RD_CTRL_SW_WIN0_YUV10B_COMPACT
@@ -703,7 +703,7 @@
(s_RGA3_WIN1_RD_CTRL_SW_WIN1_YUV10B_COMPACT(1)));
/* Only on roster mode, yuv 10bit can change to compact or set endian */
- if (msg->win1.rd_mode == RGA_RASTER_MODE && yuv10 == 1) {
+ if (msg->win1.rd_mode == 0 && yuv10 == 1) {
reg =
((reg & (~m_RGA3_WIN1_RD_CTRL_SW_WIN1_YUV10B_COMPACT)) |
(s_RGA3_WIN1_RD_CTRL_SW_WIN1_YUV10B_COMPACT

27
patches/rockchip-kernel/0002-vop2_rgba2101010_capability_fix.patch Normal file

@ -0,0 +1,27 @@
From dd8ba855bca010e281e3d69fb10c8a6600c4f541 Mon Sep 17 00:00:00 2001
From: boogie <boogiepop@gmx.com>
Date: Mon, 22 Jan 2024 12:00:42 +0100
Subject: [PATCH] vop2 rbga2101010 capability fix
---
drivers/gpu/drm/rockchip/rockchip_vop2_reg.c | 4 ----
1 file changed, 4 deletions(-)
diff --git a/drivers/gpu/drm/rockchip/rockchip_vop2_reg.c b/drivers/gpu/drm/rockchip/rockchip_vop2_reg.c
index 31069d9aebf8..a12670e1c2e7 100644
--- a/drivers/gpu/drm/rockchip/rockchip_vop2_reg.c
+++ b/drivers/gpu/drm/rockchip/rockchip_vop2_reg.c
@@ -31,10 +31,6 @@
_VOP_REG(off, _mask, s, true)
static const uint32_t formats_for_cluster[] = {
- DRM_FORMAT_XRGB2101010,
- DRM_FORMAT_ARGB2101010,
- DRM_FORMAT_XBGR2101010,
- DRM_FORMAT_ABGR2101010,
DRM_FORMAT_XRGB8888,
DRM_FORMAT_ARGB8888,
DRM_FORMAT_XBGR8888,
--
2.43.0