您的位置: 首页  >  技术问答  >  为什么opencv videowriter这么慢?

为什么opencv videowriter这么慢?

分类: 技术问答 • 2022-04-01 22:22:30

为什么opencv videowriter这么慢?

问题描述：

stackoverflow社区，我遇到了一个棘手的问题，需要您的帮助以了解此处的情况.我的程序从到目前为止仍能正常工作的视频采集卡(Blackmagic)捕获帧，同时我用opencv(cv :: imshow)显示捕获的图像，效果也很好(但CPU浪费很多).捕获的图像也应该存储在磁盘上，为此，我将捕获的帧(cv :: Mat)放在堆栈上，最后将它们与opencv异步写入:

Hi stackoverflow community, I have a tricky problem and I need your help to understand what is going on here. My program captures frames from a video grabber card (Blackmagic) which just works fine so far, at the same time I display the captured images with opencv (cv::imshow) which works good as well (But pretty cpu wasting). The captured images are supposed to be stored on the disk as well, for this I put the captured Frames (cv::Mat) on a stack, to finally write them async with opencv:

cv::VideoWriter videoWriter(path, cv::CAP_FFMPEG, fourcc, fps, *size);
videoWriter.set(cv::VIDEOWRITER_PROP_QUALITY, 100);

int id = metaDataWriter.insertNow(path);

while (this->isRunning) {

    while (!this->stackFrames.empty()) {

        cv:Mat m = this->stackFrames.pop();

        videoWriter << m;
    }
    
}

videoWriter.release();

此代码正在其他线程中运行，并且将从外部停止.到目前为止，代码可以正常工作，但是有时速度很慢，这意味着我的堆栈大小增加了，我的系统用光了内存，并被操作系统杀死了.

This code is running in an additional thread and will be stopped from outside. The code is working so far, but it is sometimes pretty slow, which means my stack size increases and my system runs out of ram and get killed by the OS.

当前它正在我的开发系统上运行:

Currently it is running on my developing system:

  • Ubuntu 18.04.05
  • 使用Cuda编译的OpenCV 4.4.0
  • 英特尔i7 10世代32GB RAM，GPU Nvidia p620，M.2 SSD

取决于编解码器(fourcc)，这会产生较高的CPU负载.到目前为止，我主要使用"MJPG"，"x264".有时，甚至MJPG也会将CPU的一个内核变成100％的负载，而我的堆栈会上升，直到程序运行不正常为止.重新启动后，有时，此问题已解决，并且负载似乎已分布在所有内核上.

Depending on the codec (fourcc) this produces a high CPU load. So far I used mainly "MJPG", "x264". Sometimes even MJPG turns one core of the CPU to 100% load, and my stack raises until the programs run out of run. After a restart, sometimes, this problem is fixed, and it seems the load is distributed over all cores.

关于英特尔文档.对于我的CPU，它为多个编解码器集成了硬件编码/解码.但是我想opencv没有使用它们.Opencv甚至使用自己的ffmpeg，而不是我的系统之一.这是我的opencv的构建命令:

Regarding to the Intel Doc. for my CPU, it has integrated hardware encoding/decoding for several codecs. But I guess opencv is not using them. Opencv even uses its own ffmpeg and not the one of my system. Here is my build command of opencv:

cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D WITH_TBB=ON \
-D WITH_CUDA=ON \
-D BUILD_opencv_cudacodec=OFF \
-D ENABLE_FAST_MATH=1 \
-D CUDA_FAST_MATH=1 \
-D WITH_CUBLAS=1 \
-D WITH_V4L=ON \
-D WITH_QT=OFF \
-D WITH_OPENGL=ON \
-D WITH_GSTREAMER=ON \
-D OPENCV_GENERATE_PKGCONFIG=ON \
-D OPENCV_ENABLE_NONFREE=ON \
-D WITH_FFMPEG=1 \
-D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib/modules \
-D WITH_CUDNN=ON \
-D OPENCV_DNN_CUDA=ON \
-D CUDA_ARCH_BIN=6.1 ..

在我使用Java/Maven之前，我才刚刚开始使用linux和C ++进行开发，因此cmake的使用仍在进行中，请放轻松.

I just started development with linux and C++, before I was working with Java/Maven, so the use of cmake is still a work in progress, pls go easy on me.

基本上我的问题是，如何才能使视频编码/写入更快，最好使用硬件加速?或者，如果您认为还有其他可疑之处，请告诉我.

Basically my question is, how can I make the video encoding/writing faster, use the hardware acceleration at best? Or if you think there is something else fishy, pls let me know.

BR迈克尔

答

--------旧-在底部查找答案--------

-------- old - look up answer on bottom --------

感谢@Micka的许多建议，我在途中找到了正确的方法.

Thank @Micka for the many advises, I found the right thing on the way.

使用cudacodec :: VideoWriter并不是那么容易，因为

Using cudacodec::VideoWriter is not that easy, after compiling I was not able to use it because of this error, and even if I can make it run, the deployment PC does not have a nvidia GPU.

由于我也将PC与AMD CPU一起使用，因此无法将cv :: CAP_INTEL_MFX用于cv :: VideoWriter的api-reference参数.但是，还有cv :: CAP_OPENCV_MJPEG，它对于MJPG编解码器工作正常(并非所有视频容器都受支持，我使用.avi，可惜.mkv不适用于此配置).如果用户不使用MJPG作为编解码器，则使用cv :: CAP_ANY，然后opencv决定使用什么.

Since I am going to use PCs with AMD CPUs as well, I can't use the cv::CAP_INTEL_MFX for the api-reference parameter of the cv::VideoWriter. But there is also the cv::CAP_OPENCV_MJPEG, which works fine for the MJPG codec (not all video container are supported, I use .avi, sadly .mkv was not working with this configuration). If the user does not use MJPG as a codec I use cv::CAP_ANY, and opencv decides what is to use.

所以

cv::VideoWriter videoWriter(path, cv::CAP_OPENCV_MJPEG, fourcc, fps, *size);

即使在我的旧系统上也可以正常工作.

works pretty well, even on my old system.

不幸的是，我之前从未更改过api-reference参数，只是从ffmpeg更改为gstreamer，我在

Unfortunately I never changed the api-reference parameter before, only from ffmpeg to gstreamer, I read in the doc of opencv only the last line "cv::CAP_FFMPEG or cv::CAP_GSTREAMER." and I did not see that there is an "e.g." before... Thank you @Micka to make me read again.

P.S.对于我的cv :: imshow性能问题，我更改为

P.S. for my performance problem with cv::imshow I changed from

cv::namedWindow(WINDOW_NAME, cv::WINDOW_NORMAL);

到

cv::namedWindow(WINDOW_NAME, cv::WINDOW_OPENGL);

显然使用OpenGL，并且做得更好.同样从cv :: Mat更改为cv :: UMat可以提高性能，

Which obviously uses OpenGL, and does a better job. Also changing from cv::Mat to cv::UMat can speed up the performance, see here

--------------编辑更好的解决方案----------------

-------------- EDIT better solution ----------------

由于某些系统的OpenCV VideoWriter仍然存在问题，因此我一直在寻找其他解决方案.现在，我用FFMPEG编写帧.对于FFMPEG，我可以使用GPU或CPU，具体取决于我使用的编解码器.如果FFMPEG是通过snapd(Ubuntu 18.04)安装的，则默认情况下启用了cuda:

Since I still had problems with the OpenCV VideoWriter for some systems, I was looking for another solution. Now I write the frames with FFMPEG. For FFMPEG I can use the GPU or CPU depending on the codec I use. If FFMPEG is installed via snapd (Ubuntu 18.04) it comes with cuda enabled by default:

sudo snap install ffmpeg --devmode

(-devmode是可选的，但是我在特定位置写入文件时遇到问题，这是我修复它的唯一方法)

(--devmode is optional, but I had problems writing files on specific location, this was the only way for me to fix it)

这是我的代码:

//this string is automatically created in my program, depending on user input and the parameters of the input frames

string ffmpegCommand = "ffmpeg -y -f rawvideo -vcodec rawvideo -framerate 50 -pix_fmt bgr24 -s 1920x1080 -i - -c:v h264_nvenc -crf 14 -maxrate:v 10M -r 50 myVideoFile.mkv";

FILE *pipeout = popen(ffmpegCommand.data(), "w");

int id = metaDataWriter.insertNow(path);

//loop will be stopped from another thread
while (this->isRunning) {
    //this->frames is a stack with cv::Mat elements in the right order
    //it is filled by another thread
    while (!this->frames.empty()) {


        cv::Mat mat = frames.front();
        frames.pop();
        fwrite(mat.data, 1, s, pipeout);

    }  
}

fflush(pipeout);
pclose(pipeout);

因此，使用文件(输出)将mat.data写入ffmpeg，ffmpeg本身在进行编码和文件写入.要的参数:

So a file (pipeout) is used to write the mat.data to ffmpeg, ffmpeg itself is doing the encoding and file writing. To the parameters:

-y =覆盖输出文件而无需询问

-y = Overwrite output files without asking

-f =格式，在这种情况下，用于输入原始视频

-f = format, in this case used for input rawvideo

-vcodec =输入的编解码器也为rawvideo，因为使用的cv :: Mat.data没有压缩/编解码器

-vcodec = codec for input which is rawvideo as well, because the used cv::Mat.data has no compression/codec

-framerate =我从采集卡/OpenCv收到的输入帧率

-framerate = the input framerate I receive from my grabber card/OpenCv

-pix_fmt =我的原始数据格式，在本例中为bgr24，因此每个通道8位，因为我使用常规的OpenCV bgr cv :: Mat

-pix_fmt = the format of my raw data, in this case bgr24, so 8 bit each channel, because I use a regular OpenCV bgr cv::Mat

-s =每帧的大小，在我的情况下为1920x1080

-s = size of each frame, in my case 1920x1080

-i =输入，在这种情况下，我们从stdinput读取，您可以在此处看到-"，因此文件(输出)由ffmpeg捕获

-i = input, in this case we read from the stdinput you can see it here "-", so the file (pipeout) is captured by ffmpeg

-c:v =输出编解码器，所以这是对视频进行编码，这里使用的是h264_nvenc，它是GPU编解码器

-c:v = output codec, so this is to encode the video, here h264_nvenc is used, which is a GPU codec

-r =帧输出速率，在这种情况下也为50 myVideoFile.mkv =这只是ffmpeg生成的文件的名称，您可以更改此文件和路径

-r = frame output rate, also 50 in this case myVideoFile.mkv = this is just the name of the file which is produced by ffmpeg, you can change this file and path

更高质量的附加参数:-crf 14 -maxrate:v 10M

Additional parameters for higher quality: -crf 14 -maxrate:v 10M

这对我来说非常有效，并使用我对GPU的硬件加速或与其他编解码器一起负责CPU.我希望这对其他开发人员也有帮助.

This works very good for me and uses my hardware acceleration of the GPU or with another codec in charge the CPU. I hope this helps other developers as well.

相关推荐