如何用硬件解码iOS上的H.264帧?-HowtodecodeaH.264frameoniOSbyhardwaredecoding?

作者： | 来源：互联网 | 2023-09-10 21:44

IhavebeenusedffmpegtodecodeeverysingleframethatIreceivedfrommyipcam.Thebriefcode

I have been used ffmpeg to decode every single frame that I received from my ip cam. The brief code looks like this:

我已经使用ffmpeg解码我从我的ip cam收到的每一个帧。简短的代码如下:

-(void) decodeFrame:(unsigned char *)frameData frameSize:(int)frameSize{
   AVFrame frame;
   AVPicture picture;
   AVPacket pkt;
   AVCodecContext *context;
   pkt.data = frameData;
   pat.size = frameSize;
   avcodec_get_frame_defaults(&frame);
   avpicture_alloc(&picture, PIX_FMT_RGB24, targetWidth, targetHeight);
   avcodec_decode_video2(&context, &frame, &got_picture, &pkt);
}

The code woks fine, but it's software decoding. I want to enhance the decoding performance by hardware decoding. After lots of research, I know it may be achieved by AVFoundation framework. The AVAssetReader class may help, but I can't figure out what's the next.Could anyone points out the following steps for me? Any help would be appreciated.

代码运行良好，但它是软件解码。我想通过硬件解码来提高解码性能。经过大量的研究，我知道这可以通过AVFoundation框架来实现。AVAssetReader类可能会有帮助，但是我不知道下一个是什么。谁能给我指出以下步骤吗?如有任何帮助，我们将不胜感激。

2 个解决方案

#1

iOS does not provide any public access directly to the hardware decode engine, because hardware is always used to decode H.264 video on iOS.

iOS不提供任何直接对硬件解码引擎的公开访问，因为在iOS上，硬件总是被用来解码H.264视频。

Therefore, session 513 gives you all the information you need to allow frame-by-frame decoding on iOS. In short, per that session:

因此，会话513提供了允许在iOS上逐帧解码所需的所有信息。简而言之，每届会议:

Generate individual network abstraction layer units (NALUs) from your H.264 elementary stream. There is much information on how this is done online. VCL NALUs (IDR and non-IDR) contain your video data and are to be fed into the decoder.
从H.264基础流中生成单独的网络抽象层单元(NALUs)。有很多关于如何在网上做这件事的信息。VCL NALUs (IDR和非IDR)包含您的视频数据，将被输入解码器。
Re-package those NALUs according to the "AVCC" format, removing NALU start codes and replacing them with a 4-byte NALU length header.
根据“AVCC”格式重新打包这些NALUs，删除NALU开始代码，并用4字节的NALU长度头替换它们。
Create a CMVideoFormatDescriptionRef from your SPS and PPS NALUs via CMVideoFormatDescriptionCreateFromH264ParameterSets()
通过cmvideoformatdescriptionfromh264parametersets()从您的SPS和PPS NALUs创建CMVideoFormatDescriptionRef ()
Package NALU frames as CMSampleBuffers per session 513.
将NALU帧打包为每个会话513的CMSampleBuffers。
Create a VTDecompressionSessionRef, and feed VTDecompressionSessionDecodeFrame() with the sample buffers
- Alternatively, use AVSampleBufferDisplayLayer, whose -enqueueSampleBuffer: method obviates the need to create your own decoder.
- 或者，使用AVSampleBufferDisplayLayer，它的-enqueueSampleBuffer:方法可以避免创建自己的解码器。
创建一个vtdemopressionsessionref，并使用示例缓冲区提供vtdemopressionsessiondecodeframe()，使用AVSampleBufferDisplayLayer，它的-enqueueSampleBuffer:方法消除了创建自己的解码器的需要。

#2

Edit:

编辑:

This link provide more detail explanation on how to decode h.264 step by step: stackoverflow.com/a/29525001/3156169

这个链接提供了关于如何解码h的更详细的解释。264步:stackoverflow.com/a/29525001/3156169

Original answer:

最初的回答:

I watched the session 513 "Direct Access to Video Encoding and Decoding" in WWDC 2014 yesterday, and got the answer of my own question.

昨天我在WWDC 2014上观看了513节“视频编码和解码的直接访问”，得到了我自己的答案。

The speaker says:

演讲者说:

We have Video Toolbox(in iOS 8). Video Toolbox has been there on OS X for a while, but now it's finally populated with headers on iOS.This provides direct access to encoders and decoders.

我们有视频工具箱(在iOS 8中)，视频工具箱在OS X上已经有一段时间了，但是现在它终于在iOS上填充了头部。这提供了对编码器和解码器的直接访问。

So, there is no way to do hardware decoding frame by frame in iOS 7, but it can be done in iOS 8.

因此，在ios7中无法逐帧进行硬件解码，但在ios8中可以实现。

Is there anyone figure out how to directly access to video encoding and decoding frame by frame in iOS 8?

在ios8系统中，有没有人知道如何直接访问视频编码和逐帧解码?