热门标签 | HotTags
当前位置:  开发笔记 > 编程语言 > 正文

利用GLSL在iOS上实现YV12到RGB的转换,并附带展示结果图像

本文介绍了如何在iOS平台上使用GLSL着色器将YV12格式的视频帧数据转换为RGB格式,并展示了转换后的图像效果。通过详细的技术实现步骤和代码示例,读者可以轻松掌握这一过程,适用于需要进行视频处理的应用开发。

referred to this question

提到这个问题

i convert the yv12 frame data to rgb data using glsl shader,the raw image below: enter image description here

我使用glsl着色器将yv12帧数据转换为rgb数据,原始图像如下:

but the result image is not same with the former,attached below: enter image description here

但结果图像与前者不相同,如下图所示:

following is my code for uploading the three planar data to textures:

以下是我上传三个平面数据到纹理的代码:

- (GLuint) textureY: (Byte*)imageData        
      widthType: (int) width       
     heightType: (int) height       
{          
    GLuint texName;    
    glGenTextures( 1, &texName );     
    glBindTexture(GL_TEXTURE_2D, texName);

    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR );    
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR );    
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);    
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);    

    glTexImage2D( GL_TEXTURE_2D, 0, GL_LUMINANCE, width, height, 0, GL_LUMINANCE, GL_UNSIGNED_BYTE, imageData );  
    //free(imageData);

    return texName;    
}    

- (GLuint) textureU: (Byte*)imageData        
          widthType: (int) width       
         heightType: (int) height       
{          
    GLuint texName;    

    glGenTextures( 1, &texName );     
    glBindTexture(GL_TEXTURE_2D, texName);


    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR );    
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR );    
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);    
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);    

    glTexImage2D( GL_TEXTURE_2D, 0, GL_LUMINANCE, width, height, 0, GL_LUMINANCE, GL_UNSIGNED_BYTE, imageData );    

    //free(imageData);
    return texName;    
}    

- (GLuint) textureV: (Byte*)imageData        
          widthType: (int) width       
         heightType: (int) height       
{          
    GLuint texName;    
    glGenTextures( 1, &texName );     
    glBindTexture(GL_TEXTURE_2D, texName);


    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR );    
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR );    
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);    
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);    

    glTexImage2D( GL_TEXTURE_2D, 0, GL_LUMINANCE, width, height, 0, GL_LUMINANCE, GL_UNSIGNED_BYTE, imageData );    

    //free(imageData);
    return texName;    
}    


- (void) readYUVFile     
{    
    NSString *file = [[NSBundle mainBundle] pathForResource:@"video" ofType:@"yv12"];
    NSLog(@"%@",file);
    NSData* fileData = [NSData dataWithContentsOfFile:file]; 
    //NSLog(@"%@",[fileData description]);
    NSInteger width  = 352;    
    NSInteger height = 288;
    NSInteger uv_width  = width  / 2;    
    NSInteger uv_height = height / 2;
    NSInteger dataSize = [fileData length];
    NSLog(@"%i\n",dataSize);

    GLint nYsize  = width * height;     
    GLint nUVsize = uv_width * uv_height;      
    GLint nCbOffSet = nYsize;    
    GLint nCrOffSet = nCbOffSet + nUVsize;    

    Byte *spriteData = (Byte *)malloc(dataSize);
    [fileData getBytes:spriteData length:dataSize];


    Byte* uData = spriteData + nCbOffSet;
    //NSLog(@"%@\n",[[NSData dataWithBytes:uData length:nUVsize] description]);
    Byte* vData = spriteData + nCrOffSet;  
    //NSLog(@"%@\n",[[NSData dataWithBytes:vData length:nUVsize] description]);
    /**
    Byte *YPlanarData = (Byte *)malloc(nYsize);
    for (int i=0; i

and my fragment shaders code:

我的片段着色器代码:

   precision highp float;
uniform sampler2D SamplerY;
uniform sampler2D SamplerU;
uniform sampler2D SamplerV;

varying highp vec2 coordinate;

void main()
{
    highp vec3 yuv,yuv1;
    highp vec3 rgb;

    yuv.x = texture2D(SamplerY, coordinate).r;

    yuv.y = texture2D(SamplerU, coordinate).r-0.5;

    yuv.z = texture2D(SamplerV, coordinate).r-0.5 ;

   rgb = mat3(      1,       1,      1,
                     0, -.34414, 1.772,
               1.402, -.71414,      0) * yuv;

    gl_FragColor = vec4(rgb, 1);
}

my confusion is the conversion formula while i using this formula directly converting the yv12 data to rgb24,and draw a image with the

我的混淆是转换公式,我使用这个公式直接将yv12数据转换为rgb24,并绘制一个图像。

CGImageCreate(iwidth, 
                                       iheight, 
                                       8, 
                                       24, 
                                       iwidth*3, 
                                       colorSpace, 
                                       bitmapInfo, 
                                       provider, 
                                       NULL, 
                                       NO, 
                                       kCGRenderingIntentDefault);

the result image is correct. but using the shader (for the direct transform approach running on iOS device is dump) turns to this problem ,i've tried some tricks(expand the UV planers to (2*uv_width)*2(uv_height) rectangle and then upload the texture),but failed in the same more red image.

结果图像是正确的。但是使用shader(在iOS设备上运行的直接转换方法是dump)转向这个问题,我尝试了一些技巧(将UV planers扩展到(2*uv_width)*2(uv_height)矩形,然后上传纹理),但是在相同的红色图像中失败了。

How to resolve this issue?

如何解决这个问题?

attached with my whole glView.m code:

连同我的整个glView。m代码:

#import "OpenGLView.h"

typedef struct {
    float Position[3];
    float TexCoord[2];
} Vertex;

const Vertex Vertices[] = {
    {{1, -1, 0},{1,1}},
    {{1, 1, 0},{1,0}},
    {{-1, 1, 0},{0,0}},
    {{-1, -1, 0},{0,1}}
};

const GLubyte Indices[] = {
    0, 1, 2,
    2, 3, 0
};



@interface OpenGLView ()
- (void)setupLayer;
- (void)setupContext;
- (void)setupRenderBuffer;
- (void)setupFrameBuffer;
- (void)render;

- (GLuint)compileShader:(NSString*)shaderName withType:(GLenum)shaderType;
- (void)setupVBOs;
- (void)compileShaders;

- (void) readYUVFile;
@end

@implementation OpenGLView

- (void)setupVBOs {

    GLuint vertexBuffer;
    glGenBuffers(1, &vertexBuffer);
    glBindBuffer(GL_ARRAY_BUFFER, vertexBuffer);
    glBufferData(GL_ARRAY_BUFFER, sizeof(Vertices), Vertices, GL_STATIC_DRAW);

    GLuint indexBuffer;
    glGenBuffers(1, &indexBuffer);
    glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, indexBuffer);
    glBufferData(GL_ELEMENT_ARRAY_BUFFER, sizeof(Indices), Indices, GL_STATIC_DRAW);

}



- (id)initWithFrame:(CGRect)frame
{
    self = [super initWithFrame:frame];
    if (self) {
        // Initialization code[]
        self.backgroundColor = [UIColor redColor];
        [self setupLayer];
        [self setupContext];
        [self setupRenderBuffer];
        [self setupFrameBuffer];

        [self setupVBOs];
        [self compileShaders];
        [self readYUVFile];

        [self render];

    }
    return self;
}

+ (Class)layerClass{
    return [CAEAGLLayer class];
}

-(void)setupLayer{
    _eaglLayer = (CAEAGLLayer *)self.layer;
    _eaglLayer.opaque = YES;
}

- (void)setupContext{
    EAGLRenderingAPI api = kEAGLRenderingAPIOpenGLES2;
    _cOntext= [[[EAGLContext alloc] initWithAPI:api] autorelease];

    if (!_context) {
        NSLog(@"Failed to initialize OpenGLES 2.0 context");
        exit(1);
    }

    if (![EAGLContext setCurrentContext:_context]) {
        NSLog(@"Failed to set current OpenGL context");
        exit(1);
    }
}

- (void)setupRenderBuffer {
    glGenRenderbuffers(1, &_colorRenderBuffer);
    glBindRenderbuffer(GL_RENDERBUFFER, _colorRenderBuffer);        
    [_context renderbufferStorage:GL_RENDERBUFFER fromDrawable:_eaglLayer];    
}

- (void)setupFrameBuffer {    
    GLuint framebuffer;
    glGenFramebuffers(1, &framebuffer);
    glBindFramebuffer(GL_FRAMEBUFFER, framebuffer);
    glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, 
                              GL_RENDERBUFFER, _colorRenderBuffer);
}

- (GLuint) textureY: (Byte*)imageData        
          widthType: (int) width       
         heightType: (int) height       
{          
    GLuint texName;    
    glGenTextures( 1, &texName );     
    glBindTexture(GL_TEXTURE_2D, texName);

    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR );    
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR );    
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);    
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);    

    glTexImage2D( GL_TEXTURE_2D, 0, GL_LUMINANCE, width, height, 0, GL_LUMINANCE, GL_UNSIGNED_BYTE, imageData );  
    //free(imageData);

    return texName;    
}    

- (GLuint) textureU: (Byte*)imageData        
          widthType: (int) width       
         heightType: (int) height       
{          
    GLuint texName;    

    glGenTextures( 1, &texName );     
    glBindTexture(GL_TEXTURE_2D, texName);


    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR );    
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR );    
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);    
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);    

    glTexImage2D( GL_TEXTURE_2D, 0, GL_RED_EXT, width, height, 0, GL_RED_EXT, GL_UNSIGNED_BYTE, imageData );    

    //free(imageData);
    return texName;    
}    

- (GLuint) textureV: (Byte*)imageData        
          widthType: (int) width       
         heightType: (int) height       
{          
    GLuint texName;    
    glGenTextures( 1, &texName );     
    glBindTexture(GL_TEXTURE_2D, texName);


    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR );    
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR );    
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);    
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);    

    glTexImage2D( GL_TEXTURE_2D, 0, GL_RED_EXT, width, height, 0, GL_RED_EXT, GL_UNSIGNED_BYTE, imageData );    

    //free(imageData);
    return texName;    
}    


- (void) readYUVFile     
{    
    NSString *file = [[NSBundle mainBundle] pathForResource:@"video" ofType:@"yv12"];
    NSLog(@"%@",file);
    NSData* fileData = [NSData dataWithContentsOfFile:file]; 
    //NSLog(@"%@",[fileData description]);
    NSInteger width  = 352;    
    NSInteger height = 288;
    NSInteger uv_width  = width  / 2;    
    NSInteger uv_height = height / 2;
    NSInteger dataSize = [fileData length];
    NSLog(@"%i\n",dataSize);

    GLint nYsize  = width * height;     
    GLint nUVsize = uv_width * uv_height;      
    GLint nCbOffSet = nYsize;    
    GLint nCrOffSet = nCbOffSet + nUVsize;    

    Byte *spriteData = (Byte *)malloc(dataSize);
    [fileData getBytes:spriteData length:dataSize];


    Byte* uData = spriteData + nCbOffSet;
    //NSLog(@"%@\n",[[NSData dataWithBytes:uData length:nUVsize] description]);
    Byte* vData = spriteData + nCrOffSet;  
    //NSLog(@"%@\n",[[NSData dataWithBytes:vData length:nUVsize] description]);

    Byte *YPlanarData = (Byte *)malloc(nYsize);
    for (int i=0; i

1 个解决方案

#1


2  

my foolish mistake,the three uploading-texture process and fragment shader are correct ,but the following code are not compatible:

我的愚蠢错误,三个上传纹理的过程和片段着色器是正确的,但是下面的代码是不兼容的:

glActiveTexture(GL_TEXTURE0); 
glBindTexture(GL_TEXTURE_2D, _YPlanarTexture);
glUniform1i(_textureUniformY, 0);

glActiveTexture(GL_TEXTURE1); 
glBindTexture(GL_TEXTURE_2D, _UPlanarTexture);
glUniform1i(_textureUniformU, 1);

glActiveTexture(GL_TEXTURE2); 
glBindTexture(GL_TEXTURE_2D, _VPlanarTexture);
glUniform1i(_textureUniformV, 2);

and below:

和下面的:

_YPlanarTexture = glGetUniformLocation(programHandle, "SamplerY");
_UPlanarTexture = glGetUniformLocation(programHandle, "SamplerU");
_VPlanarTexture = glGetUniformLocation(programHandle, "SamplerV");

so replace with this:

所以替换为:

_textureUniformY = glGetUniformLocation(programHandle, "SamplerY");
_textureUniformU = glGetUniformLocation(programHandle, "SamplerU");
_textureUniformV = glGetUniformLocation(programHandle, "SamplerV");

then it will do right thing .

这样做是对的。


推荐阅读
  • springMVC JRS303验证 ... [详细]
  • PHP 过滤器详解
    本文深入探讨了 PHP 中的过滤器机制,包括常见的 $_SERVER 变量、filter_has_var() 函数、filter_id() 函数、filter_input() 函数及其数组形式、filter_list() 函数以及 filter_var() 和其数组形式。同时,详细介绍了各种过滤器的用途和用法。 ... [详细]
  • 在Java应用程序开发过程中,FTP协议被广泛用于文件的上传和下载操作。本文通过Jakarta Commons Net库中的FTPClient类,详细介绍如何实现文件的上传和下载功能。 ... [详细]
  • 本文详细探讨了JDBC(Java数据库连接)的内部机制,重点分析其作为服务提供者接口(SPI)框架的应用。通过类图和代码示例,展示了JDBC如何注册驱动程序、建立数据库连接以及执行SQL查询的过程。 ... [详细]
  • 本文深入探讨了C++对象模型中的一些细节问题,特别是虚拟继承和析构函数的处理。通过具体代码示例和详细分析,揭示了书中某些观点的不足之处,并提供了更合理的解释。 ... [详细]
  • 本文探讨了在Oracle数据库中,动态SQL语句的执行及其对事务管理的影响,特别是关于回滚操作的有效性。重点讨论了一个具体场景:将预警短信从当前表迁移到历史表时遇到的字段长度不匹配问题及相应的异常处理。 ... [详细]
  • 深入解析 Spring Security 用户认证机制
    本文将详细介绍 Spring Security 中用户登录认证的核心流程,重点分析 AbstractAuthenticationProcessingFilter 和 AuthenticationManager 的工作原理。通过理解这些组件的实现,读者可以更好地掌握 Spring Security 的认证机制。 ... [详细]
  • 微软Exchange服务器遭遇2022年版“千年虫”漏洞
    微软Exchange服务器在新年伊始遭遇了一个类似于‘千年虫’的日期处理漏洞,导致邮件传输受阻。该问题主要影响配置了FIP-FS恶意软件引擎的Exchange 2016和2019版本。 ... [详细]
  • 深入解析TCP/IP五层协议
    本文详细介绍了TCP/IP五层协议模型,包括物理层、数据链路层、网络层、传输层和应用层。每层的功能及其相互关系将被逐一解释,帮助读者理解互联网通信的原理。此外,还特别讨论了UDP和TCP协议的特点以及三次握手、四次挥手的过程。 ... [详细]
  • 本文详细介绍了在 Windows 2000 系统中启用 TELNET 服务时需要注意的 NTLM 配置问题,帮助用户解决常见的身份验证失败错误。 ... [详细]
  • 解决FCKeditor应用主题后上传问题及优化配置
    本文介绍了在Freetextbox收费后选择FCKeditor作为替代方案时遇到的上传问题及其解决方案。通过调整配置文件和调试工具,最终解决了上传失败的问题,并对相关配置进行了优化。 ... [详细]
  • 本文详细介绍了在不同操作系统中查找和设置网卡的方法,涵盖了Windows系统的具体步骤,并提供了关于网卡位置、无线网络设置及常见问题的解答。 ... [详细]
  • 深入解析Java虚拟机(JVM)架构与原理
    本文旨在为读者提供对Java虚拟机(JVM)的全面理解,涵盖其主要组成部分、工作原理及其在不同平台上的实现。通过详细探讨JVM的结构和内部机制,帮助开发者更好地掌握Java编程的核心技术。 ... [详细]
  • 本文详细介绍了虚拟专用网(Virtual Private Network, VPN)的概念及其通过公共网络(如互联网)构建临时且安全连接的技术特点。文章探讨了不同类型的隧道协议,包括第二层和第三层隧道协议,并提供了针对IPSec、GRE以及MPLS VPN的具体配置指导。 ... [详细]
  • 2017-2018年度《网络编程与安全》第五次实验报告
    本报告详细记录了2017-2018学年《网络编程与安全》课程第五次实验的具体内容、实验过程、遇到的问题及解决方案。 ... [详细]
author-avatar
仰望天空说再见
这个家伙很懒,什么也没留下!
PHP1.CN | 中国最专业的PHP中文社区 | DevBox开发工具箱 | json解析格式化 |PHP资讯 | PHP教程 | 数据库技术 | 服务器技术 | 前端开发技术 | PHP框架 | 开发工具 | 在线工具
Copyright © 1998 - 2020 PHP1.CN. All Rights Reserved | 京公网安备 11010802041100号 | 京ICP备19059560号-4 | PHP1.CN 第一PHP社区 版权所有