HaskellGStreamertee元素(1-N)故障。-HaskellGStreamerteeelement(1-N)trouble

10  

RESOLVED

解决

I found my solution, and what follows is a lengthy post but please, bear with me. I must share my frustration with someone.

我找到了我的解决方案，下面是一个很长的帖子，但是请耐心等待。我必须和别人分享我的沮丧。

After many more buggy tries i decided to go back to testing some setups using gst-launch. This helped me to find out that after the queue element that buffers the part which goes to the filesink i needed another ffmpegcolorspace element to setup the correct video format i think. At this point i was not going back to trying this thing out it Haskell again, i thought i needed to get 'closer' so i decided to try it in C. As a side note , i don't know C , i can understand the syntax but that's about it...and for goodness sake i'm just now trying to learn Haskell. To continue, i decided to also try using 'GS.elementGetCompatiblePad' on the tee element so i can be sure that the pads will link with the queue.

在更多的bug尝试之后，我决定返回测试一些使用gst-launch的设置。这帮助我发现，在队列元素缓存到filesink的部分后，我需要另一个ffmpegcolorspace元素来设置正确的视频格式。现在我不想再用Haskell这个东西了，我想我需要更接近一些，所以我决定用C来试试。看在上帝的份上，我现在正在学习Haskell。为了继续，我决定也尝试使用“GS”。在tee元素上的elementGetCompatiblePad'，这样我可以确保pad将与队列链接。

The C code i stitched together is this :

我缝在一起的C代码是:

#include 
#include 

int
main (int argc,char *argv[])
{

    GstElement *pipeline, *source, *color, *color2 , *color3, *tee, *rQ, *vQ, *encoder,   *fSink , *sink;
    GMainLoop *loop;
    loop = g_main_loop_new (NULL,FALSE);
    /* initialize gstreamer */
    gst_init(&argc,&argv);

    /* creating elements */
    pipeline = gst_pipeline_new("stream-pipeline");

    source = gst_element_factory_make ("v4l2src","stream-source");
    color = gst_element_factory_make ("ffmpegcolorspace","video-color");
    tee = gst_element_factory_make ("tee","stream-tee");
    rQ = gst_element_factory_make ("queue","record-queue");
    vQ = gst_element_factory_make ("queue","video-queue");
    encoder = gst_element_factory_make ("theoraenc","video-encoder");
    fSink = gst_element_factory_make ("filesink","record-sink");
    sink = gst_element_factory_make ("ximagesink","video-sink");
    color2 = gst_element_factory_make ("ffmpegcolorspace","video-color2");
    color3 = gst_element_factory_make ("ffmpegcolorspace","video-color3");
    /*check that the elements were created */

    if (!source || !color || !tee || !rQ || !vQ || !encoder || !fSink || !sink){
        g_printerr("One element could not be created!");
        return -1;
    }
    /*set file output location */
    g_object_set(G_OBJECT (fSink),"location","rec",NULL);

    gst_bin_add_many (GST_BIN(pipeline),
                        source,color,color2,color3,tee,rQ,vQ,encoder,fSink,sink,NULL);

    /* get request pads */
    GstPad *dPad, *rPad, *sDPad, *sRPad;

    sDPad = gst_element_get_static_pad(vQ,"sink");
    sRPad = gst_element_get_static_pad(rQ,"sink");
    dPad = gst_element_get_compatible_pad(tee,sDPad,GST_CAPS_ANY);
    rPad = gst_element_get_compatible_pad(tee,sRPad,GST_CAPS_ANY);

    /*link pads*/
    gst_pad_link(dPad,sDPad);
    gst_pad_link(rPad,sRPad);

    /*unref pads */
    gst_object_unref(GST_OBJECT(dPad));
    gst_object_unref(GST_OBJECT(rPad));
    gst_object_unref(GST_OBJECT(sDPad));
    gst_object_unref(GST_OBJECT(sRPad));

    /*link elements */
    gst_element_link(source,tee);
    gst_element_link_many(rQ,color2,encoder,fSink,NULL);
    gst_element_link_many(vQ,color3,sink),NULL;

    /*set the pipeline state to playing */
    gst_element_set_state(pipeline,GST_STATE_PLAYING);

    g_main_loop_run (loop);

    gst_element_set_state(pipeline,GST_STATE_NULL);
    gst_object_unref(GST_OBJECT(pipeline));

    return 0;

}

In order to use 'gst_element_get_compatible_pad' i had to first get static pads from the queue elements so i hand to switch those four related lines. I try it out, and Abracadabra ...oh no, wait ... camera starts, the file is created and a window with the 'video' pops, but a black window that remains black!

为了使用“gst_element_get_compatible_pad”，我必须首先从队列元素中获取静态pad，因此我需要切换这四行相关的代码。我试了一下，然后Abracadabra…哦不,等等…摄像头启动，文件创建，窗口弹出“视频”，但黑色窗口仍然是黑色的!

No problem i say , run the program with gst-debug-level=5 ( =)) ) yea, right , try reading the whole output.I give up for the moment and i thought maybe it has something to do with the elements in my pipeline not working right together so i code another pipeline in C but this time something more simple just with audio files.
I had the same result so i decided tu debug again, this time with runlevel 3 and i started reading the whole thing,line by line.

没问题，我说，用gst-debug-level=5(=)运行程序)是的，对，试着读取整个输出。我暂时放弃了，我想也许这和我的管道中的元素有关系，不是一起工作，所以我在C中编写了另一个管道，但这次更简单的是用音频文件。我得到了相同的结果，所以我决定再次进行tu调试，这次是使用runlevel 3，我开始逐行地阅读整个过程。

Somewhere in there i found this:

我在那里的某个地方发现了这个:

trying to link stream-tee:src0 and record-queue:sink
trying to link stream-tee:src0 and video-queue:sink

尝试链接流-tee:src0和记录队列:sink尝试链接流-tee:src0和视频队列:sink

something nasty is happening here

这里发生了一些令人不快的事情

linked stream-tee:src0 and video-queue:sink,successful
trying to link stream-tee:src0 and record-queue:sink
src stream-tee:src0 was already linked with video-queue:sink

链接流-tee:src0和视频队列:sink，成功链接流-tee:src0和记录队列:sink src流-tee:src0已经与视频队列:sink链接

And it gives up!
I guess i must go back using gst_element_get_request_pad, but haven't i tried that already? So i switch back to vim and replace all occurrences of 'gst_element_get_compatible_pad with the request counterpart like so:

而且它放弃!我想我必须使用gst_element_get_request_pad返回，但是我还没有尝试过吗?因此，我切换回vim，将所有出现的“gst_element_get_compatible_pad”替换为请求对等项，如下所示:

sDPad = gst_element_get_static_pad(vQ,"sink");
sRPad = gst_element_get_static_pad(rQ,"sink");
dPad = gst_element_get_request_pad(tee,"src%d");
rPad = gst_element_get_request_pad(tee,"src%d");

I gaze upon this code and i say to myself 'you twit', this is where it all started; take a deep breath ; after all this is what the debugger complains about so i compile , i run, and Voila. I found my solution.

我凝视着这段代码，对自己说‘你这个笨蛋’，这就是一切的开始;深呼吸;毕竟这是调试器所抱怨的，所以我编译，运行，然后Voila。我发现我的解决方案。

Those four lines had to be reversed, i had to first get a reference to the static pads and then request a reference to a 'request' pad on the tee element.
I go back to haskell a happy man.I implement my solution, compile, fire up,camera starts, the file is created and ... just like that..nothing, not even the black screen.
Filled with anger i just comment out the lines where i release the request pads and decide to compile and run once more, my neck started to hurt a while ago.
Again, by magic it all works , i have video on the screen and in the file.
I guess Haskell just likes to hold tighter and sometimes you have to just go with something that makes no sense. The gstreamer docs state clearly release,release,release.

这四行必须颠倒，我必须首先获得对静态pad的引用，然后请求对tee元素上的“request”pad的引用。我回到哈斯凯尔，一个快乐的人。我实现我的解决方案，编译，启动，启动摄像头，创建文件……就像这样…什么都没有，连黑屏都没有。我非常愤怒，我只是注释掉了一些行，在这些行中，我发布了请求垫，并决定再次编译并运行。同样，神奇的是，我在屏幕上和文件中都有视频。我想Haskell只是喜欢抓得更紧一些，有时候你不得不去做一些毫无意义的事情。gstreamer文档明确地声明发布、发布、发布。

The final haskell code:

最后haskell代码:

module Main(main) where

import qualified Media.Streaming.GStreamer as GS
import Data.Maybe
import System.Exit
import System.Glib.MainLoop as Glib
import System.Glib.Signals as Glib
import System.Glib.Properties as Glib

makeElement:: String → String → IO GS.Element
makeElement elementType elementName = do
        element ← GS.elementFactoryMake elementType (Just elementName)
        case element of
            Just element' → return element'
            Nothing → do
                    putStrLn "Cannot create element!"
                    exitFailure

linkSPadToStaticSink::(GS.ElementClass object, GS.ElementClass elementT) ⇒ object →     elementT → IO (Glib.ConnectId object)
linkSPadToStaticSink elSrc elSink = do
            Glib.on elSrc GS.elementPadAdded (λpad → do
                                                    sinkPad ← GS.elementGetStaticPad elSink "sink"
                                                    GS.padLink pad (fromJust sinkPad)
                                                    return ∅)

player =  do
        GS.init
        pipeline ← GS.pipelineNew "video-stream"
        source ← makeElement "v4l2src" "video-source"
        color ← makeElement "ffmpegcolorspace" "video-color"
        color2 ← makeElement "ffmpegcolorspace" "video-color2"
        tee ← makeElement "tee" "stream-tee"
        rQ ← makeElement "queue" "record-queue"
        vQ ← makeElement "queue" "video-queue"
        encoder ← makeElement "y4menc" "video-encoder"
        rSink ← makeElement "filesink" "record-sink"
        sink ← makeElement "ximagesink" "video-sink"

        let elements = [source,color,color2,encoder,rSink,vQ,rQ,sink,tee]

        Glib.objectSetPropertyString "location" rSink "rec"

        mapM_ (GS.binAdd (GS.castToBin pipeline)) elements

        -- Get static pads from queue elements
        sDPad ← GS.elementGetStaticPad vQ "sink"
        sRPad ← GS.elementGetStaticPad rQ "sink"
        -- Request pads from tee element
        dPad ← GS.elementGetRequestPad tee "src%d"
        rPad ← GS.elementGetRequestPad tee "src%d"
        -- Link tee source to queue sink
        GS.padLink (fromJust dPad) (fromJust sDPad) 
        GS.padLink (fromJust rPad) (fromJust sRPad)

        GS.elementLink source color
        GS.elementLink color tee
        GS.elementLink vQ sink
        GS.elementLink rQ color2
        GS.elementLink color2 encoder
        GS.elementLink encoder rSink

        GS.elementSetState pipeline GS.StatePlaying

main = do
    loop ← Glib.mainLoopNew Nothing False
    player
    Glib.mainLoopRun loop

Now i ask you, should/could i have seen this ?
Was it that obvious ?

现在我问你，我能看到这个吗?这是显而易见的吗?

I'm glad this will make me be more careful and look in not so obvious places but...eww.

我很高兴这能让我更仔细地观察那些不那么明显的地方，但是……

In conclusion to all this, i learned about the gstreamer debug options, i learned that it whispers to me and i MUST listen. I learned about GDB being forced to used because when i began stitching C code all i got was a 'seg fault'.
I learned to love lazy-eval and pure Haskell code.
A little bit of Haskell, maybe a tiny bit of C and more experience. 'Lost' about half a day, three classes and several hours of sleep but after all...So it goes...

综上所述，我了解了gstreamer调试选项，了解了它对我的影响，我必须倾听。我了解到GDB被迫使用是因为当我开始缝合C代码时，我得到的只是一个“seg错误”。我学会了爱惰性的eval和纯粹的Haskell代码。一点点Haskell，也许一点点C和更多的经验。“消失”了大约半天，上了三节课，睡了几个小时，但毕竟……所以它…