Author Topic: uncompressed 14-bit RAW video recording (Read 1114533 times)

Michael Zöller · « **Reply #150 on:** May 07, 2013, 09:33:56 AM »

I've moved the posts regarding compilation and TCC to a new thread.

kgv5 · « **Reply #151 on:** May 07, 2013, 11:18:41 AM »

I made a quick comparison (with my 6D) test in AE using linear wipe: 1080p h.264 IPB encoder VBR bitrate 10, qp 6 24fps (bitrate about 80 mbit/s) vs 1080p yuv422 24fps processed with 422toimg -single file yuv to jpeg sequence - quality 100%.
IMHO there is almost no visible difference, maybe very very small improvement only visible on 400% crop. Did you guys make such comparisons?

3pointedit · « **Reply #152 on:** May 07, 2013, 12:51:29 PM »

A good way to detect artefacts is to mix between identical shots with a subtraction function. This way the only image data left will be the difference.

g3gg0 · « **Reply #153 on:** May 07, 2013, 12:59:22 PM »

thanks for your comparison.
i think JPEG might be a bad choice when you want to compare image quality, even with factor set to 100

i am no video guy, but maybe you should compare fine details with high contrast. like grass or trees.

1% · « **Reply #154 on:** May 07, 2013, 03:45:54 PM »

Low QPs are supposed to be close to "lossless".. but I see a difference in the 4:2:0 to 4:2:2 right away. I got MZ sync'd on 600D for 24P/30P and even works when 720P is overwritten to 24 frames... I still get shifted frames tho

Weird... I think burst + modules use the same EVF sync... when I tried MZ sync for evf sync it produced bad burst pics.. but normally it does not... so wtf is happening to the YUV frames if burst frames are coming out perfect. Almost 160 YUV frames on 720P today... I'm hoping when raw recording is decodable it won't be doing this. I think frame sizes are similar so the yield should be too.

Ok, double weird.... if you look at the sequences with moving subjects... after 21 frames the corruption starts, first at the bottom. Then it appears after a few frames the sequence is repeating. i.e I see the same truck that passed me in a good frame in a corrupt frame further down the line. WTF, indeed.

Issue remains, sraw trick or not.

g3gg0 · « **Reply #155 on:** May 07, 2013, 04:51:42 PM »

huh?
when recording with lv_rec you get corrupted frames?

1% · « **Reply #156 on:** May 07, 2013, 05:06:14 PM »

Yea, on 600D. Just like the guy posted. After 21 frames starts to look like mis-aligned old silent pic and repeats images. 6D I haven't noticed any issues yet. Someone was complaining about something similar with 5dII, maybe affects digic IV only?

minimimi · « **Reply #157 on:** May 07, 2013, 05:16:29 PM »

memory copy test results(without memSuite)
I'm checked it but chunk copy is fast . It's misteirous.
and, I don't know uint32_t timer counter is how many cycled.So I don't know the result is correct or not.

The code is here, and these codes located at run_test()

Code: [Select]

    void *m1 = malloc(20*1000*1000);
    void *m2 = malloc(20*1000*1000);
    void *sm1 = shoot_malloc(20*1000*1000);
    void *sm2 = shoot_malloc(20*1000*1000);
    void *ss1 = shoot_malloc_suite(20*1000*1000);
    void *ss2 = shoot_malloc_suite(20*1000*1000);

    uint32_t t1 = tskmon_get_timer_reg();
    uint32_t t2,tmp;
    int i,off;

    console_printf("malloc-ed 20M memcpy\n");
    memcpy(m1,m2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            tmp = ~tmp - t1;
            t2 = t2 + tmp;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("malloc-ed 20M memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            memcpy(m1+off,m2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("malloc-ed 20M dma_memcpy\n");
    dma_memcpy(m1,m2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("malloc-ed 20M dma_memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            dma_memcpy(m1+off,m2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc-ed 20M memcpy\n");
    memcpy(sm1,sm2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc-ed 20M memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            memcpy(sm1+off,sm2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc-ed 20M dma_memcpy\n");
    dma_memcpy(sm1,sm2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc-ed 20M dma_memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            dma_memcpy(sm1+off,sm2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("malloc to shoot_malloc-ed 20M memcpy\n");
    memcpy(sm1,m2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("malloc shoot_malloc-ed 20M memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            memcpy(sm1+off,m2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc to malloc 20M memcpy\n");
    memcpy(m1,sm2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc to malloc 20M memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            memcpy(m1+off,sm2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);





    t1 = tskmon_get_timer_reg();
    console_printf("malloc to shoot_malloc-ed 20M dma_memcpy\n");
    dma_memcpy(sm1,m2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            tmp = ~tmp - t1;
            t2 = t2 + tmp;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("malloc shoot_malloc-ed 20M dma_memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            dma_memcpy(sm1+off,m2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            tmp = ~tmp - t1;
            t2 = t2 + tmp;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc to malloc 20M dma_memcpy\n");
    dma_memcpy(m1,sm2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            tmp = ~tmp - t1;
            t2 = t2 + tmp;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc to malloc 20M dma_memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            dma_memcpy(m1+off,sm2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            tmp = ~tmp - t1;
            t2 = t2 + tmp;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

And result is: (printf has no %u so I used %d(signed))

Code: [Select]

malloc-ed 20M memcpy
-485806
malloc-ed 20M memcpy 2m each
-71147
malloc-ed 20M dma_memcpy
-272364
malloc-ed 20M dma_memcpy 2m each
-524067
shoot_malloc-ed 20M memcpy
-484381
shoot_malloc-ed 20M memcpy 2m each
-81456
shoot_malloc-ed 20M dma_memcpy
-768859
shoot_malloc-ed 20M dma_memcpy 2m each
-527604
malloc to shoot_malloc-ed 20M memcpy
-561515
malloc shoot_malloc-ed 20M memcpy 2m each
-76210
shoot_malloc to malloc 20M memcpy
-494179
shoot_malloc to malloc 20M memcpy 2m each
-77837
malloc to shoot_malloc-ed 20M dma_memcpy
-266705
malloc shoot_malloc-ed 20M dma_memcpy 2m each
-513931
shoot_malloc to malloc 20M dma_memcpy
-268545
shoot_malloc to malloc 20M dma_memcpy 2m each
-537939

Andy600 · « **Reply #158 on:** May 07, 2013, 06:50:24 PM »

Sorry for the dumb question. I have no problem working with DNGs but I haven't a clue how to convert YUV files. I downloaded 422ToImage but can't get it to work. Are there any tuts?

AlexVakulov · « **Reply #159 on:** May 07, 2013, 06:58:12 PM »

Quote from: Andy600 on May 07, 2013, 06:50:24 PM

Sorry for the dumb question. I have no problem working with DNGs but I haven't a clue how to convert YUV files. I downloaded 422ToImage but can't get it to work. Are there any tuts?

Hello! Change your file extension from YUV to 422 and use the same program.

1% · « **Reply #160 on:** May 07, 2013, 07:07:49 PM »

This is what is happening to 600D sequences:

http://imgur.com/a/fNUme#0

a1ex · « **Reply #161 on:** May 07, 2013, 07:12:45 PM »

I've noticed the same distortion on 5D3 with raw mode (480p at 12fps). First frames are pretty much OK, then it loses sync.

mindogas · « **Reply #162 on:** May 07, 2013, 07:16:33 PM »

Quote from: AlexVakulov on May 07, 2013, 06:58:12 PM

Hello! Change your file extension from YUV to 422 and use the same program.

Bad answer. *.YUV extension is for YUV422 videos and *.422 for still pictures. *.YUV files contains some extra information about its frames so this extra data is like trash for .422 conversion. converting *.YUV files is quite easy but you must know that this is experimental feature. ML and 422ToImage sources changes everyday so you should use the last versions then you experimenting with YUV videos.

g3gg0 · « **Reply #163 on:** May 07, 2013, 07:46:52 PM »

hmm, interesting.
made a few videos in .422 video mode and got no distortion at all.
this looks like there is some data lost in the beginning of frame number 21.

will look into it tonight.

deleted.account · « **Reply #164 on:** May 07, 2013, 08:57:45 PM »

Quote from: mindogas on May 07, 2013, 07:16:33 PM

Bad answer. *.YUV extension is for YUV422 videos and *.422 for still pictures. *.YUV files contains some extra information about its frames so this extra data is like trash for .422 conversion. converting *.YUV files is quite easy but you must know that this is experimental feature. ML and 422ToImage sources changes everyday so you should use the last versions then you experimenting with YUV videos.

Is it really a bad answer? I rename the .422 files to .yuv, import into avisynth, rescale chroma as the source yuv is JFIF chroma over full 8bit range and encode to h264 without creating any intermediate image sequences at all. :-)

We already know images sizes for .422 and can specify them in Avisynth so no big deal.

Its raw 8bit yuv, whether we call it image or video, whether we name it .422 or yuv.

Its down to how the host application handles it?

1% · « **Reply #165 on:** May 07, 2013, 09:47:25 PM »

Haven't seen any LV_rec issues on 6D yet... but I can't play back the raw

(yea i know, bitch bitch bitch )

kgv5 · « **Reply #166 on:** May 07, 2013, 10:14:57 PM »

Quote from: Andy600 on May 07, 2013, 06:50:24 PM

Sorry for the dumb question. I have no problem working with DNGs but I haven't a clue how to convert YUV files. I downloaded 422ToImage but can't get it to work. Are there any tuts?

Do you have the newest 1.9 version? it handles single movie files *.yuv very well, i tried today and had no problem with converting yuv to jpeg.
ver 1.8 didn't have such a feature

g3gg0 · « **Reply #167 on:** May 07, 2013, 10:31:07 PM »

weird. tried now with 1700 frames again and no problem.
maybe its due to a slow card?
i have a lexar 1000x. do you have a slower one?

mindogas · « **Reply #168 on:** May 07, 2013, 10:31:46 PM »

Quote from: y3llow on May 07, 2013, 08:57:45 PM

Is it really a bad answer? I rename the .422 files to .yuv, import into avisynth, rescale chroma as the source yuv is JFIF chroma over full 8bit range and encode to h264 without creating any intermediate image sequences at all. :-)

We already know images sizes for .422 and can specify them in Avisynth so no big deal.

Its raw 8bit yuv, whether we call it image or video, whether we name it .422 or yuv.

Its down to how the host application handles it?

Your post should start from last line i think. The thing is that question was about 422toimage software and for it the answer is clear - renaming files extension messing things up.

1% · « **Reply #169 on:** May 07, 2013, 10:36:10 PM »

6D, no issues SD card
600D, same make/model sd = issues.

theoretically raw on 6D should be really out of sync if its bad on 5d3 with "slower" card.

g3gg0 · « **Reply #170 on:** May 07, 2013, 11:07:05 PM »

about the EDMAC performance:

deleted.account · « **Reply #171 on:** May 07, 2013, 11:16:43 PM »

Quote from: mindogas on May 07, 2013, 10:31:46 PM

Your post should start from last line i think. The thing is that question was about 422toimage software and for it the answer is clear - renaming files extension messing things up.

Of coarse you're correct as the author of the application. It 'messes things up' due to the way 422toimage handles the source, my apologies.

1% · « **Reply #172 on:** May 07, 2013, 11:57:56 PM »

Edmacs from 6D.

http://imgur.com/a/u3v6q#0

http://imgur.com/a/u3v6q#1

EOS-M

http://imgur.com/a/SP2jG

shuggyrasklat · « **Reply #173 on:** May 08, 2013, 05:26:18 AM »

why cant we get this for the mkii?!?!?

g3gg0 · « **Reply #174 on:** May 08, 2013, 07:50:28 AM »

Oh come on. How often did we explain that?
Let us finish the stuff so it is working at least on one model before we port it to the other models.
....

News:

Author Topic: uncompressed 14-bit RAW video recording (Read 1114533 times)