uncompressed 14-bit RAW video recording

Started by g3gg0, April 27, 2013, 12:07:12 AM

Previous topic - Next topic

0 Members and 4 Guests are viewing this topic.

Michael Zöller

I've moved the posts regarding compilation and TCC to a new thread.
neoluxx.de
EOS 5D Mark II | EOS 600D | EF 24-70mm f/2.8 | Tascam DR-40

kgv5

I made a quick comparison (with my 6D)  test in AE using linear wipe: 1080p h.264 IPB encoder VBR bitrate 10, qp 6 24fps (bitrate about 80 mbit/s) vs 1080p yuv422 24fps processed with 422toimg -single file yuv to jpeg sequence - quality 100%.
IMHO there is almost no visible difference, maybe very very small improvement only visible on 400% crop. Did you guys make such comparisons?
www.pilotmovies.pl   5D Mark III, 6D, 550D

3pointedit

A good way to detect artefacts is to mix between identical shots with a subtraction function. This way the only image data left will be the difference.
550D on ML-roids

g3gg0

thanks for your comparison.
i think JPEG might be a bad choice when you want to compare image quality, even with factor set to 100 :)
i am no video guy, but maybe you should compare fine details with high contrast. like grass or trees.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

1%

Low QPs are supposed to be close to "lossless".. but I see a difference in the 4:2:0 to 4:2:2 right away. I got MZ sync'd on 600D for 24P/30P and even works when 720P is overwritten to 24 frames... I still get shifted frames tho :(

Weird... I think burst + modules use the same EVF sync... when I tried MZ sync for evf sync it produced bad burst pics.. but normally it does not... so wtf is happening to the YUV frames if burst frames are coming out perfect. Almost 160 YUV frames on 720P today... I'm hoping when raw recording is decodable it won't be doing this. I think frame sizes are similar so the yield should be too.


Ok, double weird.... if you look at the sequences with moving subjects... after 21 frames the corruption starts, first at the bottom. Then it appears after a few frames the sequence is repeating. i.e I see the same truck that passed me in a good frame in a corrupt frame further down the line. WTF, indeed.

Issue remains, sraw trick or not.

g3gg0

huh?
when recording with lv_rec you get corrupted frames?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

1%

Yea, on 600D. Just like the guy posted. After 21 frames starts to look like mis-aligned old silent pic and repeats images. 6D I haven't noticed any issues yet. Someone was complaining about something similar with 5dII, maybe affects digic IV only?

minimimi

memory copy test results(without memSuite)
I'm checked it but chunk copy is fast . It's misteirous.
and, I don't know uint32_t timer counter is how many cycled.So I don't know the result is correct or not.


The code is here, and these codes located at  run_test()
    void *m1 = malloc(20*1000*1000);
    void *m2 = malloc(20*1000*1000);
    void *sm1 = shoot_malloc(20*1000*1000);
    void *sm2 = shoot_malloc(20*1000*1000);
    void *ss1 = shoot_malloc_suite(20*1000*1000);
    void *ss2 = shoot_malloc_suite(20*1000*1000);

    uint32_t t1 = tskmon_get_timer_reg();
    uint32_t t2,tmp;
    int i,off;

    console_printf("malloc-ed 20M memcpy\n");
    memcpy(m1,m2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            tmp = ~tmp - t1;
            t2 = t2 + tmp;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("malloc-ed 20M memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            memcpy(m1+off,m2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("malloc-ed 20M dma_memcpy\n");
    dma_memcpy(m1,m2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("malloc-ed 20M dma_memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            dma_memcpy(m1+off,m2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc-ed 20M memcpy\n");
    memcpy(sm1,sm2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc-ed 20M memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            memcpy(sm1+off,sm2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc-ed 20M dma_memcpy\n");
    dma_memcpy(sm1,sm2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc-ed 20M dma_memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            dma_memcpy(sm1+off,sm2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("malloc to shoot_malloc-ed 20M memcpy\n");
    memcpy(sm1,m2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("malloc shoot_malloc-ed 20M memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            memcpy(sm1+off,m2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc to malloc 20M memcpy\n");
    memcpy(m1,sm2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc to malloc 20M memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            memcpy(m1+off,sm2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);





    t1 = tskmon_get_timer_reg();
    console_printf("malloc to shoot_malloc-ed 20M dma_memcpy\n");
    dma_memcpy(sm1,m2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            tmp = ~tmp - t1;
            t2 = t2 + tmp;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("malloc shoot_malloc-ed 20M dma_memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            dma_memcpy(sm1+off,m2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            tmp = ~tmp - t1;
            t2 = t2 + tmp;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc to malloc 20M dma_memcpy\n");
    dma_memcpy(m1,sm2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            tmp = ~tmp - t1;
            t2 = t2 + tmp;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc to malloc 20M dma_memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            dma_memcpy(m1+off,sm2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            tmp = ~tmp - t1;
            t2 = t2 + tmp;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);




And result is:  (printf has no %u so I used %d(signed))
malloc-ed 20M memcpy
-485806
malloc-ed 20M memcpy 2m each
-71147
malloc-ed 20M dma_memcpy
-272364
malloc-ed 20M dma_memcpy 2m each
-524067
shoot_malloc-ed 20M memcpy
-484381
shoot_malloc-ed 20M memcpy 2m each
-81456
shoot_malloc-ed 20M dma_memcpy
-768859
shoot_malloc-ed 20M dma_memcpy 2m each
-527604
malloc to shoot_malloc-ed 20M memcpy
-561515
malloc shoot_malloc-ed 20M memcpy 2m each
-76210
shoot_malloc to malloc 20M memcpy
-494179
shoot_malloc to malloc 20M memcpy 2m each
-77837
malloc to shoot_malloc-ed 20M dma_memcpy
-266705
malloc shoot_malloc-ed 20M dma_memcpy 2m each
-513931
shoot_malloc to malloc 20M dma_memcpy
-268545
shoot_malloc to malloc 20M dma_memcpy 2m each
-537939

Andy600

Sorry for the dumb question. I have no problem working with DNGs but I haven't a clue how to convert YUV files. I downloaded 422ToImage but can't get it to work. Are there any tuts?
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

AlexVakulov

Quote from: Andy600 on May 07, 2013, 06:50:24 PM
Sorry for the dumb question. I have no problem working with DNGs but I haven't a clue how to convert YUV files. I downloaded 422ToImage but can't get it to work. Are there any tuts?
Hello! Change your file extension from YUV to 422 and use the same program.

1%


a1ex

I've noticed the same distortion on 5D3 with raw mode (480p at 12fps). First frames are pretty much OK, then it loses sync.

mindogas

Quote from: AlexVakulov on May 07, 2013, 06:58:12 PM
Hello! Change your file extension from YUV to 422 and use the same program.
Bad answer. *.YUV extension is for YUV422 videos and *.422 for still pictures. *.YUV files contains some extra information about its frames so  this extra data is like trash for .422 conversion. converting *.YUV files is quite easy but you must know that this is experimental feature. ML and 422ToImage sources changes everyday so you should use the last versions then you experimenting with YUV videos.

g3gg0

hmm, interesting.
made a few videos in .422 video mode and got no distortion at all.
this looks like there is some data lost in the beginning of frame number 21.

will look into it tonight.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

deleted.account

Quote from: mindogas on May 07, 2013, 07:16:33 PM
Bad answer. *.YUV extension is for YUV422 videos and *.422 for still pictures. *.YUV files contains some extra information about its frames so  this extra data is like trash for .422 conversion. converting *.YUV files is quite easy but you must know that this is experimental feature. ML and 422ToImage sources changes everyday so you should use the last versions then you experimenting with YUV videos.

Is it really a bad answer? I rename the .422 files to .yuv, import into avisynth, rescale chroma as the source yuv is JFIF chroma over full 8bit range and encode to h264 without creating any intermediate image sequences at all. :-)

We already know images sizes for .422 and can specify them in Avisynth so no big deal.

Its raw 8bit yuv, whether we call it image or video, whether we name it .422 or yuv.

Its down to how the host application handles it?

1%

Haven't seen any LV_rec issues on 6D yet... but I can't play back the raw :( (yea i know, bitch bitch bitch ) :)

kgv5

Quote from: Andy600 on May 07, 2013, 06:50:24 PM
Sorry for the dumb question. I have no problem working with DNGs but I haven't a clue how to convert YUV files. I downloaded 422ToImage but can't get it to work. Are there any tuts?

Do you have the newest 1.9 version? it handles single movie files *.yuv very well, i tried today and had no problem with converting yuv to jpeg.
ver 1.8 didn't have such a feature
www.pilotmovies.pl   5D Mark III, 6D, 550D

g3gg0

weird. tried now with 1700 frames again and no problem.
maybe its due to a slow card?
i have a lexar 1000x. do you have a slower one?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

mindogas

Quote from: y3llow on May 07, 2013, 08:57:45 PM
Is it really a bad answer? I rename the .422 files to .yuv, import into avisynth, rescale chroma as the source yuv is JFIF chroma over full 8bit range and encode to h264 without creating any intermediate image sequences at all. :-)

We already know images sizes for .422 and can specify them in Avisynth so no big deal.

Its raw 8bit yuv, whether we call it image or video, whether we name it .422 or yuv.

Its down to how the host application handles it?
Your post should start from last line i think. The thing is that question was about 422toimage software and for it the answer is clear - renaming files extension messing things up.

1%

6D, no issues SD card
600D, same make/model sd = issues.

theoretically raw on 6D should be really out of sync if its bad on 5d3 with "slower" card.

g3gg0

Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

deleted.account

Quote from: mindogas on May 07, 2013, 10:31:46 PM
Your post should start from last line i think. The thing is that question was about 422toimage software and for it the answer is clear - renaming files extension messing things up.

Of coarse you're correct as the author of the application. It 'messes things up' due to the way 422toimage handles the source, my apologies.


shuggyrasklat


g3gg0

Oh come on. How often did we explain that?
Let us finish the stuff so it is working at least on one model before we port it to the other models.
....
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!