Author Topic: 12-bit (and 10-bit) RAW video development discussion  (Read 617113 times)

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: 12-bit (and 10-bit) RAW video
« Reply #125 on: May 24, 2013, 02:51:12 AM »
Fork whichever... I can give you write permisison too if you want.

For 6D I wouldn't fork the 'main repo as its not updated.

Gonna start merging tomorrow morning or later on tonight...

hirethestache

  • Freshman
  • **
  • Posts: 78
Re: 12-bit (and 10-bit) RAW video
« Reply #126 on: May 24, 2013, 04:22:10 AM »
Im really rooting for the development of this on my 5D3, as I just purchased two more 5D3's and cant really afford a 1000x Lexar :-p
@HireTheStache
www.HireTheStache.com
C100, 5D3, 5D2, 6D

IliasG

  • Member
  • ***
  • Posts: 124
Re: 12-bit (and 10-bit) RAW video
« Reply #127 on: May 24, 2013, 04:52:22 AM »
Wow, simply lowered black and white (*0.25f) in raw2dng before it exports

Code: [Select]
raw_info.bits_per_pixel=12;
raw_info.white_level *= 0.25f;
    raw_info.black_level *= 0.25f;

And now the file opens ACR  ;D



I think I'm done here...
1%, I'll send you my modified src files, I trust you can tidy it up and merge with yours?

Nice, thanks.

Is this posterisation down there ??.

Can you upload your dng file ??.

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: 12-bit (and 10-bit) RAW video
« Reply #128 on: May 24, 2013, 06:57:12 AM »
For some reason 12 bit stops faster than 14 bit right now.. 1200x676 I have to stop recording in 14bit but in 12 bit only 144 frames. Write speed drops to 26MB/s if LV is on and pretty much the same (28-30) if you hack it off. Global draw off seemed to help a tiny bit too. Still must be something stopping it early.

Or maybe 10 bit would be better?

vicnaum

  • Member
  • ***
  • Posts: 104
  • 600D warrior
Re: 12-bit (and 10-bit) RAW video
« Reply #129 on: May 24, 2013, 07:57:16 AM »
For some reason 12 bit stops faster than 14 bit right now.. 1200x676 I have to stop recording in 14bit but in 12 bit only 144 frames. Write speed drops to 26MB/s if LV is on and pretty much the same (28-30) if you hack it off. Global draw off seemed to help a tiny bit too. Still must be something stopping it early.

Or maybe 10 bit would be better?

Maybe good thought to try extreme values, like 2-4 bits, to find out what stops it - bit processing, or card write. And if it's okay - then slowly raise it until the problem appears. Or don't write to card at all and see how much the processing only gives (something similar to a1ex benchmark, but with real data).

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: 12-bit (and 10-bit) RAW video
« Reply #130 on: May 24, 2013, 08:12:57 AM »
It looks like a bug.. like its not clearing the buffers? 114 frames is about the size of memory or something?

a.d.

  • Member
  • ***
  • Posts: 126
Re: 12-bit (and 10-bit) RAW video
« Reply #131 on: May 24, 2013, 09:16:47 AM »
The same here on 5D2! This is the end at 114  ;D

Grunf

  • New to the forum
  • *
  • Posts: 43
Re: 12-bit (and 10-bit) RAW video
« Reply #132 on: May 24, 2013, 10:04:03 AM »
Modified reverse_bytes_order() for words instead of half-words in chdk-dng.c ,it's a bit better now.

Code: [Select]
static void FAST reverse_bytes_order(char* buf, int count)
{
long* buf32 = (long*) buf;
    int i;
    for (i = 0; i < count/4; i++)
    {
        long x = buf32[i];
buf[4*i+3] = x;
        buf[4*i+2] = x >> 8;
        buf[4*i+1] = x >> 16;
        buf[4*i] = x >>24;
    }
/*
    short* buf16 = (short*) buf;
    int i;
    for (i = 0; i < count/2; i++)
    {
        short x = buf16[i];
        buf[2*i+1] = x;
        buf[2*i] = x >> 8;
    }
*/
}

Will try this now.

Hi!


    for (i = 0; i < count/2; i++)

What about pre-calculating "count/2" into it's own variable and use that variable instead in your "for"-loop, otherwise it will waste precious CPU cycles evaluating "count/2" for each loop (giving same results over and over again)?

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: 12-bit (and 10-bit) RAW video
« Reply #133 on: May 24, 2013, 10:08:04 AM »
You mean... raw2dng feels kinda slow?

;)

Grunf

  • New to the forum
  • *
  • Posts: 43
Re: 12-bit (and 10-bit) RAW video
« Reply #134 on: May 24, 2013, 10:11:11 AM »
You mean... raw2dng feels kinda slow?

;)

Ah, I didn't know which application this snippet of code belonged to. I just reacted when i saw the code :D

mucher

  • Member
  • ***
  • Posts: 121
Re: 12-bit (and 10-bit) RAW video
« Reply #135 on: May 24, 2013, 10:15:06 AM »
I don't know programming much. But it sounds to me that the program itself stalled itself and exit somewhere(or suspended by the OS), my wild guess.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: 12-bit (and 10-bit) RAW video
« Reply #136 on: May 24, 2013, 10:21:58 AM »
FYI, gcc already moves this kind of computations outside loops.

mucher

  • Member
  • ***
  • Posts: 121
Re: 12-bit (and 10-bit) RAW video
« Reply #137 on: May 24, 2013, 10:58:34 AM »
buffer_size_compressed = (buffer_size_used * 12) / 14;

I don't know what is this, but if it changes to:

buffer_size_compressed = buffer_size_used  / 14;
buffer_size_compressed = buffer_size_compressed * 12;

It might have higher accuracy, I heard that some CPU cannot multiply too many times, or very soon it will have accuracy problem.

HA, I have been messing around. I quit.

Grunf

  • New to the forum
  • *
  • Posts: 43
Re: 12-bit (and 10-bit) RAW video
« Reply #138 on: May 24, 2013, 11:46:57 AM »
this is for ARM946E-S vs. ARM946E which we have :)

but i didnt verify

Ok! That's a pity :(

 My understanding is that Canon licensed  basic ARM946E core and then added some of their ow stuff to it. do you know if there is any way to check for procedure/op -calls done from, for example H264 encoder? I bet it has $hitload of DSP/SIMD op's hardcoded on die in order to do real-time H264 encoding, some of those could be "re-used" for 4 x small int bitshift? (Or is raw data offloaded to buffer, to be handled by some sort of co-processor involved)

Pity "d" didn't make rawc.o ASM available. It would be very interesting to take a peek...

nanomad

  • Administrator
  • Hero Member
  • *****
  • Posts: 2918
  • All your websites are belong to us
Re: 12-bit (and 10-bit) RAW video
« Reply #139 on: May 24, 2013, 12:59:31 PM »
It's available, just de-compile it  ::)

H264 encoding is done by an external chip, so there's no need for fancy SIMD instructions sets on the ARM core :)
EOS 1100D | EOS 650 (No, I didn't forget the D) | Ye Olde Canon EF Lenses ('87): 50 f/1.8 - 28 f/2.8 - 70-210 f/4 | EF-S 18-55 f/3.5-5.6 | Metz 36 AF-5

mucher

  • Member
  • ***
  • Posts: 121
Re: 12-bit (and 10-bit) RAW video
« Reply #140 on: May 24, 2013, 02:50:04 PM »
buffer_size_compressed = (buffer_size_used * 12) / 14;

I don't know what is this, but if it changes to:

buffer_size_compressed = buffer_size_used  / 14;
buffer_size_compressed = buffer_size_compressed * 12;

It might have higher accuracy, I heard that some CPU cannot multiply too many times, or very soon it will have accuracy problem.

HA, I have been messing around. I quit.

Or simply changes to:
buffer_size_compressed = (buffer_size_used / 14) *12;

Because if buffer_size_used is something big, *12 might make it very big, making it difficult to be divided by 14, so I think that it might be better to divide first, and then multiply. But this trick might not work anyway.

nanomad

  • Administrator
  • Hero Member
  • *****
  • Posts: 2918
  • All your websites are belong to us
Re: 12-bit (and 10-bit) RAW video
« Reply #141 on: May 24, 2013, 04:26:56 PM »
Don't do the compiler's job :P
He's usually good at it
EOS 1100D | EOS 650 (No, I didn't forget the D) | Ye Olde Canon EF Lenses ('87): 50 f/1.8 - 28 f/2.8 - 70-210 f/4 | EF-S 18-55 f/3.5-5.6 | Metz 36 AF-5

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: 12-bit (and 10-bit) RAW video
« Reply #142 on: May 24, 2013, 04:31:36 PM »

mucher

  • Member
  • ***
  • Posts: 121
Re: 12-bit (and 10-bit) RAW video
« Reply #143 on: May 24, 2013, 05:12:44 PM »
I have noticed another thing:

static int buffer_size_compressed = 0;

If  buffer_size_compressed is divided by 14, the result is in float point, you might need to change it back to int. That is a pain in the axx thing too I guess.

I might be wrong.

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: 12-bit (and 10-bit) RAW video
« Reply #144 on: May 24, 2013, 05:15:44 PM »
sized_used isn't static so I can try that... the frames come out smaller when I open them so 12 bit should be helping.

Code: [Select]
buffer_size_compressed = (size_used * raw_bitdepth) / 14;
This doesn't make sense to me actually... why are we multiplying this out? The *12/14 calculation is to find width normally.

Mayo

  • New to the forum
  • *
  • Posts: 29
Re: 12-bit (and 10-bit) RAW video
« Reply #145 on: May 24, 2013, 05:17:38 PM »
1%, I just realised

Code: [Select]
{
                .name = "Bit Depth",
                .priv = &raw_bitdepth,
                .min = 12,
                .max = 13,
                .choices = CHOICES("12 Bit", "14 Bit", ),                               
                .help2 = "Bit Depth",
            },

this will probably not work (can't test atm)... I wrote somewhere that raw_bitdepth would work as usual if it was anything else than 12.
It's wrong... It has to be either 12 or 14. Probably needs a menu update function and a LUT.
As for why it stops, no idea. :(
Probably

Code: [Select]
if (r != buffer_size_compressed) goto abort;
gets called, but no idea why.

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: 12-bit (and 10-bit) RAW video
« Reply #146 on: May 24, 2013, 05:21:12 PM »
It does work... when its 13 I get normal 14 bit I believe. You just have it set to if bit depth = 12 do this otherwise do that.

Also files opening in ACR just fine for me.

* Yea, both  files work fine... I just show one of each.. the 14 bit files play back in camera of course.

Mayo

  • New to the forum
  • *
  • Posts: 29
Re: 12-bit (and 10-bit) RAW video
« Reply #147 on: May 24, 2013, 05:24:59 PM »
It does work... when its 13 I get normal 14 bit I believe. You just have it set to if bit depth = 12 do this otherwise do that.

It's used as actual bitdepth in write_speed_update and lv_rec_save_footer()

int speed = (res_x * res_y * raw_bitdepth/8 / 1024) * fps / 100 / 1024;

footer.frameSize = footer.xRes * footer.yRes * raw_bitdepth/8;

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: 12-bit (and 10-bit) RAW video
« Reply #148 on: May 24, 2013, 05:30:13 PM »
But for 14 bit we don't do this?

I took the line out and it still behaves similarly... its like it just dies out after a while even at 960x540.

I see how it would be a problem then with it being 14 or not.

Mayo

  • New to the forum
  • *
  • Posts: 29
Re: 12-bit (and 10-bit) RAW video
« Reply #149 on: May 24, 2013, 05:32:16 PM »
sized_used isn't static so I can try that... the frames come out smaller when I open them so 12 bit should be helping.

Code: [Select]
buffer_size_compressed = (size_used * raw_bitdepth) / 14;
This doesn't make sense to me actually... why are we multiplying this out? The *12/14 calculation is to find width normally.

It's just the compression ratio, it's the same for width and the whole image buffer. (to tell fwrite how much to write).

@mucher: not sure if you're serious, but all your assumptions are wrong...  :P