Danne's crop_rec_4k, 5DIII

[ilia3101]

I really want to add real 12 bit to the UHD preset.

Wish I knew how😭 @Danne didn't you mention there's comments somewhere about reducing bitdepth??

[Danne]

I don't think it's less real. Quality wise probably the same. What's your issue by the way?

[ilia3101]

I have no issue, but:

The black-white range with your gain adjusted 12 bit seems to be approximately 2047-5600 (max 16383)

With real 12 bit it will probably be 512-3800 (max 4095)

So it is effectively 12 bit.

But I suspect the compression algorithm can be slightly more efficient in true 12 bit mode.

I just got an excessive amount of stripes recently shooting in crop_rec 12 bit, which got me wondering about this.

[Danne]

A1ex explained from time to time about bitdepth routines. I can't find the posts right now. I don't think those stripes are related to bitdepth in crop rec but who knows.

[ilia3101]

Ok thank you. I wasn't saying I was certain that the stripes are caused by it. Probably not. The 5D3 is really quite stripy.

[koopg]

Can i go a bit off topic here ?
I hvae bought a KB 128Gb from amazon
I have used it twice and yesterday its simply died.
I want to buy a new card, but KB feesls like out of the question with this type of quilty.
So, is there anything else to use on a 5d3 with more or less the same speeds but more reliability ?
 Used cases are danna builds 3.5k raw o  a 5d3

Thanks

Sent from my SM-N975F using Tapatalk

[Danne]

How did it die?
I use transcend. Never failed.

[Dmytro_ua]

I use Lexar. Maybe it's not a top speed but a solid proof for many years

[koopg]

Thanks

How did it die?
I use transcend. Never failed.

Pressing record and its gone, will not recognize by any system

Can you tell me which type and does it do 3.5k with danne build ?

I use Lexar. Maybe it's not a top speed but a solid proof for many years

Same .. you are saying that its not top speed, but does it work with danne build 3.5k raw on 5d3 ?

Sent from my SM-N975F using Tapatalk

[Walter Schulz]

If going Lexar 1066x: Take care, there are 2 versions: https://www.magiclantern.fm/forum/index.php?topic=24461.msg220607#msg220607

My KB card went dead some years ago and got replaced. No problems since. Same with my 2 Transcends.

[Boobie]

Dear Danne,

Is it possible to record 12/14 bit 2656x1484 (16/9)?
Or is 1308 height a hardware limitation for continuous recording 12/14 bit?

The 3072x1308 (2.35:1) mode is not useful to me as I rarely use 2.35:1 and make 99% of stuff in 16/9, so I will have to crop 3072 down to 2326 and lose 33% of the recorded pixels.

I don't have a 5D3, but I might buy one if 16/9 is working with >=4 megapixels Raw.

3072*1308=4,018,176
2688*1512=4,064,256
2656*1484=3,941,504 so it maybe possible

What is the highest resolution that 16/9 can be recorded continuously for 12/14 bit?

[IDA_ML]

Impatient to try this vey exciting card spanning feature, I borrowed the 5D3 and tested the latest February 01. build using the following two cards:

1) Transcend SDHC 32 GB 300x, (write speed: 40 MB/s, USB 3.0 card reader test)
2) Sandisk Extreme Pro 64 GB 160 MB/s, (write speed: 114 MB/s, USB 3.0 card reader test)

I.  With Card spanning ON, SD overclock OFF spanning works fine, in-camera preview too.  The 00-files get nicely recorded on the SD card and merging with the main files on the CF card goes flawlessly.  Just dump the contents of both cards in one directory and you are ready to go.  MLVApp processing works fine too.   I get continuous recording at 3072x1728 10-bit lossless resolution and 24 fps with about 1,5 stops underexposure at these settings.  With the same settings but without spanning, I get continuous recording only at 17 fps.  So, card spanning makes perfect sense and is extremely useful.

Important:
=======

To get maximum recording times, warming up cards is necessary.  With the above settings and both cards cold, I get a maximum of 200 frames recorded.  Immediately after that, I get 1000-2300 and even more frames recorded.  I guess, it's the SD card that needs warming up because, without spanning, CF card warming is not necessary.

II. With Card spanning ON and SD overclock ON, recording does not work.  Upon pressing the record button, I get three beeps along with the message "Recording stopped automatically", then recording stops and nothing gets recorded. 

------------------------------------------

Unfortunately, I had to pay a price for my impatience.  My beloved SD card died in these experiments.  Here is how this happened:

III.  I decided to check the SD-overclocking feature alone.  I removed the CF card and left only the SD card with ML on it in the camera. Then I activated SD-overclocking, waited until it is patched, (status ON) and pushed recording.  Recording did not work, I got the error message.  Then I decided to format the SD card in the camera without turning SD-overclocking off.  Formatting failed and I was prompted to replace the card.  That was it.  All further attempts to fix the card through formatting in the camera and PC via card reader failed.  The card is dead.  Not a big loss but still ...  After that, I did not try other SD cards.

----------------------------------

Excellent work Ilia3103, Danne and everyone else involved.  12-bit UHD RAW-video recording at 24 fps is a real break through for the good old 5D3 !

[ilia3101]

What is the highest resolution that 16/9 can be recorded continuously for 12/14 bit?

There is a 3K preset that does 3072x1920, so you can record 16:9 in 3072x1728 (it's continuous I think)

@IDA_ML thanks for the great confirmation that it's working! Getting quite certain it's reliable now.

Unfortunately, I had to pay a price for my impatience.  My beloved SD card died in these experiments.  Here is how this happened:

III.  I decided to check the SD-overclocking feature alone.  I removed the CF card and left only the SD card with ML on it in the camera. Then I activated SD-overclocking, waited until it is patched, (status ON) and pushed recording.  Recording did not work, I got the error message.  Then I decided to format the SD card in the camera without turning SD-overclocking off.  Formatting failed and I was prompted to replace the card.  That was it.  All further attempts to fix the card through formatting in the camera and PC via card reader failed.  The card is dead.  Not a big loss but still ...  After that, I did not try other SD cards.

Sad about your card. Sometimes it's hard to tell if a card is dead or the file system is really messed up. Sometimes mkfs.vfat can fix things like that.

[Danne]

New builds
Usual links

- Removed sd_uhs module. Too fishy. Not worth it. If anyone wants to test it check commit to compile it again:
https://bitbucket.org/Dannephoto/magic-lantern/commits/a61ec50df0059d3ecd7e2356d4e0b8ad9157d37d

[Boobie]

There is a 3K preset that does 3072x1920, so you can record 16:9 in 3072x1728 (it's continuous I think)

Can anyone please test this and confirm that it works continuously with 12/14 bit?

3072*1920 is considerably higher resolution at 5.9 megapixels, I'm surprised to hear the camera can support it with Raw.

If 3072x1920 is truly possible for 12/14 bit with 5.9 MP -- could 3264x1836 be supported?
That would be even better, pushing the max for 16/9.

3072*1920=5,898,240
3264*1836=5,992,704
3232*1818=5,875,776
3200*1800=5,760,000

Could it be possible to add 3264x1836 or 3232x1818 and/or 3200x1800 modes?
If 5.9 megapixels is the true limit, 3.2K would be the best possible 16/9 Raw output for this camera.
3200x1800 also has a nice upscale ratio as it's a perfect 6/5ths upscale to 4K.

I think it's important to add a preset supporting the maximum stable 16/9 resolution for Raw, as that's one of the most basic functionalities to get right before other unique aspect ratios.
A 3.2K 16/9 preset supporting 12/14 bit would be fabulous.

[ilia3101]

@Boobie Just tested 12 bit 3072x1728 - it's continuous with card spanning, but just on the edge. If your shot suddenly becomes overexposed, recording indicator goes orange and it will be at risk of stopping.

If 3072x1920 is truly possible for 12/14 bit with 5.9 MP -- could 3264x1836 be supported?
That would be even better, pushing the max for 16/9.

3072*1920=5,898,240
3264*1836=5,992,704
3232*1818=5,875,776
3200*1800=5,760,000

I've thought about that before, but have no idea how to implement it. It may be possible, but it's just as likely not - control of the sensor isn't very exact and there's always unpredicatble black areas coming out, they may be bigger or smaller depending on how it's configured. (at least from what I've heard from reddeercity with the 5D2, maybe its not like this on the 5D3)

[Boobie]

@Boobie Just tested 12 bit 3072x1728 - it's continuous with card spanning, but just on the edge. If your shot suddenly becomes overexposed, recording indicator goes orange and it will be at risk of stopping.

Thank you very much for testing. Did you record at 24p or 30p?

I'm uncertain if MLRAW is RGB 36 bits or RGGB 48 bits pre-compression, would love to learn more.

[masc]

I'm uncertain if MLRAW is RGB 36 bits or RGGB 48 bits pre-compression, would love to learn more.

MLRAW is 10/12/14 bit. RAW is B/W (one channel). Color comes with debayering.

[Boobie]

MLRAW is 10/12/14 bit. RAW is B/W (one channel). Color comes with debayering.

I understand, I'm saying 12+12+12 versus 12+12+12+12.

[ilia3101]

I shot 24p (23.976fps)

The raw without being compressed is not RGB, it's black and white (Bayer pattern), each pixel has only one channel, red/green/blue (nothing bad all raw cams are like that)

And it's tightly packed 12/14/10 bit, no 16 bits anywhere. Uncompressed bitrate in MB/s is (width*height*bitdepth*fps)/(8*1024*1024)

For example (3200*1736*12bit*23.976fps)/(8*1024*1024) = 190.5 MB/s

+ compression makes that smaller by ~40%

[70MM13]

hey everyone,
i'm a fan of the sd overclock!  don't drop it, it's worth keeping!  i used it to shoot my new candlelight music video continuously at 3072*1308 (4 minutes continuous!) i've also used it on the sd card alone for 1920 continuous with no cf card in the camera - all 14 bit

here's a few observations from my experiences with it...

it only works for me on the new sandisk extreme pro card i purchased for using it.  none of my other cards work with it.

it only works reliably for me if the camera is started with overclock DISABLED.  enable it after startup, and then use it.  disable it again before shutdown.

i suggest for the time being to make it disabled at startup by default.  the loss of a few seconds to enable it is far better than potentially unreliable operation...

it's a real drag that formatting the card with it enabled could fry the card.  that's valuable information!

one other thing to mention:  the camera definitely heats up when pushing it this hard.  we should really be paying close attention to this.  i shut down and let it cool down after a few takes when the temperature gets scary!  these temps could have consequences, so everyone should be aware and careful!

thanks for the great work, everyone!

check out my candlelight video to see it in action   i'll eventually post a 4k version when av1 finally gets adopted.

[Danne]

Sorry. It´s disabled. As mentioned. Follow my commit to compile the module locally. Might look into fixing it in the future.

[IDA_ML]

Removed sd_uhs module. Too fishy. Not worth it.

Not only is it worth it!  SD-overclocking is the key to the success of this entire card spanning project!  The enormous extended functionality of the EOS-M and the 100D would have been impossible without SD-overclocking.  These 10-13 MB/s on top of the 40 MB/s factory write speed were the key to the success of these two cameras.  Same for the 5D3 too.  Very little more write speed is needed to achieve continuous recording at key modes such as the anamorphic 3840x2160/12bit lossless/24 fps or the 3072x1728/12bit lossless/24 fps or 1920x1080/48 fps without the need to reduce frame rate or lower exposure.  I wish, I hadn't mentioned my damaged SD card at all. I just wanted to warn people not to repeat my mistake.  A damaged card should not be a reason to stop the development on this issue of crucial importance!

[ilia3101]

So... good opportunity to bring back the module screen 😉

[Danne]

Opportunity for others to get to learn code/compiling basics...