12-bit (and 10-bit) RAW video development discussion

[ilia3101]

@ShootMeAlready Think I've had that problem on my 5D2 for a while in the 10-12bit builds and raw_twk, lets me play back a few clips then goes black until mlv_play is restarted(was already happening in December)
@reddeercity Thanks, I didn't realise mlv_rec had black level fixed in crop mode. Just an idea: Couldn't black levels just be hardcoded for each camera so there doesn't have to be problems all the time?

[balliamo1]

Hello
about the 10-bit version, I would like to know if the 550D will be supported and who you reckon a download will be available.
Thanks

[ItsMeLenny]

Hello
about the 10-bit version, I would like to know if the 550D will be supported and who you reckon a download will be available.
Thanks

It's already there somewhere, tested it a while ago, scroll back through the comments and you should find it.

[balliamo1]

I mean is a zip file available? I am not familiar with software code testing

[reddeercity]

Doing some research & testing on 5d2 full mlv+audio 10-12bit (I'm not doing any work with mlv lite because of no Audio for now so it's low on my priority list)
I have Qemu eos emluator setup with my rom file (5d2) , I'm almost ready to step fully in to this to work on 10/12bit issue on 5d2/digic4 although I had been discourage by a few developer
I have assistance from my best friend who happen to be a programmer/coder , with are closing in on the problem I believe. When I have something to show I'll post the results
either as Code or a test private build.

On a different note I was trying selected resolution sizes to see if this has any effect to frame corruption in 12bit and it seems that any vertical resolution be on 865 has
pink noise bar corruption on the top few lines , as you increase the vertical resolution to 1114 the pink noise bar just increase in height . It seem the top part of the sensor
always as this  corruption bar but as you crop the resolution vertically you don't record it , I tried 1856x844 2.20:1 A.R. 12bit 23.976p Canon liveview and all is well no frame shearing/tearing
all frames are good plus audio. So Any one that wants to try 12bit in 1:1 FHD I have been using the test build in my bitbucket downloads 10bit_12bit_2016Dec01.5D2212

In case anyone interested in how many digic4 cam users that have download 10-12bit test builds look below


bitbucket downloads digic4 cam by RedDeerCityTV, on Flickr

5D2 with 76 downloads next 7d & 550d with 17 , that's what gives me incentive to continue on 

[RenatoPhoto]

@reddeercity
I am sure many users will be happy to know that you are working on porting this digic 4 cameras.  Make sure you post a link to the download page when you need testing.  I did not know that you had these builds! 

[reddeercity]

@reddeercity
Make sure you post a link to the download page when you need testing.

Yes I will 

[arrinkiiii]

Also for the 7D?? Many thanks   Looking forward to the links =))

[steeth]

I´m having a weird issue with the 10-12 bit version. I´m using a 5D Mark III, with the latest feb 11 1.2.3 experimental 10-12 bit version. Sorry if this has been covered before, I´ve searched all the possible combinations of words I can think of, and read thru as many threads I can but haven´t found anything regarding this issue.

I convert my MLV files using the latest version of CR2HDR, and I use Resolve as my choice of development software. After I started using the experimental 10-12 bit build to try out 12 bit recording, I noticed that some clips in the media pool inside Resolve came out as folders, and not clips. Inside those folders are still frames, and smaller corrupt parts of that particular clip making the clip useless, which I thought was weird..

I checked the folder with the dng files for that particular clip and noticed something really strange. At first I thought that there were missing frames in the sequence, but I have not checked the "frame skipping" option in the ML menu. So I checked the dng folder of one of the corrupt files, and all the dng files are there, but for some reason some of the dng files are named with the file extension as capitol letters (.DNG) and not lower case letters as usual (.dng)!? And that particular dng sequence doesn´t load in either AE or Resolve as it says it´s corrupt. In Resolve, each .DNG file comes out as a single still frame.. If I manually change the dng files that have upper case file endings (.DNG to .dng), the clip works for some reason!?

I have tried everything I can think of to re-create the issue, but it´s completely random and doesn´t happen when I use 14 bit. It only happens when I change to 12 bit, and there is nothing indicating that it has happened to a clip, nor can I figure out anything that might cause it as it seems extremely random.

Has anyone else experienced this issue? I get this issue on about an average of 2-3 clips per 64 gb card I use. And some have a lot of .DNG file endings in them, making it a real hazzle to manually re-name all of them



Kind regards

[Danne]

Not sure if 10/12 bit are optimized working with dualiso mlv files. Better stick with 14-bit if that works better. If you want resolve embedded folders you can run your mlv files through cr2hdr.app(mac).

[a1ex]

As long as the input to cr2hdr is up-converted to 14 bits, I don't see why it wouldn't work. But I've never tested this scenario.

With cr2hdr, uppercase DNG means converted files; mlv_dump outputs lowercase dng, so those files probably couldn't be converted. If their bit depth is 14, upload one of them; otherwise, try a newer mlv_dump.

[steeth]

Thanks a lot for your replies Danne and a1ex.

Sorry if I didn´t make it clear in my post, but I´m not shooting nor trying to convert dualiso files, just standard regular 25fps mlv files. And I´m using the latest cr2hdr.app to convert my .mlv files.

The strange thing is that 99% of the time, both 10 bit and 12 bit .mlv files convert properly as lowercase .dng frames that works properly in both AE and Resolve. I don´t do anything different, nor do I change any settings in either my camera or in the cr2hdr.app when converting. Sometimes I can record and fill up 2-3 64 GB cards with no problem whatsoever converting 12 bit .mlv files with cr2hdr.app. All clips are recorded and converted to regular .dng frames as they should, and then out of nowwhere, 2-3 clips has these random .DNG uppercase file extension on just some random frames in a converted clip as shown in my previous image, making both AE and Resolve think that the sequence is corrupt. And if I manually change these random .DNG frames to .dng instead, they open just fine in both AE and Resolve?

So I don´t change any settings between clips or cards, I don´t do anything different when converting my .mlv files from the card in the cr2hdr.app, and I still get these random upper case .DNG frames randomly inside a .dng sequence which I think is so strange?

Has anyone else experienced this? And if so, is there a solution or am I doing something wrong? I´m sorry to say that I´m just a filmmaker and I have no knowledge of nor any experience in how to use code, read code or search for errors etc, so I can´t contribute with anything but my experience in using Magic Lantern unfortunately.

Thanks again for taking the time to read my post and replying.

Kind regards.

[Danne]

Can you post a dng file that is turned into upper case dng in the cr2hdr.app thread?
http://www.magiclantern.fm/forum/index.php?topic=15108.msg174767#msg174767
I don,t really get why it would happen in the middle of a sequence. What the script does it tries the first file with cr2hdr to see if it will create a dual iso dng and if so it will continue to run cr2hdr on all the files. Otherwise not. This would however mean that the first dng file would be mistaken for a dualiso file and maybe that is the case here.
If the problem is what I described, cr2hdr is fooled somehow, you can easily disable the cr2hdr function for MLV files in cr2hdr.app and won,t even test for dualiso movie files. Especially if you never use dualiso. Of course this has nothing to do with this thread so just continue the c2hdr thread by posting a dng and maybe even a sample of a troubling MLV file in that thread instead.

*Update
Just added a switch in mlv_dump settings menu in cr2hdr.app which disables dualiso automation.
http://www.magiclantern.fm/forum/index.php?topic=15108.new#new

[steeth]

Thanks a lot Danne for taking the time to answer me regarding this. I´ll definitely post one of the mlv files that has this problem in the thread you mentioned. And I´ll download and test your updated cr2hdr.app build to see if the problem goes away. And now that you mention it, from what I remember, in every mlv file I´ve converted that has those upper case .DNG frames randomly in the sequence, I believe that the absolute first frame always have had a .DNG frame, and then several .dng frames, and then random .DNG frames throughout the sequence.. But I´m not sure as I haven´t looked for it on the first frame.. I´ll definitely upload a problematic .mlv file that has these issues so you can take a look. And sorry for posting in the wrong section, I didn´t know if it was caused by the 10-12 bit build or during the conversion process.

Thanks again.

And as a side note, you don´t happen to be Swedish do you? If you are, I´m from Sundsvall in Sweden Not that it has anything to do with anything, just curious

Kind regards.

[Danne]

Yup, swedish and polish. Creme de la creme .
Ok, if the first one popped then the switch will solve the issue.

[reddeercity]

I'm looking in to raw record process , 
I seems to be going in circles can someone please point me to parts of the source code that capture the raw buffer lv (which I assume hopefully , correct me if I'm wrong)
is the buffer we use for raw video . I what to step thought the lines of code for 5d2 .
Thanks in advance .

[kizza1234]

I am so impressed with the development of magic lantern throughout my short span of using it its truly amazing, and the devs are doing such an amazing job!!! Magic lantern is so so so so good, and now with 10bit 50p 1080P continuous and stable? Its literally a dream come true!

I have installed the module and so far so good on 5D III, makes 50p 1080p (vertical upscaled 1.67x) continuous from what i can test.
Although as some other people have pointed out crop mode in 1080p 50p doesnt work as half the
frame is cut off top and bottom (image here: https://drive.google.com/open?id=0B1q4BC6n6pcNVTBwcUNXamRTRmc

Main Q:
But i was just wondering if anybody know the difference in dynamic range from 10bit to 12bit to 14bit?
From what i can see there is slightly less dynamic range every step down from 14bit, biggest difference i think is from 12bit to 10bit....
It is my understanding 14bit 1080p raw is around 11.8-11.5 stops? 
Is 10bit something like 10 stops?

I might want to add that i use RawFlow to extract my files and 10bit extracts fine without any issues i can see.
Link: https://www.magiclantern.fm/forum/index.php?topic=13338.0
Its by far the easiest way i have found to extract files without any complications, and the images that get extracted are amazing!

Thanks Guys

[alejandro rumolino]

For 6D here the experimental loop https://builds.magiclantern.fm/experiments.html
I uploaded it and it works smoothly in 16/9 1600x900 10-bit, it does not have the possibility of the audio and I had a problem after filming, my canon does not start more for 1 hour ... so uninstall it ... use MLVtoMovie in .mov after

[ShootMeAlready]

Kizza the DR stops or range is a matter of sensor size and performance.
You can have 20bit raw out of a small sensor, and it will not be better than a 10bit raw from a big sensor that has wide pixel separation, low noise. 

Humans are not very good at distinguishing colour depth, so 4 bit colour verses 2bit we are not as quick. Folks cant discern any difference in a 14bit raw vs a 12bit raw photo.  Where you get the big benefit is in pushing the raw data, like adjusting exposure in post, in that you can push 14bit raw much more than 10bit raw.

I think humans can distinguish 12bit raw vs 10bit raw photos, but that's only in certain shots and not always. 10 bit raw provides greater depth than compressed video, so one should be able to distinguish these, especially in low light shooting detail.  10bit will generally do a better job with highlight details (assuming not overexposed).  So rather than quote stops of light, think more about depth and detail in low/high light.

[D_Odell]

Humans are not very good at distinguishing colour depth, so 4 bit colour verses 2bit we are not as quick. Folks cant discern any difference in a 14bit raw vs a 12bit raw photo.  Where you get the big benefit is in pushing the raw data, like adjusting exposure in post, in that you can push 14bit raw much more than 10bit raw.

I think humans can distinguish 12bit raw vs 10bit raw photos, but that's only in certain shots and not always. 10 bit raw provides greater depth than compressed video, so one should be able to distinguish these, especially in low light shooting detail.  10bit will generally do a better job with highlight details (assuming not overexposed).  So rather than quote stops of light, think more about depth and detail in low/high light.

^ Sounds like poetry.

[D_Odell]

Using 10bit_12bit.2017Feb11.5D3 build, and can't get ML to load with SND module activated (it says something like can't load modules, with a grey box). Any idea or do any of you have the same problem? Shooting a lot of sync footage tomorrow so would help a lot for to the post process. I wish not to use nightly because of the extra resolution on 10bit while doing zoom view..

Thank you!

[Teamsleepkid]

Try only loading the 2 modules you need. Nothing else.

[dfort]

You mean "failed to link modules" ?

Are you trying to load mlv_snd with mlv_lite? That won't work. You need to use mlv_snd with mlv_rec.

[D_Odell]

You mean "failed to link modules" ?

Are you trying to load mlv_snd with mlv_lite? That won't work. You need to use mlv_snd with mlv_rec.

Thats correct. Thank you for the heads up!

[MitchLally]

Hey guys I've been getting some weird behaviour from Mark III 10bit MLV files converted to DNG with MLVFS. Probably 1 in 50 MLV files seems corrupted.

It looks like the Black Level issue that we're all familiar with – but I can't correct it using exiftool. I got the black level changed to 2048 but did not solve the issue.

What might be happening here and is there a way to fix it? Would someone care to try exiftool to confirm? Here is a DNG file (untouched) from one of the corrupted 10 bit MLVs.

https://www.dropbox.com/s/m77zmz6m6h9huik/M03-1839_000008.dng?dl=0