12-bit (and 10-bit) RAW video development discussion

[budafilms]

Wonderfull guys!

[ansiivan]

Hi. Why regular chamber mov, much brighter raw vide? The settings are the same.
Canon 1100d,  magiclantern-raw_video_10bit_12bit.2017Jan13.1100D105.zip
Fps 25, iso1600.
mov
https://www.youtube.com/watch?v=2o4Umf92yLM

Fps 25, iso1600.
raw video
https://www.youtube.com/watch?v=hOEtEuKAdLs

[kaco]

Hi guys,

on Saturday, I've sucesfully shot 2TB of MLV's in 10bit. From 1400 filex, approx 150 had wrong black level. Since calling mlv_dump 150 times and then sorting all the files would be too complicated, I've wrote little windows batch script. I am not a programmer so finding the first file in a folder and even the iteration is very very messy. But works.

Code Select Expand


@echo off
setlocal EnableDelayedExpansion
set mypath=%~dp0
set workingDir=E:\footage\mlv\
set fixDir=E:\footage\cdng-fixed\
set moveDir=E:\footage\mlv-processed\


E:
cd %workingDir%
for /r %%i in (*) do (
set file=%%i

!mypath!\mlv_dump.exe --black-fix=2040 --dng "!file!"

for %%A in ("!file!") do (
set foldername=%%~dpA
set filename=%%~nxA
)
echo !filename!

cd "!fixDir!"
md "!filename!"
cd "!filename!"

for /r "%workingDir%" %%f in (*.dng) do move "%%f" >nul
for /r "%workingDir%" %%f in (*.wav) do move "%%f" >nul

move  "!file!" "!moveDir!"
)
D:
cd %mypath%


[kaco]

BTW, may I process *all* files with --black-fix=2040, even those which have correct value? If I want to process only those wrong ones, I need to extract all and visually check.

[a1ex]

On 5D3 and probably all other recent cameras, latest mlv_dump requires --black-fix=2048, regardless of ML build (since this post).

With latest ML bulid from the Experiments page AND latest mlv_dump, there should be no need to change the black level (since this post).

Checklist:
- with 10-bit files, the black level from MLV (visible with mlv_dump -v) should be 128 (127 is wrong)
- with 14-bit files, the black level should be close to 2048 (small variations are OK)
- the black level from the DNG (always upconverted to 14 bits) should be close to 2048 (e.g. 2050 is fine, 2032 is wrong)
- there should be no color difference between 10 and 14 bit files in shadows

If any of the above is not true for you with the latest experimental build and latest mlv_dump, please report.

For authors of other postprocessing tools: to avoid color casts at lower bit depths, conversion to a higher bit depth must be done by adding 0.5 LSB to the input signal (explained here). Black level scaling requires no trickery (e.g. 128 -> 2048).

Relevant commits: e3ef8c7 and a31228c.

[vstrglv]

Build  magiclantern-crop3x.2017Jan13.5D3113 (2017-01-12 23:35) – crop+10bit,12bit. MLV raw.
DNG conversion raw2cdng 1.7.9 and  mvl_dump (2017-01-13 00:55).
DNG for crop(1x1)_Xbit_ raw2cdng and DNG for crop(1x1)_Xbit_ mlv_dump look very similar (except  green colour cast for 10bit raw2cdng)  in ACR if Noise Reduction (Luminance) is set to zero. But if Noise Reduction is increased its effect is more noticeable  for mlv_dump convertion.
X is 10bit, 12bit or 14bit. This effect takes place for normal mode (not crop) too.
https://www.dropbox.com/s/q6pj4kspt8l794t/M14-1338_10b_crop_mvl_dump.dng?dl=0
https://www.dropbox.com/s/i6vpllko75d43wl/M14-1338_10b_crop_raw2cdng.dng?dl=0

M14-1338_10b_crop_0NR_raw2cdng by Vladimir Strigalev, on Flickr
M14-1338_10b_crop_0NR_mvl_dump by Vladimir Strigalev, on Flickr
M14-1338_10b_crop_30NR_raw2cdng by Vladimir Strigalev, on Flickr
M14-1338_10b_crop_30NR_mvl_dump by Vladimir Strigalev, on Flickr

[kaco]

On 5D3 and probably all other recent cameras, latest mlv_dump requires --black-fix=2048, regardless of ML build (since this post).

With latest ML bulid from the Experiments page AND latest mlv_dump, there should be no need to change the black level (since this post).

Great! However I still find batch calling mlv_dump much better then mlv-fs. Is there any way how to make real cdng from mlv_dump? So that Resolve could read audio and correct fps? Or this wouldn't work because mlv_dump is not indexing the files?

Thanks

[reddeercity]

Resolve read Cdng with audio from mlvfs as one file with correct level in real time on PC's and Mac's .
batch calling on command is very slow plus you end up with lots of dng's that fill up hard drive space .
MLVFS doesn't do that it produces Virtual Cdng so no hard drive killer here , you even read off the CF or SD cards
depending on your computer hardware .
FYI: mlv_dump produces compliance dng that work in resolve or at least used to that could be broken thou with bit reduction

[Danne]

Great! However I still find batch calling mlv_dump much better then mlv-fs. Is there any way how to make real cdng from mlv_dump?

You can process cdng files with this
http://www.magiclantern.fm/forum/index.php?topic=15108.msg146822#msg146822

The program works with a mlv_dump version which has code mixed in from a specific ml-dng branch version. Code updated to latest 10/12/14 bit the same as latest mlv_dump(g3gg0/a1ex). Works with resolve(audio embedding).
When installed head over to the mlv_dump menu (m) from the main menu and you can do setting changes from there such as black/white level etc.

[D_Odell]

Using 13th of Jan build and the frame freezes in black and white live view while shooting raw_rec with 5x. Any others experiencing the same?

[Tracerman]

@dfort

Hi, I'm up for testing 10/12 bit mlv_rec on 550D if you like (I don't know how to make a pull request either :/).

[Danne]

Is this happening with or without raw_twk enabled? Try with raw_twk disabled.

[Levas]

There 's only a windows version of MLV_dump available on the magic lantern downloads page:
https://builds.magiclantern.fm/utilities.html

Can somebody add newest version of MLV_dump for osX ?

[Danne]

mlv_dump for mac
https://bitbucket.org/Dannephoto/magic-lantern/downloads/mlv_dump

[D_Odell]

Is this happening with or without raw_twk enabled? Try with raw_twk disabled.

Still happening with raw_twk disabled. Guess its something with the build?

[dfort]

@dfort

Hi, I'm up for testing 10/12 bit mlv_rec on 550D if you like (I don't know how to make a pull request either :/).

Nothing new for the 550D yet and we know what needs to be done before we're ready for any further testing.


Sent from my iPad using Tapatalk

[CITY-U1001]

i dont see 10 bit for 50D   

[Danne]

mlv_dump for mac updated
https://bitbucket.org/Dannephoto/magic-lantern/downloads/mlv_dump

fixes here(dark/flatframe, mlv_lite)
https://bitbucket.org/hudson/magic-lantern/commits/89daf093c81fbe86be3ae27e4acce8fa79bf91f3

[DeafEyeJedi]

Thanks SO much for sharing this @Danne and for the quick fixes @a1ex!

[Tracerman]

@dfort

Thanks for your reply, feel free to let me know when/if you need testers for 550D.

Cheers

[justinbacle]

i dont see 10 bit for 50D   

50D is not supported (yet).
(Same as 5dmk2 as they don't have CONFIG_EDMAC_RAW_SLURP ?)

[bobbyshots2]

I want to try and test the t2i (550d) with 10bit - 12bit raw. If anybody has a link to  the module they could send me or the full ml download. that would be great.!
Thanks in advance!
One Love

[ShootMeAlready]

Ok, mlv_dump works to get me DNG.
I can colour correct usingbut how to get them into Ppro?  If I save as JPEG from LR, it complains that bit depth is not supported.

Is there a way to get 10bit scaled to 14bit depth?

Found that DNG works very nice in Resolve.
However LR becomes useless in this flow, since its corrections are not recognized when DNG is input to Resolve.
One can export from LR into jpeg but that limits Ppro or Resolve's abilities.

[dfort]

Looks like it is worth repeating that the following cameras need to get the CONFIG_EDMAC_RAW_SLURP treatment before it is worth doing any further tests with them.

5D2.212	-	Digic 4
7D.203	-	Dual Digic 4
50D.109	-	Digic 4
500D.111	-	Digic 4
550D.109	-	Digic 4

[reddeercity]

50D is not supported (yet).
(Same as 5dmk2 as they don't have CONFIG_EDMAC_RAW_SLURP ?)

Not really true , not sure about 50D but the 5D2 work in Crop mode 100% (10-12bit) with Audio , overlays  in MLV full  see the Proof
Even thought 1:1 liveview products some corruption in 10bit but 12bit can be used if you crop out
the top 50-100 line do to a color noise bar see here look at the 3rd image I posted .

I can't say that the 50D or 550 (T2i) will have the same level of functionality in crop mode as the 5D2 but as @dfort said

..... need to get the CONFIG_EDMAC_RAW_SLURP treatment before it is worth doing any further tests with them.

We do need more user's who can help figure the Digic 4 liveview issue (5d2,50d,7d,500,550) so I will provide the link to those builds in hope more development can be made .
My reasoning is if users have at least a limited functioning version of  raw video bit reduction , this hopefully will give more incentive to push development on the digic 4.
link to my bitbucket downloads for 10-12bit reductions 5d2,50d,7d,500,550