Author Topic: 12-bit (and 10-bit) RAW video development discussion  (Read 861621 times)

budafilms

  • Hero Member
  • *****
  • Posts: 711
  • 5D Mark a1ex
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1100 on: January 16, 2017, 01:56:31 AM »
Wonderfull guys!

ansiivan

  • Just arrived
  • *
  • Posts: 1
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1101 on: January 17, 2017, 01:20:33 AM »
Hi. Why regular chamber mov, much brighter raw vide? The settings are the same.
Canon 1100d,  magiclantern-raw_video_10bit_12bit.2017Jan13.1100D105.zip
Fps 25, iso1600.
mov

Fps 25, iso1600.
raw video

kaco

  • Member
  • ***
  • Posts: 101
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1102 on: January 17, 2017, 12:32:36 PM »
Hi guys,

on Saturday, I've sucesfully shot 2TB of MLV's in 10bit. From 1400 filex, approx 150 had wrong black level. Since calling mlv_dump 150 times and then sorting all the files would be too complicated, I've wrote little windows batch script. I am not a programmer so finding the first file in a folder and even the iteration is very very messy. But works. :)

Code: [Select]
@echo off
setlocal EnableDelayedExpansion
set mypath=%~dp0
set workingDir=E:\footage\mlv\
set fixDir=E:\footage\cdng-fixed\
set moveDir=E:\footage\mlv-processed\


E:
cd %workingDir%
for /r %%i in (*) do (
set file=%%i

!mypath!\mlv_dump.exe --black-fix=2040 --dng "!file!"

for %%A in ("!file!") do (
set foldername=%%~dpA
set filename=%%~nxA
)
echo !filename!

cd "!fixDir!"
md "!filename!"
cd "!filename!"

for /r "%workingDir%" %%f in (*.dng) do move "%%f" >nul
for /r "%workingDir%" %%f in (*.wav) do move "%%f" >nul

move  "!file!" "!moveDir!"
)
D:
cd %mypath%

kaco

  • Member
  • ***
  • Posts: 101
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1103 on: January 17, 2017, 03:04:54 PM »
BTW, may I process *all* files with --black-fix=2040, even those which have correct value? If I want to process only those wrong ones, I need to extract all and visually check.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1104 on: January 17, 2017, 05:44:17 PM »
On 5D3 and probably all other recent cameras, latest mlv_dump requires --black-fix=2048, regardless of ML build (since this post).

With latest ML bulid from the Experiments page AND latest mlv_dump, there should be no need to change the black level (since this post).

Checklist:
- with 10-bit files, the black level from MLV (visible with mlv_dump -v) should be 128 (127 is wrong)
- with 14-bit files, the black level should be close to 2048 (small variations are OK)
- the black level from the DNG (always upconverted to 14 bits) should be close to 2048 (e.g. 2050 is fine, 2032 is wrong)
- there should be no color difference between 10 and 14 bit files in shadows

If any of the above is not true for you with the latest experimental build and latest mlv_dump, please report.

For authors of other postprocessing tools: to avoid color casts at lower bit depths, conversion to a higher bit depth must be done by adding 0.5 LSB to the input signal (explained here). Black level scaling requires no trickery (e.g. 128 -> 2048).

Relevant commits: e3ef8c7 and a31228c.

vstrglv

  • Senior
  • ****
  • Posts: 252
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1105 on: January 17, 2017, 05:53:10 PM »
Build  magiclantern-crop3x.2017Jan13.5D3113 (2017-01-12 23:35) – crop+10bit,12bit. MLV raw.
DNG conversion raw2cdng 1.7.9 and  mvl_dump (2017-01-13 00:55).
DNG for crop(1x1)_Xbit_ raw2cdng and DNG for crop(1x1)_Xbit_ mlv_dump look very similar (except  green colour cast for 10bit raw2cdng)  in ACR if Noise Reduction (Luminance) is set to zero. But if Noise Reduction is increased its effect is more noticeable  for mlv_dump convertion.
X is 10bit, 12bit or 14bit. This effect takes place for normal mode (not crop) too.
https://www.dropbox.com/s/q6pj4kspt8l794t/M14-1338_10b_crop_mvl_dump.dng?dl=0
https://www.dropbox.com/s/i6vpllko75d43wl/M14-1338_10b_crop_raw2cdng.dng?dl=0

M14-1338_10b_crop_0NR_raw2cdng by Vladimir Strigalev, on Flickr
M14-1338_10b_crop_0NR_mvl_dump by Vladimir Strigalev, on Flickr
M14-1338_10b_crop_30NR_raw2cdng by Vladimir Strigalev, on Flickr
M14-1338_10b_crop_30NR_mvl_dump by Vladimir Strigalev, on Flickr
Canon 5D3,1.1.3; Canon EOS M,202,  CF-SanDisk Extreme PRO,160MB/s, 256GB, SD-SanDisk Extreme Pro, 170MB/s, 128GB.

kaco

  • Member
  • ***
  • Posts: 101
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1106 on: January 17, 2017, 07:09:42 PM »
On 5D3 and probably all other recent cameras, latest mlv_dump requires --black-fix=2048, regardless of ML build (since this post).

With latest ML bulid from the Experiments page AND latest mlv_dump, there should be no need to change the black level (since this post).

Great! However I still find batch calling mlv_dump much better then mlv-fs. Is there any way how to make real cdng from mlv_dump? So that Resolve could read audio and correct fps? Or this wouldn't work because mlv_dump is not indexing the files?

Thanks

reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2303
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1107 on: January 18, 2017, 02:10:40 AM »
Resolve read Cdng with audio from mlvfs as one file with correct level in real time on PC's and Mac's .
batch calling on command is very slow plus you end up with lots of dng's that fill up hard drive space .
MLVFS doesn't do that it produces Virtual Cdng so no hard drive killer here , you even read off the CF or SD cards
depending on your computer hardware .
FYI: mlv_dump produces compliance dng that work in resolve or at least used to that could be broken thou with bit reduction

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7701
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1108 on: January 18, 2017, 02:48:08 AM »
Quote
Great! However I still find batch calling mlv_dump much better then mlv-fs. Is there any way how to make real cdng from mlv_dump?
You can process cdng files with this
http://www.magiclantern.fm/forum/index.php?topic=15108.msg146822#msg146822
 
The program works with a mlv_dump version which has code mixed in from a specific ml-dng branch version. Code updated to latest 10/12/14 bit the same as latest mlv_dump(g3gg0/a1ex). Works with resolve(audio embedding).
When installed head over to the mlv_dump menu (m) from the main menu and you can do setting changes from there such as black/white level etc.

D_Odell

  • Member
  • ***
  • Posts: 187
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1109 on: January 19, 2017, 12:24:50 PM »
Using 13th of Jan build and the frame freezes in black and white live view while shooting raw_rec with 5x. Any others experiencing the same?
5D3 (OLPF removed) :: 1.1.3 :: Canon FD L Serie

Tracerman

  • New to the forum
  • *
  • Posts: 4
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1110 on: January 19, 2017, 01:11:37 PM »
@dfort

Hi, I'm up for testing 10/12 bit mlv_rec on 550D if you like (I don't know how to make a pull request either :/).


Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7701
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1111 on: January 19, 2017, 01:19:19 PM »
Is this happening with or without raw_twk enabled? Try with raw_twk disabled.

Levas

  • Contributor
  • Hero Member
  • *****
  • Posts: 1741
  • 6d - Nightly build user
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1112 on: January 19, 2017, 03:53:58 PM »
There 's only a windows version of MLV_dump available on the magic lantern downloads page:
https://builds.magiclantern.fm/utilities.html

Can somebody add newest version of MLV_dump for osX ?

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7701

D_Odell

  • Member
  • ***
  • Posts: 187
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1114 on: January 19, 2017, 10:36:42 PM »
Is this happening with or without raw_twk enabled? Try with raw_twk disabled.
Still happening with raw_twk disabled. Guess its something with the build?
5D3 (OLPF removed) :: 1.1.3 :: Canon FD L Serie

dfort

  • Guest
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1115 on: January 20, 2017, 08:33:57 AM »
@dfort

Hi, I'm up for testing 10/12 bit mlv_rec on 550D if you like (I don't know how to make a pull request either :/).

Nothing new for the 550D yet and we know what needs to be done before we're ready for any further testing.


Sent from my iPad using Tapatalk

CITY-U1001

  • New to the forum
  • *
  • Posts: 46
  • 50D in future
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1116 on: January 20, 2017, 02:01:04 PM »
i dont see 10 bit for 50D   ???
50D | EFS 18-55 | last build crop_rec-3744x1080_24fps_50D-eXperimental.4.57pm.2020May06.50D109.zip

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7701

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3413
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1118 on: January 20, 2017, 07:31:31 PM »
Thanks SO much for sharing this @Danne and for the quick fixes @a1ex!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Tracerman

  • New to the forum
  • *
  • Posts: 4
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1119 on: January 21, 2017, 12:30:33 PM »
@dfort

Thanks for your reply, feel free to let me know when/if you need testers for 550D.

Cheers

justinbacle

  • New to the forum
  • *
  • Posts: 40
  • DP
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1120 on: January 21, 2017, 04:44:04 PM »
i dont see 10 bit for 50D   ???
50D is not supported (yet).
(Same as 5dmk2 as they don't have CONFIG_EDMAC_RAW_SLURP ?)

bobbyshots2

  • New to the forum
  • *
  • Posts: 12
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1121 on: January 21, 2017, 07:14:30 PM »
I want to try and test the t2i (550d) with 10bit - 12bit raw. If anybody has a link to  the module they could send me or the full ml download. that would be great.!
Thanks in advance!
One Love

ShootMeAlready

  • Senior
  • ****
  • Posts: 264
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1122 on: January 21, 2017, 07:35:51 PM »
Ok, mlv_dump works to get me DNG.
I can colour correct usingbut how to get them into Ppro?  If I save as JPEG from LR, it complains that bit depth is not supported.

Is there a way to get 10bit scaled to 14bit depth?

Found that DNG works very nice in Resolve.
However LR becomes useless in this flow, since its corrections are not recognized when DNG is input to Resolve.
One can export from LR into jpeg but that limits Ppro or Resolve's abilities.

T3i+ML & 70D.112+ML, Tokina 11-16 2.8, Sigma 18-35 1.8, 50-150 II 2.8, 50 1.4, Canon 28 1.8, 35 2, 85 1.8 "Shoot Wide and Prosper"

dfort

  • Guest
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1123 on: January 21, 2017, 08:05:18 PM »
Looks like it is worth repeating that the following cameras need to get the CONFIG_EDMAC_RAW_SLURP treatment before it is worth doing any further tests with them.

5D2.212-Digic 4
7D.203-Dual Digic 4
50D.109-Digic 4
500D.111-Digic 4
550D.109-Digic 4

reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2303
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1124 on: January 22, 2017, 02:29:02 AM »
50D is not supported (yet).
(Same as 5dmk2 as they don't have CONFIG_EDMAC_RAW_SLURP ?)
Not really true , not sure about 50D but the 5D2 work in Crop mode 100% (10-12bit) with Audio , overlays  in MLV full  see the Proof
Even thought 1:1 liveview products some corruption in 10bit but 12bit can be used if you crop out
the top 50-100 line do to a color noise bar see here look at the 3rd image I posted .

I can't say that the 50D or 550 (T2i) will have the same level of functionality in crop mode as the 5D2 but as @dfort said
..... need to get the CONFIG_EDMAC_RAW_SLURP treatment before it is worth doing any further tests with them.

We do need more user's who can help figure the Digic 4 liveview issue (5d2,50d,7d,500,550) so I will provide the link to those builds in hope more development can be made .
My reasoning is if users have at least a limited functioning version of  raw video bit reduction , this hopefully will give more incentive to push development on the digic 4.
link to my bitbucket downloads for 10-12bit reductions 5d2,50d,7d,500,550