Author Topic: 12-bit (and 10-bit) RAW video development discussion  (Read 260752 times)

Danne

  • Hero Member
  • *****
  • Posts: 3140
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1225 on: March 27, 2017, 11:27:59 PM »
Yes, classic black level issue. From your file after extracting with mlv_dump
Black Level                     : 7312
White Level                     : 14992

When reprocessing and specifying numbers below picture is perfect.
Black Level                     : 2049
White Level                     : 12277

In mlvp you specify 10bit levels so they differ. Above numbers are for upressed to 14bit. Visually the picture should look the same for both 10bit and 14bit after fixing.

andy kh

  • Senior
  • ****
  • Posts: 301
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1226 on: March 27, 2017, 11:39:27 PM »
@Danne thanks for checking it out. corrupted files are so many. as i mention earlier atleast3 to 4 out of 10 files get corrupted everytime i record in 10bit

Danne

  • Hero Member
  • *****
  • Posts: 3140
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1227 on: March 27, 2017, 11:55:41 PM »
I don,t know the status of 70D and 10/12bit recording but you could use mlv_dump for windows and specify both white and black level for all files that you export. Or stick to mlvp if that,s more convenient. Here is mlv_dump for windows
https://builds.magiclantern.fm/jenkins/job/mlv_dump/20/artifact/src/modules/mlv_rec/mlv_dump.exe

Specify like this
Code: [Select]
mlv_dump --dng --white-fix=12277 --black-fix=2049 Input.MLV
ANd why not do the same command with bouncyball´s version mlv_dump on steroids :)
http://www.magiclantern.fm/forum/index.php?topic=18975.msg180083#msg180083

andy kh

  • Senior
  • ****
  • Posts: 301
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1228 on: March 28, 2017, 12:11:43 AM »
@Danne
i wil try bounchyball's version as i prefer to export in cdng
thanks for your help

reddeercity

  • Hero Member
  • *****
  • Posts: 1368
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1229 on: March 28, 2017, 04:22:19 AM »
If on Windows use MLVProducer , support 10 & 12 bit full mlv , mlv lite & old raw plus exports as Cdng if needed .
Also has slider for black level with real time preview like A.E. ACR plus many more grading options , I MUST have tool in windows along mlvfs .

andy kh

  • Senior
  • ****
  • Posts: 301
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1230 on: March 28, 2017, 10:05:06 AM »
@reddeercity i have been using mlvp for sometime and wil continue to use it. it has some problems though. cdng exported from mlvp has a decrete amount of magenta cast while exporting to resolve or premiere pro. it can be fixed by adjusting the white balance though. awpstar is aware of the magenta cast as someone in the forum pointed out with image. and after converting to h264 422 or 444, im not able to import them in resolve. while clicking the folder containing converted h264 422 in resolve, all i can see is red icons written offline. i wil share with more details in mlvp forum
thanks

beauchampy

  • Senior
  • ****
  • Posts: 297
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1231 on: March 28, 2017, 12:50:08 PM »
@Danne I've been trying to find out more information for black level / white level on the 5d3. I recently forced white level to 1163 using cr2hdr (is that your app btw!?). Can it be pushed around even more for increased dynamic range?

Danne

  • Hero Member
  • *****
  • Posts: 3140
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1232 on: March 28, 2017, 01:00:23 PM »
You can push whitelevel to 16383, it,s maximum white point.
Black level 2048. I do think default for 5D mark III is set to 16383 already.
Yes, my scripts bundled in automator. Utilizes a mlv_dump version with code from all over the place.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10086
  • 5D Mark Free
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1233 on: March 28, 2017, 01:10:46 PM »
The black level is the signal level you get in the absence of light. It's a positive value because:
- the value is not the same for every sensel (hence the need for dark frame subtraction)
- values below black level do appear (because of the additive noise); a noise reduction or a frame stacking step can make use of those sub-black values.

"Pushing" the black level to a lower value would be the same as changing the input curve (rendering deep shadows in a lighter tone, rather than black). You probably liked the look, but there's no additional dynamic range to gain this way.

If the black level is incorrectly set above the real value, you will see additional detail from lowering it. Otherwise, there's little or no point in changing it.

Changing the black level also changes the colors in shadows, because the white balance multipliers are usually not equal. If you set all of them to 1, the color cast introduced by changing black level disappears.

On the other hand, there might be additional dynamic range to gain from pushing white level to a higher value. With recent versions, this additional gain will be less than 0.02 EV.
Code: [Select]
/**
 * White level
 *
 * With PREFERRED_RAW_TYPE set to CCD, most cameras appear to clip above 16300
 * (most of them actually use the full range, until 16382)
 *
 * one size fits all: 16200 may sacrifice up to 0.02 EV of highlights
 * that is, log2((16382-2048) / (16000-2048))
 */
#define WHITE_LEVEL 16200

However, if you find pink highlights with this value, I'd be interested to see a sample MLV.

@Danne: pushing white to 16383 may be a little optimistic; there are exposure settings that clip to this value and there are exposure settings that clip to a slightly lower level (including 5D3). Usually, if white level is above clipping point, you get pink highlights; but in some cases, the result is pretty good anyway, so it's a bit of an edge case.

You can check the white level in the samples in this thread: http://www.magiclantern.fm/forum/index.php?topic=11899.0

e.g. with octave (requires read_raw.m):
Code: [Select]
a = read_raw("frame.dng");
prctile(a(:), [90 95 99 99.9 99.99 99.999])

Note the max value from the image can exceed the clipping point from one sensel (happens with hot pixels), that's why I prefer checking it with percentiles.

beauchampy

  • Senior
  • ****
  • Posts: 297
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1234 on: March 28, 2017, 04:52:16 PM »
@a1ex @danne

Just to clarify. What should the black and white levels be for a) true colour representation and b) maximum dynamic range on the 5d3?

Danne

  • Hero Member
  • *****
  • Posts: 3140
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1235 on: March 28, 2017, 05:43:34 PM »
I would go with this for white balance. Thanks a1ex.
Quote
one size fits all: 16200 may sacrifice up to 0.02 EV of highlights
Black level 2048

However Davinci resolve probably expects 5D mark III around 15000 if you start messing with their highlight recovery so you simply have to check what looks alright, how you choose to work with raw material and what NLE you are working in.

Do have a nice evening read in Baldavengers resolve frenzy over here
http://www.magiclantern.fm/forum/index.php?topic=15801.msg164822#msg164822

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 2942
  • 5D3 / M1 / 7D / 70D / SL1
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1236 on: March 28, 2017, 06:35:00 PM »
Been getting decent results w 5D3 footage by using @Danne's cr2hdr.app (black levels @ 2048 and white levels @ 16383) but now according to @a1ex it seems the new 'sweet spot' would be 16200 for the whites, correct?
5D3.113 • 5D3.123 • EOSM.203 • 7D.203 • 70D.112 • 100D.101

Danne

  • Hero Member
  • *****
  • Posts: 3140
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1237 on: March 28, 2017, 06:48:07 PM »
Safety margin, not sweet spot. Very well explained.

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 2942
  • 5D3 / M1 / 7D / 70D / SL1
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1238 on: March 28, 2017, 07:15:24 PM »
Well perhaps I should have mentioned 'safety spot' rather than. This is all great to know and thanks guys!
5D3.113 • 5D3.123 • EOSM.203 • 7D.203 • 70D.112 • 100D.101

Haeri

  • New to the forum
  • *
  • Posts: 3
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1239 on: March 28, 2017, 10:49:15 PM »
Any compiled 550D builds around? Would really like to try it out :D

MitchLally

  • New to the forum
  • *
  • Posts: 30
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1240 on: March 30, 2017, 06:38:33 PM »
Theres a few others here having the black level issues popping up with 10bit. I have a suspicion that it could be a result of having raw_twk loaded or "extra_hacks" selected in the mlv menu. Its one of the two so far I've had no false black levels with both switched off. I will test extensively over the weekend. Anyone else who has had black level issues tried this? And is it even possible that another module would interfere? I'm clueless on the coding so all I can do is test for you guys.

reddeercity

  • Hero Member
  • *****
  • Posts: 1368
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1241 on: March 31, 2017, 06:44:07 AM »
a Little update for the 5D2 , Dual ISO Video Does work with 10-12bit Full MLV. the file corruption for 1:1 is still there thou 3x Crop work OK
did produce a few Hot pixels , by fine tuning the ISO range and a few more setting there can be eliminated . I was mostly looking to see
if it would work and it does . I only had 1 liveview freeze the needed a battery pull ,but I did a Super Fast pan before recording and it was in
3x crop mode with mlv_lite  before recording , after recording started it didn't seem to be a problem.
So at this point Dual ISO Video is functional in 3x Crop mode and all frame have proper black level with the 20bit option toggled in MLVFS PC Version Web GUI.
If you what to try it out here is the link 5D2212_Video_dual_iso.zip !!! Beware !!! This could brick your camera (will probably not :D)  I had to say that !
If anything happen like a liveview freeze just pull the battery out asap.
One more thing this is for just the 5D2 not sure if it will work with other Digic IV cams
Until next time  8)

Ilia3101

  • Member
  • ***
  • Posts: 175
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1242 on: March 31, 2017, 08:47:42 AM »
I tried the dual ISO recently too, and it was really difficult to get the freeze to happen(maybe it's improved?). I still think the freeze needs to be fixed for it to be included in main build though.
5D2

domo94

  • New to the forum
  • *
  • Posts: 6
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1243 on: April 04, 2017, 03:29:48 PM »
So I was using the Dec build for 10-12 bit raw video on the 7D; here's a little update to show some love for the camera.

While I was recording 10bit (various resolutions) my screen is killed instantly until it stops recording and writing, but it still records at normal FPS, with and without override.
When I was recording 12bit, the camera functioned normally and I had no issues manning my camera as I normally do; but the video output only gets a rough 15-20 frames a second, making the video look like a Chaplin film with color.

14 bit works just as fine. I did both 5x and normal modes.

Also, while recording 12, and I believe 10, The recording frame was not aligned properly- here's what I mean: http://imgur.com/a/sIb9W

To anybody working on the 7d, or to who can point me in the right direction, I'd like to contribute and accelerate the development of this firmware for the 7D as much as I can. So if you'd like some info or logs from me, please let me know what I can do.


Check my signature if you want to see an example of the footage and the quality (fixed from tech errors)

rduquematos

  • New to the forum
  • *
  • Posts: 3
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1244 on: April 07, 2017, 08:31:26 AM »
is 10 bit raw available on the 50d? thanks

CITY-U1001

  • New to the forum
  • *
  • Posts: 35
  • 50D in future
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1245 on: April 11, 2017, 11:59:30 PM »
is 10 bit raw available on the 50d? thanks

yes, work
50D | Helios-44M 2/58 | last build 10bit_12bit.2016Dec18.50D109

dfort

  • Hero Member
  • *****
  • Posts: 1759
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1246 on: April 12, 2017, 12:25:05 AM »
Nested quote --

is 10 bit raw available on the 50d? thanks
yes, work

Really? Since when? It isn't on the Experiments download page and I haven't seen any commits on bitbucket for it.

It would be nice to get it working properly on the 50D -- and the 5D2, 7D, 500D and 550D but those who have tried can tell you, it isn't that easy.
EOSM.202 EOSM.203 EOSM2.103 700D.115 5D3.*

reddeercity

  • Hero Member
  • *****
  • Posts: 1368
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1247 on: April 12, 2017, 03:45:50 AM »
yes, work
Really? Since when? ...... It would be nice to get it working properly on the 50D -- and the 5D2, 7D, 500D and 550D but those who have tried can tell you, it isn't that easy.
+1  what dfort said ,
@CITY-U1001 if you mean 3x crop will that's not working fully is it , when 1:1 works that's when you can say it works.
I've been working on the 5d2 for mouths now and I still can't get it to work right in 1:1 , 3x crop work @ 10bit but that's not working fully.
If you have it working  then Please share the code/info so I can advance the 5d2 .   

mirko91

  • Just arrived
  • *
  • Posts: 2
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1248 on: April 15, 2017, 10:59:37 PM »
Hi guys, where can I download a compiled version with 10/12bit module for a 5d3 1.2.3? There were some links around but they are all missing. Thanks to anyone!

pc_bel

  • Member
  • ***
  • Posts: 108