What About Black Shading ?

[jcbouden]

Black Shading is a very useful tool that the Red Owners already know. It allows to avoid a tons of noise in your image : the Sony F65 is very high demanded because of its ability of doing a black shading before capturing every frame. The results, noise-free, are pretty amazing.

Is it possible for Magic Lantern to do something about it ? To allow us to do a black shading when the temperature changes, etc ?

More about it…

Thanks guys !

[g3gg0]

MLV basically supports this.
you have to record a MLV video with a black image and the settings you use.
then use 'mlv_dump -s black.mlv -o out.mlv in.mlv' to subtract that black from your image.

never tried though

-s mlv_file         subtract the reference frame in given file from every single frame during processing

[jcbouden]

Thanks for the quick answer. I didn't buy my 5D yet but I will definitely try this Take care

[g3gg0]

i ran a test with the 5D3 and these settings:

ISO 25600, 1920x1080, 24 fps

M07-1813.MLV, my room
M07-1825.MLV, black scene with lens cap on (~500 frames)


# dump room video without any correction (two frames only)
mlv_dump --dng -f 1 M07-1813.MLV

# sum up all frames of the black image and write single frame into the darkframe avg.mlv
mlv_dump -o avg.mlv -a M07-1825.MLV
mlv_dump --dng avg.MLV

# process M07-1813.MLV into sub.mlv by subtracting the frame in avg.mlv from every single frame (darkframe)
mlv_dump -o sub.mlv -s avg.mlv M07-1813.MLV
mlv_dump --dng -f 1 sub.MLV


here the result:

[jcbouden]

This is very interesting. Is the result convincing when it comes to watch the video with the correction ?

Is it possible to do the same thing with RAW ? Thanks

[g3gg0]

this kind of image processing is very common for years for astrophotography.
(see e.g. http://deepskystacker.free.fr/english/faq.htm)

didnt try a video yet

[SpcCb]

this kind of image processing is very common for years for astrophotography.
(see e.g. http://deepskystacker.free.fr/english/faq.htm)

Indeed.
But here it's only a simple calibration with dark frame, as the original Canon FW do in long exposure noise reduction.
In astrophotography the calibration is a bit more complex (bias/dark/flat/defect/etc. frames).

didnt try a video yet

I do it all time for scientific video imaging but I'm not sure this is friendly and/or useful for most people (?). And sure it is too much for real processing via the camera CPU.

[timbytheriver]

Is 'Black shading' calibration (used by RED, Blackmagic URSA, et al.) the same as 'dark frame subtraction' which we have in MLVApp?

There is such a mine of info generated by raw_diag.mo that it would be useful to plough this data back into a proper way to calibrate each user's camera individually. I'm starting to believe that there is quite a lot of variance between people's camera sensors which might explain test results not being matched across the community. 

Where might one begin…    ?

[Luther]

Is 'Black shading' calibration (used by RED, Blackmagic URSA, et al.) the same as 'dark frame subtraction' which we have in MLVApp?

Yes, "black shading" is just a fancy name RED uses. Dark frame will also remove dead pixels. Some (more aggressive) FPN might not be completely removed using dark frame, though. There's another thread where a1ex and g3gg0 worked on FPN removal, try to find it...

I'm starting to believe that there is quite a lot of variance between people's camera sensors

Temperature is a big factor for CMOS sensor's SNR.

[70MM13]

yes, luther!, 100% on the sensor but also the preamps are affected by temperature too!

they are easier to keep cool, and i am considering drilling some holes in my 5d3 body!

[timbytheriver]

@Luther Thanks for the clarification! Think the RED does the calibration in-camera, which would be useful…

There's another thread where a1ex and g3gg0 worked on FPN removal, try to find it...

Seen that! That'd be an awesome module to see come alive! 

[Luther]

@Luther Thanks for the clarification! Think the RED does the calibration in-camera, which would be useful…

Aparently there's already a block in MLV format for dark and flat frame:
https://www.magiclantern.fm/forum/index.php?topic=24470.msg220691#msg220691

So, someone "just" needs to automate the process with some module and embed the frame in each .MLV file (MLVApp should also be modificed to recognize automatically).

[DeafEyeJedi]

...someone "just" needs to automate the process with some module and embed the frame in each .MLV file (MLVApp should also be modificed to recognize automatically).

+1

[timbytheriver]

So, someone "just" needs to automate the process with some module and embed the frame in each .MLV file

Such an interesting rabbit hole to disappear down!

@Luther Do you think there would be a hit to performance if it's encoding another frame?

[Kharak]

I think BM Ursa does the black shading as a Sensor Calibration as you need to keep the camera with lens cap on for 5-10 mins for it to finish, this will remove the magenta cast that develops over time. I don't know how the Red equivalent works. But Ursa is not like DFA where you need to do it in post, you probably can, but that is not how the built in black shading works.

I remember some Sony TV's also had built in Image cleaning (maybe they still do), basically it was "White Shading" with a built in White Screen that you should leave on for 5 mins to get clean blacks and colours.

[Luther]

@Luther Do you think there would be a hit to performance if it's encoding another frame?

Don't think so, as it would just copy the frame inside each MLV. Another solution would be to have a "link" inside MLV metadata to point to the dark frame file. This way MLVApp and other software would know where the frame is, without the need to copy it on each MLV (it would also save some CF space). I like the second solution more because of efficiency.
Another idea would be to automate the creation of such dark frames, because it's more effective when the frame is taken shortly before the shot it will subtract to (for thermal reasons, I think). For example, if you have a event to photograph/record, you could do multiple dark frames using the module throughout the duration of such event. Based on the date of each MLV, the module could link the closest dark frame in this period of time. This would increase the dark frame effectiveness, as noise changes when the camera heats.
The module should also copy the current camera settings (shutter speed and ISO, mainly), because the noise also vary with exposure time and sensibility.
Don't know if averaging would be possible inside the camera, though. But folders could be created to store multiple dark frames at the same time, and then leave this task of averaging for MLVApp or other software (the averaging could be automated too).
Just some ideas.

But Ursa is not like DFA where you need to do it in post, you probably can, but that is not how the built in black shading works.

It works even on Raw? Hard to believe they are doing such low-level processing inside the camera. Applying that to debayered data inside the camera is easier, but on Raw it gets more complicated...