White balance issue.

Started by extremelypoorfilmaker, September 28, 2016, 06:32:26 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

extremelypoorfilmaker

Hi Everyone, for quite some time I started noticing one thing: my MLV files seems not to register the white balance I set at the time of filming.

At first, I didn't really pay much attention as I used to take advantage of changing it easily in Lightroom.
But recently some footage I shot at 3000k turned out to be at 5400k once converted through MLVrawviewer. And the same happened for other footage that was supposed to be shot at 6000k and 5500k.

for normal h264 videos and CR2 files the white balance remains set as correctly shot with the camera.
I did a couple of searches to see if anyone had the same "issue" (sorta) but nothing came up.
Is that supposed to be normal?

Thanks! :)

a1ex

Output of "mlv_dump -v file.mlv" ?

extremelypoorfilmaker

Quote from: a1ex on September 28, 2016, 06:38:05 PM
Output of "mlv_dump -v file.mlv" ?

err.. Care to explain a little bit? :)

what do I need to produce? the .MRX files? the .idx files? the resulting .dng?
I am very sorry but I haven't yet got around using mlv_dump.

I have to apologize for forgetting to include all the details regarding the camera and build I am using.

Camera: 5D3
build: August 2014 firmware version 1.2.3


DeafEyeJedi

Care to share the original MLV file for us to check on our end?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

Isn,t dng files exported with default 5500k from regular mlv_dump output?
MLVFS (dmilligan) solves this problem and Bouncyball implemented this change in a version of mlv_dump. I use it in MLP.
On a sidenote I don,t think MlRawViewer respects white balance tag anyway.

Here is Bouncyball on bitbucket
https://bitbucket.org/bouncyball/mlv-tools/downloads

DeafEyeJedi

Quote from: Danne on September 28, 2016, 07:53:03 PM
On a sidenote I don,t think MlRawViewer respects white balance tag anyway.

Indeed it doesn't. Best use either MLVFS or MLP as both does it's job well.
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

extremelypoorfilmaker

https://drive.google.com/file/d/0B4V3oyfM-f2YWlg5VGpnMkRTN1E/view?usp=sharing

Here's the direct link to one of the .mlv files, as you will see, the white balance it's supposed to be 2800k.

I am not really concerned if Magic Lantern .dng files don't have a baked in white balance, like odyssey's .dngs or blackmagic .dngs

I still would like to know what: Output of "mlv_dump -v file.mlv" means. :) 

a1ex

Quote from: extremelypoorfilmaker on September 28, 2016, 10:48:45 PM
I still would like to know what: Output of "mlv_dump -v file.mlv" means. :) 

Type it in your command prompt.

extremelypoorfilmaker


After properly understand what the great A1ex meant, I can happily announce that there is nothing wrong with my MLVs.
I was guilty of ignorance not knowing that mlvrawviewer export every .dng with a default white balance of 5500k.

Thanks everyone to clarify my doubt and to allow me to learn a little bit more. :)

dfort

By the way, here's the relevant info for the file you posted.

The first white balance block comes just before the STYL block:

Block: WBAL
  Offset: 0x00000200
    Size: 44
    Time: 0.005000 ms
     Mode:   9
     Kelvin:   3000
     Gain R:   491
     Gain G:   1024
     Gain B:   638
     Shift GM:   0
     Shift BA:   0


Interestingly there's a second white balance block before the first VIDF block. There is a VIDF and a NULL block for every frame in your MLV file.

Block: WBAL
  Offset: 0x000003b0
    Size: 48
    Time: 514.177000 ms
     Mode:   9
     Kelvin:   3000
     Gain R:   491
     Gain G:   1024
     Gain B:   638
     Shift GM:   0
     Shift BA:   0


So according to this information it looks like mlv_dump is reporting 3000K as your color temperature. I believe white balance also requires a green/magenta tint offset value.

dfort

Interesting, I created DNG files from your MLV using mlv_dump and it opened up in Adobe Camera Raw with a color temperature of 5500 and a tint of +19, not the 3000K reported by mlv_dump -v. Running exiftool I can't find the white balance information. Is it calculated from Color Matrix 1 and As Shot Neutral? Whatever it is it isn't being honored by ACR.

ExifTool Version Number         : 10.13
File Name                       : M26-1832_frame_000000.dng
Directory                       : /Users/rosiefort/Desktop/mlv test
File Size                       : 3.4 MB
File Modification Date/Time     : 2016:10:07 19:34:20-07:00
File Access Date/Time           : 2016:10:07 19:35:45-07:00
File Inode Change Date/Time     : 2016:10:07 19:34:20-07:00
File Permissions                : rw-r--r--
File Type                       : DNG
File Type Extension             : dng
MIME Type                       : image/x-adobe-dng
Exif Byte Order                 : Little-endian (Intel, II)
Image Description               : (MLV Video without INFO blocks)
Make                            : Canon
Camera Model Name               : Canon EOS 5D Mark III
Orientation                     : Horizontal (normal)
Software                        : Magic Lantern
Modify Date                     : 2016:09:26 19:32:44
Artist                          :
Subfile Type                    : Full-resolution Image
Image Width                     : 1920
Image Height                    : 1038
Bits Per Sample                 : 14
Compression                     : Uncompressed
Photometric Interpretation      : Color Filter Array
Strip Offsets                   : 33792
Samples Per Pixel               : 1
Rows Per Strip                  : 1038
Strip Byte Counts               : 3487680
X Resolution                    : 180
Y Resolution                    : 180
Planar Configuration            : Chunky
Resolution Unit                 : inches
CFA Repeat Pattern Dim          : 2 2
CFA Pattern 2                   : 0 1 1 2
Black Level                     : 2046
White Level                     : 15000
Default Crop Origin             : 0 0
Default Crop Size               : 1920 1038
Active Area                     : 0 0 1038 1920
Opcode List 1                   : (Binary data 28 bytes, use -b option to extract)
Copyright                       :
Exposure Time                   : 1/41
F Number                        : 4.0
Exposure Program                : Not Defined
ISO                             : 800
Exif Version                    : 0221
Date/Time Original              : 2016:09:26 19:32:44
Shutter Speed Value             : 1
Aperture Value                  : 1.0
Exposure Compensation           : 0
Max Aperture Value              : 1.0
Metering Mode                   : Unknown
Flash                           : No Flash
Focal Length                    : 105.0 mm
Sub Sec Time                    : 507
Sub Sec Time Original           : 507
Focal Length In 35mm Format     : 0 mm
TIFF-EP Standard ID             : 1 0 0 0
Serial Number                   : 293022002387
Lens Model                      : EF24-105mm f/4L IS USM
DNG Version                     : 1.3.0.0
DNG Backward Version            : 1.3.0.0
Unique Camera Model             : Canon EOS 5D Mark III
Color Matrix 1                  : 0.6722 -0.0635 -0.0963 -0.4287 1.246 0.2028 -0.0908 0.2162 0.5668
Analog Balance                  : 1 1 1
As Shot Neutral                 : 0.473635 1 0.624
Baseline Exposure               : undef
Baseline Noise                  : 1
Baseline Sharpness              : 1.333333333
Linear Response Limit           : 1
Calibration Illuminant 1        : D65
Frame Rate                      : 23.976
Aperture                        : 4.0
CFA Pattern                     : [Red,Green][Green,Blue]
Image Size                      : 1920x1038
Megapixels                      : 2.0
Shutter Speed                   : 1/41
Date/Time Original              : 2016:09:26 19:32:44.507
Modify Date                     : 2016:09:26 19:32:44.507
Focal Length                    : 105.0 mm
Light Value                     : 6.4


Same MLV opened in MLVFS shows a temp of 3050 and tint +9 in ACR and shows more detailed information in exiftool:

ExifTool Version Number         : 10.13
File Name                       : M26-1832_000000.dng
Directory                       : /Users/rosiefort/Desktop/mount/M26-1832.MLV
File Size                       : 3.9 MB
File Modification Date/Time     : 2016:09:26 19:32:44-07:00
File Access Date/Time           : 2016:09:26 19:32:44-07:00
File Inode Change Date/Time     : 2016:09:26 19:32:44-07:00
File Permissions                : rw-rw-rw-
File Type                       : DNG
File Type Extension             : dng
MIME Type                       : image/x-adobe-dng
Exif Byte Order                 : Little-endian (Intel, II)
Subfile Type                    : Full-resolution Image
Image Width                     : 1920
Image Height                    : 1038
Bits Per Sample                 : 16
Compression                     : Uncompressed
Photometric Interpretation      : Color Filter Array
Fill Order                      : Normal
Make                            : Canon
Camera Model Name               : Canon EOS 5D Mark III
Strip Offsets                   : 65536
Orientation                     : Horizontal (normal)
Samples Per Pixel               : 1
Rows Per Strip                  : 1038
Strip Byte Counts               : 3985920
Planar Configuration            : Chunky
Software                        : MLVFS
Modify Date                     : 2016:09:26 18:32:44
CFA Repeat Pattern Dim          : 2 2
CFA Pattern 2                   : 0 1 1 2
Exposure Time                   : 1/42
F Number                        : 4.0
ISO                             : 800
Sensitivity Type                : ISO Speed
Exif Version                    : 0230
Subject Distance                : 45 m
Focal Length                    : 105.0 mm
Focal Plane X Resolution        : 1314.168378
Focal Plane Y Resolution        : 1316.872428
Focal Plane Resolution Unit     : inches
Lens Model                      : EF24-105mm f/4L IS USM
DNG Version                     : 1.4.0.0
Unique Camera Model             : Canon EOS 5D Mark III
Black Level                     : 2046
White Level                     : 15000
Default Scale                   : 1 1
Default Crop Origin             : 0 0
Default Crop Size               : 1920 1038
Color Matrix 1                  : 0.7234 -0.1413 -0.06 -0.3631 1.115 0.285 -0.0382 0.1335 0.6437
Color Matrix 2                  : 0.6722 -0.0635 -0.0963 -0.4287 1.246 0.2028 -0.0908 0.2162 0.5668
As Shot Neutral                 : 0.7243757511 1 0.3887548764
Baseline Exposure               : 0
Calibration Illuminant 1        : Standard Light A
Calibration Illuminant 2        : D65
Active Area                     : 0 0 1038 1920
Forward Matrix 1                : 0.7868 0.0092 0.1683 0.2291 0.8615 -0.0906 0.0027 -0.4752 1.2976
Forward Matrix 2                : 0.7637 0.0805 0.1201 0.2649 0.9179 -0.1828 0.0137 -0.2456 1.057
Time Codes                      : 00:00:00.00
Frame Rate                      : 23.976
Reel Name                       : /M26-1832.MLV
Baseline Exposure Offset        : 0
Aperture                        : 4.0
CFA Pattern                     : [Red,Green][Green,Blue]
Image Size                      : 1920x1038
Megapixels                      : 2.0
Scale Factor To 35 mm Equivalent: 1.0
Shutter Speed                   : 1/42
Circle Of Confusion             : 0.029 mm
Depth Of Field                  : 55.57 m (30.47 - 86.03 m)
Field Of View                   : 19.0 deg
Focal Length                    : 105.0 mm (35 mm equivalent: 107.7 mm)
Hyperfocal Distance             : 94.13 m
Light Value                     : 6.4

dfort

Sorry for the multiple posts but I think there's something interesting happening here.

First of all, a screenshot showing a frame of the file provided by @extremelypoorfilmaker processed by the current mlv_dump and opened in Adobe Camera Raw. Note that Temperature is 5500 and Tint is +19.



Next is the same frame but using a modified mlv_dump that fixes the white balance issue.



Note that the Temperature on this one is 3050 and Tint +9, the same as MLVFS.

The "fixed" mlv_dump came from compiling a version from @bouncyball - his repository is here:
https://bitbucket.org/bouncyball/ml_dng-branch

Danne

It,s forked from dmilligan ml-dng branch. Bouncyball put in the time to implement his(dmilligan) code from which I understand is based on code used in ufraw.

DeafEyeJedi

That's right @Danne. This is all great stuff @dfort and should eventually make its way to mlv_dump!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

extremelypoorfilmaker

Thank you @dfort for your detailed analysis of my file.

I did try to convert the same file with raw2cdng 1.6.5 after usinv mlv_dump (the modfied version of bouncyball) and the converted .cdng files had the correct white balance.


...Stupid question:
Is it thanks to the 2 extra bit, why mlv white balance is not baked in like Odyssey's 7Q+ dngs?

:)
Thanks!

g3gg0

Quote from: dfort on October 08, 2016, 02:41:11 AM
Interestingly there's a second white balance block before the first VIDF block. There is a VIDF and a NULL block for every frame in your MLV file.

not so uncommon imho.
didn't check the source, but mlv_rec should write the usual status blocks once on start and
also insert them on every update in some VIDF frame's empty space.

so it seems the camera "updates" WBAL right after recording start for some reason.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

dfort

@g2gg0 what is more interesting is that I just ran the same test using the file that @extremelypoorfilmmaker provided, this time with the current unified mlv_dump comparing it with one from the the ml_dng pull request by @dmilligan and the results were the same as my previous test using that obscure fork of a fork of a pull request version.

This is what motivated me to champion the ml_dng pull request that was first put up in January 2015. That and having working cdng files that open in Adobe Premiere, the current version creates files that raise an "Unsupported format or damaged file" error message when imported into Premiere. I understand that things will happen in ML when they are ready, but isn't this ready yet?

g3gg0

i don't have a problem with the dng thing whatsoever.

but i dont want "this is a useful addon"-pull-requests with changes that are not explainable but are said to fix something.

so my only wish is that separate concerns are split into separate pull requests.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

Danne

This could be a fix for the chroma smooth bug I reported about long ago.
http://www.magiclantern.fm/forum/index.php?topic=7122.msg167481#msg167481
Feedback stopped and I pointed out the issue to bouncyball who fixed it. I then didn,t know how to do a pull request(probably still don,t) so I messaged you pointing to the solution. Probably didn,t come through.

Yes, here,s the original commit
https://bitbucket.org/bouncyball/ml_dng-branch/commits/58ae1f6e8f580ff0dc63989d464edf193cd23af9

dfort

Quote from: Danne on October 18, 2016, 10:23:23 PM
This could be a fix for the chroma smooth bug I reported about long ago.

Confirmed that the current unified has the chroma smooth segfault issue and once the changes in the commit that Danne pointed out are applied it fixes the issue.

It looks like what happened was that the changes were applied to the dmilligan ml_dng branch and that was passed along to the main unified branch but it wasn't merged because the pull request didn't explain the chroma smooth segfault issue.

We're going off topic with this discussion so I'll link back to this in the appropriate topic.