Green and magenta artefacts on DUAL ISO output

[jml2017]

Hello Dual ISO gurus,

I am trying the DUAL ISO module and I would like to understand what is the optimal way of processing the DNG created by cr2hdr.

- I shot 2 pictures of the same building, one with no Dual ISO (Standard ETTR iso 100), and one with Dual ISO 100/1600. 
- I processed the Dual ISO CR2 with cr2hdr default settings (drag and drop on the exe)
- I developed the two pictures using Rawtherapee 5.3 with the following settings:
Processing Profile Neutral, AMaZE Demosaicing, White Balance Daylight, Auto Levels

Below is a screen cap of some details zoomed at 100%



Left CR2 developed/ Right DualISO DNG developed


As you can hopefully see, the Dual ISO shot shows some green and magenta vertical artefacts (see for example the balconies, the window frames, or the vertical pattern in the facade). I would upload the cr2 files but I am not sure how since they are so big.

My question is :

1. What can be done to minimize these effects? Where is it the best to address them?
a. During the shot: Different settings of Dual ISO (maybe a smaller range of ISO like 100-400, or 100-800?)
b. During the conversion with cr2hdr: Different settings of cr2hdr?
c. During development of the dng: Different postprocessing in Rawtherapee (different demosaicing algorithm?)


In addition I have a few other questions with respect to Dual ISO:
2. The white balance in the Dual ISO CR2 is not the white balance that I have selected on the camera, whereas for a standard CR2 it is. Is this the expected behavior? Does this affect the processing by cr2hdr?
3. I am not sure I can evaluate in the field whether I really need Dual ISO or not. So I would like to always take 2 images of the same scene (one standard and one Dual ISO). So far I need to manually change the settings in the menu between shots to achieve this. Is there a possibility to set up the camera to take automatically 2 images in a row: one standard image with current settings (base ISO) and one Dual ISO image? That would make the process more convenient.

Thanks for your help.





Config:
ML for 6D Build 2017-12-07
cr2hdr Build 2017-12-04

[Walter Schulz]

3: Expo tab -> Dual ISO -> [Q] -> Alternate frames only ON

[mothaibaphoto]

1. DualISO artifacts usually horisontal due to it's nature. What you see is most probably CA/Fringing. The reasons you see it only in DualISO processed files may be   due to cr2hdr eliminates lens info from raw files. Try to choose lens manually in your convertor.
2. WB does not affect cr2hdr.
3. AutoETTR can activate/disable DualISO according to scene.

[jml2017]

@ Walter: Thanks  It was in plain sight there...

@ mothaibaphoto. With respect to your comment 1. I believe that if it was lens induced, it should appear on both the standard shot and the Dual ISO shot. I purposely chose minimal development in Rawtherapee disabling any lens dependent correction or chromatic aberration/fringing correction. The lens information present in the cr2 file is not used. And sorry, I forgot to mention, I shot this building in portrait mode, so it would make sense that the Dual ISO artefacts are vertical on this picture according to your comment.

[a1ex]

To answer #1, please upload the CR2. The artifacts do look like aliasing, to my eye.

For #3, raw-based zebras will tell you where you will get such artifacts (those areas covered only by the lower ISO). A lower ISO range usually helps. If the semi-clipped areas only contain clouds or out-of-focus subjects or other low-frequency detail, it should be fine; otherwise, consider reducing the ISO range, or using a single ISO properly exposed to the right, or look into the ISO research modules.

WB is still unsolved; there was a half-successful attempt to fix it. There are some processing steps affected by it in minimal ways.

[Danne]

Why can´t this CR2 tag work WB RGGB Levels As Shot? It works all the time for all cameras. I ran hundreds of dualiso files through Switch always delivering correct wb(minor discrepancy when camera calibration tag isn´t included).
http://www.magiclantern.fm/forum/index.php?topic=7139.msg172751#msg172751

Sidenote:
The final "fine tuning"  piece of information is still the camera specific camera calibration tag which I havn´t seen anyone finding yet. Workaround is to develop a dng with adobe dng converter and grab your cameras calibration tag.
http://www.magiclantern.fm/forum/index.php?topic=7139.msg172807#msg172807
Sidenote 2:
Camera calibration tag itself can be baked into the final dng AsShotNeutral tag number.

[jml2017]

@ a1ex: Here are the 2 cr2 files:
https://drive.google.com/file/d/1F8BpKInMfmRPr0mcbFVPJ6FtQ9O5wh2B/view?usp=sharing
https://drive.google.com/file/d/1hX5gfC-36i-N1pwVa3QI3bq44z7c84V7/view?usp=sharing

I will do some tests by varying the ISO range for a given scene and will look at raw zebras when shooting. Is there a way to review these zebras after the fact other than right after having taken the picture (for example while reviewing pictures in front of a computer?).

@ a1ex and Danne
I would only be really worried about the WB if this had an impact on the image quality. According to a1ex it has very little impact on the cr2hdr processing. I guess that it does not create artefacts (correct me if this is wrong), so the impact is probably hard to detect by eye. This means that I can safely correct it in Rawtherapee.

I will not enter the debate to know whether it is solved or half solved as I do not know what is exactly going on under the hood

[Danne]

Here is the file after it´s been developed through Switch. It´s really cr2hdr with corrected wb and camera calibration tag to match original CR2 wb:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/IMG_2047.DNG

[jml2017]

@ Danne

It looks pretty good with respect to white balance. Unfortunately I am on Windows, so I cannot use Switch.

[Danne]

It's the exact white balance from the original file

[a1ex]

@jml2017: when you took the Dual ISO image, you should have seen something like this on the screen (preview colors might not be accurate, but the raw-based zebras and histogram should match what you've got back then):



That shows diagonal lines all over the building (in other words, it's saying "expect aliasing there").

@Danne: don't have the older test files with me right now, will look later; would be nice if you can submit a PR meanwhile.

[Danne]

Since I parse the AsShotNeutral tag in post with exiftool I can´t really add any new stuff into cr2hdr itself and a PR wouldn´t work for post production steps.
I think it would be possible to simply snatch the WB RGGB Levels As Shot from the dualiso CR2 itself and then convert the numbers and parse it into the processed DNG file after or during cr2hdr processing.

For instance:

Inside the CR2(e.g):
WB RGGB Levels As Shot          : 1739 1024 1024 1931

Calculate into AsShotNeutral tag:
1024/1739=0,588844163312248
1024/1931=0,528107271789582

Then inject the tag with exiftool.
exiftool "-AsShotNeutral=0,588844163312248 1 0,528107271789582" dualiso.dng

I grab the multipliers with dcraw in Switch something like this:

Code: [Select]

#Place a CR2 file in a folder and run the following script to end up with the AsShotNeutral dng white balance tag

#Fetch the first pair of multipliers from a CR2 with dcraw
  CR2_0=$(dcraw -i -v "$(ls *.CR2 | head -1)" | awk '/Camera multipliers/ { print $4; exit }')
  CR2_1=$(dcraw -i -v "$(ls *.CR2 | head -1)" | awk '/Camera multipliers/ { print $3; exit }')

#Fetch the second pair of multipliers from a CR2 with dcraw
  CR2_2=$(dcraw -i -v "$(ls *.CR2 | head -1)" | awk '/Camera multipliers/ { print $6; exit }')
  CR2_3=$(dcraw -i -v "$(ls *.CR2 | head -1)" | awk '/Camera multipliers/ { print $5; exit }')

#Calculate into DNG spec AsShotNeutral values

#First result to be used in the dng file(AsShotNeutral) 
  CR2_01=$(echo "$CR2_0"/"$CR2_1" | bc -l | awk 'FNR == 1 {print}')
#Second result to be used in the dng file(AsShotNeutral)
  CR2_02=$(echo "$CR2_2"/"$CR2_3" | bc -l | awk 'FNR == 1 {print}')

#Inject into the DNG accordingly, uncomment if matched with processed DNG
  #exiftool "-AsShotNeutral=$CR2_01 1 $CR2_02" "$(ls *.DNG | head -1)" -overwrite_original

#The final resulting AsShotNeutral tag to be used in the DNG file
  echo "-AsShotNeutral=$CR2_01 1 $CR2_02"

[a1ex]

Have you looked in cr2hdr source?

https://bitbucket.org/hudson/magic-lantern/src/06fae84faeee81d6368091205d10868cf8d4cbe3/modules/dual_iso/exiftool-bridge.c?at=cr2hdr&fileviewer=file-view-default#exiftool-bridge.c-69

It does pretty much the same thing, so I've assumed you know how to tweak it and you have files from a few different camera models to test on (I don't have them right now with me). This code was contributed by ayshih.

[Danne]

I´ll do some digging/testing with the specific code parts. Got some free time during christmas.

Think this needs to be verified from the code parts in question:

//If WB mode is Auto, read WB_RGGBLevelsMeasured values

I would say that WB_RGGBLevelsAsShot is what to use for auto as well.

I tested against 50D, 100D, eosm, 5D mark III, maybe a 70D file can´t remember. I´ll see if I can find my files again or I just go with 100D, eosm and the 5D mark III. Conclusion was always the same. The WB RGGB Levels As Shot would work for any wb mode including auto wb. That is we still need to accept slight discrepancies for the missing CameraCalibration tag. Or find it.

cr2hdr --wb=exif will not work reliably.
http://www.magiclantern.fm/forum/index.php?topic=16024.msg155670#msg155670
Testfile in question is gone. I´ll see if I can find it.

*Update
Tested the uploaded file here and it looks alright to me white balance wise straight from cr2hdr.
http://www.magiclantern.fm/forum/index.php?topic=21370.msg195006#msg195006
I will need to do some more testing around this. Seems there have been a change or two regarding wb since last I tried this. Will try all wb modes to begin with.

[Danne]

Ok, my tests reveal that it´s only AWB(auto white balance) that will fail when using:

Code: [Select]

cr2hdr --wb=exif *.CR2
All other wb modes are working.

The fix would be to use WB_RGGBLevelsAsShot and skip the idea about WB_RGGBLevelsMeasured when it comes to Auto White Balance. My suggestion is that cr2hdr by default applies --wb=exif when processing CR2 files and that --wb=exif is simply deleted from command line options.

Will start a PR around this.

[jml2017]

@ a1ex:

Thanks for the screencap. Which tool did you use to generate that picture with zebras and raw histogram?

I understand that there is no free lunch and in the regions where only 1 iso is present, there is a risk of aliasing. I also see that aliasing is reduced when shooting with a smaller ISO range. I will consider a smaller iso range for my next shots.

My question is: for the pictures that I have already taken (like that one). Can I do something about aliasing either by changing the cr2hdr settings or by applying some specific postprocessing in Rawtherapee (different demosaicing, other tool?). In other words, and according to your experience, what can be done in postprocessing to mitigate these effects?

Thanks for your advice.

[a1ex]

Thanks for the screencap. Which tool did you use to generate that picture with zebras and raw histogram?

QEMU (with a few patches), dcraw -e (to extract Canon's JPEG preview), imagemagick (compose). I've got the raw zebras and histogram overlays from emulator and combined them with the (resized) JPEG preview to approximate how the screen would have looked in camera.

My question is: for the pictures that I have already taken (like that one). Can I do something about aliasing either by changing the cr2hdr settings or by applying some specific postprocessing in Rawtherapee (different demosaicing, other tool?). In other words, and according to your experience, what can be done in postprocessing to mitigate these effects?

For the end user, probably not much; try, for example, LMMSE.

The problem of recovering missing detail is somewhat similar to debayering - which I don't fully grasp yet - except a lot more data is missing (you've got two lines of Bayer data followed by two lines of clipped - i.e. missing - data). I'm pretty sure there are better algorithms out there - however, that brings us deep into image processing research, maybe neural networks, maybe deconvolution - if you have a solid math background (unlike me), you are welcome to give it a try.

[Danne]

Been busy with this:
https://bitbucket.org/hudson/magic-lantern/pull-requests/890/crop-rec-4k-cr2hdr-wb-fix/diff#comment-None

Auto wb seems fixed but we are still missing refinements from the CameraCalibration tag

Code: [Select]

{ &exif_wb,               1, "--wb=exif",       "set AsShotNeutral from EXIF WB (close but missing CameraCalibration tag)" },
Let´s see what exiftool can find. I use the following CR2 file:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/IMG_5906.CR2

Run exiftool with -u and grep

Code: [Select]

exiftool -u IMG_5906.CR2 | grep 'Camera Color Calibration'
Camera Color Calibration 01     : -340  374  931 (10900K)
Camera Color Calibration 02     : -322  382  908 (10000K)
Camera Color Calibration 03     : -276  404  847 (8300K)
Camera Color Calibration 04     : -221  431  783 (7000K)
Camera Color Calibration 05     : -166  463  724 (6000K)
Camera Color Calibration 06     : -139  478  696 (5600K)
Camera Color Calibration 07     : -108  496  665 (5200K)
Camera Color Calibration 08     :  -59  524  614 (4700K)
Camera Color Calibration 09     :    0  563  564 (4200K)
Camera Color Calibration 10     :   54  605  520 (3800K)
Camera Color Calibration 11     :  103  643  482 (3500K)
Camera Color Calibration 12     :  163  693  439 (3200K)
Camera Color Calibration 13     :  208  732  407 (3000K)
Camera Color Calibration 14     :  254  786  386 (2800K)
Camera Color Calibration 15     :  371  926  327 (2400K)
Might be the numbers we are after. After converting the CR2 file in adobe dng converter we find the CameraCalibration tags inside the dng file:

Code: [Select]

Camera Calibration 1            : 1.0336 0 0 0 1 0 0 0 1.0044
Camera Calibration 2            : 1.0336 0 0 0 1 0 0 0 1.0044
We also find the:

Code: [Select]

As Shot Neutral                 : 0.48439 1 0.649334
If we use the Calibration 1 tag and do some divison with the As Shot Neutral tag we get the calibrated number we are looking for:
0.48439/1.0336 = 0.4686435759
0.649334/1.0044 = 0.6464894464

New corrected tag:

Code: [Select]

As Shot Neutral                 : 0.4686435759 1 0.6464894464
Of course what we want is to obtain the Camera Calibration 1 tag without having to roundtrip through adobe dng converter.

So if those Camera Color Calibration numbers are what we are looking for we need a way to transform it to Camera Calibration number and then we will have exact wb match...

[DeafEyeJedi]

Great work @Danne on getting this closer to being resolved!

[Danne]

I uploaded cr2hdr for windows and mac(could not compile on ubuntu) and like to test the --wb=exif setting.
https://bitbucket.org/Dannephoto/magic-lantern/downloads/cr2hdr_mac_win.zip

If anybody could test and report do as follows:

Lazy test:
Shoot a few dual iso CR2 stills in AWB(auto white balance) in different lighting, a room, outside, etc and then process those files with cr2hdr then compare the CR2 white balance against the DNG produced. White balance should be close to same:

Code: [Select]

cr2hdr --wb=exif *.CR2
Extensive test:
Shoot a dual iso CR2 still in every wb mode on your camera and then process and compare the files white balance. Test custom, kelvin, and the other modes and report if it works:

Code: [Select]

cr2hdr --wb=exif *.CR2
As stated before we still miss camera calibration tags so expect results to be close but not perfect when viewing the files.

[jml2017]

I tried the cr2hdr posted by Danne.

The procedure is as follow:
I took pictures of an indoor scene on a tripod with controlled constant lighting.
For each white balance I shot a standard CR2 and a Dual ISO 100-400 CR2.
I converted the dual iso, either using the cr2hdr compiled on Dec 4th, or with Danne's cr2hdr.
In both cases I used the option --wb=exif.
I read the color temperature after loading each picture in Rawtherapee using its Neutral preprocessing profile.

White Balance   "Standard exposure CR2"   "DUAL ISO CR2"   " 2017-12-04 00:01 cr2hdr –wb=exif DNG"   "Danne cr2hdr –wb=exif DNG"
Auto                              3046   3152   1691   3152
Daylight                         5203   5203   5203   5203
Shade                            6913   6913   6913   6913
Cloudy                           5978   5978   5978   5978
Tungsten                       3152   3152   3152   3152
White fluo                      3606   3606   3606   3606
Flash                             6296   6296   6296   6296
Custom                          5203   5203   5203   5203
Color Temp set to 2600   2594   2594   2594   2594

Conclusion: Danne's cr2hdr works well for AWB. Maybe the main download page for cr2hdr should be updated with that version?

Extra comment:
I also checked how repeatable the Color temperature was in AWB with this setup. I shot 8 standard images (no Dual ISO) in AWB. The standard deviation in Color temperature I see is only 4 K.

The small discrepancy (106 K) between the color temperature from the standard exposure and dual iso image in AWB is then probably due to the actual dual iso capture itself.

 

[elenhil]

I uploaded cr2hdr for windows and mac(could not compile on ubuntu) and like to test the --wb=exif setting.
https://bitbucket.org/Dannephoto/magic-lantern/downloads/cr2hdr_mac_win.zip

Where can I still find this newer version of cr2hdr?