Magic Lantern Forum

Using Magic Lantern => Post-processing Workflow => HDR and Dual ISO Postprocessing => Topic started by: kidfob on April 26, 2021, 01:58:37 AM

Title: cr2hdr error on some DNGs
Post by: kidfob on April 26, 2021, 01:58:37 AM
Im noticing cr2hdr crashes on certain DNGs with cr2hdr. DNG attached so others can reproduce:
https://drive.google.com/file/d/1iRSCLh4NO1iwsMqrAmdfMm4KOCFazAia/view?usp=sharing
Title: Re: cr2hdr error on some DNGs
Post by: names_are_hard on April 26, 2021, 04:05:27 AM
Confirmed, thanks for the sample file.


Input file      : M22-1343_000000.dng
Camera          : Canon EOS 5D Mark III
Full size       : 1808 x 2268
Active area     : 5424 x 2268
Black borders   : N/A
ISO pattern     : dBBd RGGB
==759831== Invalid read of size 2
==759831==    at 0x116922: raw_get_pixel16 (cr2hdr.c:467)
==759831==    by 0x116922: white_detect (cr2hdr.c:943)
==759831==    by 0x10B65B: hdr_interpolate (cr2hdr.c:2041)
==759831==    by 0x10B65B: main (cr2hdr.c:773)
==759831==  Address 0x4f613e8 is 1,824,752 bytes inside an unallocated block of size 1,831,920 in arena "client"


raw_info.buffer is not valid at point of use in raw_get_pixel16().  In hdr_interpolate(), we do this:


    if (!rggb) /* this code assumes RGGB, so we need to skip one line */
    {
        raw_info.buffer += raw_info.pitch;


For this file, rggb is true, so we don't adjust the buffer...  but it's not clear to me where it's initially set (it's a global and annoying to track, and I'm unfamiliar with this code).
Title: Re: cr2hdr error on some DNGs
Post by: a1ex on April 26, 2021, 11:56:32 AM
The buffer is valid, but... for some reason, cr2hdr is trying to access some pixels at negative coordinates:


buf:0x4771028 x:-3616 y:0
==11898== Invalid read of size 2
==11898==    at 0x804F2E8: raw_get_pixel16 (cr2hdr.c:473)
...
buf:0x4771028 x:-3601 y:15
==11898== Invalid write of size 4
==11898==    at 0x804F360: white_balance_gray (cr2hdr.c:3589)
...


The problem comes from here (output size larger than full image size):

dcraw -i -v M22-1343_000000.dng
...
Pixel Aspect Ratio: 3.000000
Full size:   1808 x 2268
Image size:  1808 x 2268
Output size: 5424 x 2268


Note that exiftool identifies the active area correctly:

Active Area                     : 0 0 2268 1808
Image Size                      : 1808x2268


As cr2hdr attempts to detect the active area by parsing dcraw output, that's where it fails (that's the reason for trying to access pixels at negative coordinates). Using exiftool for autodetection is technically possible, and probably a better approach, but somebody has to write the code. I'm not able to help with this for the time being, but I'll take a look if somebody comes up with a patch.

Edit: apparently, using the "Image size" field from dcraw instead of "Output size" fixes the issue. Will commit the fix later this week.

In the meantime, you may want to try mlv_dump for preprocessing, as recommended in the Dual ISO topic:

mlv_dump clip.mlv --dng --no-fixcp --no-stripes
Title: Re: cr2hdr error on some DNGs
Post by: Danne on April 26, 2021, 01:55:12 PM
Nice. How to keep track of coming commit? Will you post it here?
Title: Re: cr2hdr error on some DNGs
Post by: theBilalFakhouri on April 26, 2021, 05:46:27 PM
If you still want to use MLVApp, another temporarily solution is to set "Height Stretch" in "Transformation" tab back to 1.00 (In MLVApp), export as DNG sequence and process the DNGs in cr2hdr.

You need to stretch the footage manually in your editing software to get correct aspect ratio.
Title: Re: cr2hdr error on some DNGs
Post by: kidfob on April 26, 2021, 10:23:22 PM
Thanks for the great feedback all!

One other thing I noticed was the EXIF resolution data seems to change to low res for processed DNGs. The DNGs open fine in Lightroom (maybe Lightroom ignores the EXIF data), but open low res in Windows photo app.

Images attached:
https://drive.google.com/file/d/1_1Znd47wZz-ShT8BMHlt1pjOM45RFjIn/view?usp=sharing
https://drive.google.com/file/d/1ytaFNuc-bLgsvMoHumez-toTzGUvooMS/view?usp=sharing
Title: Re: cr2hdr error on some DNGs
Post by: kidfob on April 29, 2021, 04:25:27 AM
Any thoughts on the EXIF data issue?
Title: Re: cr2hdr error on some DNGs
Post by: names_are_hard on April 29, 2021, 04:46:16 PM
I don't see any EXIF data problem.  I suspect what's happening here is that some applications choose to display the embedded jpg preview (which is 384x252 in size and correctly described as such in EXIF).  The raw image is described in EXIF as 3584x1500.  Compare the output of "exiftool sample.dng" and "exiftool -ee sample.dng".

I would guess Windows photo app can't handle raw so opens the jpg.
Title: Re: cr2hdr error on some DNGs
Post by: kidfob on May 02, 2021, 09:48:47 PM
@Danne and @a1ex, just checking to see if the bug fix has been committed yet.

Thanks much!
Title: Re: cr2hdr error on some DNGs
Post by: Danne on May 02, 2021, 10:09:09 PM
I have no time checking this issue.
Cannot remember last I saw a commit from a1ex so better keep expectations low. I am sure he writes back if he finds time for a fix.