Author Topic: Lightroom plugin cr2hdr v3.0 DEV (WIP)  (Read 692591 times)

kichetof

  • Contributor
  • Senior
  • *****
  • Posts: 469
  • Take a beer and enjoy it!
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1000 on: December 15, 2015, 05:06:19 PM »
That's a good question !

Code: [Select]
C:\Users\tof\Scripts\lr_cr2hdr\cr2hdr.lrplugin\bin>cr2hdr.exe --same-levels "E:\Photographies LCAP\Photographies\ML\same-levels\*-dualiso.dng" > "E:\Photographies LCAP\Photographies\ML\same-levels\test.txt"
E:\Photographies LCAP\Photographies\ML\same-levels\*-dualiso.dng: Invalid argument

test.txt
Code: [Select]
cr2hdr: a post processing tool for Dual ISO images

Last update: 8b9190b on 2015-11-14 16:44:35 UTC by a1ex:
cr2hdr: identify camera model from string (should fix issue #2403 ...

Active options:
--amaze-edge    : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs2x2         : apply 2x2 chroma smoothing in noisy and aliased areas (default)
--wb=graymax    : set AsShotNeutral by maximizing the number of gray pixels (default)
--same-levels   : Adjust output white levels to keep the same overall exposure
                  for all frames passed in a single command line
                  (useful to avoid flicker - for video or panoramas)

Input file      : E:\Photographies LCAP\Photographies\ML\same-levels\*-dualiso.dng
Camera          : Canon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mar
Matrix not found: EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mar
No table found for camera model: EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mar
dcraw could not open this file

so ... you're right! sh*""%t!

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3411
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1001 on: December 15, 2015, 05:14:04 PM »
Love how Walter's usually right. He'd be one heck of a Detective! [emoji4]
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

kichetof

  • Contributor
  • Senior
  • *****
  • Posts: 469
  • Take a beer and enjoy it!
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1002 on: December 15, 2015, 05:32:27 PM »
So... I need to find a way to pass a wildcard to cr2hdr --same-levels on Windows... Always windows!

With some beers, perhaps a genius idea will come!

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7401
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1003 on: December 15, 2015, 07:50:30 PM »
I was thinking. Could you create a list of the files selected? Find command would work, ls command would not. Will break script after around 5000 files selected. From the list you could grab and diminish the list with a while loop until all files have gotten the same white level. This way same levels would work as it was supposed to from the beginning. I use this list while looping in MLP. Could write the code here when I, m at the computer.
Happy beering :D

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 8495
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1004 on: December 15, 2015, 07:58:34 PM »
@a1ex: Would it be a big deal to implement wildcards for --same-level into cr2hdr for windows? Or something like "--same-level <dir>"?

BestLittleStudio

  • New to the forum
  • *
  • Posts: 5
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1005 on: December 15, 2015, 08:03:26 PM »
Man, you guys run fast around here.

dfort, kichetof,

I did find the file location and located it in lightroom, the file did not auto import but I just went back to the full collection and it was there with the dualiso name. Unfortunately there is no difference between the two images.

I originally used the non beta, I saw no change, so tried the beta and saw the same issue, no change.

If I download the files from the camera to a sort folder and import from there then they are CR2 files. However, if I import directly from the camera then they come in as DNG files.

The CR2 files do not convert with the beta, and the DNG files, though they do convert, show no changes.

I have uploaded the original CR2 and DNG Files, along with the converted DNG files. All setting are at defaults.
You can find them here.

https://dl.dropboxusercontent.com/u/10158027/MagicLantern/IMG_3483-dualiso.TXT
https://dl.dropboxusercontent.com/u/10158027/MagicLantern/IMG_3483-dualiso.DNG
https://dl.dropboxusercontent.com/u/10158027/MagicLantern/IMG_3483.dng
https://dl.dropboxusercontent.com/u/10158027/MagicLantern/IMG_3480-dualiso.TXT
https://dl.dropboxusercontent.com/u/10158027/MagicLantern/IMG_3480-dualiso.DNG
https://dl.dropboxusercontent.com/u/10158027/MagicLantern/IMG_3480.dng
https://dl.dropboxusercontent.com/u/10158027/MagicLantern/151215-IMG_3483-70D.CR2
https://dl.dropboxusercontent.com/u/10158027/MagicLantern/151215-IMG_3480-70D.CR2

I included both an overexposed and an underexposed image.
70D.111A

dfort

  • Guest
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1006 on: December 15, 2015, 11:06:55 PM »
I tried the files BestLittleStudio posted with cr2hdr on the command line and got the same errors he's reporting. I think it is an issue with 70D files. I've never seen that error before.

Code: [Select]
No camera model found with id 0x325

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3411
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1007 on: December 15, 2015, 11:19:46 PM »
So does this mean I should try to do some similar test with 70D files to confirm this as well?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

kichetof

  • Contributor
  • Senior
  • *****
  • Posts: 469
  • Take a beer and enjoy it!
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1008 on: December 15, 2015, 11:28:22 PM »
@Danne yes I can export a list of file, easy! But if we found a command line to simplify the process, I'll be happy!
Especially that's work on Mac... Windows always the same story! I've an idea, thanks beer, loop inside file with cmd for like for dcraw white balance! Maybe after a little night it will be more clearer!

@Walter I think that cr2hdr is capable to accept wildcard but it's the way that I launch the command that fail to works because on Mac it works well!

@dfort thanks to find the bug! Glad to heard that's not the plugin! Pfiou  8)

dfort

  • Guest
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1009 on: December 15, 2015, 11:48:47 PM »
So does this mean I should try to do some similar test with 70D files to confirm this as well?

That would be great--and please post the bug report with a sample file on bitbucket. I would do it but since I don't own a 70D and already have too many projects going on....

BestLittleStudio

  • New to the forum
  • *
  • Posts: 5
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1010 on: December 16, 2015, 02:32:36 AM »
Woohoo, I found a bug, wait, damn I found a bug.

Anything you need from me let me know but treat me like a toddler with this.
70D.111A

kichetof

  • Contributor
  • Senior
  • *****
  • Posts: 469
  • Take a beer and enjoy it!
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1011 on: December 16, 2015, 02:39:12 PM »
@a1ex: Would it be a big deal to implement wildcards for --same-level into cr2hdr for windows? Or something like "--same-level <dir>"?

Nothing different with or without --same-levels:
Code: [Select]
if (same_levels && num_files > 1)
    {
        /* Equalize white-black for all shots.
         *
         * Assuming all the pictures were shot at the same exposure settings,
         * this step will make sure they are all rendered identically (without flicker).
         *
         * However, for this to work, all the files must be passed in the same command line.
         *
         * We will use something close to maximum range among all files (with outlier filter).
         *
         * This should work even if the black level is not the same in all shots.
         */
       
        printf("\nEqualizing levels...\n");
       
        int* ranges = malloc(num_files * sizeof(ranges[0]));
        for (int i = 0; i < num_files; i++)
        {
            ranges[i] = whites[i] - blacks[i];
        }
        int new_range = kth_smallest_int(ranges, num_files, num_files * 8 / 9 - 1);

        for (int i = 0; i < num_files; i++)
        {
            char* input_file = argv[file_indices[i]];

            /* fixme: duplicate code */
            char out_filename[1000];
            snprintf(out_filename, sizeof(out_filename), "%s", input_file);
            int len = strlen(out_filename);
            out_filename[len-3] = 'D';
            out_filename[len-2] = 'N';
            out_filename[len-1] = 'G';

            int new_white = blacks[i] + new_range;
            printf("%-16s: %d ... %d\n", out_filename, blacks[i], new_white);
            set_white_level(out_filename, new_white);
        }
       
        free(ranges);
    }
https://bitbucket.org/hudson/magic-lantern/src/ab1e90c88b9515856f77bff0a88714d900918f39/modules/dual_iso/cr2hdr.c?at=cr2hdr&fileviewer=file-view-default#cr2hdr.c-886

So, I need to find the way to pass a wildcard to windows command through Lua ... (normally that should works fine... I don't know why not!)

Otherwise, I can use same way as Danne for MLP, with exiftool.

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 8495
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1012 on: December 16, 2015, 02:43:37 PM »
You're saying you can't pass something like "cr2hdr.exe *.cr2" to Windows CLI?

kichetof

  • Contributor
  • Senior
  • *****
  • Posts: 469
  • Take a beer and enjoy it!
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1013 on: December 16, 2015, 02:59:52 PM »
I thought that the bug came from my external hard drive, but I just test it now, and confirm that wildcard cr2hdr.exe *.dng doesn't work.

Code: [Select]
C:\Users\tof\Scripts\lr_cr2hdr\cr2hdr.lrplugin\bin>cr2hdr.exe C:\Users\tof\Scripts\dng\*.dng
cr2hdr: a post processing tool for Dual ISO images

Last update: 8b9190b on 2015-11-14 16:44:35 UTC by a1ex:
cr2hdr: identify camera model from string (should fix issue #2403 ...

Active options:
--amaze-edge    : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs2x2         : apply 2x2 chroma smoothing in noisy and aliased areas (default)--wb=graymax    : set AsShotNeutral by maximizing the number of gray pixels (default)

Input file      : C:\Users\tof\Scripts\dng\*.dng
C:\Users\tof\Scripts\dng\*.dng: Invalid argument
Camera          : Canon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D MarMatrix not found: EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mar
No table found for camera model: EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mar
dcraw could not open this file

From cr2hdr source , no bug appear, loop inside all parsed files.

I think that the Windows command that bug with wildcard... need to be investigate!

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 8495
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1014 on: December 16, 2015, 03:07:17 PM »
Confused ...
Does "cr2hdr.exe *.cr2" work when called from the commandline (just CMD.EXE and cr2hdr.exe (+dcraw.exe + exiftool.exe) placed in dir where CRs are located?
Or are we just talking about calls via LUA?

kichetof

  • Contributor
  • Senior
  • *****
  • Posts: 469
  • Take a beer and enjoy it!
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1015 on: December 16, 2015, 03:23:34 PM »
Sorry for confusion. My latest test was running on Windows CMD, not Lua.

Code: [Select]
dcraw.exe *.dngdoesn't work too!

Code: [Select]
dir *.dng
dir *00.dng
works well!
Maybe a bug from dcraw ?

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 8495
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1016 on: December 16, 2015, 03:27:05 PM »
Testing now ... hold the line ...

You're right. Replaced dcraw.exe with older version and wildcard problem is gone.

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 8495
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1017 on: December 16, 2015, 03:57:20 PM »
Downloaded dcraw for windows from here and it looks like working:
http://www.centrostudiprogressofotografico.it/en/dcraw/
Read remark "Wildcards * are supported". ;-)

kichetof

  • Contributor
  • Senior
  • *****
  • Posts: 469
  • Take a beer and enjoy it!
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1018 on: December 16, 2015, 04:05:45 PM »
Thanks Walter!! You're the boss!

I'll update bin and test it before continue to solve same-levels on entire path to avoid external bug !

We need to check with @dfort why compile doesn't works with wildcard

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 8495
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1019 on: December 16, 2015, 04:23:33 PM »
Seems to be a known issue. You read the "Wildcards * are supported" part on linked page?

kichetof

  • Contributor
  • Senior
  • *****
  • Posts: 469
  • Take a beer and enjoy it!
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1020 on: December 16, 2015, 04:29:04 PM »
Oh yeah effectively! Maybe CS fix this bug.

Now it's time to find the way to pass wildcard to command:

Code: [Select]
cr2hdr.exe --amaze-edge --cs2x2 --no-bad-pix --fullres --alias-map --stripe-fix --wb=graymax --same-levels "E:\Photographies LCAP\Photographies\ML\dualiso\*.DNG" > "E:\Photographies LCAP\Photographies\ML\dualiso\same-levels2.TXT"
bad syntax on folder.. I don't have eyes in front of hole! Glad the weekend arrives to get some sleep

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 8495
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1021 on: December 16, 2015, 07:23:32 PM »
It's a long way to Tipperary ...

Running --same-levels without "> [...]" to avoid cr2hdr messing up. And using <Path\Filenames>. Result: Good!
Code: [Select]
E:\Temp\tofbin>cr2hdr.exe --amaze-edge --cs2x2 --no-bad-pix --fullres --alias-map --stripe-fix --wb=graymax --same-levels e:\temp\tof1\IMG_1985.CR2 e:\temp\tof1\IMG_1986.CR2
cr2hdr: a post processing tool for Dual ISO images

Last update: 8b9190b on 2015-11-14 16:44:35 UTC by a1ex:
cr2hdr: identify camera model from string (should fix issue #2403 ...

Active options:
--amaze-edge    : use a temporary demosaic step (AMaZE) followed by edge-directe
d interpolation (default)
--cs2x2         : apply 2x2 chroma smoothing in noisy and aliased areas (default
)
--no-bad-pix    : disable bad pixel fixing (try it if you shoot stars)
--wb=graymax    : set AsShotNeutral by maximizing the number of gray pixels (def
ault)
--same-levels   : Adjust output white levels to keep the same overall exposure
                  for all frames passed in a single command line
                  (useful to avoid flicker - for video or panoramas)

Input file      : e:\temp\tof1\IMG_1985.CR2
Camera          : Canon EOS 7D
Full size       : 5360 x 3516
Active area     : 5202 x 3465
Black borders   : 158 left, 51 top
Black level     : 2047
ISO pattern     : dBBd GBRG
White levels    : 10000 12328
Noise levels    : 5.30 11.12 11.98 5.15 (14-bit)
ISO difference  : 3.04 EV (824)
Black delta     : 3.62
Black adjust    : -20
Dynamic range   : 10.59 (+) 9.74 => 12.79 EV (in theory)
AMaZE interpolation ...
Amaze took 1.72 s
Edge-directed interpolation...
Semi-overexposed: 35.40%
Deep shadows    : 35.76%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap     : 4.6 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level     : 72.31 (20-bit), ideally 71.88
Dynamic range   : 12.78 EV (cooked)
Black adjust    : -15
AsShotNeutral   : 0.44 1 0.80, 6796K/g=1.12 (gray max)
Output file     : e:\temp\tof1\IMG_1985.DNG
e:\temp\tof1\IMG_1985.DNG: copying EXIF from e:\temp\tof1\IMG_1985.CR2

Input file      : e:\temp\tof1\IMG_1986.CR2
Camera          : Canon EOS 7D
Full size       : 5360 x 3516
Active area     : 5202 x 3465
Black borders   : 158 left, 51 top
Black level     : 2047
ISO pattern     : dBBd GBRG
White levels    : 10000 12328
Noise levels    : 5.30 10.85 11.99 5.18 (14-bit)
ISO difference  : 3.05 EV (826)
Black delta     : 3.83
Black adjust    : -21
Dynamic range   : 10.59 (+) 9.74 => 12.79 EV (in theory)
AMaZE interpolation ...
Amaze took 1.72 s
Edge-directed interpolation...
Semi-overexposed: 35.39%
Deep shadows    : 35.33%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap     : 4.5 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level     : 71.64 (20-bit), ideally 71.03
Dynamic range   : 12.79 EV (cooked)
Black adjust    : -16
AsShotNeutral   : 0.44 1 0.80, 6796K/g=1.12 (custom)
Output file     : e:\temp\tof1\IMG_1986.DNG
e:\temp\tof1\IMG_1986.DNG: copying EXIF from e:\temp\tof1\IMG_1986.CR2

Equalizing levels...
e:\temp\tof1\IMG_1985.DNG: 8190 ... 40002
e:\temp\tof1\IMG_1986.DNG: 8190 ... 40002

Now with wildcard: Nope, not looking any good!
Code: [Select]
E:\Temp\tofbin>cr2hdr.exe --amaze-edge --cs2x2 --no-bad-pix --fullres --alias-map --stripe-fix --wb=graymax --same-levels e:\temp\tof1\*.cr2
cr2hdr: a post processing tool for Dual ISO images

Last update: 8b9190b on 2015-11-14 16:44:35 UTC by a1ex:
cr2hdr: identify camera model from string (should fix issue #2403 ...

Active options:
--amaze-edge    : use a temporary demosaic step (AMaZE) followed by edge-directe
d interpolation (default)
--cs2x2         : apply 2x2 chroma smoothing in noisy and aliased areas (default
)
--no-bad-pix    : disable bad pixel fixing (try it if you shoot stars)
--wb=graymax    : set AsShotNeutral by maximizing the number of gray pixels (def
ault)
--same-levels   : Adjust output white levels to keep the same overall exposure
                  for all frames passed in a single command line
                  (useful to avoid flicker - for video or panoramas)

Input file      : e:\temp\tof1\*.cr2
Camera          : Canon EOS 7DCanon EOS 7D
Matrix not found: EOS 7DCanon EOS 7D
No table found for camera model: EOS 7DCanon EOS 7D
Full size       : 5360 x 3516
Active area     : 5202 x 3465
Black borders   : 158 left, 51 top
Black level     : 2047
ISO pattern     : dBBd GBRG
White levels    : 10000 12328
Noise levels    : 5.30 11.12 11.98 5.15 (14-bit)
ISO difference  : 3.04 EV (824)
Black delta     : 3.62
Black adjust    : -20
Dynamic range   : 10.59 (+) 9.74 => 12.79 EV (in theory)
AMaZE interpolation ...
Amaze took 1.72 s
Edge-directed interpolation...
Semi-overexposed: 35.40%
Deep shadows    : 35.76%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap     : 4.6 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level     : 72.31 (20-bit), ideally 71.88
Dynamic range   : 12.78 EV (cooked)
Black adjust    : -15
AsShotNeutral   : 0.44 1 0.81, 22999K/g=-1.#J (gray max)
Output file     : e:\temp\tof1\*.DNG
e:\temp\tof1\*.DNG: copying EXIF from e:\temp\tof1\*.cr2
No matching files
**WARNING** exiftool couldn't update DNG tag data

Way to go and I think we need a word from a1ex.



a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1022 on: December 16, 2015, 07:27:41 PM »
On Linux and Mac, wildcards are expanded by the shell.

On Windows, they are not...

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 8495
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1023 on: December 16, 2015, 07:36:28 PM »
Thanks! Is there a suitable way to bypass this limit in cr2hdr code? Dcraw seems to have had the same/related error (? IANAP!) but it was solved.
EDIT: On second thought ... not that related. Dcraw.exe had problems running "dcraw *.cr2" and cr2hdr.exe will do just fine in this regard.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Lightroom plugin cr2hdr v3.0 BETA3 (WIP)
« Reply #1024 on: December 16, 2015, 07:50:23 PM »
Yes, by expanding the wildcards manually, or by configuring the C library to do it for you. I'm not sure what's the best way to do this under Windows (a few minutes of searching didn't bring a clear answer), but here are some useful links:
https://msdn.microsoft.com/en-us/library/8bch7bkk.aspx
http://mingw.5.n7.nabble.com/command-line-wildcard-expansion-gets-turned-off-by-wxWidgets-td1889.html
http://stackoverflow.com/questions/12501761/passing-multple-files-with-asterisk-to-python-shell-in-windows

Expanding the wildcards externally will cause trouble, because, on Windows, command-line length is limited.

Dcraw seems to have had the same/related error (? AINAP!) but it was solved.

Latest dcraw compiled for Windows:
Code: [Select]
Z:\dcraw>dcraw *.DNG
*.DNG: Invalid argument