Lightroom plugin cr2hdr v3.0 DEV (WIP)

Started by kichetof, March 18, 2014, 05:04:33 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

kichetof

That's a good question !


C:\Users\tof\Scripts\lr_cr2hdr\cr2hdr.lrplugin\bin>cr2hdr.exe --same-levels "E:\Photographies LCAP\Photographies\ML\same-levels\*-dualiso.dng" > "E:\Photographies LCAP\Photographies\ML\same-levels\test.txt"
E:\Photographies LCAP\Photographies\ML\same-levels\*-dualiso.dng: Invalid argument


test.txt

cr2hdr: a post processing tool for Dual ISO images

Last update: 8b9190b on 2015-11-14 16:44:35 UTC by a1ex:
cr2hdr: identify camera model from string (should fix issue #2403 ...

Active options:
--amaze-edge    : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs2x2         : apply 2x2 chroma smoothing in noisy and aliased areas (default)
--wb=graymax    : set AsShotNeutral by maximizing the number of gray pixels (default)
--same-levels   : Adjust output white levels to keep the same overall exposure
                  for all frames passed in a single command line
                  (useful to avoid flicker - for video or panoramas)

Input file      : E:\Photographies LCAP\Photographies\ML\same-levels\*-dualiso.dng
Camera          : Canon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mar
Matrix not found: EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mar
No table found for camera model: EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mar
dcraw could not open this file


so ... you're right! sh*""%t!

DeafEyeJedi

Love how Walter's usually right. He'd be one heck of a Detective! [emoji4]
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

kichetof

So... I need to find a way to pass a wildcard to cr2hdr --same-levels on Windows... Always windows!

With some beers, perhaps a genius idea will come!

Danne

I was thinking. Could you create a list of the files selected? Find command would work, ls command would not. Will break script after around 5000 files selected. From the list you could grab and diminish the list with a while loop until all files have gotten the same white level. This way same levels would work as it was supposed to from the beginning. I use this list while looping in MLP. Could write the code here when I, m at the computer.
Happy beering :D

Walter Schulz

@a1ex: Would it be a big deal to implement wildcards for --same-level into cr2hdr for windows? Or something like "--same-level <dir>"?

BestLittleStudio

Man, you guys run fast around here.

dfort, kichetof,

I did find the file location and located it in lightroom, the file did not auto import but I just went back to the full collection and it was there with the dualiso name. Unfortunately there is no difference between the two images.

I originally used the non beta, I saw no change, so tried the beta and saw the same issue, no change.

If I download the files from the camera to a sort folder and import from there then they are CR2 files. However, if I import directly from the camera then they come in as DNG files.

The CR2 files do not convert with the beta, and the DNG files, though they do convert, show no changes.

I have uploaded the original CR2 and DNG Files, along with the converted DNG files. All setting are at defaults.
You can find them here.

https://dl.dropboxusercontent.com/u/10158027/MagicLantern/IMG_3483-dualiso.TXT
https://dl.dropboxusercontent.com/u/10158027/MagicLantern/IMG_3483-dualiso.DNG
https://dl.dropboxusercontent.com/u/10158027/MagicLantern/IMG_3483.dng
https://dl.dropboxusercontent.com/u/10158027/MagicLantern/IMG_3480-dualiso.TXT
https://dl.dropboxusercontent.com/u/10158027/MagicLantern/IMG_3480-dualiso.DNG
https://dl.dropboxusercontent.com/u/10158027/MagicLantern/IMG_3480.dng
https://dl.dropboxusercontent.com/u/10158027/MagicLantern/151215-IMG_3483-70D.CR2
https://dl.dropboxusercontent.com/u/10158027/MagicLantern/151215-IMG_3480-70D.CR2

I included both an overexposed and an underexposed image.
70D.111A

dfort

I tried the files BestLittleStudio posted with cr2hdr on the command line and got the same errors he's reporting. I think it is an issue with 70D files. I've never seen that error before.

No camera model found with id 0x325

DeafEyeJedi

So does this mean I should try to do some similar test with 70D files to confirm this as well?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

kichetof

@Danne yes I can export a list of file, easy! But if we found a command line to simplify the process, I'll be happy!
Especially that's work on Mac... Windows always the same story! I've an idea, thanks beer, loop inside file with cmd for like for dcraw white balance! Maybe after a little night it will be more clearer!

@Walter I think that cr2hdr is capable to accept wildcard but it's the way that I launch the command that fail to works because on Mac it works well!

@dfort thanks to find the bug! Glad to heard that's not the plugin! Pfiou  8)

dfort

Quote from: DeafEyeJedi on December 15, 2015, 11:19:46 PM
So does this mean I should try to do some similar test with 70D files to confirm this as well?

That would be great--and please post the bug report with a sample file on bitbucket. I would do it but since I don't own a 70D and already have too many projects going on....

BestLittleStudio

Woohoo, I found a bug, wait, damn I found a bug.

Anything you need from me let me know but treat me like a toddler with this.
70D.111A

kichetof

Quote from: Walter Schulz on December 15, 2015, 07:58:34 PM
@a1ex: Would it be a big deal to implement wildcards for --same-level into cr2hdr for windows? Or something like "--same-level <dir>"?

Nothing different with or without --same-levels:

if (same_levels && num_files > 1)
    {
        /* Equalize white-black for all shots.
         *
         * Assuming all the pictures were shot at the same exposure settings,
         * this step will make sure they are all rendered identically (without flicker).
         *
         * However, for this to work, all the files must be passed in the same command line.
         *
         * We will use something close to maximum range among all files (with outlier filter).
         *
         * This should work even if the black level is not the same in all shots.
         */
       
        printf("\nEqualizing levels...\n");
       
        int* ranges = malloc(num_files * sizeof(ranges[0]));
        for (int i = 0; i < num_files; i++)
        {
            ranges[i] = whites[i] - blacks[i];
        }
        int new_range = kth_smallest_int(ranges, num_files, num_files * 8 / 9 - 1);

        for (int i = 0; i < num_files; i++)
        {
            char* input_file = argv[file_indices[i]];

            /* fixme: duplicate code */
            char out_filename[1000];
            snprintf(out_filename, sizeof(out_filename), "%s", input_file);
            int len = strlen(out_filename);
            out_filename[len-3] = 'D';
            out_filename[len-2] = 'N';
            out_filename[len-1] = 'G';

            int new_white = blacks[i] + new_range;
            printf("%-16s: %d ... %d\n", out_filename, blacks[i], new_white);
            set_white_level(out_filename, new_white);
        }
       
        free(ranges);
    }

https://bitbucket.org/hudson/magic-lantern/src/ab1e90c88b9515856f77bff0a88714d900918f39/modules/dual_iso/cr2hdr.c?at=cr2hdr&fileviewer=file-view-default#cr2hdr.c-886

So, I need to find the way to pass a wildcard to windows command through Lua ... (normally that should works fine... I don't know why not!)

Otherwise, I can use same way as Danne for MLP, with exiftool.

Walter Schulz

You're saying you can't pass something like "cr2hdr.exe *.cr2" to Windows CLI?

kichetof

I thought that the bug came from my external hard drive, but I just test it now, and confirm that wildcard cr2hdr.exe *.dng doesn't work.


C:\Users\tof\Scripts\lr_cr2hdr\cr2hdr.lrplugin\bin>cr2hdr.exe C:\Users\tof\Scripts\dng\*.dng
cr2hdr: a post processing tool for Dual ISO images

Last update: 8b9190b on 2015-11-14 16:44:35 UTC by a1ex:
cr2hdr: identify camera model from string (should fix issue #2403 ...

Active options:
--amaze-edge    : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs2x2         : apply 2x2 chroma smoothing in noisy and aliased areas (default)--wb=graymax    : set AsShotNeutral by maximizing the number of gray pixels (default)

Input file      : C:\Users\tof\Scripts\dng\*.dng
C:\Users\tof\Scripts\dng\*.dng: Invalid argument
Camera          : Canon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D MarMatrix not found: EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mar
No table found for camera model: EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mark IIICanon EOS 5D Mar
dcraw could not open this file


From cr2hdr source , no bug appear, loop inside all parsed files.

I think that the Windows command that bug with wildcard... need to be investigate!

Walter Schulz

Confused ...
Does "cr2hdr.exe *.cr2" work when called from the commandline (just CMD.EXE and cr2hdr.exe (+dcraw.exe + exiftool.exe) placed in dir where CRs are located?
Or are we just talking about calls via LUA?

kichetof

Sorry for confusion. My latest test was running on Windows CMD, not Lua.

dcraw.exe *.dng
doesn't work too!

dir *.dng
dir *00.dng

works well!
Maybe a bug from dcraw ?

Walter Schulz

Testing now ... hold the line ...

You're right. Replaced dcraw.exe with older version and wildcard problem is gone.

Walter Schulz

Downloaded dcraw for windows from here and it looks like working:
http://www.centrostudiprogressofotografico.it/en/dcraw/
Read remark "Wildcards * are supported". ;-)

kichetof

Thanks Walter!! You're the boss!

I'll update bin and test it before continue to solve same-levels on entire path to avoid external bug !

We need to check with @dfort why compile doesn't works with wildcard

Walter Schulz

Seems to be a known issue. You read the "Wildcards * are supported" part on linked page?

kichetof

Oh yeah effectively! Maybe CS fix this bug.

Now it's time to find the way to pass wildcard to command:

cr2hdr.exe --amaze-edge --cs2x2 --no-bad-pix --fullres --alias-map --stripe-fix --wb=graymax --same-levels "E:\Photographies LCAP\Photographies\ML\dualiso\*.DNG" > "E:\Photographies LCAP\Photographies\ML\dualiso\same-levels2.TXT"

bad syntax on folder.. I don't have eyes in front of hole! Glad the weekend arrives to get some sleep

Walter Schulz

It's a long way to Tipperary ...

Running --same-levels without "> [...]" to avoid cr2hdr messing up. And using <Path\Filenames>. Result: Good!

E:\Temp\tofbin>cr2hdr.exe --amaze-edge --cs2x2 --no-bad-pix --fullres --alias-map --stripe-fix --wb=graymax --same-levels e:\temp\tof1\IMG_1985.CR2 e:\temp\tof1\IMG_1986.CR2
cr2hdr: a post processing tool for Dual ISO images

Last update: 8b9190b on 2015-11-14 16:44:35 UTC by a1ex:
cr2hdr: identify camera model from string (should fix issue #2403 ...

Active options:
--amaze-edge    : use a temporary demosaic step (AMaZE) followed by edge-directe
d interpolation (default)
--cs2x2         : apply 2x2 chroma smoothing in noisy and aliased areas (default
)
--no-bad-pix    : disable bad pixel fixing (try it if you shoot stars)
--wb=graymax    : set AsShotNeutral by maximizing the number of gray pixels (def
ault)
--same-levels   : Adjust output white levels to keep the same overall exposure
                  for all frames passed in a single command line
                  (useful to avoid flicker - for video or panoramas)

Input file      : e:\temp\tof1\IMG_1985.CR2
Camera          : Canon EOS 7D
Full size       : 5360 x 3516
Active area     : 5202 x 3465
Black borders   : 158 left, 51 top
Black level     : 2047
ISO pattern     : dBBd GBRG
White levels    : 10000 12328
Noise levels    : 5.30 11.12 11.98 5.15 (14-bit)
ISO difference  : 3.04 EV (824)
Black delta     : 3.62
Black adjust    : -20
Dynamic range   : 10.59 (+) 9.74 => 12.79 EV (in theory)
AMaZE interpolation ...
Amaze took 1.72 s
Edge-directed interpolation...
Semi-overexposed: 35.40%
Deep shadows    : 35.76%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap     : 4.6 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level     : 72.31 (20-bit), ideally 71.88
Dynamic range   : 12.78 EV (cooked)
Black adjust    : -15
AsShotNeutral   : 0.44 1 0.80, 6796K/g=1.12 (gray max)
Output file     : e:\temp\tof1\IMG_1985.DNG
e:\temp\tof1\IMG_1985.DNG: copying EXIF from e:\temp\tof1\IMG_1985.CR2

Input file      : e:\temp\tof1\IMG_1986.CR2
Camera          : Canon EOS 7D
Full size       : 5360 x 3516
Active area     : 5202 x 3465
Black borders   : 158 left, 51 top
Black level     : 2047
ISO pattern     : dBBd GBRG
White levels    : 10000 12328
Noise levels    : 5.30 10.85 11.99 5.18 (14-bit)
ISO difference  : 3.05 EV (826)
Black delta     : 3.83
Black adjust    : -21
Dynamic range   : 10.59 (+) 9.74 => 12.79 EV (in theory)
AMaZE interpolation ...
Amaze took 1.72 s
Edge-directed interpolation...
Semi-overexposed: 35.39%
Deep shadows    : 35.33%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap     : 4.5 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level     : 71.64 (20-bit), ideally 71.03
Dynamic range   : 12.79 EV (cooked)
Black adjust    : -16
AsShotNeutral   : 0.44 1 0.80, 6796K/g=1.12 (custom)
Output file     : e:\temp\tof1\IMG_1986.DNG
e:\temp\tof1\IMG_1986.DNG: copying EXIF from e:\temp\tof1\IMG_1986.CR2

Equalizing levels...
e:\temp\tof1\IMG_1985.DNG: 8190 ... 40002
e:\temp\tof1\IMG_1986.DNG: 8190 ... 40002


Now with wildcard: Nope, not looking any good!

E:\Temp\tofbin>cr2hdr.exe --amaze-edge --cs2x2 --no-bad-pix --fullres --alias-map --stripe-fix --wb=graymax --same-levels e:\temp\tof1\*.cr2
cr2hdr: a post processing tool for Dual ISO images

Last update: 8b9190b on 2015-11-14 16:44:35 UTC by a1ex:
cr2hdr: identify camera model from string (should fix issue #2403 ...

Active options:
--amaze-edge    : use a temporary demosaic step (AMaZE) followed by edge-directe
d interpolation (default)
--cs2x2         : apply 2x2 chroma smoothing in noisy and aliased areas (default
)
--no-bad-pix    : disable bad pixel fixing (try it if you shoot stars)
--wb=graymax    : set AsShotNeutral by maximizing the number of gray pixels (def
ault)
--same-levels   : Adjust output white levels to keep the same overall exposure
                  for all frames passed in a single command line
                  (useful to avoid flicker - for video or panoramas)

Input file      : e:\temp\tof1\*.cr2
Camera          : Canon EOS 7DCanon EOS 7D
Matrix not found: EOS 7DCanon EOS 7D
No table found for camera model: EOS 7DCanon EOS 7D
Full size       : 5360 x 3516
Active area     : 5202 x 3465
Black borders   : 158 left, 51 top
Black level     : 2047
ISO pattern     : dBBd GBRG
White levels    : 10000 12328
Noise levels    : 5.30 11.12 11.98 5.15 (14-bit)
ISO difference  : 3.04 EV (824)
Black delta     : 3.62
Black adjust    : -20
Dynamic range   : 10.59 (+) 9.74 => 12.79 EV (in theory)
AMaZE interpolation ...
Amaze took 1.72 s
Edge-directed interpolation...
Semi-overexposed: 35.40%
Deep shadows    : 35.76%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap     : 4.6 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level     : 72.31 (20-bit), ideally 71.88
Dynamic range   : 12.78 EV (cooked)
Black adjust    : -15
AsShotNeutral   : 0.44 1 0.81, 22999K/g=-1.#J (gray max)
Output file     : e:\temp\tof1\*.DNG
e:\temp\tof1\*.DNG: copying EXIF from e:\temp\tof1\*.cr2
No matching files
**WARNING** exiftool couldn't update DNG tag data


Way to go and I think we need a word from a1ex.



a1ex

On Linux and Mac, wildcards are expanded by the shell.

On Windows, they are not...

Walter Schulz

Thanks! Is there a suitable way to bypass this limit in cr2hdr code? Dcraw seems to have had the same/related error (? IANAP!) but it was solved.
EDIT: On second thought ... not that related. Dcraw.exe had problems running "dcraw *.cr2" and cr2hdr.exe will do just fine in this regard.

a1ex

Yes, by expanding the wildcards manually, or by configuring the C library to do it for you. I'm not sure what's the best way to do this under Windows (a few minutes of searching didn't bring a clear answer), but here are some useful links:
https://msdn.microsoft.com/en-us/library/8bch7bkk.aspx
http://mingw.5.n7.nabble.com/command-line-wildcard-expansion-gets-turned-off-by-wxWidgets-td1889.html
http://stackoverflow.com/questions/12501761/passing-multple-files-with-asterisk-to-python-shell-in-windows

Expanding the wildcards externally will cause trouble, because, on Windows, command-line length is limited.

Quote from: Walter Schulz on December 16, 2015, 07:36:28 PM
Dcraw seems to have had the same/related error (? AINAP!) but it was solved.

Latest dcraw compiled for Windows:

Z:\dcraw>dcraw *.DNG
*.DNG: Invalid argument