Lightroom plugin cr2hdr v3.0 DEV (WIP)

[kichetof]

uh ‽ I don't know what the problem ! Maybe try to convert your dual-iso with cr2hdr in the Terminal and compare both

[Danne]

Yea I tried another converter and the problem is in the cr2hdr binary I think. I showed this in my reply #371 here

[kichetof]

Could you run cr2hdr without any parameters and no pictures. Tell us the 2 version of the binary

[Danne]

Hi Kitchehof!

Here is the original file 2560x1090 3x zoom mode

https://drive.google.com/file/d/0B4tCJMlOYfirOWtiSWstNWUteVk/edit?usp=sharing

With both lightroom binary (310e39f on 2014-02-27) and other tools like cr2hdr 2.0 (2f4e2df 20113-12-02) for mac put together from A.D I get a thin white line at the bottom. Earlier versions than 2f4e2df 20113-12-02 seems not to have the problem.
https://bitbucket.org/a_d_/magic-lantern/downloads/



Here,s some text from the mac app 2.0. It says cr2hdr version 2f4e2df 20113-12-02 (gives a white line at the bottom)



With an older mac app, 1.4 I don,t get any white lines. Couldn,t find out what cr2hdr version. No white line with MLVmystic either which are giving the best fullres results at the moment.

No white line



text from 1.4 version(no white line)



Conclusion. Something happened with the later binary versions, probably starting from this version 2f4e2df 20113-12-02 and it shows when converting dual iso movie files in 3x zoom mode every now and then. Could this be fixed you think?


/D

[ayshih]

First, don't use a cr2hdr built in 2013(!)

Second, don't refer to "2.0" and 1.4" when referring to the version of cr2hdr, because it's not versioned like that.  I presume you are referring to the version of the Lightroom plugin, but that doesn't immediately reveal what version of cr2hdr is actually being used.

Third, please do what kichetof suggested: in your two cases with different behavior, run the respective cr2hdr executables from Terminal, without giving it any input files.  The first few lines should look like:

Code: [Select]

cr2hdr: a post processing tool for Dual ISO images

Last update: f381b87 on 2014-07-06 09:46:41 UTC by phil:
Fix output (-o) filename value passed to Adobe DNG Converter wh...

No input files.


[Danne]

I don, t know how to execute from terminal. This is my way of trying to explain.
I provided the original dng. My point is that after the 2013 date that I found ALL cr2hdr versions I used are giving a strange white bottom line when converting dual iso movie files filmed in 3x mode. Not in every picture. It comes and goes about avery third or fourth picture. Both in the lightroom plugin and in the mac converter. Maybe something to do with the interpolation method, I don, t know.
If you like Ayshih or Kitchehof please run the provided original in the plugin and see for yourself.
I modified some of my text according to Ayshis response above.
Thanks Kitchehof.
//D

[kichetof]

@Danne
Try to use this cr2hdr binary https://bitbucket.org/kichetof/lr_cr2hdr/downloads/cr2hdr.30a5132.zip (replace the binary into the LR Plugin bin folder, make a copy of the old binary, or rename it).

Code: [Select]

cr2hdr: a post processing tool for Dual ISO images

Last update: 30a5132 on 2014-05-07 10:34:25 UTC by a1ex:
Maybe it will solve your problem !

[Danne]

Thanks for sharing but no success though. Still the white bottom line.


Did you try converting the provided original to see if it gives the same result? Maybe  Have to post an issue report. Do you perhaps know where to find cr2hdr on bitbucket? Been searching.
Thanks for helping.
/D

[Danne]

Here is two more. One with purple colored line. These are zoomed in crops. Any idea what is causing this?

[kichetof]

I've converted your image with cr2hdr 310e39f 140227 and cr2hdr-20bit 30a5132 140507 and indeed there are on both the white line at the bottom. Strange. Maybe A1ex has a solution !
Do you try to convert a classic picture took from Dual ISO and not from a Dual ISO Movie ?

You can't found cr2hdr from bitbucket outside MagicLantern Project. You need to compile yourself from the source if you want the latest changes. Or use one of the binary compiled by A_D_ or myself.

I won't be at home for 2 weeks and without access to internet, so I hope a1ex solve this problem

[Danne]

Thanks man for listening. I, ll do some more searching on this and see what comes up.
//D

[kichetof]

If someone have some times and knowledges to help me to resolve this pull request, Version 3 will come out

Target:

• Use dcraw and exiftool from local binary folder, not from system folder (uniformity of versions)
• I don't know how to send this information to exiftool-bridge.c

Big thanks for your great help !

[lureb74]

New GUI
...
Download Source branch default and rename the folder like cr2hdr.lrplugin or what you want.
If everything seems good, I'll release the v3 in second week of june.
...

Hi,

I just installed this version, and report this error that appear when I try to process a dual-iso image:

[string "MLProcess.lua"]:157: attempt to perform arithmetic on local 'isoFirst' (a nil value)

However, the image is created anyway, but LR stops elaborating (when select multiple files).

Hope it whould be useful, CHEERS!

[Danne]

Hi Kitchehof. Would be possible to solve the same levels problem by using a manual setting somehow? In Alex code you can change the numbers in whitelevel. Would be sweet to have it set on "standard 50000" and then try different settings if needed.
 

Right, using --same-levels with parallel processing is a little harder (because all the files are analyzed at the end, and then it decides a white level that hopefully matches all of them). But it can be done after a regular conversion, like this:

Code: [Select]

exiftool *.DNG "-WhiteLevel<BlackLevel"
exiftool *.DNG "-WhiteLevel+=50000"

You may need to fiddle with the values though; lower the value if you get pink highlights, increase it if you think you can squeeze some more highlight detail.

[kichetof]

[string "MLProcess.lua"]:157: attempt to perform arithmetic on local 'isoFirst' (a nil value)

Hi Lureb74, could you post a screenshot of Dual ISO Metadata from the picture that is problematic (only if not empty) ? Could you post exif ISO ? Thanks !

@Danne
Yes this is a workaround to "fix" it

To add --same-levels I need to know the list of pictures that have been selected for rendering. I need to investigate into exportContext:renditions.

I'm back from vacation and I'll have some time to try to implement this function.

[cfmcmillan]

It seems that something is broken in the export of DNG files.  I always get the "Wrong Format" message when I try to export a DNG - while there's a dualiso file when I get done, it hasn't been converted.

OS = Win 7
LR = 5.6

Love it when it worked.  I took a look at MLExportServiceProvider.lua but I can't quite follow the logic to see what's going wrong. 

[kichetof]

@cfmcmillan
Which version of plugin do you use ?
Could you check into Metadata with EXIF & IPTC selected, the row File Format ? And tell me exactly what's wrote (case sensitive!) because I don't understand why it doesn't work...

[kichetof]

@Danne & all --same-levels user

It's sound good !

Code: [Select]

for photo in exportSession:photosToExport() do mlLog:debugf("%s", photo:getRawMetadata("path")) end
I've found the way to list pictures path before export now I need some time to rewrite the plugin to add --same-levels and maybe, I'll remove the export (speed enhancement) but I need some try and I don't know if I can

Stay tuned!

[chris_overseas]

Currently the amount of temp space required is about 2.5x the total size of all the dual ISO photos being converted because copies of the CR2 and DNG files are kept until the entire export is complete. Would it be possible to delete the temp CR2 and DNG files after each file is processed? I'm currently only able to process a fraction of my dual ISO photos at a time, otherwise I run out of disk space.

[kichetof]

@chris I'll rewrite the plugin and there were no export to temp folder. I'll use original cr2 or dng file

So, I need some help.

How do you think the rendering to cr2hdr:

- export all pictures to cr2hdr with args (same-levels or not) in one time, like:
cr2hdr [args...] picture1 picture2 pictureN
And after reimport one by one picture and add metadata

Or

- export one picture to cr2hdr with args (no same-levels) for each picture, like:
cr2hdr [args...] picture1

The first idea is the only possibility to use same-levels, so I'll use this for it but for pictures without this arg, I don't know if it's a good idea.

I explain: if cr2hdr crash all processed files will not be reimported into LR, because the return of the execution will fail.

So, this is my idea:

Process by batch if same-levels are selected and process individually if not selected.

What do you think?

[chris_overseas]

I don't have any use for same-levels personally so can't really comment on that side of things, but without same-levels I'd prefer to see the files processed individually if that means less temp space is used along with more robust recovery in the event of an error.

[Levas]

Found out about the Lightroom plugin cr2hdr v2.2 BETA 6.
I am amazed by the dual iso output results by 20 bit export 

Is there a newer plug in version available for download and if so, where can i find the newest version ?

[kichetof]

Actually the beta 6 is the latest

[Levas]

Ah, so I have the latest version, in that case I'm cool 

[kichetof]

hum, I've a little problem with --same-levels for reading the log file to apply metadata...

Coding skill are welcome

I've a table with all path of each pictures, and I've a log file like that:

Code: [Select]

cr2hdr: a post processing tool for Dual ISO images

Last update: 0eabcb0 on 2014-08-29 12:42:54 UTC by a1ex:
cr2hdr exposure matching: when all else fails, brute force prevails...

Active options:
--amaze-edge    : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs2x2         : apply 2x2 chroma smoothing in noisy and aliased areas (default)
--wb=graymax    : set AsShotNeutral by maximizing the number of gray pixels (default)
--same-levels   : Adjust output white levels to keep the same overall exposure
                  for all frames passed in a single command line
                  (useful to avoid flicker - for video or panoramas)

Input file      : /Users/tof/Documents/MagicLantern/Dual ISO try/_MG_8353.CR2
Camera          : Canon EOS 5D Mark III
Camera model    : Canon EOS 5D Mark III
Full size       : 5920 x 3950
Active area     : 5796 x 3870
Black borders   : 124 left, 80 top
Black level     : 2046
ISO pattern     : BddB RGGB
White levels    : 15180 13787
Noise levels    : 8.43 6.73 6.70 8.42 (14-bit)
ISO difference  : 2.98 EV (789)
Black delta     : 3.55
Dynamic range   : 10.94 (+) 10.44 => 13.42 EV (in theory)
Looking for hot/cold pixels...
Hot pixels      : 310
Cold pixels     : 308
AMaZE interpolation ...
Amaze took 2.84 s
Edge-directed interpolation...
Semi-overexposed: 1.17%
Deep shadows    : 54.64%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap     : 5.0 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level     : 55.98 (20-bit), ideally 55.97
Dynamic range   : 13.87 EV (cooked)
Black adjust    : -25
AsShotNeutral   : 0.47 1 0.68, 5867K/g=1.05 (gray max)
Output file     : /Users/tof/Documents/MagicLantern/Dual ISO try/_MG_8353.DNG
/Users/tof/Documents/MagicLantern/Dual ISO try/_MG_8353.DNG: copying EXIF from /Users/tof/Documents/MagicLantern/Dual ISO try/_MG_8353.CR2

Input file      : /Users/tof/Documents/MagicLantern/Dual ISO try/_MG_8354.CR2
Camera          : Canon EOS 5D Mark III
Camera model    : Canon EOS 5D Mark III
Full size       : 5920 x 3950
Active area     : 5796 x 3870
Black borders   : 124 left, 80 top
Black level     : 2046
ISO pattern     : BddB RGGB
White levels    : 10086 13785
Noise levels    : 7.30 6.70 6.70 7.28 (14-bit)
ISO difference  : 1.98 EV (394)
Black delta     : 5.47
Dynamic range   : 10.23 (+) 10.65 => 12.63 EV (in theory)
Looking for hot/cold pixels...
Hot pixels      : 23
Cold pixels     : 340
AMaZE interpolation ...
Amaze took 2.80 s
Edge-directed interpolation...
Semi-overexposed: 1.33%
Deep shadows    : 23.53%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap     : 5.3 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level     : 99.76 (20-bit), ideally 97.64
Dynamic range   : 12.33 EV (cooked)
Black adjust    : -97
AsShotNeutral   : 0.52 1 0.64, 5145K/g=1.13 (gray max)
Output file     : /Users/tof/Documents/MagicLantern/Dual ISO try/_MG_8354.DNG
/Users/tof/Documents/MagicLantern/Dual ISO try/_MG_8354.DNG: copying EXIF from /Users/tof/Documents/MagicLantern/Dual ISO try/_MG_8354.CR2

Equalizing levels...
/Users/tof/Documents/MagicLantern/Dual ISO try/_MG_8353.DNG: 8185 ... 40345
/Users/tof/Documents/MagicLantern/Dual ISO try/_MG_8354.DNG: 8190 ... 40350

How can I read this log file to apply metadata to each picture ? Each log "block" are delimited by a return before "Input file".

This is the actual parse log file function (thanks akry!):

Code: [Select]

-- parse logfile to read info
function MLProcess.parseLogFileToArray(filename)

local logFileOpen = assert(io.open(filename, 'r'))

local logfile = logFileOpen:read("*all")
logFileOpen:close()

info = {}
info_index = {}

-- parse cr2hdr output for "xxx : yyy" and store keys/values in temp_t
for k, v in string.gmatch(logfile, '%s*([%w%s-]+)%s*:%s*([%S ]+)%s*') do

k = MLProcess.trim(k)

-- there are duplicating keys in output (for instance, "Dynamic range")
if info[k] ~= nil then

local t = k
info_index[t] = info_index[t] + 1
k = k .. info_index[t]

else

info_index[k] = 1

end

info[k] = MLProcess.trim(v)

end

return info

end
Maybe return an array with log info ?
Feel free to pull request into the main repo !

Thanks a lot for your help!
Tof