Lightroom plugin cr2hdr v3.0 DEV (WIP)

Started by kichetof, March 18, 2014, 05:04:33 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

kichetof

Quote from: Danne on September 14, 2014, 12:32:41 PM
You nailed it Kitchehof! The updated  MLProcess.lua totally worked! So nice to see this.

Yeah so nice !

Quote from: Danne on September 14, 2014, 12:32:41 PM
Question. Is it a good idea to grey out the"add the converted file to this subfolder" when dual iso dng same levels is selected? As for now the originals are all converted and of course will not move to a subfolder for that reason. Instead creates a message "no such file or directory" if you specify a folder.

Please update your MLProcess.lua file with this one. I fixed the problem... :)
Mistake of my brain!

Let me know!

Danne

So fast :). I, ll place the new lua after conversion of 30gb of dual iso movie files which is running my mac to the max in 4 parallell conversions at the moment :). Running nicely though.
Will be home again in a few hours. Coolest stuff.

kontrakatze

I still have a problem, which I think is not directly related to your script, but maybe you know a solution.

After importing a dualiso raw file into lightroom and conversion using your script (either way using 20-bit or classic) the dng's open fine in lightroom, but not in other programs. I very often use Photomatix for example which opens the dng's with a terrible magenta cast. To work with it I have to export the files after conversion using the dng profile from Lightroom, making a dng from a dng. I already contacted the developers of Photomatix and send them an example file. The problem seem to be some missing tags in the dngs created by cr2hdr, I guess. I don't know what to with it, but this is what they told me:

If you are going to pass on the needed tags to the Magic Lantern people, these are the missing tags:

CFAPlaneColor tag 50710
CFALayout tag 50711


The magenta cast is also visible in the finders preview. Maybe it is of some help anyhow.
By the way, your new version of the script is a huge improvement, chapeau!

Regards,
kontrakatze

kichetof

Unfortunately, I can't do anything with this plugin to solve the "problem".

I remember a discussion about these tags with a1ex and he said that it's optional but needed by photomatix.

I don't use photomatix but it's seem to be "easy" to add it! Looks at DNG specs and add it to chdk-dng.c

Maybe something like that (not tested, and I don't know how to to it)


CFAPlaneColor
Tag 50710 (C616.H)
Type BYTE
Count ColorPlanes
Value See below
Default 0, 1, 2 (red, green, blue) Usage Raw IFD
Description
CFAPlaneColor provides a mapping between the values in the CFAPattern tag and the plane numbers in LinearRaw space. This is a required tag for non-RGB CFA images.


CFALayout
Tag 50711 (C617.H) Type SHORT
Count 1
Value See below Default 1
Usage Raw IFD
Description
CFALayout describes the spatial layout of the CFA. The currently defined values are: 1 = Rectangular (or square) layout
2 = Staggered layout A: even columns are offset down by 1/2 row 3 = Staggered layout B: even columns are offset up by 1/2 row
4 = Staggered layout C: even rows are offset right by 1/2 column 5 = Staggered layout D: even rows are offset left by 1/2 column
6 = Staggered layout E: even rows are offset up by 1/2 row, even columns are offset left by 1/2 column
7 = Staggered layout F: even rows are offset up by 1/2 row, even columns are offset right by 1/2 column
8 = Staggered layout G: even rows are offset down by 1/2 row, even columns are offset left by 1/2 column
9 = Staggered layout H: even rows are offset down by 1/2 row, even columns are offset right by 1/2 column
Note that for the purposes of this tag, rows and columns are numbered starting with one. Layout values 6 through 9 were added with DNG version 1.3.0.0.


struct dir_entry raw_ifd[]={
    {0xC616,  T_BYTE,       4,  0x???},              // CFAPlaneColor tag 50710
    {0xC617,  T_SHORT,     1,  1},                    // CFAPlaneColor tag 50710; value: 1 to 9
}


and change

struct
    {
        struct dir_entry* entry;
        int count;                  // Number of entries to be saved
        int entry_count;            // Total number of entries
    } ifd_list[] =
    {
        {ifd0,      DIR_SIZE(ifd0),     DIR_SIZE(ifd0)},
        {ifd1,      DIR_SIZE(ifd1),     DIR_SIZE(ifd1)},
        {exif_ifd,  DIR_SIZE(exif_ifd), DIR_SIZE(exif_ifd)},
    };


with

struct
    {
        struct dir_entry* entry;
        int count;                  // Number of entries to be saved
        int entry_count;            // Total number of entries
    } ifd_list[] =
    {
        {ifd0,      DIR_SIZE(ifd0),     DIR_SIZE(ifd0)},
        {ifd1,      DIR_SIZE(ifd1),     DIR_SIZE(ifd1)},
        {exif_ifd,  DIR_SIZE(exif_ifd), DIR_SIZE(exif_ifd)},
        {raw_ifd,  DIR_SIZE(raw_ifd), DIR_SIZE(raw_ifd)},
    };


Something like that I think. It's out of my knowledges!

Danne

Hi Kitchehof. Copy to folder works! Tried going deep in my hard drive but the "no such path" dialogoue appeared going this way. ML(hard drive)-3xdualiso-Vasteras-2014-09-14, Gideonsberg, Anna-(to folders). I think it is just to much folders.

Well, what worked was going straight to my hard drive and put the folders there.
Observations. Exiftool is not too happy about parallell conversions. In almost every txt file I get a message that white balance is not changed. But it does seem that it actually change it anyway so it could be a none issue. I observed this exiftool issue before when I ran terminal in four parallells. I checked metadata and all seemed ok though this time. Gotta to some more testing.

When queuing up and running the conversion in one big row exiftool will set the whitelevel the same on all the folders and without any white level errors.


So gotta do the 30gb test once more since I got the folder issues before. I,ll be back.

some pics
[img=http://s29.postimg.org/yulzny8nn/Screen_Shot_2014_09_14_at_16_06_52.png]

[img=http://s29.postimg.org/yvvxhdahf/Screen_Shot_2014_09_14_at_16_17_23.png]

[img=http://s29.postimg.org/x5cwfvsyb/Screen_Shot_2014_09_14_at_16_23_26.png]

kichetof

Hi Danne,

If I understand right, you have specified a subfolder and you had a problem with this folder: ML(hard drive)-3xdualiso-Vasteras-2014-09-14, Gideonsberg, Anna-(to folders) ?

Could you paste the entire path ? Maybe there are some illegal char or it's too long or ... I don't know!

Danne

Yes, thats right. I, ll get the path in a while, not at home :).

Danne

Hi again! Had another go and directly to hard drive disc.

Didn,t get the multiple conversions to work fully and tried exporting a converted dng sequence with mlrawviewer but it was flickery so the exiftool command was random in multiple conversions. Will try in one long go as well.

What I did now, I select three folders and run the plugin, then three new folders, etc, etc and let them run in parallell. In total around 5000 dng:s lined up in 4 queues.
[img=http://s15.postimg.org/ftxyoyz07/Screen_Shot_2014_09_14_at_19_12_45.png]

This is the path but I don,t think that is the problem here. The multiple conversion method works when same levels is not selected so it has to be exiftool related.
Last login: Sun Sep 14 19:16:56 on ttys000
Daniels-MacBook-Pro:~ Daniel$ /Volumes/ML/cr2hdr /Volumes/ML/cr2hdr/M14-1012_DNG_000001 /Volumes/ML/cr2hdr/M14-1014_DNG_000001 /Volumes/ML/cr2hdr/M14-1015_DNG_000001 etc, etc.
 
I didn,t try running this as one long row but one killer feature with this plugin is being able to run multiple folders through lightroom.
A solution could be to be able to run the dng:s with cr2hdr without the same level command in the first run and in second step apply the exiftool same level command in one long row wich seems to suit exiftool better?

Thanks for listening Kitchehof, and for all hard work.

error codes:

[img=http://s15.postimg.org/mbgv8h9dj/Screen_Shot_2014_09_14_at_19_01_29.png]

[img=http://s15.postimg.org/43k17l67r/Screen_Shot_2014_09_14_at_19_04_03.png]

Something interrups
[img=http://s15.postimg.org/t8b1l05o7/Screen_Shot_2014_09_14_at_19_04_17.png]

[img=http://s15.postimg.org/4vmp6saev/Screen_Shot_2014_09_14_at_19_04_36.png]

Something interrups
[img=http://s15.postimg.org/uidgdsotz/Screen_Shot_2014_09_14_at_19_06_05.png]

[img=http://s15.postimg.org/83r6jtwon/Screen_Shot_2014_09_14_at_19_11_40.png]




kontrakatze

@kichetof: Thank you very much for your help! I can't do this as I simply don't have the knowledge, but I read the passage you were talking about. Having in mind that this happened about 6 month ago, I don't think they are going to change anything. A workaround is to export the dng from lightroom as new dng. Then the missing "optional" tags are in the new dng. Not smooth, but working.
(in the end, it is quite funny that most applications that create dng's add these "optional" flags, which makes it quite a standard?).

Regards,
kontrakatze


kichetof

Quote from: Danne on September 14, 2014, 07:23:44 PM
What I did now, I select three folders and run the plugin

Okay !! Now I understand well what you're doing!
I never tried this! I'll try in the week! When I've a solution, I tell you!

Danne


Danne

Hi Kitchehof.
I,m trying to narrow down when and why my exiftool stops working. What I notice is I get errors from my external harddrive which don,t happen when files are directly on internal hd. This makes me wonder if the plugin could work as is.
What is said about exiftool overwrite command is that if something else is occupied with the file somehow these errors might occur.
So, I will try to run your plugin later today from the internal disc and let you know...

Danne

Hi Kitchehof! Tried some multiple conversions today as suggested before. This time from internal harddrive. This worked without flaws.  Will try all 5000 dng;s later but this already makes me thinks that exiftool is not the problem here but my hard drive. Either it has some permission flaws or something else giving this random problems. Have been doing som repairs and so on and will test both internal and external drive.
Sorry for being unspecific but it is hard to test all possibilities.
Conclusion so far. The app is fully working!

kichetof


Danne

Sucess  8) You,re app finished converting 5148 dng:s with same leves commands in multiple of four conversions without a single problem :):)
Fantastic!
I,m using Mlrawviewer to export to dng:s and your lighroom app for the smoothest conversion worflow ever.
Amazing!

kichetof

Hi Danne,

Glad to help you and make you happy  8)
I'll update the beta with corrected MLProcess.lua when I'm at home!

No more feedback?

75 downloads and one feedback, so it's stable  :D

Some translation file needs a review!

Danne

Yeah, and I tried runnning the same level option on my external hard drive again just now and the random problems with exiftool started again. This time I,m almost certain this is a bug from apple that has to do with permission handling. Found a lot of info on this.  I,m also getting error code -50 when deleting. This flaw will make exiftool confused and actually erase files randomly.
On internal drive all is working good though.

I could try to export copy to a specific folder though. This migh work better since it doesn,t overwrite the original. I,ll get back on this.

Were are the translation files? I could check for Swedish language.


Danne

The new folder didn,t take care of the problem.
Conclusion. There could be a problem with exiftool on external hard drives when --same-level is selected. Apple bug. Internal drives are ok :)

kichetof

Which version of exiftool do you use?

You can find source code here.

Feel free to pull a request on bitbucket (see explanations post from Audionut) or post here!

Thanks for your feedback!

Danne

Hi all! dual iso movie file conversion now works in conjunction with MLVFS workflow from here http://www.magiclantern.fm/forum/index.php?topic=13152.0

And of course with the lightroom app as well :)

This means there is no need to export mlv since they,re read straight through fuse app :). Huge timesaver

kichetof


Danne

Yes, and with the lightroom app as well which is terrific :)

senzazn12


Quote from: Danne on September 22, 2014, 03:51:31 PM
Yes, and with the lightroom app as well which is terrific :)

Danne is it possible for you to make a video tutorial on using MLVFS with Lightroom  on Windows? Willing to donate for the tutorial since it will be time and energy on your part.

Danne

Sorry, I work in mac but if you get a setup done with fuse in window it should work the same. Simply select the mlvfs folder mount. Could make a tutorial for mac.

senzazn12


Quote from: Danne on September 22, 2014, 04:37:32 PM
Sorry, I work in mac but if you get a setup done with fuse in window it should work the same. Simply select the mlvfs folder mount. Could make a tutorial for mac.

If you could that would be great since I could get the general idea.