raw2dng.app 0.13 Osx GUI development, bugs and updates

Started by scrax, May 19, 2013, 03:15:09 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

CaptainHook

Quote from: scrax on May 21, 2013, 10:26:59 AMGood to hear that, meanwhile I've recompiled from last source and now dng extracted from my raw can't be opened anymore in ACR

I just found photoshop doesn't like 5610 dng files and hangs. :D
Lightroom is currently spitting out tif files for me instead, although it seems much slower than PS would do it which is a shame...

Here are my "LOG" settings by the way that i knocked together VERY quickly.. seems to work well so far with alexa LogC->709/bmcc/my bmcc lut shared via John Brawley (swing log offset tint/gain around)/etc in Resolve (haven't tested thoroughly yet):

PS:
http://cl.ly/1R1W0O1i362z

LR:
http://cl.ly/3x2M2H2n1E3r

squig

Tried a 52Gb file

/Applications/raw2dng.app/Contents/Resources/script: line 10: /bin/ls: Argument list too long
Nothing converted

:'(

Danne

Quote from: squig on May 21, 2013, 11:22:52 AM
Tried a 52Gb file

/Applications/raw2dng.app/Contents/Resources/script: line 10: /bin/ls: Argument list too long
Nothing converted

:'(

52Gb file, now that is one massive moviefile :)!!

squig

Now I'm getting the same error with every file, even under 2Gb.  :'(

squig

Ok I think I get what's going on now. raw2dng swallowed up the 52Gb file, the app is now 53.05Gb hehe.

I trashed the app. reinstalled and converted a 21Gb file, opened in ACR 7, no issues.


Danne

Gotta get home from work and some serious raw-encoding, seriously, this is so funky

squig


menoc

So if the App is ballooning when saving large files, perhaps Raw2Dng should show some kind of dialog box to save to a location other than the Raw2Dng app?? Will this overcome the file size errors and not have to delete the app everytime?

squig

The app only ballooned with the 52Gb file (twice). Saving to the app isn't a feature, it's a bug triggered by the 52Gb file, everything 42Gb and under is being saved where it's supposed to be. I haven't tested between 42 and 52Gb.

It's quite slow with the larger files so a 64bit version would be a welcome addition.

sicetime

THIS IS WHAT I HAVE BEEN WATING FOR, THE NEWEST VERSION ROCKS! THANK YOU ALL FOR YOUR HARD WORK!!

12georgiadis

Hello,

First of all, thank you for the tremendous work done on the 5D Raw and the raw2dng.app !!

I wanted to know if it was possible to incorporate CinemaDNG specifications in the conversion of raw to DNG. In fact, Da Vinci Resolve does not recognize DNG or partially because they do not contain the following metadatas : Time Code, Frame Rate, T-Stop, Reel Name, Camera Label.
For rushes of BMCC (also in Cinema DNG), these metadatas are only stored in the first DNG, which allows it to be read in Resolve and help us to create proxies (id. ProRes LT) to mount on any NLE. Then we can make a XML to conform and send it to Da Vinci. This would avoid making color corrections before the editing and would give to the 5D Raw video, access to use for a feature film workflow.

squig

It's being looked at, nobody's figured out how to do it yet.


sicetime

My tests have shown that with .5 batch is working on batch 4gb files, I have yet to test beyond that, no bugs, no hickups, I'll fill up my card tonight and report back, thanks again scrax!

Fulgencio Martinez

I do use resolve as a grading tool
regular dng files are not working on resolve.. so i have to go through photoshop, AE or any other conversion method
Could raw2dng make cinema dng files?
This would make the workflow easier
thanks
f

dngrhm

Quote from: squig on May 21, 2013, 06:18:22 PM
It's being looked at, nobody's figured out how to do it yet.

No progress since then  :D Welcome to the bleeding edge.  The workflow might suck, but the view is nice.
EOS 650D + 620 | Canon EF-S 18-135mm f/3.5-5.6 STM + 55-250mm f/4-5.6 | Canon EF 50mm f/1.8
Mac OS 10.9 | PinkDotRemoval Tool | RAWMagic | DaVinci Resolve | FCP X

12georgiadis


scrax

Quote from: squig on May 21, 2013, 03:56:28 PM
The app only ballooned with the 52Gb file (twice). Saving to the app isn't a feature, it's a bug triggered by the 52Gb file, everything 42Gb and under is being saved where it's supposed to be. I haven't tested between 42 and 52Gb.

It's quite slow with the larger files so a 64bit version would be a welcome addition.
I suppose that for the 52Gb it converted most of the stuff but since the dng are moved after finishing conversion they were not moved out from the app
In future without trashing all the app just right click it and select show package contents, then navigate to Resources folder and you will find all the dng converted.
To solve that wee need to investigate more the issue, so far I se two problems one is the swallowing of the app and another is the error from raw2dng, for the first I'll try to save dng in the final folder directly, but I don't know yet how.

Quote from: CaptainHook on May 21, 2013, 12:38:15 PM
Scrax - may need to replace "ls" with "find" or something similar?
where? I don't get the difference but explain me more please, since probably it could be improved a lot.
I'm using ML2.3 for photography with:
EOS 600DML | EOS 400Dplus | EOS 5D MLbeta5- EF 100mm f/2.8 USM Macro  - EF-S 17-85mm f4-5.6 IS USM - EF 70-200mm f/4 L USM - 580EXII - OsX, PS, LR, RawTherapee, LightZone -no video experience-

CaptainHook

Quote from: 12georgiadis on May 22, 2013, 12:15:40 AM
I found this for the Cinema DNG's related tags : http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/en/devnet/cinemadng/pdfs/CinemaDNG_Format_Specification_v1_1.pdf

This is the wrong thread i think, but cinemaDNG can simply be a folder of DNG files with sequential numbers so the spec that needs to be followed in just the plain DNG spec.

http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec_1.4.0.0.pdf

Currently using dng_validate (which is in the DNG SDK) gives these errors with DNG's from raw2dng which i've already let a1ex know about (this thread is about an OSX wrapper for that app):

*** Warning: IFD 0 Model is not NULL terminated ***
*** Warning: IFD 0 UniqueCameraModel is not NULL terminated ***
*** Warning: The ExposureTime is <= 0 ***
*** Warning: Too little padding on left edge of CFA image (possible interpolation artifacts) ***
*** Warning: Too little padding on top edge of CFA image (possible interpolation artifacts) ***
*** Warning: Too little padding on right edge of CFA image (possible interpolation artifacts) ***
*** Warning: Too little padding on bottom edge of CFA image (possible interpolation artifacts) ***


Quote from: scrax on May 22, 2013, 12:42:59 AM
To solve that wee need to investigate more the issue, so far I se two problems one is the swallowing of the app and another is the error from raw2dng, for the first I'll try to save dng in the final folder directly, but I don't know yet how.

Yeah, the script is using "ls" to check to see if the folder has any DNG files, but with Squig's large 52gb file this produces so many dng files that ls can't deal with them all, so the script fails before moving all the DNG's out of the resources directory (they actually get converted fine).

I don't have any large files to test (32gb lexar 1000x is largest i have), Squig, could you try replacing your script file with this and test with your 52gb file? (http://www.magiclantern.fm/forum/index.php?topic=5404.msg37139#msg37139) I'm not sure what performance will be like and definitely ideal is for raw2dng to be modified to accept destination directory as an argument (ask a1ex?).

#!/bin/sh

importFILE() {
   if [ "${file_is: -4}" == ".RAW" ] ; then
#      echo "File $file_is Supported"
      name=$(basename -s .RAW "$file_is")
      fldr="$(dirname "$file_is")"/

      ./raw2dng "$file_is"
    
      mkdir "$fldr$name"
     find . -type f -name "*.dng" -print | \
     while read filename
     do
           mv "${filename}" "$fldr$name"/
     done
   
   else
   echo "File not Supported"
   fi
}
   
checkFLDR() {
   for d in $(ls "$FILES_FLDR"); do
      if [ -d "$FILES_FLDR/$d/" ];then
         FILES_FLDR="$FILES_FLDR$d"
         echo "we have: $FILES_FLDR"
         for f in $(ls $FILES_FLDR); do
            file_is="$FILES_FLDR$f"
            echo "SEARCHING: $file_is"
            importFILE
         done
      else
         file_is="$FILES_FLDR/$d"
         echo "SEARCHING file: $file_is"
         importFILE
      fi
   done
}

#################


echo "raw2dng converter GUI for OsX"
echo "Beta ver.0.5"
if [ -d "$1" ];then
   FILES_FLDR="$1"/
   echo "SEARCHING: $FILES_FLDR"
   checkFLDR
elif [ -f "$1" ]; then
   file_is="$1"
   importFILE
else
   echo "Drop a ML .RAW video file or folder here"
fi
exit 0

a1ex

Quote from: CaptainHook on May 22, 2013, 06:09:49 AM
*** Warning: IFD 0 Model is not NULL terminated ***
*** Warning: IFD 0 UniqueCameraModel is not NULL terminated ***

I don't get those with dng_validate, can you send me a broken dng?

squig

It worked  :) I just opened a 60Gb file. Thx Captain

CaptainHook

Quote from: a1ex on May 22, 2013, 06:44:53 AM
I don't get those with dng_validate, can you send me a broken dng?

This is using Scrax's app 0.5 - which i think he said uses and older raw2dng compile? Maybe you've fixed and it's not in Scrax's share yet, but here's a file anyway i just converted:

http://cl.ly/232r0t3P352O

Quote from: squig on May 22, 2013, 07:08:28 AM
It worked  :) I just opened a 60Gb file. Thx Captain

Great, i'll leave the rest to Scrax. :)

12georgiadis

Quote from: CaptainHook on May 22, 2013, 06:09:49 AM
This is the wrong thread i think, but cinemaDNG can simply be a folder of DNG files with sequential numbers so the spec that needs to be followed in just the plain DNG spec.

http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec_1.4.0.0.pdf

Currently using dng_validate (which is in the DNG SDK) gives these errors with DNG's from raw2dng which i've already let a1ex know about (this thread is about an OSX wrapper for that app):

*** Warning: IFD 0 Model is not NULL terminated ***
*** Warning: IFD 0 UniqueCameraModel is not NULL terminated ***
*** Warning: The ExposureTime is <= 0 ***
*** Warning: Too little padding on left edge of CFA image (possible interpolation artifacts) ***
*** Warning: Too little padding on top edge of CFA image (possible interpolation artifacts) ***
*** Warning: Too little padding on right edge of CFA image (possible interpolation artifacts) ***
*** Warning: Too little padding on bottom edge of CFA image (possible interpolation artifacts) ***



Ok, sorry if I'm in the wrong thread (I'm new), I did not know it had been discussed elsewhere. Do you have the link to the right thread?
If I understand correctly, the files that come out of raw2DNG contain metadata errors that are the reason why Da Vinci Resolve does not recognize the result of DNG files (in a folder) as Cinema DNG?
With Black Magic Cinema Camera for example, additional metadata (timecode, Frame Rate, T-Stop, Reel Name, Camera Label) are contained in the first DNG but not in the other DNG file.

If there is a way for the DNG to be recognized in Da Vinci Resolve, the workflow would Become the same as the BMCC and could be used to make feature films. But it's more complicated if you have to go through Photoshop and After Effects before the editing.

squig

Quote from: 12georgiadis on May 22, 2013, 10:15:00 AM

If there is a way for the DNG to be recognized in Da Vinci Resolve, the workflow would Become the same as the BMCC and could be used to make feature films. But it's more complicated if you have to go through Photoshop and After Effects before the editing.

We know. It's a work in progress.