Author Topic: raw2dng.app 0.13 Osx GUI development, bugs and updates  (Read 387921 times)

CaptainHook

  • Freshman
  • **
  • Posts: 52
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #50 on: May 21, 2013, 10:55:29 AM »
Good to hear that, meanwhile I've recompiled from last source and now dng extracted from my raw can't be opened anymore in ACR

I just found photoshop doesn't like 5610 dng files and hangs. :D
Lightroom is currently spitting out tif files for me instead, although it seems much slower than PS would do it which is a shame...

Here are my "LOG" settings by the way that i knocked together VERY quickly.. seems to work well so far with alexa LogC->709/bmcc/my bmcc lut shared via John Brawley (swing log offset tint/gain around)/etc in Resolve (haven't tested thoroughly yet):

PS:
http://cl.ly/1R1W0O1i362z

LR:
http://cl.ly/3x2M2H2n1E3r

squig

  • Hero Member
  • *****
  • Posts: 518
  • Crash test dummy MK3
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #51 on: May 21, 2013, 11:22:52 AM »
Tried a 52Gb file

/Applications/raw2dng.app/Contents/Resources/script: line 10: /bin/ls: Argument list too long
Nothing converted

 :'(

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7702
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #52 on: May 21, 2013, 11:44:47 AM »
Tried a 52Gb file

/Applications/raw2dng.app/Contents/Resources/script: line 10: /bin/ls: Argument list too long
Nothing converted

 :'(

52Gb file, now that is one massive moviefile :)!!

squig

  • Hero Member
  • *****
  • Posts: 518
  • Crash test dummy MK3
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #53 on: May 21, 2013, 12:01:40 PM »
Now I'm getting the same error with every file, even under 2Gb.  :'(

squig

  • Hero Member
  • *****
  • Posts: 518
  • Crash test dummy MK3
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #54 on: May 21, 2013, 12:31:59 PM »
Ok I think I get what's going on now. raw2dng swallowed up the 52Gb file, the app is now 53.05Gb hehe.

I trashed the app. reinstalled and converted a 21Gb file, opened in ACR 7, no issues.

CaptainHook

  • Freshman
  • **
  • Posts: 52
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #55 on: May 21, 2013, 12:38:15 PM »
Scrax - may need to replace "ls" with "find" or something similar?

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7702
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #56 on: May 21, 2013, 12:54:11 PM »
Gotta get home from work and some serious raw-encoding, seriously, this is so funky

squig

  • Hero Member
  • *****
  • Posts: 518
  • Crash test dummy MK3
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #57 on: May 21, 2013, 02:56:04 PM »
Tested 42Gb and 37Gb conversion, no issues.

menoc

  • Senior
  • ****
  • Posts: 273
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #58 on: May 21, 2013, 03:27:12 PM »
So if the App is ballooning when saving large files, perhaps Raw2Dng should show some kind of dialog box to save to a location other than the Raw2Dng app?? Will this overcome the file size errors and not have to delete the app everytime?

squig

  • Hero Member
  • *****
  • Posts: 518
  • Crash test dummy MK3
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #59 on: May 21, 2013, 03:56:28 PM »
The app only ballooned with the 52Gb file (twice). Saving to the app isn't a feature, it's a bug triggered by the 52Gb file, everything 42Gb and under is being saved where it's supposed to be. I haven't tested between 42 and 52Gb.

It's quite slow with the larger files so a 64bit version would be a welcome addition.

sicetime

  • Freshman
  • **
  • Posts: 53
  • 5D III
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #60 on: May 21, 2013, 04:50:23 PM »
THIS IS WHAT I HAVE BEEN WATING FOR, THE NEWEST VERSION ROCKS! THANK YOU ALL FOR YOUR HARD WORK!!

12georgiadis

  • Member
  • ***
  • Posts: 213
  • 5DmkIII - 7D - EOS-M
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #61 on: May 21, 2013, 05:22:06 PM »
Hello,

First of all, thank you for the tremendous work done on the 5D Raw and the raw2dng.app !!

I wanted to know if it was possible to incorporate CinemaDNG specifications in the conversion of raw to DNG. In fact, Da Vinci Resolve does not recognize DNG or partially because they do not contain the following metadatas : Time Code, Frame Rate, T-Stop, Reel Name, Camera Label.
For rushes of BMCC (also in Cinema DNG), these metadatas are only stored in the first DNG, which allows it to be read in Resolve and help us to create proxies (id. ProRes LT) to mount on any NLE. Then we can make a XML to conform and send it to Da Vinci. This would avoid making color corrections before the editing and would give to the 5D Raw video, access to use for a feature film workflow.

squig

  • Hero Member
  • *****
  • Posts: 518
  • Crash test dummy MK3
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #62 on: May 21, 2013, 06:18:22 PM »
It's being looked at, nobody's figured out how to do it yet.

mattmvpmedia

  • New to the forum
  • *
  • Posts: 24
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #63 on: May 21, 2013, 07:10:17 PM »
Thank you Scrax!

sicetime

  • Freshman
  • **
  • Posts: 53
  • 5D III
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #64 on: May 21, 2013, 07:11:54 PM »
My tests have shown that with .5 batch is working on batch 4gb files, I have yet to test beyond that, no bugs, no hickups, I'll fill up my card tonight and report back, thanks again scrax!

Fulgencio Martinez

  • Just arrived
  • *
  • Posts: 1
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #65 on: May 21, 2013, 09:45:06 PM »
I do use resolve as a grading tool
regular dng files are not working on resolve.. so i have to go through photoshop, AE or any other conversion method
Could raw2dng make cinema dng files?
This would make the workflow easier
thanks
f

dngrhm

  • Freshman
  • **
  • Posts: 83
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #66 on: May 21, 2013, 10:28:03 PM »
It's being looked at, nobody's figured out how to do it yet.

No progress since then  :D Welcome to the bleeding edge.  The workflow might suck, but the view is nice.
EOS 650D + 620 | Canon EF-S 18-135mm f/3.5-5.6 STM + 55-250mm f/4-5.6 | Canon EF 50mm f/1.8
Mac OS 10.9 | PinkDotRemoval Tool | RAWMagic | DaVinci Resolve | FCP X

12georgiadis

  • Member
  • ***
  • Posts: 213
  • 5DmkIII - 7D - EOS-M
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #67 on: May 22, 2013, 12:15:40 AM »
Perhaps Adobe has released an SDK Cinema DNG
I found this for the Cinema DNG's related tags : http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/en/devnet/cinemadng/pdfs/CinemaDNG_Format_Specification_v1_1.pdf (P.20)

scrax

  • Contributor
  • Hero Member
  • *****
  • Posts: 1543
  • Code monkey
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #68 on: May 22, 2013, 12:42:59 AM »
The app only ballooned with the 52Gb file (twice). Saving to the app isn't a feature, it's a bug triggered by the 52Gb file, everything 42Gb and under is being saved where it's supposed to be. I haven't tested between 42 and 52Gb.

It's quite slow with the larger files so a 64bit version would be a welcome addition.
I suppose that for the 52Gb it converted most of the stuff but since the dng are moved after finishing conversion they were not moved out from the app
In future without trashing all the app just right click it and select show package contents, then navigate to Resources folder and you will find all the dng converted.
To solve that wee need to investigate more the issue, so far I se two problems one is the swallowing of the app and another is the error from raw2dng, for the first I'll try to save dng in the final folder directly, but I don't know yet how.

Scrax - may need to replace "ls" with "find" or something similar?
where? I don't get the difference but explain me more please, since probably it could be improved a lot.
I'm using ML2.3 for photography with:
EOS 600DML | EOS 400Dplus | EOS 5D MLbeta5- EF 100mm f/2.8 USM Macro  - EF-S 17-85mm f4-5.6 IS USM - EF 70-200mm f/4 L USM - 580EXII - OsX, PS, LR, RawTherapee, LightZone -no video experience-

CaptainHook

  • Freshman
  • **
  • Posts: 52
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #69 on: May 22, 2013, 06:09:49 AM »
I found this for the Cinema DNG's related tags : http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/en/devnet/cinemadng/pdfs/CinemaDNG_Format_Specification_v1_1.pdf

This is the wrong thread i think, but cinemaDNG can simply be a folder of DNG files with sequential numbers so the spec that needs to be followed in just the plain DNG spec.

http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec_1.4.0.0.pdf

Currently using dng_validate (which is in the DNG SDK) gives these errors with DNG's from raw2dng which i've already let a1ex know about (this thread is about an OSX wrapper for that app):

*** Warning: IFD 0 Model is not NULL terminated ***
*** Warning: IFD 0 UniqueCameraModel is not NULL terminated ***
*** Warning: The ExposureTime is <= 0 ***
*** Warning: Too little padding on left edge of CFA image (possible interpolation artifacts) ***
*** Warning: Too little padding on top edge of CFA image (possible interpolation artifacts) ***
*** Warning: Too little padding on right edge of CFA image (possible interpolation artifacts) ***
*** Warning: Too little padding on bottom edge of CFA image (possible interpolation artifacts) ***


To solve that wee need to investigate more the issue, so far I se two problems one is the swallowing of the app and another is the error from raw2dng, for the first I'll try to save dng in the final folder directly, but I don't know yet how.

Yeah, the script is using "ls" to check to see if the folder has any DNG files, but with Squig's large 52gb file this produces so many dng files that ls can't deal with them all, so the script fails before moving all the DNG's out of the resources directory (they actually get converted fine).

I don't have any large files to test (32gb lexar 1000x is largest i have), Squig, could you try replacing your script file with this and test with your 52gb file? (http://www.magiclantern.fm/forum/index.php?topic=5404.msg37139#msg37139) I'm not sure what performance will be like and definitely ideal is for raw2dng to be modified to accept destination directory as an argument (ask a1ex?).

#!/bin/sh

importFILE() {
   if [ "${file_is: -4}" == ".RAW" ] ; then
#      echo "File $file_is Supported"
      name=$(basename -s .RAW "$file_is")
      fldr="$(dirname "$file_is")"/

      ./raw2dng "$file_is"
    
      mkdir "$fldr$name"
     find . -type f -name "*.dng" -print | \
     while read filename
     do
           mv "${filename}" "$fldr$name"/
     done
   
   else
   echo "File not Supported"
   fi
}
   
checkFLDR() {
   for d in $(ls "$FILES_FLDR"); do
      if [ -d "$FILES_FLDR/$d/" ];then
         FILES_FLDR="$FILES_FLDR$d"
         echo "we have: $FILES_FLDR"
         for f in $(ls $FILES_FLDR); do
            file_is="$FILES_FLDR$f"
            echo "SEARCHING: $file_is"
            importFILE
         done
      else
         file_is="$FILES_FLDR/$d"
         echo "SEARCHING file: $file_is"
         importFILE
      fi
   done
}

#################


echo "raw2dng converter GUI for OsX"
echo "Beta ver.0.5"
if [ -d "$1" ];then
   FILES_FLDR="$1"/
   echo "SEARCHING: $FILES_FLDR"
   checkFLDR
elif [ -f "$1" ]; then
   file_is="$1"
   importFILE
else
   echo "Drop a ML .RAW video file or folder here"
fi
exit 0

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #70 on: May 22, 2013, 06:44:53 AM »
*** Warning: IFD 0 Model is not NULL terminated ***
*** Warning: IFD 0 UniqueCameraModel is not NULL terminated ***

I don't get those with dng_validate, can you send me a broken dng?

squig

  • Hero Member
  • *****
  • Posts: 518
  • Crash test dummy MK3
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #71 on: May 22, 2013, 07:08:28 AM »
It worked  :) I just opened a 60Gb file. Thx Captain

CaptainHook

  • Freshman
  • **
  • Posts: 52
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #72 on: May 22, 2013, 07:12:14 AM »
I don't get those with dng_validate, can you send me a broken dng?

This is using Scrax's app 0.5 - which i think he said uses and older raw2dng compile? Maybe you've fixed and it's not in Scrax's share yet, but here's a file anyway i just converted:

http://cl.ly/232r0t3P352O

It worked  :) I just opened a 60Gb file. Thx Captain

Great, i'll leave the rest to Scrax. :)

12georgiadis

  • Member
  • ***
  • Posts: 213
  • 5DmkIII - 7D - EOS-M
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #73 on: May 22, 2013, 10:15:00 AM »
This is the wrong thread i think, but cinemaDNG can simply be a folder of DNG files with sequential numbers so the spec that needs to be followed in just the plain DNG spec.

http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec_1.4.0.0.pdf

Currently using dng_validate (which is in the DNG SDK) gives these errors with DNG's from raw2dng which i've already let a1ex know about (this thread is about an OSX wrapper for that app):

*** Warning: IFD 0 Model is not NULL terminated ***
*** Warning: IFD 0 UniqueCameraModel is not NULL terminated ***
*** Warning: The ExposureTime is <= 0 ***
*** Warning: Too little padding on left edge of CFA image (possible interpolation artifacts) ***
*** Warning: Too little padding on top edge of CFA image (possible interpolation artifacts) ***
*** Warning: Too little padding on right edge of CFA image (possible interpolation artifacts) ***
*** Warning: Too little padding on bottom edge of CFA image (possible interpolation artifacts) ***



Ok, sorry if I'm in the wrong thread (I'm new), I did not know it had been discussed elsewhere. Do you have the link to the right thread?
If I understand correctly, the files that come out of raw2DNG contain metadata errors that are the reason why Da Vinci Resolve does not recognize the result of DNG files (in a folder) as Cinema DNG?
With Black Magic Cinema Camera for example, additional metadata (timecode, Frame Rate, T-Stop, Reel Name, Camera Label) are contained in the first DNG but not in the other DNG file.

If there is a way for the DNG to be recognized in Da Vinci Resolve, the workflow would Become the same as the BMCC and could be used to make feature films. But it's more complicated if you have to go through Photoshop and After Effects before the editing.

squig

  • Hero Member
  • *****
  • Posts: 518
  • Crash test dummy MK3
Re: raw2dng.app 0.5 Osx GUI development, bugs and updates
« Reply #74 on: May 22, 2013, 10:18:04 AM »

If there is a way for the DNG to be recognized in Da Vinci Resolve, the workflow would Become the same as the BMCC and could be used to make feature films. But it's more complicated if you have to go through Photoshop and After Effects before the editing.

We know. It's a work in progress.