Author Topic: raw2dng.app 0.13 Osx GUI development, bugs and updates  (Read 369660 times)

bashi

  • Freshman
  • **
  • Posts: 53
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #225 on: June 16, 2013, 01:44:56 PM »
downloading now, Good to hear that you are working on this too :)
do you think it will be better to have a repo to share our code?

this might make sense at some point. but first have a look at it and then we can see. my coding isn't really great ;-)
note: the wb multipliers in my app are wrong.


bashi

  • Freshman
  • **
  • Posts: 53
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #226 on: June 16, 2013, 01:46:49 PM »
TO DO:
Extract dead pixel list from dng and use it in dcraw to remove dead pixels.

Dcraw wont read the dead pixel information from the source file but needs an external list file.

this is a feature request?


bashi

  • Freshman
  • **
  • Posts: 53
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #227 on: June 16, 2013, 01:54:09 PM »
here the canon 5d III dcraw white balance rgbg multipliers:

Daylight
1.950195 1.000000 1.605469 1.000000
Shade
2.226562 1.000000 1.369141 1.000000
Cloudy
2.089844 1.000000 1.477539 1.000000
Tungsten
1.412109 1.000000 2.392578 1.000000
White Fluorescent
1.720703 1.000000 2.291016 1.000000

mkorho

  • New to the forum
  • *
  • Posts: 13
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #228 on: June 16, 2013, 02:18:57 PM »
this is a feature request?
Yes it is.

All the 3x and 5x zoom mode raw recording have more than 5 dead pixels, at least on my camera.
It is funny that nobody else is seeing them. (or dont care)
To me red, green and blue pixels in video is a NO NO. :)
And manual removal is too painful process.

Adobe camera raw and Ufraw can read the dead (hot) pixel list from the source file but dcraw cant.

bashi

  • Freshman
  • **
  • Posts: 53
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #229 on: June 16, 2013, 02:57:55 PM »
@scrax

here your script edited for fps and white balance dialog with cocoadialog working:

Code: [Select]
set_wb(){

  "$CD" standard-dropdown \
  --text "Pick a wb" --title "wb Options" --items "Auto" "Daylight" "Shade" "Cloudy" "Tungsten" "White Fluorescent"\
  --string-output | tail -1 > wb.txt
   
  wb=`awk ' { print $1; }' "$FLDR$FILE_NAME"_dng/wb.txt`
  rm wb.txt
  #Set rgb multiplier
  if [[ $wb = Auto ]]
  then
      Wtemp="-a"
 
  elif [[ $wb = *Daylight* ]]
  then
      Wtemp="-r 1.950195 1.000000 1.605469 1.000000"
 
  elif [[ $wb = *Shade* ]]
  then
  Wtemp="-r 2.226562 1.000000 1.369141 1.000000"
 
  elif [[ $wb = *Cloudy* ]]
  then
  Wtemp="-r 2.089844 1.000000 1.477539 1.000000"
 
  elif [[ $wb = *Tungsten* ]]
  then
  Wtemp="-r 1.412109 1.000000 2.392578 1.000000"

  elif [[ $wb = *White* ]]
  then
  Wtemp="-r 1.720703 1.000000 2.291016 1.000000"
 
   
  fi
}

convert_file() {
   cd "$FLDR$FILE_NAME"_dng
   "$RAW2DNG" "$FLDR$FILE_NAME".RAW > r2dlog.txt
   
   FPS=`awk '/FPS/ { print $3; }' "$FLDR$FILE_NAME"_dng/r2dlog.txt`
   
   rm r2dlog.txt
   
   set_wb
   
   printf "Generating ProResHQ 422\n"
   #find "$FLDR$FILE_NAME"_dng -type f -name "*.dng" -print | \
   #while read dng
   #do
      "$DCRAW" -c -H 0 -6 $Wtemp -W -q 3 *.dng | "$FFMPEG" -f image2pipe -vcodec ppm -r $FPS -i pipe:0 -vcodec prores_kostya -profile:v 3 -vendor ap10 -pix_fmt yuv422p10le -y -r $FPS "$FLDR$FILE_NAME".mov
   #done
   printf "Done\n"
}

bashi

  • Freshman
  • **
  • Posts: 53
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #230 on: June 16, 2013, 03:00:47 PM »
Yes it is.

All the 3x and 5x zoom mode raw recording have more than 5 dead pixels, at least on my camera.
It is funny that nobody else is seeing them. (or dont care)
To me red, green and blue pixels in video is a NO NO. :)
And manual removal is too painful process.

Adobe camera raw and Ufraw can read the dead (hot) pixel list from the source file but dcraw cant.

will look into it... (we just switched ufraw to dcraw - might switch back then ;-)
it's not too high of a priority for me though.

scrax

  • Contributor
  • Hero Member
  • *****
  • Posts: 1543
  • Code monkey
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #231 on: June 16, 2013, 03:14:41 PM »
@scrax

here your script edited for fps and white balance dialog with cocoadialog working:

I've added fps as you suggested early, now I'm looking into a way for having some options to set, but they should be optional (for advanced use only).
I'm using ML2.3 for photography with:
EOS 600DML | EOS 400Dplus | EOS 5D MLbeta5- EF 100mm f/2.8 USM Macro  - EF-S 17-85mm f4-5.6 IS USM - EF 70-200mm f/4 L USM - 580EXII - OsX, PS, LR, RawTherapee, LightZone -no video experience-

bashi

  • Freshman
  • **
  • Posts: 53
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #232 on: June 16, 2013, 03:35:02 PM »
I've added fps as you suggested early, now I'm looking into a way for having some options to set, but they should be optional (for advanced use only).

something like an advanced checkbox on first window perhaps... not really sure how to change platypus default window to do that. so far i'm not getting "warm" with platypus and cocoadialog. as far as i see you can only trigger dialog popup (i might be wrong) which is not great to use.

i think the app (the ideal app) should be something like:
-be as easy as possible
-give as much control over the image (raw->img process) as possible (if desired, advanced option). Since those settings gets "backed" into resulting img seq or video.
-be suitable for batch process

btw, did my app work on your computer?

i don't like on mine:
-no drag drop on app (in file selection does work)
-not nice ui (but hey, a ui ;-)
-no feedback about what it is doing ( while raw2dng and dcraw-video)

scrax

  • Contributor
  • Hero Member
  • *****
  • Posts: 1543
  • Code monkey
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #233 on: June 16, 2013, 03:51:06 PM »
We should consider use pashua instead of cocoadialog and keep Platypus as wrapper. Pashua will be good for the preference page maybe.
 Your app is working in making the dng, then it show the options for video rendering but video is not generated.

The big issue I see so far is that in batch conversion setting wb for each file will be not ideal.

My idea is that settings should be something that is related to app and not to raw file. For example if we enable ProRes conversion in settings all the file imported will be converted, if we set a working folder all file go in that folder and so on.
Things like wb, fps etc. should be automatic and right. Maybe ML could make a video.log of raw captures too like for normal video, and we can use that for finding out needed settings...

EDIT: I can open a settings dialog by dropping raw2dng.app into his window ;)
Code: [Select]
set_pref(){
    printf "Preference panel to be added!\n"
}

##### Main

if [[ ! "$1" ]]; then
   usage
else
   if [[ "$1" == *raw2dng.app ]]; then
      set_pref
   fi
   #set_workdir
...
I'm using ML2.3 for photography with:
EOS 600DML | EOS 400Dplus | EOS 5D MLbeta5- EF 100mm f/2.8 USM Macro  - EF-S 17-85mm f4-5.6 IS USM - EF 70-200mm f/4 L USM - 580EXII - OsX, PS, LR, RawTherapee, LightZone -no video experience-

bashi

  • Freshman
  • **
  • Posts: 53
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #234 on: June 16, 2013, 04:53:45 PM »
We should consider use pashua instead of cocoadialog and keep Platypus as wrapper. Pashua will be good for the preference page maybe.
 Your app is working in making the dng, then it show the options for video rendering but video is not generated.

The big issue I see so far is that in batch conversion setting wb for each file will be not ideal.

My idea is that settings should be something that is related to app and not to raw file. For example if we enable ProRes conversion in settings all the file imported will be converted, if we set a working folder all file go in that folder and so on.
Things like wb, fps etc. should be automatic and right. Maybe ML could make a video.log of raw captures too like for normal video, and we can use that for finding out needed settings...

EDIT: I can open a settings dialog by dropping raw2dng.app into his window ;)
Code: [Select]
set_pref(){
    printf "Preference panel to be added!\n"
}

##### Main

if [[ ! "$1" ]]; then
   usage
else
   if [[ "$1" == *raw2dng.app ]]; then
      set_pref
   fi
   #set_workdir
...

use platypus and pasha seems ideal. will try this.

what i think of:
on first dialog have checkbox advanced and combo box White Balance with default Auto. if advanced unchecked it will work as usual and convert all raw with this White Balance setting.
if advanced checked it will open for each raw a preview thing to make settings and then render.

additionally on first dialog checkbox image and video, to make img seq and/or video.

this would allow to batch process but as well fine tuning of each. 

EDIT: I can see that it can make sense to edit all videos with the same settings. At the same time i see cases where it would be great to edit each. So an option to edit settings for all or for each could make sense.

scrax

  • Contributor
  • Hero Member
  • *****
  • Posts: 1543
  • Code monkey
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #235 on: June 16, 2013, 05:06:16 PM »
So far I've a 0.13 candidate that will use cocoadialog to set working folder and if Prores should be generated or not.
I'm using ML2.3 for photography with:
EOS 600DML | EOS 400Dplus | EOS 5D MLbeta5- EF 100mm f/2.8 USM Macro  - EF-S 17-85mm f4-5.6 IS USM - EF 70-200mm f/4 L USM - 580EXII - OsX, PS, LR, RawTherapee, LightZone -no video experience-

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7368
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #236 on: June 16, 2013, 06:55:13 PM »
Since we,re on the ProRes topic. Will the files contain similar dynamic range possibilities as with the dng,s? I compared the raw imported with the ginger plugin(trial) and the generated ProRes file. It seems contrasts and highlight information differs. Maybe one could get a flatter profile ProRes file somehow? The first three are the ginger plugin one and the other three are the ProRes. Might it be a white balance thing?












bashi

  • Freshman
  • **
  • Posts: 53
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #237 on: June 16, 2013, 08:29:18 PM »
Since we,re on the ProRes topic. Will the files contain similar dynamic range possibilities as with the dng,s? I compared the raw imported with the ginger plugin(trial) and the generated ProRes file. It seems contrasts and highlight information differs. Maybe one could get a flatter profile ProRes file somehow? The first three are the ginger plugin one and the other three are the ProRes. Might it be a white balance thing?

Magic Lantern RAW is 14bit, ProRes is 10bit. There is no way (i know of) to increase ProRes bit depth.
There might be the possibility to output raw video (not sure if it would be 14bit), but FileSize is similar to CinemaDNG, so i don't see it make sense, better work of CinemaDNG sequence then.

the raw converter in use (dcraw) can do gamma correction, so we should be able to get a flatter profile. It can also do Highlight blending and reconstruction. But those things are not in raw2dng.app for now.

White balance has influence on Highlights too.

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7368
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #238 on: June 16, 2013, 08:31:17 PM »
Thanks a lot for answering

Jason Montalvo

  • Freshman
  • **
  • Posts: 54
  • 5D3/5D2 & Some BlackMagic
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #239 on: June 17, 2013, 01:20:50 PM »
Hello and thank you for the new version of raw2dng. Just wanted to report that when the app is generating the prores file it doesn't seem to be able to make a file greater than 645 frames. Any else experienced this ?

Thank You Again !!

bashi

  • Freshman
  • **
  • Posts: 53
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #240 on: June 17, 2013, 03:27:13 PM »
Hello and thank you for the new version of raw2dng. Just wanted to report that when the app is generating the prores file it doesn't seem to be able to make a file greater than 645 frames. Any else experienced this ?

Thank You Again !!

thanks for report. not good ;-) will look into it.
If you click in raw2dng.app on the Details button left, after it failed, can you copy the last part of the text output bellow to here (or pasteall.org)?

Jason Montalvo

  • Freshman
  • **
  • Posts: 54
  • 5D3/5D2 & Some BlackMagic
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #241 on: June 17, 2013, 04:23:39 PM »
@Bashi

I just tried it again and now it seems to have converted the entire clip to a prores file. I'll try more and let you know if I discover anything.

Doyle4

  • Hero Member
  • *****
  • Posts: 690
  • Canon 5DRAWii & 600Dual-Iso
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #242 on: June 17, 2013, 04:58:18 PM »
Sweet this version is working well for me and the ProRes output is a nice feature too, drag n drop into afters, change color bit from 8bit to 16 and using adjustment layers all seems sweet!

Iv noticed that it dosnt convert the very last frame so i got 187 out of 188, this isnt a major problem at all but i thought id let you know.

Thanks!

bashi

  • Freshman
  • **
  • Posts: 53
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #243 on: June 17, 2013, 05:17:08 PM »
@Bashi

I just tried it again and now it seems to have converted the entire clip to a prores file. I'll try more and let you know if I discover anything.

it's bashi ;-) Good to hear, and please let us know of any issues.

@Doyle4: Thanks for the info.


noix222

  • Freshman
  • **
  • Posts: 78
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #244 on: June 17, 2013, 10:59:52 PM »
just converted some files with 0.12 to me it worked pretty good... i still prefer to proRes4444 with AE. It would be amazing if we could batch convert with raw2dng and have the option to create just DNG's or DNG's and proRes... anyway its killing it. thanks guys!

bashi

  • Freshman
  • **
  • Posts: 53
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #245 on: June 18, 2013, 12:35:40 AM »
just converted some files with 0.12 to me its worked pretty good... i still prefer to proRes4444 with AE. It would be amazing if we could batch convert with raw2dng and have the option to create just DNG's or DNG's and proRes... anyway its killing it. thanks guys!

“-pix_fmt yuv444p10″ specifies that you want to use the ProRes 4444 codec specifically

we can do that. ;-)

so this would be prores 4444 12bit.
(until now i had some confusion with those pixel formats)

the option to make dng and/or video will come soon.

noix222

  • Freshman
  • **
  • Posts: 78
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #246 on: June 18, 2013, 04:58:43 AM »
we can do that. ;-)

so this would be prores 4444 12bit.
(until now i had some confusion with those pixel formats)

the option to make dng and/or video will come soon.

that's awesome thanks!!! its possible to batch convert multiple .RAW files with raw2dng? I went through a lot of time going one by one :P

8beeeaaat

  • New to the forum
  • *
  • Posts: 16
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #247 on: June 18, 2013, 05:50:45 AM »
I using RAW module for 60D made by marekk.
http://www.magiclantern.fm/forum/index.php?topic=5653.0

UPDATE #9:
Hot/dead pixels removed automatically by ACR (you need fresh raw2dng)
ML with raw_rec compiled hourly by rocket's scripts:
https://www.dropbox.com/sh/1kitzbmanifb6sf/qgkHzW3gn2


The ProRes files were made by raw2dng.app 0.12 have some Hot/dead pixels.

Thanks.

budafilms

  • Hero Member
  • *****
  • Posts: 711
  • 5D Mark a1ex
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #248 on: June 18, 2013, 07:55:14 AM »
Hi Guys, what kind of Pro Res is the file created by RAW2DNG: 422, 4444, LT? Because in Quicktime information only said the creator´s name. Thanks!

bashi

  • Freshman
  • **
  • Posts: 53
Re: raw2dng.app 0.12 Osx GUI development, bugs and updates
« Reply #249 on: June 18, 2013, 11:16:01 AM »
that's awesome thanks!!! its possible to batch convert multiple .RAW files with raw2dng? I went through a lot of time going one by one :P

just drag 'n drop all files on it, or folder containing them.

Hi Guys, what kind of Pro Res is the file created by RAW2DNG: 422, 4444, LT? Because in Quicktime information only said the creator´s name. Thanks!

currently it's 422 HQ, but 4444 is possible and will be enabled.