raw2dng.app 0.13 Osx GUI development, bugs and updates

Started by scrax, May 19, 2013, 03:15:09 PM

Previous topic - Next topic

0 Members and 3 Guests are viewing this topic.

bashi

Quote from: scrax on June 16, 2013, 01:08:37 PM
downloading now, Good to hear that you are working on this too :)
do you think it will be better to have a repo to share our code?

this might make sense at some point. but first have a look at it and then we can see. my coding isn't really great ;-)
note: the wb multipliers in my app are wrong.


bashi

Quote from: mkorho on June 16, 2013, 04:45:09 AM
TO DO:
Extract dead pixel list from dng and use it in dcraw to remove dead pixels.

Dcraw wont read the dead pixel information from the source file but needs an external list file.

this is a feature request?


bashi

here the canon 5d III dcraw white balance rgbg multipliers:

Daylight
1.950195 1.000000 1.605469 1.000000
Shade
2.226562 1.000000 1.369141 1.000000
Cloudy
2.089844 1.000000 1.477539 1.000000
Tungsten
1.412109 1.000000 2.392578 1.000000
White Fluorescent
1.720703 1.000000 2.291016 1.000000

mkorho

Quote from: bashi on June 16, 2013, 01:46:49 PM
this is a feature request?
Yes it is.

All the 3x and 5x zoom mode raw recording have more than 5 dead pixels, at least on my camera.
It is funny that nobody else is seeing them. (or dont care)
To me red, green and blue pixels in video is a NO NO. :)
And manual removal is too painful process.

Adobe camera raw and Ufraw can read the dead (hot) pixel list from the source file but dcraw cant.

bashi

@scrax

here your script edited for fps and white balance dialog with cocoadialog working:

set_wb(){

  "$CD" standard-dropdown \
  --text "Pick a wb" --title "wb Options" --items "Auto" "Daylight" "Shade" "Cloudy" "Tungsten" "White Fluorescent"\
  --string-output | tail -1 > wb.txt
   
  wb=`awk ' { print $1; }' "$FLDR$FILE_NAME"_dng/wb.txt`
  rm wb.txt
  #Set rgb multiplier
  if [[ $wb = Auto ]]
  then
      Wtemp="-a"
 
  elif [[ $wb = *Daylight* ]]
  then
      Wtemp="-r 1.950195 1.000000 1.605469 1.000000"
 
  elif [[ $wb = *Shade* ]]
  then
  Wtemp="-r 2.226562 1.000000 1.369141 1.000000"
 
  elif [[ $wb = *Cloudy* ]]
  then
  Wtemp="-r 2.089844 1.000000 1.477539 1.000000"
 
  elif [[ $wb = *Tungsten* ]]
  then
  Wtemp="-r 1.412109 1.000000 2.392578 1.000000"

  elif [[ $wb = *White* ]]
  then
  Wtemp="-r 1.720703 1.000000 2.291016 1.000000"
 
   
  fi
}

convert_file() {
   cd "$FLDR$FILE_NAME"_dng
   "$RAW2DNG" "$FLDR$FILE_NAME".RAW > r2dlog.txt
   
   FPS=`awk '/FPS/ { print $3; }' "$FLDR$FILE_NAME"_dng/r2dlog.txt`
   
   rm r2dlog.txt
   
   set_wb
   
   printf "Generating ProResHQ 422\n"
   #find "$FLDR$FILE_NAME"_dng -type f -name "*.dng" -print | \
   #while read dng
   #do
      "$DCRAW" -c -H 0 -6 $Wtemp -W -q 3 *.dng | "$FFMPEG" -f image2pipe -vcodec ppm -r $FPS -i pipe:0 -vcodec prores_kostya -profile:v 3 -vendor ap10 -pix_fmt yuv422p10le -y -r $FPS "$FLDR$FILE_NAME".mov
   #done
   printf "Done\n"
}

bashi

Quote from: mkorho on June 16, 2013, 02:18:57 PM
Yes it is.

All the 3x and 5x zoom mode raw recording have more than 5 dead pixels, at least on my camera.
It is funny that nobody else is seeing them. (or dont care)
To me red, green and blue pixels in video is a NO NO. :)
And manual removal is too painful process.

Adobe camera raw and Ufraw can read the dead (hot) pixel list from the source file but dcraw cant.

will look into it... (we just switched ufraw to dcraw - might switch back then ;-)
it's not too high of a priority for me though.

scrax

Quote from: bashi on June 16, 2013, 02:57:55 PM
@scrax

here your script edited for fps and white balance dialog with cocoadialog working:

I've added fps as you suggested early, now I'm looking into a way for having some options to set, but they should be optional (for advanced use only).
I'm using ML2.3 for photography with:
EOS 600DML | EOS 400Dplus | EOS 5D MLbeta5- EF 100mm f/2.8 USM Macro  - EF-S 17-85mm f4-5.6 IS USM - EF 70-200mm f/4 L USM - 580EXII - OsX, PS, LR, RawTherapee, LightZone -no video experience-

bashi

Quote from: scrax on June 16, 2013, 03:14:41 PM
I've added fps as you suggested early, now I'm looking into a way for having some options to set, but they should be optional (for advanced use only).

something like an advanced checkbox on first window perhaps... not really sure how to change platypus default window to do that. so far i'm not getting "warm" with platypus and cocoadialog. as far as i see you can only trigger dialog popup (i might be wrong) which is not great to use.

i think the app (the ideal app) should be something like:
-be as easy as possible
-give as much control over the image (raw->img process) as possible (if desired, advanced option). Since those settings gets "backed" into resulting img seq or video.
-be suitable for batch process

btw, did my app work on your computer?

i don't like on mine:
-no drag drop on app (in file selection does work)
-not nice ui (but hey, a ui ;-)
-no feedback about what it is doing ( while raw2dng and dcraw-video)

scrax

We should consider use pashua instead of cocoadialog and keep Platypus as wrapper. Pashua will be good for the preference page maybe.
Your app is working in making the dng, then it show the options for video rendering but video is not generated.

The big issue I see so far is that in batch conversion setting wb for each file will be not ideal.

My idea is that settings should be something that is related to app and not to raw file. For example if we enable ProRes conversion in settings all the file imported will be converted, if we set a working folder all file go in that folder and so on.
Things like wb, fps etc. should be automatic and right. Maybe ML could make a video.log of raw captures too like for normal video, and we can use that for finding out needed settings...

EDIT: I can open a settings dialog by dropping raw2dng.app into his window ;)

set_pref(){
    printf "Preference panel to be added!\n"
}

##### Main

if [[ ! "$1" ]]; then
   usage
else
   if [[ "$1" == *raw2dng.app ]]; then
      set_pref
   fi
   #set_workdir
...
I'm using ML2.3 for photography with:
EOS 600DML | EOS 400Dplus | EOS 5D MLbeta5- EF 100mm f/2.8 USM Macro  - EF-S 17-85mm f4-5.6 IS USM - EF 70-200mm f/4 L USM - 580EXII - OsX, PS, LR, RawTherapee, LightZone -no video experience-

bashi

Quote from: scrax on June 16, 2013, 03:51:06 PM
We should consider use pashua instead of cocoadialog and keep Platypus as wrapper. Pashua will be good for the preference page maybe.
Your app is working in making the dng, then it show the options for video rendering but video is not generated.

The big issue I see so far is that in batch conversion setting wb for each file will be not ideal.

My idea is that settings should be something that is related to app and not to raw file. For example if we enable ProRes conversion in settings all the file imported will be converted, if we set a working folder all file go in that folder and so on.
Things like wb, fps etc. should be automatic and right. Maybe ML could make a video.log of raw captures too like for normal video, and we can use that for finding out needed settings...

EDIT: I can open a settings dialog by dropping raw2dng.app into his window ;)

set_pref(){
    printf "Preference panel to be added!\n"
}

##### Main

if [[ ! "$1" ]]; then
   usage
else
   if [[ "$1" == *raw2dng.app ]]; then
      set_pref
   fi
   #set_workdir
...


use platypus and pasha seems ideal. will try this.

what i think of:
on first dialog have checkbox advanced and combo box White Balance with default Auto. if advanced unchecked it will work as usual and convert all raw with this White Balance setting.
if advanced checked it will open for each raw a preview thing to make settings and then render.

additionally on first dialog checkbox image and video, to make img seq and/or video.

this would allow to batch process but as well fine tuning of each. 

EDIT: I can see that it can make sense to edit all videos with the same settings. At the same time i see cases where it would be great to edit each. So an option to edit settings for all or for each could make sense.

scrax

So far I've a 0.13 candidate that will use cocoadialog to set working folder and if Prores should be generated or not.
I'm using ML2.3 for photography with:
EOS 600DML | EOS 400Dplus | EOS 5D MLbeta5- EF 100mm f/2.8 USM Macro  - EF-S 17-85mm f4-5.6 IS USM - EF 70-200mm f/4 L USM - 580EXII - OsX, PS, LR, RawTherapee, LightZone -no video experience-

Danne

Since we,re on the ProRes topic. Will the files contain similar dynamic range possibilities as with the dng,s? I compared the raw imported with the ginger plugin(trial) and the generated ProRes file. It seems contrasts and highlight information differs. Maybe one could get a flatter profile ProRes file somehow? The first three are the ginger plugin one and the other three are the ProRes. Might it be a white balance thing?












bashi

Quote from: Danne on June 16, 2013, 06:55:13 PM
Since we,re on the ProRes topic. Will the files contain similar dynamic range possibilities as with the dng,s? I compared the raw imported with the ginger plugin(trial) and the generated ProRes file. It seems contrasts and highlight information differs. Maybe one could get a flatter profile ProRes file somehow? The first three are the ginger plugin one and the other three are the ProRes. Might it be a white balance thing?

Magic Lantern RAW is 14bit, ProRes is 10bit. There is no way (i know of) to increase ProRes bit depth.
There might be the possibility to output raw video (not sure if it would be 14bit), but FileSize is similar to CinemaDNG, so i don't see it make sense, better work of CinemaDNG sequence then.

the raw converter in use (dcraw) can do gamma correction, so we should be able to get a flatter profile. It can also do Highlight blending and reconstruction. But those things are not in raw2dng.app for now.

White balance has influence on Highlights too.

Danne


Jason Montalvo

Hello and thank you for the new version of raw2dng. Just wanted to report that when the app is generating the prores file it doesn't seem to be able to make a file greater than 645 frames. Any else experienced this ?

Thank You Again !!

bashi

Quote from: Jason Montalvo on June 17, 2013, 01:20:50 PM
Hello and thank you for the new version of raw2dng. Just wanted to report that when the app is generating the prores file it doesn't seem to be able to make a file greater than 645 frames. Any else experienced this ?

Thank You Again !!

thanks for report. not good ;-) will look into it.
If you click in raw2dng.app on the Details button left, after it failed, can you copy the last part of the text output bellow to here (or pasteall.org)?

Jason Montalvo

@Bashi

I just tried it again and now it seems to have converted the entire clip to a prores file. I'll try more and let you know if I discover anything.

Doyle4

Sweet this version is working well for me and the ProRes output is a nice feature too, drag n drop into afters, change color bit from 8bit to 16 and using adjustment layers all seems sweet!

Iv noticed that it dosnt convert the very last frame so i got 187 out of 188, this isnt a major problem at all but i thought id let you know.

Thanks!

bashi

Quote from: Jason Montalvo on June 17, 2013, 04:23:39 PM
@Bashi

I just tried it again and now it seems to have converted the entire clip to a prores file. I'll try more and let you know if I discover anything.

it's bashi ;-) Good to hear, and please let us know of any issues.

@Doyle4: Thanks for the info.


noix222

just converted some files with 0.12 to me it worked pretty good... i still prefer to proRes4444 with AE. It would be amazing if we could batch convert with raw2dng and have the option to create just DNG's or DNG's and proRes... anyway its killing it. thanks guys!

bashi

Quote from: noix222 on June 17, 2013, 10:59:52 PM
just converted some files with 0.12 to me its worked pretty good... i still prefer to proRes4444 with AE. It would be amazing if we could batch convert with raw2dng and have the option to create just DNG's or DNG's and proRes... anyway its killing it. thanks guys!

"-pix_fmt yuv444p10″ specifies that you want to use the ProRes 4444 codec specifically

we can do that. ;-)

so this would be prores 4444 12bit.
(until now i had some confusion with those pixel formats)

the option to make dng and/or video will come soon.

noix222

Quote from: bashi on June 18, 2013, 12:35:40 AM
we can do that. ;-)

so this would be prores 4444 12bit.
(until now i had some confusion with those pixel formats)

the option to make dng and/or video will come soon.

that's awesome thanks!!! its possible to batch convert multiple .RAW files with raw2dng? I went through a lot of time going one by one :P

8beeeaaat

I using RAW module for 60D made by marekk.
http://www.magiclantern.fm/forum/index.php?topic=5653.0

UPDATE #9:
Hot/dead pixels removed automatically by ACR (you need fresh raw2dng)
ML with raw_rec compiled hourly by rocket's scripts:
https://www.dropbox.com/sh/1kitzbmanifb6sf/qgkHzW3gn2


The ProRes files were made by raw2dng.app 0.12 have some Hot/dead pixels.

Thanks.

budafilms

Hi Guys, what kind of Pro Res is the file created by RAW2DNG: 422, 4444, LT? Because in Quicktime information only said the creator´s name. Thanks!

bashi

Quote from: noix222 on June 18, 2013, 04:58:43 AM
that's awesome thanks!!! its possible to batch convert multiple .RAW files with raw2dng? I went through a lot of time going one by one :P

just drag 'n drop all files on it, or folder containing them.

Quote from: budafilms on June 18, 2013, 07:55:14 AM
Hi Guys, what kind of Pro Res is the file created by RAW2DNG: 422, 4444, LT? Because in Quicktime information only said the creator´s name. Thanks!

currently it's 422 HQ, but 4444 is possible and will be enabled.