Magic Lantern Forum

Using Magic Lantern => Post-processing Workflow => HDR and Dual ISO Postprocessing => Topic started by: SuperJackRabbit on August 28, 2013, 04:06:00 PM

Title: Dual ISO Raw Video processing on Mac?
Post by: SuperJackRabbit on August 28, 2013, 04:06:00 PM
I spent several hours to figure out how to process the CR2 files with Dual ISO feature,but it's different when it came to Raw videos.... :'(

After shooting raw video in Dual ISO mode,I use the RAW2DNG(0.13 version)to process the XXXX.RAW files,but the lines are still in every frame...it just convert the RAW files into DNG sequences and do nothing to Dual ISO...
Title: Re: Dual ISO Raw Video processing?
Post by: PressureFM on August 28, 2013, 05:00:06 PM
Hint: cr2hdr.
Title: Re: Dual ISO Raw Video processing?
Post by: Danne on August 28, 2013, 05:05:24 PM
All info here http://www.magiclantern.fm/forum/index.php?topic=7139.0
Title: Re: Dual ISO Raw Video processing?
Post by: 1% on August 28, 2013, 05:25:00 PM
Serious pain right now:

.Raw/MLV -> DNG -> cr2hdr
Title: Re: Dual ISO Raw Video processing?
Post by: nandoide on August 28, 2013, 05:50:49 PM
No, it's similar than processing monoiso raw video. You only need a specific version of raw2dng.exe from first post of the thread http://www.magiclantern.fm/forum/index.php?topic=7139.0. No need of cr2hdr.

Title: Re: Dual ISO Raw Video processing?
Post by: 1% on August 28, 2013, 06:31:36 PM
So latest raw2dng will do the dual ISO raw files? I haven't tried yet, only using it for stills.

/** HDR video interpolation */
/** Technical details: http://a1ex.magiclantern.fm/bleeding-edge/isoless/dual_iso.pdf */

/* choose interpolation method (define only one of these) */
#define INTERP_MEAN23
#undef INTERP_MEDIAN_6



Yea, its in there it appears.
Title: Re: Dual ISO Raw Video processing?
Post by: a1ex on August 28, 2013, 07:10:49 PM
The algorithm is not that good as with latest cr2hdr, but should work.
Title: Re: Dual ISO Raw Video processing?
Post by: nandoide on August 28, 2013, 07:56:12 PM
Yes, it works very well in crop mode. In normal mode we obtain coloured groups of dots, I think probably aliasing. An example.

https://www.dropbox.com/s/823nmk7d7zwvsmh/M24-1127000012.dng

Same result if I process with raw2dng.exe mono_iso and apply cr2hdr or cr2hdr_exp to dngs, obtained by raw2dng.exe dual_iso.

(6D)
Title: Re: Dual ISO Raw Video processing?
Post by: a1ex on August 28, 2013, 07:58:36 PM
Can you upload an interlaced DNG?
Title: Re: Dual ISO Raw Video processing?
Post by: nandoide on August 28, 2013, 08:34:03 PM
Of course. The same frame processed with raw2dng.exe mono_iso:

https://www.dropbox.com/s/br7jllyqe6rjpu9/M24-1127000012.interlaced.dng

I've been trying differents methods of demosaicing in rawtherapee whitout luck.
Title: Re: Dual ISO Raw Video processing?
Post by: a1ex on August 28, 2013, 09:44:06 PM
Can you take a silent picture? That one includes black level and noise info, which is missing from the raw video for size (speed) reasons.

edit: are you using lv_af_raw? these artifacts don't look like aliasing...

edit 2: ISO difference : 6.28 EV (7779)

You went too far, try 100/1600 or 100/400...

edit 3: I think I've solved it (http://acoutts.com/a1ex/M24-1127000012.solved.DNG).

(http://acoutts.com/a1ex/M24-1127000012.solved.jpg)

Raise the ISO 100 by 64 before ISO matching (this value is HUGE, usual values are 2-5, and the Batman shot had 20 iirc). Or wait until I'll implement a fix (most likely tomorrow).
Title: Re: Dual ISO Raw Video processing?
Post by: nandoide on August 28, 2013, 11:24:45 PM
Buf, yes is a big difference in isos, but no perceived issues in crop mode (indeed very powerful tool for macro video (invaluable) : raw+dualiso+3x crop, no significant detail loss and good control of highlights in flowers and so on) It's the mode that I are using mainly.

At this folder I put another test ("mike wasorsrki" test with silent pics and iso 100/6400), The dots are far less perceived than  the previous example (only in the edges of the speaker holes in the right). The silent pics are better.

https://www.dropbox.com/sh/mlawr8tgfsb25zm/C1CcLNwK2M

edit 1: I don't use lv_af_raw (I supose because I don't know what is this  ;) , another module?)

Thanks a lot.
Title: Re: Dual ISO Raw Video processing?
Post by: a1ex on August 28, 2013, 11:27:00 PM
This is because silent pics contain all the info to find and fix the black offset.

Fortunately, most of it can be guessed by linear regression.
Title: Re: Dual ISO Raw Video processing?
Post by: 1% on August 29, 2013, 12:01:45 AM
AF raw has been off forever.. those look like af raw marks but sideways. I've never seen them but also haven't done too much 100/6400
Title: Re: Dual ISO Raw Video processing?
Post by: SuperJackRabbit on August 29, 2013, 04:31:17 AM
I do use the CR2HDR to convert CR2 files into DNG files, and it works very well....And I try to use it to convert Raw files,it says"/Users/MnZnP/Desktop//M28-1959.RAW File not Supported",Then I use the RAW2DNG 0.13version as the latest version on Mac,I get all DNG files with those lines,and the put the DNG files into CR2HDR,it also says:/Users/MnZnP/Desktop/M28-1959_dng/000082.dng File not Supported" :-\

I ask several people and everyone told me the answer is in http://www.magiclantern.fm/forum/index.php?topic=7139.0 ,but I am a Mac user and I can't run exe files like latest raw2dng.exe,any solution to work on Mac?    ???
Title: Re: Dual ISO Raw Video processing?
Post by: elarturex on August 29, 2013, 04:39:48 AM
Fun Weekend At Timothy Lake, Oregon (Video + Timelapse)

Shot with GoPro HERO2, GoPro HERO3, Canon 5D Mark II with Magic Lantern (14 bit RAW Video) and Canon 7D with Magic Lantern (Dual ISO RAW Timelapse)

Title: Re: Dual ISO Raw Video processing on Mac?
Post by: jbarkey on September 06, 2013, 05:55:42 AM
Been shooting RAW for a while on my 5D3, but first stab at the dual ISO. How can I run the CR2HDR on my Mac? I've converted my .RAW files, and I want to get rid of everyone's favourite lines. If you can just send me to a post or someone's comment that might enlighten me, it'd be much appreciated!
Title: Re: Dual ISO Raw Video processing on Mac?
Post by: geodorno on September 13, 2013, 07:01:43 PM
Can someone please compile a mac version of the newest cr2hdr, which can process DNGs?
I did some testing with dual ISO raw recording on 60d, but can't get my hands on a PC...

Thanks so much for making my camera more fun!!!
Title: Re: Dual ISO Raw Video processing on Mac?
Post by: geodorno on September 13, 2013, 07:09:35 PM
Also, it would be awesome to have a option for extracting the bright and dark frames separately. So you can make HDR with other software, like Photomatix.
But i really like the dng cr2hdr outputs from cr2 files, you can use the fill light slider in camera raw a lot!!! so much DR, and now motion artifacts like in HDR video mode...

thanks so much, guys!!!!
Title: Re: Dual ISO Raw Video processing on Mac?
Post by: N/A on September 14, 2013, 09:59:26 AM
To process dual iso video files on mac, you need the latest cr2hdr mac gui here-
https://bitbucket.org/a_d_/magic-lantern/downloads

and an app to change the extensions of the .dng's to CR2 in one batch, like this one-
http://www.mrrsoftware.com/MRRSoftware/NameChanger.html

I converted my .raw files into dng with RawMagic, but I'm seeing a lot of debayering issues. Might Raw2Dng instead. Once you have your .dng's in one folder, open NameChanger and pull all of them into the app. Type in .dng in the original text box and .CR2 in the new text box, and process them. Then drag the entire folder to the cr2hdr app and convert them back to .dng processed into the dual iso version. Edit like normal.
Title: Re: Dual ISO Raw Video processing on Mac?
Post by: geodorno on September 14, 2013, 01:00:31 PM
unfortunately, this doesn't work for me.
I tried to extract the raw file with rawmagic and raw2dng, works fine.
than after renaming with this nice tool you linked, I can open the cr2 files in camera raw (photoshop), but cr2hdr still says "file not supported".
I tried older version of cr2hdr as well as the the version 1.1 you linked...
I'm on 60D, by the way.
any ideas?

thanks
Title: Re: Dual ISO Raw Video processing on Mac?
Post by: N/A on September 14, 2013, 01:31:34 PM
Try .CR2 instead of .cr2

I THINK that was my problem too, I'll have to double-check
Title: Re: Dual ISO Raw Video processing on Mac?
Post by: khurra on September 14, 2013, 05:49:30 PM
this is the error I'm receiving:

Input file : 000000.CR2
sh: exiftool: command not found sh: dcraw: command not found sh: dcraw: command not found Error: dcraw output is not a valid PGM file Logfile    : 000000.txt

Title: Re: Dual ISO Raw Video processing on Mac?
Post by: RenatoPhoto on September 14, 2013, 06:07:20 PM
Quote from: khurra on September 14, 2013, 05:49:30 PM
this is the error I'm receiving:

Input file : 000000.CR2
sh: exiftool: command not found sh: dcraw: command not found sh: dcraw: command not found Error: dcraw output is not a valid PGM file Logfile    : 000000.txt
Forum Rules: no double posting!
Title: Re: Dual ISO Raw Video processing on Mac?
Post by: a.d. on September 15, 2013, 01:34:42 PM
@khurra
You also need to download OSX_cr2hdr.zip, a few links below. I've already provided a pkg installer for dcraw and exiftools.
Title: Re: Dual ISO Raw Video processing on Mac?
Post by: khurra on September 15, 2013, 06:00:18 PM
hey yah!

when i download it, and unzip. i get the cr2hdr app.

I moved that app to applications folder.

Here are my workflow and error messages:

take test file of dual iso RAW shot on canon 650d, unpackage raw to dng using raw2dng executable.

change file names of .dng to end with .CR2.

open cr2hdr add files. Error:

Input file : 1.CR2
sh: exiftool: command not found sh: dcraw: command not found sh: dcraw: command not found Error: dcraw output is not a valid PGM file Logfile    : 1.txt

___

i then opened terminal and installed dcraw using macports "sudo apt-get install dcraw"

macports installed it, i can find dcraw and its components in HD/opt/local/bin/dcraw

so im not exactly sure what im doing wrong.

Thank you for your help! would LOVE to shoot some dual iso.
Title: Re: Dual ISO Raw Video processing on Mac?
Post by: khurra on September 15, 2013, 07:10:05 PM
update: read the note from a.d. like actually READ it and realized i was missing components.

it works for me now!!

thanks everyone for all your hard work and help.

Title: Re: Dual ISO Raw Video processing on Mac?
Post by: geodorno on September 16, 2013, 11:50:27 AM
Yes it works, with renaming to "CR2"...

here is an example:
https://vimeo.com/74517601

still got dead pixels, any tips for this?
the dynamic range is incredible!!!
and a lot of moire, aliasing artifacts... (I realized that i did a great moire test, too with all the small details in the walls...

thanks
Title: Re: Dual ISO Raw Video processing on Mac?
Post by: a1ex on September 16, 2013, 11:52:28 AM
Can you upload a CR2?

Try using a less aggressive setting (100/400 maybe?)
Title: Re: Dual ISO Raw Video processing on Mac?
Post by: geodorno on September 16, 2013, 12:04:29 PM
processed
https://dl.dropboxusercontent.com/u/10328620/M13-1644_C0000_00153.DNG
raw
https://dl.dropboxusercontent.com/u/10328620/M13-1644_C0000_00109.CR2

processed
https://dl.dropboxusercontent.com/u/10328620/000069.DNG
raw
https://dl.dropboxusercontent.com/u/10328620/000069.CR2

less agressive better for dead pixels??
Title: Re: Dual ISO Raw Video processing on Mac?
Post by: mkrjf on September 18, 2013, 09:40:44 PM
I have used the 'process' referenced here and elsewhere for DualIso 5DMk3 latest build and compared to plain ML Raw and also camera jpeg

If you look at some sample output from a low light test (my kitchen with a few practicals):
https://plus.google.com/photos/109692674038873146393/albums/5924621701487097649
Sorry, I have not labeled all the photos, it would be obvious from the file names. Anyway - I see lots of banding in both raw and dual iso raw much more than in a pic with the camera. I tried to pick an optimal exposure range that would not clip the white toaster.
Looking at the pics in google plus the banding is not so obvious, but looking at original output at HD resolution it reminds me of old 256 color graphics cards :(
Is this the expected result? It would not be usable, I think. Also - does DualISO only provide big improvement for non-optimal exposure?
For the sunset test with June build I only noticed banding in the sky if I pushed too hard in grading - not right out of the process (raw*/cdr2hdr)

I have not tried raw2cdng as suggested by redder yet. Does that support CDR?
Thx
Mike
Title: Re: Dual ISO Raw Video processing on Mac?
Post by: a1ex on September 18, 2013, 09:58:29 PM
These shots are not processed, you must postprocess them with cr2hdr.

www.magiclantern.fm/forum/index.php?topic=7139
Title: Re: Dual ISO Raw Video processing on Mac?
Post by: mkrjf on September 19, 2013, 04:25:57 AM
I did post process.
I included before and after as well as raw video of same scene with no dual ISO.
There are probably too many images to look at - I will simplify and repost.
But basically jpeg from camera shows no banding and ml raw and dual ISO show banding. Dual ISO does not show much improvement in dynamic range from raw video (no dual ISO).
But like I said I certainly could have done something wrong. There are no problems with latest version of ml or the 'process' described for dualiso on Mac?
Thx
Title: Re: Dual ISO Raw Video processing on Mac?
Post by: a1ex on September 19, 2013, 07:20:28 AM
http://www.chiark.greenend.org.uk/~sgtatham/bugs.html
Title: Re: Dual ISO Raw Video processing on Mac?
Post by: Throck on December 22, 2013, 09:56:42 AM
Hi, brand new user here.  I'm still getting the,

"Error: dcraw output is not a valid PGM file"

I have downloaded and installed:

OSX_cr2hrd.zip (and run the dcraw & exiftool installer)
&
OSX_cr2hdr_app.zip

Am I still missing something?  I'm on a 7D and OSX 10.9

---EDIT---

I didn't have alternate frames only enabled.  You'll get this error if the .CR2 isn't a dual ISO shot.
Title: Re: Dual ISO Raw Video processing on Mac?
Post by: kyrobb on July 05, 2014, 08:51:36 PM
So everything seems to go well when I post-process on my Mac. I have cr2hdr with dcraw and exiftool. I convert the raw file to dngs with RawMagic and then drag into cr2hdr. This is the response.

Input file : M05-1135_00003.dng
1 image files updated cr2hdr: a post processing tool for Dual ISO images Last update: 2f4e2df on 2013-12-02 17:45:36 UTC by alex: cr2hdr: fix DNG compression under Windows Active options: --amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default) --cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default) Input file : M05-1135_00003.CR2 Camera : Canon EOS 5D Mark III Full size : 1504 x 630 Active area : 1504 x 630 White level : 12500 Black borders : N/A ISO pattern : dBBd RGGB Noise levels : 8.00 8.00 8.00 8.00 (14-bit) ISO difference : 4.06 EV (1664) Black delta : 32.43 Dynamic range : 10.35 (+) 10.35 => 14.41 EV (in theory) Looking for hot/cold pixels... Hot pixels : 8 AMaZE interpolation ... Amaze took 0.32 s Edge-directed interpolation... Semi-overexposed: 10.70% Deep shadows : 68.15% Horizontal stripe fix... Full-res reconstruction... ISO overlap : 3.3 EV (approx) Half-res blending... Chroma smoothing... Building alias map... Filtering alias map... Smoothing alias map... Final blending... Noise level : 8.00 (16-bit), ideally 8.00 Dynamic range : 12.35 EV (cooked) Output file : M05-1135_00003.DNG
THE END

But after this, my files cannot be read by after effects, or any other program for that matter. What am I doing wrong?
Title: Re: Dual ISO Raw Video processing on Mac?
Post by: kyrobb on July 20, 2014, 08:38:28 PM
Well I've played some more, mostly using Auto ETTR linked to dual iso, but I've used Adobe DNG Converter to make my files readable by Camera Raw, which seems to work. I'm now getting pink frames and a flickering exposure, and no noticeable impact on my dynamic range. Thoughts?
Title: Re: Dual ISO Raw Video processing on Mac?
Post by: Danne on July 21, 2014, 02:16:57 AM
You could try MLVmystic for dual iso files. Settings in there does a decent job.
http://www.magiclantern.fm/forum/index.php?topic=10466.0
Title: Re: Dual ISO Raw Video processing on Mac?
Post by: kyrobb on July 22, 2014, 07:49:06 AM
A step in the right direction thanks! I'm noticing the dynamic range increase without flickering using Mystic. Unfortunately, now all bright areas are turning pink and pixelated. Odd...