(Linux/Mac) convmlv: Featureful MLV Developer

[Danne]

For dual iso movie sequences I use wildcard and 4 parallell conversions. Find command and dual iso processing what was started all that ended up in MLP(command tip from dmilligan).

Code: [Select]

find . '*.dng' -print0 | xargs -0 -P 4 -n 1 cr2hdr
You want to apply same white level(avoid flicker) to all files in post and I use exiftool to grab that info. This should work.

Code: [Select]

wl=$(exiftool my.dng | awk '/White Level/ { print $4; }')
Apply white level

Code: [Select]

find . -maxdepth 1 -mindepth 1 -name '*.dng' -print0 | xargs -0 exiftool "-WhiteLevel=$wl" -overwrite_original

Since you will have to use cr2hdr you might wanna consider developing CR2 files as well. This is also working to grab the correct white balance multiplier from the CR2 file. You could check this source file from MLP
dualiso_to_DNG(CR2 images)
https://drive.google.com/file/d/0B4tCJMlOYfirUTlfWlFFa3BpeE0/view?usp=sharing

Basically it develops the same as for movie sequences

Code: [Select]

find . -maxdepth 1 -mindepth 1 -name '*.CR2' -print0 | xargs -0 -P 4 -n 1 ~/Library/Services/MLP.workflow/Contents/cr2hdr20bit $CR2 $co_01
You could grab the white balance from the CR2 file itself.
I do this by creating a list and calculate for whitebalance asShotNeutral values(courtesy chmee). Check the script. You could probably make this a lot nicer than in my script. There is also the export path solution if you include binaries outside the system paths(cr2hdr,dcraw,exiftool) but seems you,re dependencies might be put in the system itself. Solution from dfort ,dmilligan ,stackoverflow put in MLP

Code: [Select]

export PATH=~/Library/Services/MLP.workflow/Contents:$PATH
By the way. I don,t think 7D can shoot dual iso movie sequences.

[dfort]

An open question: How does Dual ISO video processing work? And does anyone have any samples...? (My 7D refuses to cooperate; I think I need to put a new ML build in there)

Here are some sample Dual ISO files. One from a 100D and another from an EOSM.
https://www.dropbox.com/sh/kkrx3k2a2hz8nl2/AADAQS5GoTBNPbfP9bFR4OAZa?dl=0

I believe that the 7D has the same issue with Dual ISO video as the 5D2. There has been some recent breakthroughs with that on the 5D2. If you can compile ML (easy as π on Linux) you might try adapting it for the 7D.
http://magiclantern.fm/forum/index.php?topic=16854

[so-rose]

All right, so I got Dual ISO video to work. However, I'm getting what seems like blinking focus pixels. Here's the proxy video output from the 100D sample:

https://drive.google.com/open?id=0Bx0YaKfeuWSrLU9iY2tUZjJqZFE

I tried running mlv2badpixels.sh on the original MLV file, as well as on a developed (with cr2hdr) DNG, using $raw_buffer --> $video_mode (from @dfort's code) from the MLV to specify the -m option, but they both have the exact same artifacts as above. What the heck is going on?

Besides the blinking pixels  , the output looks good.

Thanks @Danne for exposing me to the -P option of xargs - I managed to parallelize image sequence and Dual ISO development . Things are much faster now.



Finally, the script can now output to TIFF, EXR, DPX, and PNG sequences, all automatically compressed if -c is specified. It's trivial to add more; any requests? 

[so-rose]

One more question: How does footage archival work with mlv_dump? As in, compressing the MLV file (and being able to decompress it for use)?

[Danne]

Nice stuff. On a train atm so can,t check anything yet.
I,m planning to add darkframe subtraction something like this to MLP. Might be of interest here.
http://magiclantern.fm/forum/index.php?topic=15801.msg164620#msg164620

Gotta check how you export to exr and dpx later. How is it done ?

Exr are created with imagemagick with convert command right?

[dfort]

I tried running mlv2badpixels.sh on the original MLV file, as well as on a developed (with cr2hdr) DNG, using $raw_buffer --> $video_mode (from @dfort's code) from the MLV to specify the -m option, but they both have the exact same artifacts as above. What the heck is going on?

What is going on is that the way dcraw is filling in the "badpixels" isn't working very well with that particular Dual ISO example. I just checked it out and it is also happening on the Mac using MLP with the mlv_dump -> cr2hdr method. This isn't the case when using MLP in combination with MLVFS. dmilligan worked out a pixel blending system specifically for Dual ISO. It should be possible to do something similar with dcraw but that involves some C coding and creating a custom dcraw build. Better yet, it is also be possible to add the focus pixel fixing code into mlv_dump. That would be a better solution because it would fix the focus pixels on the DNG files like MLVFS does.

By the way I shot that Dual ISO shot on the 100D when testing MLVFS. That was before I created the mlv2badpixels.sh script which is used in MLP. Guess I never ran that shot through mlv_dump -> cr2hdr -> dcraw -P -> ffmpeg.

[so-rose]

@Danne I figured out darkframe subtraction! It's in 1.7.1; thanks for the pointer!

dmilligan worked out a pixel blending system specifically for Dual ISO. It should be possible to do something similar with dcraw but that involves some C coding and creating a custom dcraw build. Better yet, it is also be possible to add the focus pixel fixing code into mlv_dump. That would be a better solution because it would fix the focus pixels on the DNG files like MLVFS does.

A custom dcraw build specifically for Dual ISO is starting to sound a little scary, though it makes a lot of sense if focus pixel removal could be integrated into mlv_dump - or perhaps cr2hdr?


A question for @dmilligan: Looking through the MLVFS source code, I notice a function in 'cs.c' named 'fix_focus_pixels' and another named 'fix_bad_pixels'; I wonder how possible/painful it would be to isolate these little stumps of magic, using the resulting binary to simply take in a bad DNG (from the output of cr2hdr) and output a fixed one?



Also, open question, is there any difference between cr2hdr and cr2hdr20bit? If so where can I find/compile cr2hdr20bit?

[dfort]

Also, open question, is there any difference between cr2hdr and cr2hdr20bit? If so where can I find/compile cr2hdr20bit?

No difference. cr2hdr20bit was merged into the main branch several months ago. People should stop using the old name to avoid confusion.

[DeafEyeJedi]

Nice work guys. Been following this thread and as it seems to get best results with Dual-ISO and Focus Pixel to work together is to combine MLP and MLVFS atm, correct?

[so-rose]

Nice work guys. Been following this thread and as it seems to get best results with Dual-ISO and Focus Pixel to work together is to combine MLP and MLVFS atm, correct?

Thanks .

It would seem that MLVFS has the golden ticket to badpixel removal in Dual ISO footage, so in a sense yes. However, I'm not quite sure what you mean by combining the two programs - if you're suggesting working on integrating the pixel blending code, then I agree fully! convmlv would benefit from such work as well.

[DeafEyeJedi]

Sorry I forgot to also mention these in conjunction with DarkFraming Avg Process (Dual-ISO, Focus Pixel Fix and DF) all together ... My fault for the typo so-rose!

[dmilligan]

A question for @dmilligan: Looking through the MLVFS source code, I notice a function in 'cs.c' named 'fix_focus_pixels' and another named 'fix_bad_pixels'; I wonder how possible/painful it would be to isolate these little stumps of magic, using the resulting binary to simply take in a bad DNG (from the output of cr2hdr) and output a fixed one?

the bad pixel fix was pulled straight from cr2hdr, the focus pixel fixing probably needs to be added directly into cr2hdr

[so-rose]

the focus pixel fixing probably needs to be added directly into cr2hdr

I'd be happy to give this a go at some point in time, but as I've never coded in C (only simple code in C++, and never image processing outside of Python), I'm not sure I'd have much success - would anyone be up for trying this?

in conjunction with DarkFraming Avg Process (Dual-ISO, Focus Pixel Fix and DF) all together ... My fault for the typo so-rose!

No problems  . Combining MLP and MLVFS does, at the moment, give the best results as far as I can tell; it seems like a bit of a roundabout way to do it, however. Implementing the focus pixel fix in cr2hdr would allow MLP (and convmlv for that matter) to process those three features - Dual-ISO, Focus Pixel Fix and DF - all by itself!

[so-rose]

If anyone's interested, I wrote some documentation for convmlv! It's a PDF found in docs->docs.pdf in the source code (https://bitbucket.org/so-rose/convmlv/src).

[DeafEyeJedi]

No problems  . Combining MLP and MLVFS does, at the moment, give the best results as far as I can tell; it seems like a bit of a roundabout way to do it, however. Implementing the focus pixel fix in cr2hdr would allow MLP (and convmlv for that matter) to process those three features - Dual-ISO, Focus Pixel Fix and DF - all by itself!

Exactly. Glad you followed me and I feel better that I'm not alone in this either. Looking forward to this implementation in the near future.

May the force be with you, @so-rose!

[so-rose]

May the force be with you, @so-rose!

Many thanks!

the focus pixel fixing probably needs to be added directly into cr2hdr

Seems like a challenge. I shall dive. No promises...



...I opened up cr2hdr.c, and cs.c from mlvfs, and I now fear for my life ! Though I'm incredibly unsure how to do it, I have an idea of what I need to do (and quite certain I can do it due to the magic of GPL):

From cs.c, I need to take the function fix_focus_pixels(struct frame_headers * frame_headers, uint16_t * image_data, int dual_iso) and all that it needs to function.

This I need to plop into cr2hdr.c next to (or in?) find_and_fix_bad_pixels(int dark_noise, int bright_noise, int* raw2ev, int* ev2raw), in such a way that fix_focus_pixels executed as a step.




The issues I have even at this point are numerous:

• What is the uint16 array image_data, and how might I set the output's pixel data in cr2hdr.c?
• Are the raw2ev and ev2raw integer arrays the same in cs.c as in the argument of find_and_fix_bad_pixels()? Oh and what is it  ?
• Several attributes of the pointer frame_headers are referenced, including what seems to be crop values (frame_headers->vidf_hdr.panPos<X or Y>). raw_info in cr2hdr seems to be the corresponding struct, but I can't find any "pan" values. Something that looks promising is found on Line 774 in cr2hdr.c, where the script sets an attribute called raw_info.active_area.<x/y 1/2>.
  
  I assume focus pixel handling is different based on whether the footage is taken with a cropped frame or not (that's how it works in mlv2badpixels.sh). If so, where can I find or get/how can I use what I have in raw_info to get these "panPos" values that I don't have, but seemingly need, to power the focus pixels script?
• What is the parameter dual_iso?

@dmilligan and @anyone Any ideas?

[dfort]

The pan and crop metadata isn't saved in the DNG files. Since cr2hdr starts with either the CR2 or DNG files it doesn't have access to this information. In order to find crop and pan you'll have to go back to the original MLV file. Actually you only need to find the pan because the crop can be derived from the pan--but not vice versa.

dmilligan provided some instructions on how you can read headers and footers from MLV and RAW files.
http://magiclantern.fm/forum/index.php?topic=16054.msg163145#msg163145

Since you are crossing the line from shell script to C programming I'd suggest abandoning my mlv2badpixels.sh as a reference and looking at the source code for MLVFS.

https://bitbucket.org/dmilligan/mlvfs

If you have any further questions, you already know who to ask!

[dmilligan]

I assume focus pixel handling is different based on whether the footage is taken with a cropped frame or not (that's how it works in mlv2badpixels.sh). If so, where can I find or get/how can I use what I have in raw_info to get these "panPos" values that I don't have, but seemingly need, to power the focus pixels script?[/li][/list]

Sorry, it's not possible to add the focus pixel stuff to cr2hdr (cr2hdr does not have the required metadata), it needs to be added to mlv_dump instead, where you still have crop information.

What is the uint16 array image_data, and how might I set the output's pixel data in cr2hdr.c?

read pixel at (x,y): p = image_data[x + y * w];
set pixel at (x,y): image_date[x + y * w] = p;

Are the raw2ev and ev2raw integer arrays the same in cs.c as in the argument of find_and_fix_bad_pixels()? Oh and what is it  ?

LUTs that convert linear raw pixel values to a log scale (EVs in fixed point) and back.

What is the parameter dual_iso?

Whether or not the footage is dual ISO. If it is dual ISO we have to interpolate pixels differently, we can't use the pixels above or below for interpolation because they will be at a different ISO.

[so-rose]

Status update: I'm trying to modify mlv_dump! No idea when it's going to work, but work it shall!

In any case, I finally made my script aware of Black Levels. It may interest people to know that dcraw apparently doesn't read Black Level info from the DNG file it's processing, leaving a magenta/green color cast. It's all fixed now, though; I compared the output to Darktable and MLRawViewer to be certain.


Thanks for all the support everyone! 

[Danne]

Good luck on modifying mlv_dump!

I havn,t had any problems with dcraw black levels. I recall though, some greenish output files on some early messing around of another script I was working on but I changed dcraw build and suddenly the problem was gone. Did you try the setting +M?

[a1ex]

I've noticed the dcraw bug with the prepackaged version that came with Ubuntu Wily; recompiled from latest source and the bug was gone.

The bug was also present in ufraw.

[so-rose]

@Danne I did, with no luck... I ended up exiftool'ing the Black Level from the developed DNG (works on Dual ISO footage as well), so that the user doesn't need to deal with it - this seems to be a universally usable solution.

It's interesting that the bug should be present in ufraw too. The dcraw on Debian at least has the bug.

[Danne]

The non universal solution(but faster) would be adding black level through hex. Dfort already cracked those numbers for white level.
Did you compile dcraw yourself?

*that was fo mlv files not dng files. My bad.

[so-rose]

The non universal solution(but faster) would be adding black level through hex. Dfort already cracked those numbers for white level.
Did you compile dcraw yourself?

*that was fo mlv files not dng files. My bad.

I'm just using the latest dcraw available in the Debian repository (testing), inserting the extracted BlackLevel tag after the -k option.

I only extract that value from one DNG, however - does it change? You mention that adding the black level through hex would be faster, but wouldn't I still have to specify -k, thus slowing it all down regardless?

Sent from my Nexus 6 using Tapatalk

[Danne]

Are you adding black level from dcraw or exiftool. Wasn,t aware you could change black level with dcraw. Wouldn,t that be faster than exiftool?