Dealing with Focus Pixels in raw video

[Danne]

if it doesn't have an impact on the quality of the image or processing speed maybe that would be a way to get crop_rec working in MLVFS without having to swap out the map files?

+1

[dfort]

Committed bouncyball's 100D crop_rec map file to the ML Focus Pixels repository.

Now for the instructions how to use this in MLVFS.

Grab that map file any way you want--here's a direct link to it. Right click, save link. Make sure it is named 80000346_1808x726.fpm.

Now copy it to your MLVFS contents. On the Mac MLVFS is located in your home Library/Services. Right click (control click on Mac) and choose Show Package Contents. Copy the map file in there. Sometimes the raw buffer is 727, sometimes 726 so make a copy of that map file and rename it 80000346_1808x727.fpm. Yes, I know there's already a file in there by that name, save it somewhere safe because you might want to go back to it if this experiment doesn't work.

Ok now--100D owners, shoot some crop_rec and some mv720 footage and check for focus pixels. Please report back.

[bouncyball]

Oops--I missed that when I tried to re-create your map. It does clean up nicely in MLVFS using your 2-pass map and doesn't need any chroma smoothing in MLVFS.

Nice to hear. I was sure that you gone the way I did by yourself and just missed some details .

I'm having problems with some of my scripts that I use to convert fpm to pbm and back but once I get your map converted to an image file I'll save that too.

Give a try to my 'fpmutil' it converts fpm->pbm->fpm in no second. Run './fpmutil -h' to get detailed help with examples. Note: you have to compile it because existing mac binary not updated and does not include the latest 100D crop_rec changes updated.

Also I'm sure you noticed header in the FPM. Header format is:

Code Select Expand


Sig  CamID    W    H   Crop program vers
----------------------------------------
#FPM 80000346 1808 726 1 -- fpmutil v0.5

and it's needed to convert fpm->pbm

If you want to convert your FPM to PBM you just need to insert this header as the 1st line of FPM and fill with appropriate values after '#FPM' signature with white spaces in between.

I could implement this header to MLVFS and make file name format like this: CamID_Width_x_Height_croprec.fpm. Had no time to do this yet.

Edit: unfortunately detection of crop_rec by RAWC header is not implemented anywhere yet.

[dfort]

Wow, you took this topic to a higher level. I'm just now trying it out and it is lightning fast compared to my scripts. Have you considered submitting it to the contrib section of the official ML source?

[Danne]

Even better. Bouncyball put fpmutil code into mlv_dump_on_steroids already:
https://bitbucket.org/bouncyball/magic-lantern/commits/408e300eb5841a90479300a7289086eca3b3f15b?at=mlv_dump-on-steroids

[bouncyball]

@dfort
Glad you liked it
Now try the link Danne posted above. On the fly map generation is implemented to the mlv_dump itself. No more fpm files in in the mlv_dump directory needed.

Have you considered submitting it to the contrib section of the official ML source?

Nope, I have not yet.

[dfort]

The 2-pass crop_rec map file is working so well with the 100D that I decided to do the same with the EOSM/650D/700D crop_rec map. The changes made the code much simpler.

Need to do some more testing but it should solve the hassle of having to swap out map files in MLVFS when switching between mv720 and crop_rec footage.

[DeafEyeJedi]

Damn @bouncyball's at it again. Keep on rolling and please never stop bouncing around! 

To be able to get clean footage from cameras affected by Focus Pixels without the need of Chroma Smoothing is just godsend!

[bouncyball]

Keep on rolling and please never stop bouncing around! 

Well... I'm trying

[bouncyball]

I was testing 2pass method on cameras other than 100D and discovered that 1st pass is not needed for 700D (only mv1080, no mv720). EOSM definitely needs it. What about 650D? I don't have the sample MLV. I guess it does not support crop_rec at all.

[Danne]

On its way...
https://bitbucket.org/daniel_fort/magic-lantern/pull-requests/6/crop-rec-4k-650d/diff#chg-modules/silent/lossless.c

[bouncyball]

Good. Do you have some samples?

[dfort]

...1st pass is not needed for 700D (only mv1080, no mv720). EOSM definitely needs it...

Interesting. Usually what works on on the EOSM works on the 700D and vice versa. We're close to getting the 650D working in crop_rec and my bet is that it behaves the same as the 700D. These are all of the "pattern A" cameras so once the tests are in we can make specific cases for the cameras that need the 2-pass treatment and those that don't need it--though with zoom mode I'm hitting areas that don't need it and it doesn't seem to hurt. Then there's the zoom mode on the 100D that doesn't show any focus pixels at all. Go figure.

[dfort]

On second thought--it makes sense that the EOSM needs an mv720 pixel map pass because that camera is "normally" in mv720 mode except when recording H.264.

[bouncyball]

I need more 700D (crop_rec/no crop_rec) samples. Can someone upload them?

[bouncyball]

The funny thing is I never possessed/borrowed the camera affected by focus pixels

[saulbass]

Hi Bouncyball - I have a 300mb mlv from a 650D running dforts latest crop rec experiment crop_rec_4k.2017Oct11.650D104.zip - it's 2512x1072. 14bit lossless compressed.
How should I pass it over to you - Hightail?

[Danne]

Hi saulbass. Noticed you used Switch? You can shrink this file by creating a sample file package(zip). You do it from either (ms) mlv_dump_on_steroids or (m) mlv_dump menus:

[saulbass]

Danne - thanks for the pointer - really useful to know - cheers!

[dfort]

@saulbass - could you check if this is happening on the 650D?

https://bitbucket.org/hudson/magic-lantern/pull-requests/872/crop_rec_4k-for-650d104/diff#comment-47051072

[bouncyball]

Hi Bouncyball - I have a 300mb mlv from a 650D running dforts latest crop rec experiment crop_rec_4k.2017Oct11.650D104.zip - it's 2512x1072. 14bit lossless compressed.
How should I pass it over to you - Hightail?

Sorry about very late response. It does not matter, just upload it somewhere and give the link.

Thx
bb

[theBilalFakhouri]

Hello
I noticed today a small problem with 12/11-8bit lossless pixel dots (14bit lossless works fine)
The focus pixels only appears in Contrast edge image and in the whole image except flat dark or flat white parts
Used camera: Canon 700D with croprec4kmlvsnd.2017Oct23.700D115 build
ISO: 100
All was Shooted at 12bit lossless on mv1080 mode 1736x976
Examples with no focus map applied :
White out of focus:

Dark image:

Contrast Edge image:

And this with focus map applied: 80000326_1808x1190.fpm focus map (only the middle cleard)

Orginal MLV, DNG samples and Images here:
https://drive.google.com/open?id=1-TaNgr0ZTjQ_Ea4Q6tPjAZVTIW_7qSG0
If this problem discoverd before, Can you give me the Map for this pattern? I tried all the maps didn't work.

[dfort]

Wow, fantastic testing!

I've been going though your sample files and have tried a few different methods. The best results I've gotten so far is using mlv_dump_on_steroids (via Switch) and adding a 2x2 chroma smooth. It looks like you are using MLVFS and it should work the same there but I didn't test it because I broke it on my system, need to look into that.

I made a tiff from the dng file using dcraw. This is tends to show focus pixels more than Resolve, Adobe Camera Raw and other raw converters. dcraw is also showing some artifacts that can be smoothed out with the other apps but I wanted to show a worse case scenario.



Click on the image to see it full sized, zoom in, download, pixel peep, whatever.

Note that the test pattern that you are using is really challenging the pixel removal. Hard contrasty edges will often show the focus pixels even when running through the map file and that's why I added some chroma smoothing.

Ok--so we know that this worked with your 14-bit file but the problem is with the 12-bit file, right? Where's the 12-bit MLV? From your examples it looks like you found focus pixels that are outside of the area where we know there are focus pixels. Not surprising because there are settings that can show strange patterns all over the image even on cameras that don't normally show focus pixels.

mlv_dump -v

Code Select Expand

    raw_info:
      api_version      0x00000001
      height           1188
      width            1808
      pitch            3164
      frame_size       0x0039674C
  --> bits_per_pixel   14
      black_level      2047
      white_level      5586
      active_area.y1   28
      active_area.x1   72
      active_area.y2   1185
      active_area.x2   1808
      exposure_bias    0, 0
      cfa_pattern      0x02010100
      calibration_ill  1


[a1ex]

Ok--so we know that this worked with your 14-bit file but the problem is with the 12-bit file, right? Where's the 12-bit MLV?

Right here (explanation):

Code Select Expand


  --> bits_per_pixel   14
      black_level      2047
      white_level      5586


Note the 8...12-bit lossless mode uses a different LV raw type. There might be other raw types that allow image darkening via digital gain (to "reduce" the bit depth - actually the range of useful levels - while keeping the "old" focus pixels) - no idea how to find them other than brute-forcing (or whether they exist).

[dfort]

Learn something new every day. I've been doing lots of testing with 10-bit and mlv_dump shows bits_per_pixel as 10.

@theBilalFakhouri - I can clearly see focus pixels in areas where they haven't appeared before so I'll do some more digging into this. I can make a fpm file that covers the entire frame, like it does on the 100D, but it will take me some time before I can get to it.