Dealing with Focus Pixels in raw video

[Jonit]

Awesome dfort! Thank you for your work. Exactly what I need for my two cams  .

[2blackbar]

Hey dfort, im trying to fix 1736x738 resolution in mv1080 , 12 bit mode on canon M but cant find focus pixel map for it to use in mlvapp, downloaded this https://bitbucket.org/daniel_fort/ml-focus-pixels/downloads/
Heres the MLV file :
https://drive.google.com/open?id=1zAL1oTAm5TqExNadjr9zxIHQpGm0OHGu
https://drive.google.com/open?id=1FY50wEnIhZY8FwU2XcnSzq7RLdcwcVZP
From what i see these focus pixels are moving up/down every second .
trying to work on it found some tools with gui
https://www.magiclantern.fm/forum/index.php?topic=19460.0
http://meguidcraw.blogspot.com/2009/07/descarga-download.html

The file is 1808x1188 and i do see it in folder 80000331_1808x1188.fpm
But it sill doesnt work, why is that ?

[dfort]

@blackbar - The files you uploaded have been run through MLV App and it appears that most of the focus pixels have already been processed.

Code: [Select]

Block: VERS
  Offset: 0x0000023c
  Number: 8
    Size: 40
    Time: 18446744073709552.000000 ms
  String: 'MLV App version 1.6'

Could you shoot a short sample and upload an unprocessed MLV file? You can also trim down to just a couple of frames using mlv_dump.

Code: [Select]

  -f frames           frames to save. e.g. '12' saves frames 0 to 12, '12-40' saves frames 12 to 40

For example, this saves the first couple of frames:

Code: [Select]

mlv_dump -o output.mlv -f 2 input.mlv

From what i see these focus pixels are moving up/down every second .

In that case can you try to trim down to a range where they show the shift in position?

If you can't get mlv_dump working, simply switch off the "Fix Focus Dots" option in MLV App.

[2blackbar]

Hey thanks for reply ! i doscovered something, we have 3 different focus options to choose from canon menu , what if you need yo make only pattern for flexizone , i had it set as tracking.When i set it to flexizone i can clearly see the zone in the middle and foxus dots pattern looks totally different.
Ill reupload ! Maybe you dont have to make focys maps for every focus modes and just tell people to use the simplest one that isnt moving so much.
Ill do some tests now.
--
Ahh i cant get it to change now... i swear ive seen different pattern onscreen it was something like this but i dont know how to get it again

[2blackbar]

https://drive.google.com/open?id=1RnltM0HBnD_6ST-9kh9H8X5e2I6uulkd
https://drive.google.com/open?id=1sQ_JbnVaZYs3EninR7FyIJnEXRuq0r9E
Files, from what i see there is 2 image pattern with 2 variations moved up/down and i cannot see simialr pattern in focus_pixel_image_files or im just blind.

[dfort]

Got me stumped on this one. I played around with the blank wall shots and it looks like there is an mv720 focus pixel pattern moving over this in addition to the mv1080 and full resolution (1:1) pattern. Crazy. I got something that looked like maybe it was working but when I tried it on the baby shot--Yikes! Not even the map file I started with is working well with it.

Here is what it looks like without any focus pixel correction--obviously pushed the saturation, contrast, etc. to the max to make the focus pixels pop.



With the focus pixel map file no matter what I did it would put a pattern over the areas that don't need correction. Yeah, the focus pixels are gone but yuck!



There is another interpolation method in MLV App and it is a bit better for this image but note the focus pixels that show up in the high contrast border areas:



I tried chroma smoothing but these focus pixels are just too stubborn.

Maybe my map files are too aggressive? Maybe, but the extra passes with different patterns are still needed because the focus pixels are showing up on various areas of the image:



Some of that is noise but the repeating pattern looks like focus pixels to me.

This was shot at 12bit lossless which should be a good setting but I've had issues trying to remove focus pixels from files shot with reduced bit depth and lossless compression. Maybe play around with dark frame averaging or some other tricks?

[2blackbar]

This is bizarre, thanks a lot and do You have canon M ? Maybe Danne could help and point me how should i record cinemascope croprec on canon m to get focus pixels that would be removable, its quite popular aspect ratio, ive seen 1808x1088 resolution in your folder for canon M but they look different.
Is there something i can do on my side to make existing focus maps work for my footage in the future ?
It looks like mcm rewire will give me different focus pixels , ill try that and post here so others would know if that made a difference.

[dfort]

Maybe I'm trying to do too much with my map files. I'm trying to cover every possible situation. Maybe try going through the tutorial and create a map file that has the minimum number of coordinates. You can download the map files from MLVFS and see if any of the old map files work for you.

Keep the samples of the baby online and I'll see if I can get back to this when I have some time. Seems like the problem shows up more on that shot than on the blank wall shots you made. Those just look like a bunch of noise. If you look back through this topic you'll see we had a hard time tracking down the focus pixels on reduced bit depth lossless compression settings.

[2blackbar]

Daniel i tested MCM rewire mode on canon M and it gives the same resolution and quality in raw video like the files i provided but they have no issue with focus pixels !
Thats what Danne suggested and i tried, footage doesnt have focus pixels issues after export from mlvapp , so i would like to thank You a lot for trying , it looks like problem was already solved and i used raw mode improperly
I dont know about you spending time on solving this one, it might be waste of time unless this mode has some advantage over mcm rewire on canon M, Danne probably knows this, i just prefere if people recording with EOS M knew in what mode they should record to avoid focus pixels pattern that i had , otherwise they endup like me with 100 raw videos with pixels dancing onscreen and will be back here asking you for some help  im just glad that it was solved by You and Danne long time ago and it was mistake on my part.Thanks a lot!

[dfort]

Check this out--no focus pixels:



This is using the current focus pixel map file. The difference is that I used dcraw with the median filter to zap out the artifacts created by the map file.



I used one of my archived scripts called mlv2badpixels.sh to create trimmed map file (a.k.a. "Bad Pixels list") that can be used in dcraw. That script reads the metadata in the MLV file then trims the 80000331_1808x1190.fpm map file to the right size and names it (in this case) EOSM_mv1080_1736x738.txt then that can be used by dcraw to remove the focus pixels. The median filter is sort of a chroma smoothing on steroids filter. I made no color correcting on the image.

Code: [Select]

./mlv2badpixels.sh M06-1210foxusd.MLV
mlv_dump --dng --no-fixfp M06-1210foxusd.MLV
for x in *.dng; do dcraw -m 5 -T -P EOSM_mv1080_1736x738.txt $x; done
The MLV file was trimmed down to make it an easy file transfer so when it is looped on the timeline it looks like the baby is rocking out.

[2blackbar]

Amazing , ahah nice edit  , i will give it a try cause i have this script of Yours.
Youre fast, about the dead pixel, chroma smoothing 3x3 takes care of it but id prefere to fix this in cam.

[aulden]

Hmm im having problems focus pixels with MLVFS. Im on mac and using Dannes latest build with 1080p MCM rewire. 1736 x 736 resolution.

I managed do get rid of the focuspixels on 1080p MCM rewire with mlvfs by adding the pixelmaps to the mlvfs script before installing.

BUT when using the "3x crop" with the 1080p MCM rewire mode the pixels persist? Any ideas?

https://drive.google.com/open?id=1tgKlaKWNmw9j3fmjcJmr0xSAJyycg5Vh

[dfort]

Looks like a new one. I'll post on this topic when I have a new map file.

[dfort]

@aulden - New focus pixel map file uploaded. Look for 80000331_1872x1110.fpm.

Note that these map files are named according to the full raw buffer size. You can find it like this:

mlv_dump -v M12-2025.MLV | less

Code: [Select]

...
      height           1110
      width            1872
...


[aulden]

@dfort

YESSSSS! Thank you so much, works like a charm!

[Danne]

Dfort delivers once again. Great!

[dfort]

Realized that there are several newcomers trying to figure out how to use the focus pixel map files so here's a quick tutorial:

The easiest way to get the map files is to download the whole repository from the downloads page:

https://bitbucket.org/daniel_fort/ml-focus-pixels/downloads/

Take all of the .fmp files in the focus_pixel_map_files directory and place them next to the MLVFS binary. I'm on a Mac so it is located in my home directory > Library > Services > MLVFS > Contents (doing Control click on the MLVFS in order to get into the package contents). Never used MLVFS on Windows or Linux so I'm not sure how that works.

Same thing with MLV App, put the .fpm files next to the binary. MLV App is currently the best way to work with 5K Anamorphic files and yes, it is possible to export cdng using that application with fixed focus pixels and adjusted aspect ratio.

Either MLVFS or MLV App is a great way to get your raw footage into editing systems that can handle cdng files like DaVinci Resolve or After Effects. However -- only editing apps that use Adobe Camera Raw (like After Effects) seem to respect the resize and aspect ratio metadata in the cdng files so you'll have to adjust the image size if you want to use these dng files directly in Premiere or Resolve. You can also export to Prores or other several other file formats from MLV App and not worry about the resize/aspect ratio issues.

[aulden]

Hmm, can somebody please help me out.

There are white/red/blue dancing pixels when shooting in low light. Might be related to the focus pixels?

When importing to resolve with MLVFS, they persist. But when importing into mlvapp - gone! Any idea how to deal with these pesky pixels with resolve + MLVFS?


https://drive.google.com/open?id=1PXmbhbjuwDBi2DnYoW-TfqLEcZ6jIKWj

[dfort]

Whoa--that shot is way underexposed.



Looking at the metadata I see you're playing around with Danne's EOSM crop_rec settings. You'll need to load the focus pixel map files in both MLVFS and MLV App to work with these settings. Simply download the project repository and place the .fpm files next to the executable. Check back a few posts to see how it is done. Sorry--limited time, I'm still on vacation in Costa Rica.

This is what I'm seeing -- note that turning off "Fix Focus Dots" then turning up the contrast, vibrance, saturation all the way up usually shows the focus pixels but in this case all I see is noise.

[aulden]

dfort, please enjoy your vacation!!!

Hehe yes, i was shooting at night and tried to recreate it in my bathroom. So sorry if i was unclear. The focus pixels are not the issue here, its those bright red, white and blue pixels that seem to appear randomly. I thought they might be a result of the focus pixel maps but upon further inspection they seem to be embedded in the original DNG files.







There are quite a few of them at times, maybe 8ish at most. They dont appear in MLVapp, but they do in Davinci Resolve which is my program of choice. I realize now this might be the wrong the wrong place to post it, but  if you could perhaps point me in the right direction i would be greatful!

Oh yeah, Dannes newest build, shot with MCMrewire 1736x736 14bit

[dfort]

So you are saying that these colored specks appear in Resolve when looking at the exported DNG files? These appear in both MLVFS and MLV App but you can't see them in the MLV App user interface? I think you are looking at noise but they might be hot/dead/whatever pixels. I wrote a post on how you can map these out:

https://www.magiclantern.fm/forum/index.php?topic=16054.msg212690#msg212690

I see you also posted this on the MLV App discussion. They are very good about answering user questions.

[kayman1021]

Hello

I have a Xiaomi A1 smartphone, which is able to take .DNG raw files with firmware modification. This sadly also introduces 200k focus pixels in my pictures,that also affects phone generated .JPEGs.
I have a few thousand pictures taken already. RawTherapee does very good job removing them, also I made a .BADPIXELS file to average them out. It was made by following the forum thread. Many thanks for that.

I've decided to get rid of them by modifying the DNG's copy. I've made a simple C# program, that is able to extract the raw data from my dng-s to a greyscale 16bit tiff file without any loss of raw information. Quite sure it is not compatible with anything else. Also, filepaths are hardcoded.

When fiddling about with self made pixel maps, I've noticed that under the focus pixels, the raw data's histogram is veeery similar to the original picture's. My guess would be focus pixel values are just offset from the optimal values, by a constant or maybe a*b+c.
I am a bit excited, maybe it could help removing focus pixels from ML videos, by not averaging, but backtracking the original value, eliminating the averaging articacts or chroma smooth, which affects the whole picture.
I can continue next monday afternoon if it is true to ML expoted dngs.
My phone uses 16bit BGGR pattern.
https://drive.google.com/drive/folders/1I9eSsfgxjZnuUc3sbdGCCVxIzhCa0MBX?usp=sharing

[dfort]

Interesting findings especially if it could apply to the focus pixels in ML supported cameras. Doing adjacent pixel interpolation doesn't work on the edges of the frame and sometimes in high contrast borders. Dual ISO is also a problem.

Your C# program uses the Microsoft .NET libraries which I'm not familiar with but noticed that you have embedded a sample image and your pixel map file. I can't see any focus pixels in your sample even when turning the vibrance and saturation all the way up:



Here's a detail of your focus pixel map file.



Interested in what you come up with. Please post here if you need any sample files from the various cameras that show focus pixels.

[kayman1021]

On my x64 Windows 5.5 RawTherapee makes kid of pop. Near the right corner of the building it is visible as yellow dots.

Neutral profile:
https://flic.kr/p/2fThTrf

And with .badpixel file:
https://flic.kr/p/2fXT1vn

I can make them more visible by separating B,G1,G2,R values. Here is an output from my program. The tiff is too dark by default, so i needed to adjust the level. I have focus pixels only in the blue data = even column, even row coordinates in the original.

https://flic.kr/p/2fXTtvV


I bought a 100D camera, and i made some ML videos a few weeks ago, mainly anamorphic. I'll try my luck with some exported dng from them.

[Danne]

I bought a 100D camera, and i made some ML videos a few weeks ago, mainly anamorphic. I'll try my luck with some exported dng from them.

Cool. Make sure to run the latest build. I put in some refinements:
https://www.magiclantern.fm/forum/index.php?topic=16040.msg216728#msg216728
and:
https://www.magiclantern.fm/forum/index.php?topic=16040.msg216554#msg216554