Dealing with Focus Pixels in raw video

[IDA_ML]

Amazing work, Dfort and Bouncyball! Congratulations!  I can't wait to test the new MLV_Dump on the 100D.   

Dfort, could you please post a download link to the latest focus pixel maps, so I can test with MLVFS?  Do I just replace the old maps with the new ones in the MLVFS directory or there is more to it?  Thank you.

[dfort]

The latest map files are in the ML Focus Pixels Bitbucket repository. Just replace the ones that are in MLVFS with these.

[IDA_ML]

Thanks a lot, Dfort.  I will test as soon as I can.

[bouncyball]

MLV App also updated with:

1. Latest focus dots fix (kudos @dfort).
2. Correct aspect ratio according to RAWC block info.

[dfort]

Made some more changes on the fpm.sh script.

When in doubt whether to use mv720 or crop_rec map files, default to crop_rec because it will work with either video mode.

Using the -n option to create full raw buffer sized maps will also default to crop_rec because it is assumed that these map files will be used in MLVFS that can't can't yet differentiate between crop_rec and mv720.

I also looked into removing dependency of the Crop metadata but it turns out that it is necessary to line up the map files properly.

Still todo -- run that 1-pixel offset map for mv1080crop mode only if it is 8...12bit lossless compression. Not a high priority because running additional passes that aren't really needed doesn't seem to hurt the image. At least not much.

[dfort]

Did some testing on the 100D. The 1:1 video modes, mv1080crop and zoom, could also have the focus pixels offset 8-pixels to the right. Discussion on the 100D topic.

Updated the map files for MLVFS, updated the MLVFS pull request and updated the fpm.sh script.

[IDA_ML]

Dfort,

You are a genius!  Amazing work:

https://www.magiclantern.fm/forum/index.php?topic=16040.new#new

[IDA_ML]

Dfort,

Testing the latest January 20-th build for the 100D which allows handheld FRSPs with stabilized lenses, I noticed some focus pixels when shooting at Dual ISO 100/800.  Here are the original shot (size reduced) and a 100% crop from the area where they are clearly seen:





This is the original image.

Focus pixel area (100% crop) -> Here the "Add image to post" does not embed the 100% crop.  I don't know why.  So please magnify the above image and you will see the focus pixels along the edges of the cupboard doors.

[dfort]

Are you referring to the colored area under the knobs?



These aren't focus pixels. You should get similar artifacts on the 7D (camera no focus pixels) as you do on the 100D (camera with focus pixels). That's why some ML users won't shoot without a VAF filter on their camera. Well, except for 5D3 users, note that you can't even get a VAF filter for the 5D3.

[IDA_ML]

No, I am referring to the long vertical colored edges of the cupboard doors.  They look like a string of red/magenta  dots.  Please have a look at the photo at 100% magnification and you should be able to see them.  When I get back home, I will try to find the 100% crop from the original where these dots are clearly seen. 

[dfort]

Ok. This is a Dual ISO image? I'm surprised it looks this good. My understanding of how MLVFS fills in the focus pixels for Dual ISO is that it only samples the neighboring pixels on the horizontal axis. This is because Dual ISO has a different ISO every 2-pixels vertically. That's why it is difficult to remove the focus pixels on high contrast vertical lines. Someone please correct me if I'm wrong on this.

[IDA_ML]

Dfort,

Following your request, I have now prepared a complete (almost) set of clips of the Pixel Scanner pattern for you to test and see whether you have totally and once for all killed all focus pixels on the 100D or there are still a few left.  Using the October 23-rd, 2017 build I shot the clips in the RAW-video mode as follows:

1) Uncropped (at 1736x976 resolution)
2) Movie Crop Mode (at 1736x976 resolution) and
3) 5x-magnification mode (at 2520x1080 resolution)

In all these three modes I shot a clip at 14, 12 and 10 bit uncompressed and 14, 12 and 11...8 bit losslessly compressed bit rates.  Here are the download links for the original MLV files that will be active for the next 7 days:

For the 5x_Magnification_100D_FP.rar and the Movie_Crop_Mode_100D_FP.rar archives:

https://we.tl/Kr6ltVMayi

And for the Uncropped_100D_FP.rar archive:

https://we.tl/ttpzKYjXHy

Please note that some of the files (e. g. the ones shot at 10 and 12 bit uncompressed in the Movie Crop Mode) contain corrupt frames but there should still be enough sound frames for you to check for focus pixels.  Shooting the moving test pattern was quite a challenge for the 100D!  I had to repeat the shots several times to get a few OK.

I hope, this helps.

Happy birthday to you, Dfort and thank you so much for all the presents we received from you lately!

[IDA_ML]

Ok. This is a Dual ISO image? I'm surprised it looks this good.

Oh, don't be surprised!  This is the 100D - a hell of a camera!  Even more so now, that A1ex has added working image stabilization to the silent module which allows for hand held low-light Dual ISO FRSP-pictures with amazingly clean results using stabilized lenses.  Once the hiccup issue is fixed, we should be able to shoot even Dual ISO video with comparably clean results at 2520x1080 resolution!   For now, the 100D is second best after the 5DMkIII for Magic Lantern work.  Now you understand why I am so eager to get the 100D working properly. 

[dfort]

Please note that some of the files (e. g. the ones shot at 10 and 12 bit uncompressed in the Movie Crop Mode) contain corrupt frames...

Add to that 14 bit uncompressed. That's strange, all of the uncompressed settings show corrupted frames in Movie Crop Mode. You should probably report that on the 100D topic. Maybe that "hiccup" build I made also fixes this issue?

I just processed everything into movie files and need to take a closer look but first impression is very good--at least as far as focus pixels are concerned.

...Now you understand why I am so eager to get the 100D working properly. 

Maybe a little less eager on the 7D? Getting 10/12bit working on that camera is a tough nut to crack.

[IDA_ML]

Absolutely!  I can't wait for 7D to become Number 2!

By the way, you did a hell of of a job with the focus pixels on the 100D.  Congratulations!  Did you notice that also sound is working on the 8...11bit ant 12bit lossless?

[dfort]

I looked closely at the latest 100D tests and the new maps are working quite well when pixel peeping still frames but look what happens when you put things in motion:

Fixing the focus pixels actually seems to help a bit with some of the aliasing issues. I didn't expect that 14-bit lossless exaggerated the aliasing, maybe the camera moved slightly on that take? Not much seems to happen in the 1:1 modes in video but we are seeing the "hiccup" issue in 5x zoom mode. The 14-bit lossless take is a bit short, for obvious reasons, so I slowed it down 50%. Note some glitches in zoom mode but it is perfectly useable. Uncompressed mv1080crop (a.k.a. Movie Crop Mode) was unusable but the lossless footage was quite good--yet another unexpected surprise.

What isn't surprising is the moiré pattern in the 3x3 sampling mv1080 mode caused by the vertical line skipping. The 1:1 sampling modes show almost no moiré pattern or aliasing. The rolling shutter, evident by the slanted lines, is also less in zoom and Movie Crop Mode.

The "no focus pixel fix" footage was processed through mlv_dump and the "with focus pixel fix" was done with MLVFS. I was just trying different workflows in Danne's Switch app. That's part of the reason why there's a slight color shift between the the two. Another reason is because we're removing a bunch of green and magenta dots. In some modes it is predominantly green, in others magenta so even though these are complimentary colors they don't exactly cancel each other out.

Overall, quite an interesting test. I think the take away from this is to avoid shooting moving vertical lines if at all possible! Don't run that video too many times or you might get hypnotized.

[IDA_ML]

Dfort,

All clips that I provided were recorded with the camera on a stable tripod.  The only shake that could have occured would be when pressing the record button.  This is the reason why I recorded at least 3-4s. of every clip so that you have enough useable frames.

I am happy to see that the test clips were helpfull to gain new knowledge on the focus pixel/aliasing behavior of the 100D.  I have to say that the camera struggled a lot with this test - the high-contrast periodically moving pattern seemed to bug the camera and very often it would refuse to record the clip, so I had to repeat the recordings several times until I got one recorded, sometimes with lots of corrupt frames.  This never hapened with normal scenes, except those shot at 12 and 10 bits uncompressed in the Movie Crop Mode.  I don't know how to explain this behavior. 

Anyway, you seem to be right by saying that the focus pixel fairy tale is a never ending story, not to mention  Dual ISO!  What I am very happy about though is the fact that under normal shooting conditions, the focus pixels do not seem to be a problem any more.  And I thank you for that!

[theBilalFakhouri]

Found the two registers that's related to the lovely pixels dots:

ADTG[8196] N    (dwSrFstAdtg1[2], Line count + 1)      mv1080=0x12e       x5=0x0
ADTG[8197] N    (dwSrFstAdtg1[3])                             mv1080=0x38c       x5=0x0

1-You can make any pattern you want!
Mode=mv1080 | 14bit-lossless | No focus map applied |Original MLVs Here

Normal


Tweaked registers (vertical pattern on the whole image) click and zoom in.


at 0x0 it's become very dim


2-You can shift it myabe. (I lost the settings).

In x5 mode maybe more focus pixels appears and I got corrupted frames while recording when I pushed the values up.
In mv720 mode it's really hard to see what is going on because adtg_gui makes the camera freezes in that mode.
In photo mode these registers disappeared.
Make try on your camera (set preview mode to Framing to see the changes directly).

@dfort

Affected cameras: EOSM, 100D, 600D, 650D, 700D, (and more?)
[EDIT: Pretty sure now that the 600D doesn't show focus pixels in raw video.]

How the 600D was affected in these pixels (same patterns as 700d?) and what is the way that solved it?

@a1ex or @dfort (anyone have 5D3)
Can you check what is the values on 5D3 ? and if can try 700D's values above in mv1080 mode ?
And what is these registers job?

[dfort]

How the 600D was affected in these pixels (same patterns as 700d?) and what is the way that solved it?

Actually, nothing was done. The 600D is a Digic 4 camera and only the EOSM/100D/650D/700D (Digic 5 cameras) show focus pixels, as far as we know. The other Digic 5 cameras don't show focus pixels. OK, maybe the EOSM2 because it is from the same family but that has yet to be seen.

Note that if you use the silent module and shoot a simple silent DNG while the camera is in movie mode at the settings that you are trying to test, the saved DNG should also show the focus pixels. Processed through dcraw using the -4 -E options and you'll be able to have a clear idea what needs to be done in order to remove the focus pixels. I know this is a lot to go through but if you read through this discussion you'll see how focus pixel "fixing" works.

You're more than welcome to play around with my script and focus pixel map files:

https://bitbucket.org/daniel_fort/ml-focus-pixels

or try out the more advanced mlv-tools by @bouncyball :

https://bitbucket.org/bouncyball/mlv-tools

[theBilalFakhouri]

Thanks @dfort
I am trying to find what are these pixels and why we see them in RAW video.

[dfort]

Seems like these mysterious focus pixels keep coming back to haunt me. Another user pointed out that the full raw buffer has changed and MLVFS is showing focus pixels again. That should be fixed with some of the recent commits to the crop_rec_4k branch. In addition, 1920x1080 zoom mode should once again be working on the 100D/650D/700D/EOSM.

These commits will also allow us to simplify the focus pixel map files but if we fix them now anything shot with the crop_rec_4k branch in the last few months might show focus pixels so this is just a note to say that we're aware of the issue and a reminder to once again revise the focus pixel map files.

[dfort]

4K and other resolutions are starting to become a reality on the crop sensor cameras including the ones with the focus pixel issue so it is time to start updating the focus pixel maps.

What we need are some frames that show focus pixels at these new crop_rec settings. In addition, we need to capture full raw buffer images of the various settings. The way to do this is to turn on the silent module and set it to simple silent. Next, set the camera to movie settings using the crop_rec settings you want to use. Now simply take some simple silent images by doing half-shutter presses. Make sure to keep the camera in movie mode for this. Upload the MLV and DNG files and post links on this discussion so we can start working on this.

[theBilalFakhouri]

This is a good idea I was thinking the same but it's too early for now because of some presets need to adjust the horizontal and vertical offset to make it centered in the right way (now it's close to center) and this will change the position of focus dots.

I will try to do this ASAP.

EDIT: Of course if this will not affect the progress ? I mean we can map the focus dots now and shift it later?

[a1ex]

I'd like to automate the centering part (to use generic code for all models, rather than hardcoding values for every single preset), so the cropped position might change a bit. Just FYI.

[theBilalFakhouri]

Hello @dfort

Here a files for 1x3 Binning with full sensor height 1736x3478 it should cover all the focus pixels in this mode. And Here Full-Resolution LiveView MLV it should cover all focus pixels in 1x1 Binning mode maybe we should calculate only the offsets in MLVApp and etc..

We don't have to capture it in Pixel Scanner pattern. We are using 14-bit lossless and the focus pixels is static, I tried to capture it in the pattern but it's didn't show it better than the above samples.

Can you please generate the focus pixels maps?

@a1ex
Something I didn't get it very well, why the Full-Resolution LiveView MLV file has size of 64 MB (EDIT: It's only 1 frame)? and also it's 10-bit uncompressed. When using silent picture in 14-bit uncompressed it gives Max size ~30 MB . Of course when exporting the MLV to DNG file it gives ~21 MB, but why the MLV gives 3 times larger size?

Thanks!