Dealing with Focus Pixels in raw video

Started by dfort, October 22, 2015, 11:09:10 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

2blackbar

can i get it fixed by chopping one pixel or 2 in ML code  for crop mode ? so its 1736x976 vs 1736x734 , so both modes have different resolution

masc

Your final resolution don't cares. Identification is done with raw buffer resolution (metadata).
5D3.113 | EOSM.202

2blackbar

Best at the moment without fix would be if i could choose manually which focus pixel map should be used

Danne

I could fix it like this and upload a new build.

For non crop mode it stays at this(buffer):
case 0xC0F06804: return 0x4a601e4
When selecting crop mode it will record following:
case 0xC0F06804: return 0x4a701e4
Should give same resolution but it seems to activate the correct maps.

It won´t fix the old files though.

masc

What will this do? Change the metadata in the MLV?
5D3.113 | EOSM.202

Danne

Yes. For x3crop it will look like this:
      active_area.y2   1189
      active_area.x2   1812

For non crop:
      active_area.y2   1188
      active_area.x2   1812

This nudge seems to activate all the right dotmagic.


2blackbar

Great, i applied croprec.c changes to my custom build.
---
Well, new code is interfering with my custom 2.5k mode settings and i have black bar on right side of the screen, why is that? Or what should i remove to not interfere with 2.5k settings at all ?

It works fine before i add this new code :
    //needs to differ raw buffer to fit pixel map files in mlv app
    if (ratios == 0x0 && x3crop == 0x1)
    {
        switch (reg)
        {
            case 0xC0F06804: return 0x4a701e4 + reg_6804_width + (reg_6804_height << 16);
            case 0xC0F0713c: return 0x4a7 + reg_713c;
            case 0xC0F07150: return 0x430 + reg_7150;
        }
    }
   


When i add only this and remove that ratios line from code a bit above it ( line 3710) then 2.5k mode is changed, i will test if its broken in your build.
--
Looks like its fine in Your build, ill adjust it on my end.
But the fix does work !

Danne

Please test my build first and see if it works properly in x3crop mode without any borders etc. Then you can post anything you want but I want to be sure the original intent is working as it should.
The modified 2.5k mode? Is that modification also with mcm rewired but in x3crop mode? HArd to follow this.

Edit: Ok so it seems you are on to something. Saw you updated your post.

2blackbar

Yup, your build is all fine, ill start from copying your addresses and try to get it working for my cctv lenses again with minimum vignette like before !
Great work !
I use different addresses for 2.5k mode to get framing right for my cctv lenses, its a matter of finding them again with this new change.
I use this :
            case CROP_PRESET_2K_EOSM:
  cmos_new[5] = 0x2c0;             /* vertical (first|last) */
              cmos_new[7] = 0x2a9;    /* pink highlights without this */
                break;


And this is Yours:             case CROP_PRESET_2K_EOSM:
                cmos_new[7] = 0xaa9;    /* pink highlights without this */
                break;

Ill figure it out
--
Fixed it on my end! All fine and no black bar.Thanks again Danne!
This fixed it for me
           
            case CROP_PRESET_2K_EOSM:
  cmos_new[5] = 0x300;             /* vertical (first|last) */
              cmos_new[7] = 0x2a9;    /* pink highlights without this */
                break;

Danne


dfort

Danne came up with yet another new setting for the EOSM that has a different full raw buffer size and it required a new focus pixel map file. He did a quick fix by simply copying and renaming the closest map file and it worked. However, I thought this might be a good opportunity to show how I create a new map file when this comes up because it seems to be a recurring theme.

The new setting has a full raw buffer size of 1808x769 and this is for the EOSM so we're looking for a file named 80000331_1808x769.fpm. That 80000331 is the camera code Canon assigned to the EOSM. When using MLVFS the map file has to fit exactly but with MLV App there is a little bit of wiggle room with the vertical size because we found that it sometimes varies slightly and it is always plus or minus on the bottom few rows. The closest match in the current repository is 80000331_1808x759.fpm which is outside of the tolerance allowed by MLV App so we need a new map file. Danne found out that by simply changing the name of that map file it worked. We could just leave that as is but if for any reason there are some focus pixels in the new area at the bottom of the frame we'll miss mapping them out.

First of all, copy the 80000331_1808x759.fpm file along with the fpm2img.sh and img2fpm.sh scripts from the focus pixels repository into a working directory. Next, rename the file to 80000331_1808x769.fpm. This will allow us to create a graphic image file with the right dimensions using the fpm2img.sh script.


rosiefort@Rosie-Forts-Computer ~ % cd Desktop/wip
rosiefort@Rosie-Forts-Computer wip % ls
80000331_1808x769.fpm fpm2img.sh img2fpm.sh
rosiefort@Rosie-Forts-Computer wip % ./fpm2img.sh 80000331_1808x769.fpm
output_file     = 80000331_1808x769_1.pbm
input_file      = 80000331_1808x769.fpm
input file type = fpm
width           = 1808
height          = 769
Mapping focus pixel x,y coordinates:
1806 0758
Filling in remaining pixels
00000000
output_file     = 80000331_1808x769_2.pbm
1803 0755
Filling in remaining pixels
00000000
output_file     = 80000331_1808x769_3.pbm
Filling in remaining pixels
00000000
Converting from PBM to PNG image file format


Ok, what just happened? You got not one but three graphic files?

It so happens that this is a multi-pass map file. Perhaps we had two different settings with the same full raw buffer size that showed the focus pixels in different locations. In addition, this being the EOSM which is always in Live View mode it will sometime overlay focus pixels for the "native" 720x480 resolution of the LCD screen. At least that's what we believe might be happening. We need to run the map files one at a time otherwise the pixel blending algorithm that is hiding the focus pixels gets overwhelmed and will possibly start contaminating the pixels it is trying to hide with nearby focus pixels and it turns into quite a mess. What the script did was to create separate graphic image files for each pass. Let's take a close look at the bottom few rows in an image editor, Photoshop in this case:



Those bottom few rows are blank and we should fill them in. I copied the background layer, changed the blending mode to "Darken" and moved down the new layer until the pattern matched up and filled in the bottom rows. Then I exported a PNG file. That format seems to work fine for this purpose. If you have ImageMagick installed on your system the scripts I wrote should work with it and save small PNG files, the script itself works with plain text PBM files which is an ancient format that creates huge files but it is easy to manipulate with very simple shell scripts.

Next, turn each of these PNG files back into FPM files:


rosiefort@Rosie-Forts-Computer wip % ./img2fpm.sh 80000331_1808x769_1.png
input_file = 80000331_1808x769_1.pbm
output_file = 80000331_1808x769_1.fpm
width = 1808
height = 769
Finding mapped pixel x,y coordinates:
Converting pbm file to png to save space.
rosiefort@Rosie-Forts-Computer wip % ./img2fpm.sh 80000331_1808x769_2.png
input_file = 80000331_1808x769_2.pbm
output_file = 80000331_1808x769_2.fpm
width = 1808
height = 769
Finding mapped pixel x,y coordinates:
Converting pbm file to png to save space.
rosiefort@Rosie-Forts-Computer wip % ./img2fpm.sh 80000331_1808x769_3.png
input_file = 80000331_1808x769_3.pbm
output_file = 80000331_1808x769_3.fpm
width = 1808
height = 769
Finding mapped pixel x,y coordinates:
Converting pbm file to png to save space.


Finally, copy these map files one after the other in order to create a multi-pass map file:


rosiefort@Rosie-Forts-Computer wip % cat 80000331_1808x769_1.fpm 80000331_1808x769_2.fpm 80000331_1808x769_3.fpm > 80000331_1808x769.fpm


I'll save you the trouble on this one and update the repository with this new map file.

Note that next month Bitbucket will no longer support Mercurial repositories. Both the main Magic Lantern repository as well as my ML Focus Pixels repository are Mercurial so they need to be converted or moved to another hosting site in order to continue this project. I would appreciate it if someone takes over the focus pixel repository because I have moved on to other projects and don't have much reason to continue maintaining it.

Danne

Quote from: dfort on June 03, 2020, 11:55:05 PM
I would appreciate it if someone takes over the focus pixel repository because I have moved on to other projects and don't have much reason to continue maintaining it.
:'(

DeafEyeJedi

Excellent follow up @dfort. I know we are each have our own projects lined up and whatnot. But please keep lurking on your rear end window occasionally from time to time.

I don't mind making the effort to have you groom me up with your talents. I even sold one of mine 5D3's in favor of keeping the EOSM's and 100D respectively.  :o

Quote from: dfort on June 03, 2020, 11:55:05 PM
Note that next month Bitbucket will no longer support Mercurial repositories. Both the main Magic Lantern repository as well as my ML Focus Pixels repository are Mercurial so they need to be converted or moved to another hosting site in order to continue this project. I would appreciate it if someone takes over the focus pixel repository because I have moved on to other projects and don't have much reason to continue maintaining it.

+1 - as I am definitely interested in partaking this role in any way shape or form. I'm very eager in wanting to keep ML alive well beyond Bitbucket years.
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Wlad81

Could you, please, be so kind to tell me if there are any focus pixel maps for 5D3 1.2.3 1080 full sensor?
Canon EOS 5D Mk III + Canon 24-105 F/4 L IS USM + SanDisk Exreme Pro 64 GB (SD, ML Nightly.2021Feb07.5D3113) + SanDisk Extreme Pro 128 GB (CF).

Walter Schulz

Quote from: Wlad81 on September 16, 2020, 11:21:08 PM
Could you, please, be so kind to tell me if there are any focus pixel maps for 5D3 1.2.3 1080 full sensor?

Thou shalt not crosspost! Question killed in https://www.magiclantern.fm/forum/index.php?topic=14704.msg230983#msg230983

SponaxSpinax

The bitbucket repository was deleted and I can't find the files anywhere.  :'(

Does anyone knows where I can find them?

Walter Schulz


masc

@SponaxSpinax: in the case you're talking about focus pixel map files, they are here:
https://github.com/ilia3101/MLV-App/tree/master/pixel_maps
5D3.113 | EOSM.202

capt.du

Hello, I recently downloaded the most recent focus maps and the focus dots won't go away. They only go away when I change the chroma smooth to 3x3

here is a link to the photo: https://cdn.discordapp.com/attachments/671076275665895504/811807211834572830/image0.jpg

tominator_66

There doesn't seem to be a focus pixel map for 2800 x 1190 for the eos m. Is it difficult to make one?

Edit: I've been trying to use bash in Win 10 Powershell as outlined by DFort in reply #736, but I keep getting error Line 180: permission denied, or similar. It creates the file, and it looks like it has the correct pattern for focus pixels, but doesn't seem to terminate correctly and just keeps making the file bigger. It got up to 800MB over two days before I killed it. Does anyone have an idea what I'm doing wrong?

Cheers.

dragonsfire1981

For some reason, I am sometimes also getting the focus pixels on standard HD H264 mode when I have magic lantern function turned on. I did not load any modules except Focus peaking, Global liveview. Is this normal?

names_are_hard

Are you seeing this in Liveview only, or do you see them when you play back recordings?  It is normal for focus peaking indicators to show in liveview (they should not be recorded).  Also, focus peaking is not the same things as focus pixels, so you're asking in the wrong thread ;)

theBilalFakhouri

@dragonsfire1981

Which camera, which build? share a image showing the problem, also a short H.264 clip has the problem

dragonsfire1981

They are definately focus pixels because when I played them back on the computer they are seen, so no not just live view.

Quote from: names_are_hard on September 23, 2021, 06:55:30 PM
Are you seeing this in Liveview only, or do you see them when you play back recordings?  It is normal for focus peaking indicators to show in liveview (they should not be recorded).  Also, focus peaking is not the same things as focus pixels, so you're asking in the wrong thread ;)