Author Topic: Dealing with Focus Pixels in raw video  (Read 349869 times)

2blackbar

  • Hero Member
  • *****
  • Posts: 508
Re: Dealing with Focus Pixels in raw video
« Reply #725 on: May 04, 2020, 07:48:37 PM »
can i get it fixed by chopping one pixel or 2 in ML code  for crop mode ? so its 1736x976 vs 1736x734 , so both modes have different resolution

masc

  • Contributor
  • Hero Member
  • *****
  • Posts: 2069
Re: Dealing with Focus Pixels in raw video
« Reply #726 on: May 04, 2020, 07:56:32 PM »
Your final resolution don't cares. Identification is done with raw buffer resolution (metadata).
5D3.113 | EOSM.202

2blackbar

  • Hero Member
  • *****
  • Posts: 508
Re: Dealing with Focus Pixels in raw video
« Reply #727 on: May 04, 2020, 08:03:27 PM »
Best at the moment without fix would be if i could choose manually which focus pixel map should be used

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7659
Re: Dealing with Focus Pixels in raw video
« Reply #728 on: May 04, 2020, 08:10:31 PM »
I could fix it like this and upload a new build.

For non crop mode it stays at this(buffer):
Code: [Select]
case 0xC0F06804: return 0x4a601e4When selecting crop mode it will record following:
Code: [Select]
case 0xC0F06804: return 0x4a701e4Should give same resolution but it seems to activate the correct maps.

It won´t fix the old files though.

masc

  • Contributor
  • Hero Member
  • *****
  • Posts: 2069
Re: Dealing with Focus Pixels in raw video
« Reply #729 on: May 04, 2020, 08:12:36 PM »
What will this do? Change the metadata in the MLV?
5D3.113 | EOSM.202

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7659
Re: Dealing with Focus Pixels in raw video
« Reply #730 on: May 04, 2020, 08:20:07 PM »
Yes. For x3crop it will look like this:
Code: [Select]
      active_area.y2   1189
      active_area.x2   1812
For non crop:
Code: [Select]
      active_area.y2   1188
      active_area.x2   1812
This nudge seems to activate all the right dotmagic.


2blackbar

  • Hero Member
  • *****
  • Posts: 508
Re: Dealing with Focus Pixels in raw video
« Reply #732 on: May 04, 2020, 09:05:38 PM »
Great, i applied croprec.c changes to my custom build.
---
Well, new code is interfering with my custom 2.5k mode settings and i have black bar on right side of the screen, why is that? Or what should i remove to not interfere with 2.5k settings at all ?

It works fine before i add this new code :
Code: [Select]
    //needs to differ raw buffer to fit pixel map files in mlv app
    if (ratios == 0x0 && x3crop == 0x1)
    {
        switch (reg)
        {
            case 0xC0F06804: return 0x4a701e4 + reg_6804_width + (reg_6804_height << 16);
            case 0xC0F0713c: return 0x4a7 + reg_713c;
            case 0xC0F07150: return 0x430 + reg_7150;
        }
    }
   

When i add only this and remove that ratios line from code a bit above it ( line 3710) then 2.5k mode is changed, i will test if its broken in your build.
--
Looks like its fine in Your build, ill adjust it on my end.
But the fix does work !

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7659
Re: Dealing with Focus Pixels in raw video
« Reply #733 on: May 04, 2020, 09:44:16 PM »
Please test my build first and see if it works properly in x3crop mode without any borders etc. Then you can post anything you want but I want to be sure the original intent is working as it should.
The modified 2.5k mode? Is that modification also with mcm rewired but in x3crop mode? HArd to follow this.

Edit: Ok so it seems you are on to something. Saw you updated your post.

2blackbar

  • Hero Member
  • *****
  • Posts: 508
Re: Dealing with Focus Pixels in raw video
« Reply #734 on: May 04, 2020, 09:46:12 PM »
Yup, your build is all fine, ill start from copying your addresses and try to get it working for my cctv lenses again with minimum vignette like before !
Great work !
I use different addresses for 2.5k mode to get framing right for my cctv lenses, its a matter of finding them again with this new change.
I use this :
Code: [Select]
            case CROP_PRESET_2K_EOSM:
  cmos_new[5] = 0x2c0;             /* vertical (first|last) */
              cmos_new[7] = 0x2a9;    /* pink highlights without this */
                break;

And this is Yours:
Code: [Select]
            case CROP_PRESET_2K_EOSM:
                cmos_new[7] = 0xaa9;    /* pink highlights without this */
                break;
Ill figure it out
--
Fixed it on my end! All fine and no black bar.Thanks again Danne!
This fixed it for me
Code: [Select]
           
            case CROP_PRESET_2K_EOSM:
  cmos_new[5] = 0x300;             /* vertical (first|last) */
              cmos_new[7] = 0x2a9;    /* pink highlights without this */
                break;

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7659
Re: Dealing with Focus Pixels in raw video
« Reply #735 on: May 04, 2020, 10:17:26 PM »
Great!

dfort

  • Guest
Re: Dealing with Focus Pixels in raw video
« Reply #736 on: June 03, 2020, 11:55:05 PM »
Danne came up with yet another new setting for the EOSM that has a different full raw buffer size and it required a new focus pixel map file. He did a quick fix by simply copying and renaming the closest map file and it worked. However, I thought this might be a good opportunity to show how I create a new map file when this comes up because it seems to be a recurring theme.

The new setting has a full raw buffer size of 1808x769 and this is for the EOSM so we're looking for a file named 80000331_1808x769.fpm. That 80000331 is the camera code Canon assigned to the EOSM. When using MLVFS the map file has to fit exactly but with MLV App there is a little bit of wiggle room with the vertical size because we found that it sometimes varies slightly and it is always plus or minus on the bottom few rows. The closest match in the current repository is 80000331_1808x759.fpm which is outside of the tolerance allowed by MLV App so we need a new map file. Danne found out that by simply changing the name of that map file it worked. We could just leave that as is but if for any reason there are some focus pixels in the new area at the bottom of the frame we'll miss mapping them out.

First of all, copy the 80000331_1808x759.fpm file along with the fpm2img.sh and img2fpm.sh scripts from the focus pixels repository into a working directory. Next, rename the file to 80000331_1808x769.fpm. This will allow us to create a graphic image file with the right dimensions using the fpm2img.sh script.

Code: [Select]
rosiefort@Rosie-Forts-Computer ~ % cd Desktop/wip
rosiefort@Rosie-Forts-Computer wip % ls
80000331_1808x769.fpm fpm2img.sh img2fpm.sh
rosiefort@Rosie-Forts-Computer wip % ./fpm2img.sh 80000331_1808x769.fpm
output_file     = 80000331_1808x769_1.pbm
input_file      = 80000331_1808x769.fpm
input file type = fpm
width           = 1808
height          = 769
Mapping focus pixel x,y coordinates:
1806 0758
Filling in remaining pixels
00000000
output_file     = 80000331_1808x769_2.pbm
1803 0755
Filling in remaining pixels
00000000
output_file     = 80000331_1808x769_3.pbm
Filling in remaining pixels
00000000
Converting from PBM to PNG image file format

Ok, what just happened? You got not one but three graphic files?

It so happens that this is a multi-pass map file. Perhaps we had two different settings with the same full raw buffer size that showed the focus pixels in different locations. In addition, this being the EOSM which is always in Live View mode it will sometime overlay focus pixels for the "native" 720x480 resolution of the LCD screen. At least that's what we believe might be happening. We need to run the map files one at a time otherwise the pixel blending algorithm that is hiding the focus pixels gets overwhelmed and will possibly start contaminating the pixels it is trying to hide with nearby focus pixels and it turns into quite a mess. What the script did was to create separate graphic image files for each pass. Let's take a close look at the bottom few rows in an image editor, Photoshop in this case:



Those bottom few rows are blank and we should fill them in. I copied the background layer, changed the blending mode to "Darken" and moved down the new layer until the pattern matched up and filled in the bottom rows. Then I exported a PNG file. That format seems to work fine for this purpose. If you have ImageMagick installed on your system the scripts I wrote should work with it and save small PNG files, the script itself works with plain text PBM files which is an ancient format that creates huge files but it is easy to manipulate with very simple shell scripts.

Next, turn each of these PNG files back into FPM files:

Code: [Select]
rosiefort@Rosie-Forts-Computer wip % ./img2fpm.sh 80000331_1808x769_1.png
input_file = 80000331_1808x769_1.pbm
output_file = 80000331_1808x769_1.fpm
width = 1808
height = 769
Finding mapped pixel x,y coordinates:
Converting pbm file to png to save space.
rosiefort@Rosie-Forts-Computer wip % ./img2fpm.sh 80000331_1808x769_2.png
input_file = 80000331_1808x769_2.pbm
output_file = 80000331_1808x769_2.fpm
width = 1808
height = 769
Finding mapped pixel x,y coordinates:
Converting pbm file to png to save space.
rosiefort@Rosie-Forts-Computer wip % ./img2fpm.sh 80000331_1808x769_3.png
input_file = 80000331_1808x769_3.pbm
output_file = 80000331_1808x769_3.fpm
width = 1808
height = 769
Finding mapped pixel x,y coordinates:
Converting pbm file to png to save space.

Finally, copy these map files one after the other in order to create a multi-pass map file:

Code: [Select]
rosiefort@Rosie-Forts-Computer wip % cat 80000331_1808x769_1.fpm 80000331_1808x769_2.fpm 80000331_1808x769_3.fpm > 80000331_1808x769.fpm

I'll save you the trouble on this one and update the repository with this new map file.

Note that next month Bitbucket will no longer support Mercurial repositories. Both the main Magic Lantern repository as well as my ML Focus Pixels repository are Mercurial so they need to be converted or moved to another hosting site in order to continue this project. I would appreciate it if someone takes over the focus pixel repository because I have moved on to other projects and don't have much reason to continue maintaining it.

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7659
Re: Dealing with Focus Pixels in raw video
« Reply #737 on: June 04, 2020, 09:18:35 PM »
I would appreciate it if someone takes over the focus pixel repository because I have moved on to other projects and don't have much reason to continue maintaining it.
:'(

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3411
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: Dealing with Focus Pixels in raw video
« Reply #738 on: June 04, 2020, 11:34:42 PM »
Excellent follow up @dfort. I know we are each have our own projects lined up and whatnot. But please keep lurking on your rear end window occasionally from time to time.

I don't mind making the effort to have you groom me up with your talents. I even sold one of mine 5D3's in favor of keeping the EOSM's and 100D respectively.  :o

Note that next month Bitbucket will no longer support Mercurial repositories. Both the main Magic Lantern repository as well as my ML Focus Pixels repository are Mercurial so they need to be converted or moved to another hosting site in order to continue this project. I would appreciate it if someone takes over the focus pixel repository because I have moved on to other projects and don't have much reason to continue maintaining it.

+1 - as I am definitely interested in partaking this role in any way shape or form. I'm very eager in wanting to keep ML alive well beyond Bitbucket years.
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Wlad81

  • Member
  • ***
  • Posts: 136
Re: Dealing with Focus Pixels in raw video
« Reply #739 on: September 16, 2020, 11:21:08 PM »
Could you, please, be so kind to tell me if there are any focus pixel maps for 5D3 1.2.3 1080 full sensor?
Canon EOS 5D Mk III + Canon 24-105 F/4 L IS USM + SanDisk Exreme Pro 64 GB (SD, ML Nightly.2021Feb07.5D3113) + SanDisk Extreme Pro 128 GB (CF).

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 8609
Re: Dealing with Focus Pixels in raw video
« Reply #740 on: September 18, 2020, 07:54:32 AM »
Could you, please, be so kind to tell me if there are any focus pixel maps for 5D3 1.2.3 1080 full sensor?

Thou shalt not crosspost! Question killed in https://www.magiclantern.fm/forum/index.php?topic=14704.msg230983#msg230983

SponaxSpinax

  • New to the forum
  • *
  • Posts: 2
Re: Dealing with Focus Pixels in raw video
« Reply #741 on: October 29, 2020, 12:22:26 AM »
The bitbucket repository was deleted and I can't find the files anywhere.  :'(

Does anyone knows where I can find them?

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 8609
Re: Dealing with Focus Pixels in raw video
« Reply #742 on: November 14, 2020, 12:31:02 AM »
Which files are you looking for?

masc

  • Contributor
  • Hero Member
  • *****
  • Posts: 2069
Re: Dealing with Focus Pixels in raw video
« Reply #743 on: November 14, 2020, 08:54:58 AM »
@SponaxSpinax: in the case you're talking about focus pixel map files, they are here:
https://github.com/ilia3101/MLV-App/tree/master/pixel_maps
5D3.113 | EOSM.202

capt.du

  • Just arrived
  • *
  • Posts: 1
Re: Dealing with Focus Pixels in raw video
« Reply #744 on: February 18, 2021, 05:10:05 AM »
Hello, I recently downloaded the most recent focus maps and the focus dots won't go away. They only go away when I change the chroma smooth to 3x3

here is a link to the photo: https://cdn.discordapp.com/attachments/671076275665895504/811807211834572830/image0.jpg

tominator_66

  • New to the forum
  • *
  • Posts: 2
Re: Dealing with Focus Pixels in raw video
« Reply #745 on: July 11, 2021, 04:38:37 AM »
There doesn't seem to be a focus pixel map for 2800 x 1190 for the eos m. Is it difficult to make one?

Edit: I've been trying to use bash in Win 10 Powershell as outlined by DFort in reply #736, but I keep getting error Line 180: permission denied, or similar. It creates the file, and it looks like it has the correct pattern for focus pixels, but doesn't seem to terminate correctly and just keeps making the file bigger. It got up to 800MB over two days before I killed it. Does anyone have an idea what I'm doing wrong?

Cheers.

dragonsfire1981

  • New to the forum
  • *
  • Posts: 13
Re: Dealing with Focus Pixels in raw video
« Reply #746 on: September 23, 2021, 05:46:29 PM »
For some reason, I am sometimes also getting the focus pixels on standard HD H264 mode when I have magic lantern function turned on. I did not load any modules except Focus peaking, Global liveview. Is this normal?

names_are_hard

  • Developer
  • Hero Member
  • *****
  • Posts: 638
  • Dev: 200D, 750D, 850D, 7D2
Re: Dealing with Focus Pixels in raw video
« Reply #747 on: September 23, 2021, 06:55:30 PM »
Are you seeing this in Liveview only, or do you see them when you play back recordings?  It is normal for focus peaking indicators to show in liveview (they should not be recorded).  Also, focus peaking is not the same things as focus pixels, so you're asking in the wrong thread ;)

theBilalFakhouri

  • Developer
  • Hero Member
  • *****
  • Posts: 1148
  • UHS-I
Re: Dealing with Focus Pixels in raw video
« Reply #748 on: September 23, 2021, 07:11:24 PM »
@dragonsfire1981

Which camera, which build? share a image showing the problem, also a short H.264 clip has the problem
Cooking some features . . .

dragonsfire1981

  • New to the forum
  • *
  • Posts: 13
Re: Dealing with Focus Pixels in raw video
« Reply #749 on: September 28, 2021, 10:31:18 AM »
They are definately focus pixels because when I played them back on the computer they are seen, so no not just live view.

Are you seeing this in Liveview only, or do you see them when you play back recordings?  It is normal for focus peaking indicators to show in liveview (they should not be recorded).  Also, focus peaking is not the same things as focus pixels, so you're asking in the wrong thread ;)