Dealing with Focus Pixels in raw video

Started by dfort, October 22, 2015, 11:09:10 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

dfort

One more change to the focus pixel script -- it can now automatically determine crop_rec footage.

Another interesting thing turned up with the latest round of focus pixel map updates. The non-crop maps for all the cameras are now identical. Why so many files for MLVFS? Because that app needs a different file for each full raw buffer size from each of the supported cameras even thought the files themselves have the same contents.

bouncyball

Is detecting only 3x3 sampling from RAWC is sufficient? Croprec module for affected cameras always uses 3x3? What about 1x1, is that used in mv1080crop?

I never had any other canon cameras than 7D and 5D3 :)

dfort

Quote from: bouncyball on January 01, 2018, 04:36:46 PM
Is detecting only 3x3 sampling from RAWC is sufficient? Croprec module for affected cameras always uses 3x3?

At this stage of development of the crop_rec module, yes. The 100D/650D/700D/EOSM cameras use only one of the crop_rec module options so it is rather easy to figure it out. If it uses the mv720 full raw buffer -- 5x3 sampling is mv720 and 3x3 sampling is crop_rec.

@rbrune first figured out how to get 3x3 sampling on the mv720 mode raw buffer to get around the lack of mv1080 mode on the EOSM. At some point maybe @mk11174 or someone else will figure out how to get some of the other crop_rec options working on these cameras (proof of concept) and then it will be back to the drawing board on how to deal with those new focus pixels.

Quote from: bouncyball on January 01, 2018, 04:36:46 PM
What about 1x1, is that used in mv1080crop?

Yes, 1x1 is also used in zoom mode. Note that zoom mode cleaned up nicely with the latest map files:



The problem is with mv1080crop. It works fine on 14-bit files but on 8...12-bit, yuck!



I'm taking a close look at it but it is difficult to see exactly what is going on. Both of these modes are using the same focus pixel pattern.

BTW--I had to make one more change in the fpm.sh script. On older MLV files without the RAWC block the script was picking up the audio "samplingRate". Fixed using the whole word (-w) option in grep.

Danne

Different stream for 8-12bit recordings causing pixels in random places?

dfort

Turns out that the 12-bit lossless mv1080crop focus pixels also have a pattern to them and can be cleaned up just as well as all the other video modes.

First, how about a flashy video to demonstrate the new focus pixel maps. Note how well they work with focus pixels but doesn't affect the dust on the sensor.



This was done from the 12-bit lossless compression samples @theBilalFakhouri supplied. The video modes that use 3x3 and 5x3 sampling show the focus pixels the most while the 1:1 sampling, zoom and mv1080crop (a.k.a. Movie crop mode or CROP_MODE_HACK) show the least. The focus pixel map file for zoom mode is working fine with 8...12-bit lossless compression but the mv1080crop still had some leftover pixels that seemed to show up in random places--or are they random?

Here is a closeup of the mv1080crop 12-bit lossless sample after running it through the focus pixel map, not all the leftover pixels we missed.



Let's take a closer look. Note how these focus pixels pop out as either bright green or magenta dots. The area around these pixels are also affected because of the blending used in the app--dcraw in this case.



Laying an image file of the focus pixel map file it is clear that we are missing these pixels.



It is like this all over the image. So did the focus pixels somehow move one pixel to the right? I made another map file with all of the coordinates shifted one pixel to the right and that took care of these two but others showed up. How about combining both map files? This would give a pattern that would hit two adjacent pixels:



Of course if you are averaging the area around the focus pixel to hide it, this won't work. However, using the 2-pass method that @bouncyball came up with, it worked:



I know that some users will note that this isn't perfect. I'm using "dcraw -T" to create tiff files from the DNG's and there is no chroma smoothing. It looks better in Adobe Camera Raw but I wanted to show a worst-case scenario.

So what is going on here? I can only speculate. Maybe this has something to do with the lossless compression? Why is it not an issue in zoom mode? I don't really know but mv1080crop is only available on the cameras that have the focus pixel issue so maybe it has something to do with FEATURE_CROP_MODE_HACK.

Still to do: Update the script and the 100D map files for mv1080crop mode. [EDIT] Done!

dfort

The new focus pixel map files are ready for testing in MLVFS. You can find them in the ML Focus Pixels repository under Source/focus_pixel_map_files.

The fpm.sh script is also updated and ready for testing.

bouncyball

Thanks for your time Dan, really appreciate what you are doing. I'll take a look at your updated script today (hopefully) :)

dfort

A couple more updates. In the ML Focus Pixels repository there is a scripts_archive directory with some scripts that I have found useful. A couple of them, fpm2pbm.sh and pbm2fpm.sh have been particularly useful. The updates improve processing speed though they are still excruciatingly slow. The octave version created by @a1ex, fpm2pbm.m, is much faster but the latest Homebrew octave installation doesn't work with the it so I have fallen back to using my bash scripts.

I choose to work with the plain text version of the pbm (Portable Bit Map) image file format because the file format is very simple, yet I was using ImageMagick in my scripts. The updated scripts no longer require ImageMagic though if it is installed you can now convert pretty much any graphic file format to a focus pixel map file with pbm2fpm.sh.

The main script is still fpm.sh which can create focus pixel map files as needed. It is more of a prototyping script and not too many people will actually use it. @bouncyball converted it to C and enhanced it in his mlv-tools project and it is used in his mlv_dump on steroids. These tools are also used in Switch and other MLV processing apps.

bouncyball

Quote from: dfort on January 06, 2018, 01:24:37 AM
pbm2fpm.sh pbm2fpm.sh have been particularly useful. The updates improve processing speed though they are still excruciatingly slow. The octave version created by @a1ex, fpm2pbm.m, is much faster but the latest Homebrew octave installation doesn't work with the it so I have fallen back to using my bash scripts.

I choose to work with the plain text version of the pbm (Portable Bit Map) image file format because the file format is very simple, yet I was using ImageMagick in my scripts. The updated scripts no longer require ImageMagic though if it is installed you can now convert pretty much any graphic file format to a focus pixel map file with pbm2fpm.sh.
I thought for fpm->pbm->fpm conversion you use 'fpmutil' (it does conversion almost in no time, and if you just did not figure out the procedure of doing this I can help). There are some examples printed out in console when "-h" or "--help" command line switch is used.

Yes I know it exports P4 (binary) PBMs, but if you need I can add P1 (ascii) support (have not seen real reason for adding this before). Are you directly editing this ascii pbms in text editor?

Also I can add option to export fpm without a header to simplify things further. Because MLVFS does not support this header and you should delete it manually after every export if MLVFS is the target not mlv_dump on steroids.

Edit: I could add this fpm header support to MLVFS itself... The purpose of this FPM header is the need of knowing exact resolution for the backward conversion to PBM and 'fpmutil' gets this information from my humble header of those fpms ;)

bb

dfort

I still use my scripts even though they are slower than molasses in January partially because they are the Devil I know. Ok--enough with the idioms.

I did a little experiment on those specks that are showing up on the @theBilalFakhouri test files and this might be the beginning of a feature that developers may want to add to their MLP processing applications. I was going to hold off publishing it until I could optimize the speed of my scripts a bit more but maybe I should just dump the scripts and see if I can get fpmutil to do the tricks that the scripts can do.

We determined that the specks on the footage were not focus pixels but possibly dead/hot/cold/stuck whatever pixels or maybe just dust on the sensor. What if this happened on a shot that we want to use? Maybe there was a speck on the sensor that ruined an entire day's shoot? What if our camera has a few dead pixels? dcraw has a feature to deal with problem pixels. When I started this project I made dcraw "badpixels" files to map out the focus pixels and it worked quite well.

So how about adding the locations of these pixels to the map file? This can be easily done manually but what if you have a bunch of spots to remove and you're more of a visual person--like me, I studied photography not computer science.

This is what I used to spot my photograhs:



Kidding aside, one of my college classmates was Russel Brown, a Photoshop pioneer and I've managed to keep up with technology. Enough of the name-dropping and on with the spotting exercise.

Here is one of the spots that the focus pixel map file didn't remove:



This isn't a Photoshop tutorial so I won't go into all of the details but I made a new layer and used the pencil tool set to one pixel to spot out that problem pixel.



After hitting all the problem areas I made the background white. Here's a closeup of one of the spots, notice the x,y coordinates in Photoshop:



If we add these coordinates to the appropriate focus pixel map file in MLVFS it won't work because the map file covers the full raw buffer and the image we used is cropped out of that buffer. It is easy enough to find out how much to offset the x,y coordinates with mlv_dump using the "-v" option. Tip, pipe it into less so you won't have to scroll back through the entire output.

mlv_dump -v 1736x1152.MLV | less
...
Block: RAWI
...
    Res:  1736x1152
    raw_info:
...
      height           1189
      width            1808
...
Block: IDNT
...
     Camera Name:   'Canon EOS 700D'
...
     Camera Model:  0x80000326
...
Block: VIDF
...
     Pan: 72x30
...


There's all the information we need. MLVFS will use the map file named "80000326_1808x1189.fpm" and we need to offset our coordinates 72x30. Since I'm more comfortable with Photoshop than basic arithmetic --

Change the canvas size to the full raw buffer size:



and offset the layer we created to "Pan: 72x30":



So that same pixel we just looked at is now at the coordinates based on the full raw buffer:



Fill in the edges with white, save the file. The pbm2fpm.sh script needs a plain text pbm file which surprise, Photoshop doesn't do but if ImageMagick is installed the script can work with pretty much any image file format. Here is a section of the output, note the pixel at 310x290:

667 262
245 263
643 276
310 290
1744 302
1724 314
1706 556


Like I said the script is slower than... well let's just say very slow so I tried fpmutil to convert This is where I had a problem:

./fpmutil fixed_pixels.pbm -o fixed_pixels.fpm

Focus Pixel Map Utility v0.5
****************************
Error: 'fixed_pixels.pbm' header corrupted, map can not be converted


This was a basic (a.k.a. P4 or non-text) Portable Bit Map file exported from Photoshop. Apparently fpmutil requires a specific header in the the pbm file. Speaking of headers, I'm not using any in my fpm files and my scripts strip out any headers. I don't have anything against the use of headers but if the file doesn't have a header we should probably be taking it from the filename like MLVFS and fpm2pbm.sh are doing.

Back to the spotting exercise. Add the list of fixes to the appropriate map file, "80000326_1808x1189.fpm" in this case and save it in MLVFS.

Before:


After:


Sure, I could have done it like this: (x + Pan_X) (y + Pan_Y) = (238 + 72)  (260 + 30) = 310 290 and just added that to the map file with the same results but imagine adding a feature on an MLV processing app where you can spot out specks as easily as using Spotone.

Another idea is to add a feature in MLVFS where the problem pixels for your camera can be saved in a map file. Maybe using the camera's serial number in the map file name? That way the "dead" pixels will be fixed automatically, sort of the way it was designed in dcraw.

Quote from: bouncyball on January 06, 2018, 09:00:39 AM
Yes I know it exports P4 (binary) PBMs, but if you need I can add P1 (ascii) support (have not seen real reason for adding this before). Are you directly editing this ascii pbms in text editor?

No real need to support Plain PBM. According to the file specifications:

QuotePlain PBM actually came first, but even its inventor couldn't stand its recklessly squanderous use of resources after a while and switched to what we now know as the regular PBM format.

I'm going to continue to support Plain PBM in my scripts unless I can figure out how to write binary files in bash or learn C enough to modify fpmutil to do what I want. Then again maybe I'll just stick with shell scripts and say I'm a die hard follower of the Unix Philosophy where all data should be text streams and worse is better.

Quote from: bouncyball on January 06, 2018, 09:00:39 AM
Edit: I could add this fpm header support to MLVFS itself... The purpose of this FPM header is the need of knowing exact resolution for the backward conversion to PBM and 'fpmutil' gets this information from my humble header of those fpms ;)

I like the idea of having a header because it could eliminate duplicate map files. I would suggest having fpmutil fall back on the filename if the FPM header is missing.

@dmilligan and I had a philosophical discussion about having duplicate fpm files and code to generate map files on the fly. Maybe it is time to re-think this now that we've got 32 map files, most of them with duplicate data, taking up nearly 30 MB of space.

bouncyball

Quote from: dfort on January 06, 2018, 10:30:29 PM
... because they are the Devil I know.
Yup, I know this is the main reason :)

Quote from: dfort on January 06, 2018, 10:30:29 PM
Apparently fpmutil requires a specific header in the the pbm file.
Not that much but alongside with image resolution in the header's standard line it needs camera ID and crop flag info which should be written in the #comment line of the header.

In short fpm and pbm both have this info in the header: camID, Crop flag, Xres, Yres. And the maps can have whatever name you desire. However, if name is not specified, 'fpmutil' names maps according to MLVFS established standard.

FPM header

#FPM 80000326 1808 726 0 -- fpmutil v0.5


PBM header

P4
# 80000326 0 -- fpmutil v0.5
1808 726


Quote from: dfort on January 06, 2018, 10:30:29 PM
I like the idea of having a header because it could eliminate duplicate map files. I would suggest having fpmutil fall back on the filename if the FPM header is missing.
Yes right, fall back is exactly what have to be implemented in addition to the option to export FPMs without header.

Quote from: dfort on January 06, 2018, 10:30:29 PM
... code to generate ... Maybe it is time to re-think this now that we've got 32 map files, most of them with duplicate data, taking up nearly 30 MB of space.
On the fly map generation is implemented to mlv_dump on steroids and MLV App.

Selecting bad/whatever pixels by mouse and adding it to the map, easily can be implemented to the MLV App. And they could be added as a 2nd (or even 3rd) pass coordinates.

I did not add your new unified pattern generators to the 'fpmutil' for now because I have not decided yet, fully substitute old pattern generators with newer ones or leave both. As I said I don't like the idea of touching unaffected healthy pixels by the map.

BTW, we could detect 8-10,12bit lossless raw by the white level, if it's lower then 15000 (or 14000) then it is most likely amplified 14bit lossless (aka 8-10,12bit lossless).

Edit: changes pushed to the 'fpmutil' repository: LINK

regards
bb

Danne

QuoteSelecting bad/whatever pixels by mouse and adding it to the map, easily can be implemented to the MLV App. And they could be added as a 2nd (or even 3rd) pass coordinates.
Sounds good :)

theBilalFakhouri

Hey @dfort How did you show raw information?
It's look like this after applying mlv_dump -v



Quote from: dfort on December 23, 2017, 07:14:03 PM
mlv_dump -v
    raw_info:
      api_version      0x00000001
      height           1188
      width            1808
      pitch            3164
      frame_size       0x0039674C
  --> bits_per_pixel   14
      black_level      2047
      white_level      5586
      active_area.y1   28
      active_area.x1   72
      active_area.y2   1185
      active_area.x2   1808
      exposure_bias    0, 0
      cfa_pattern      0x02010100
      calibration_ill  1


Walter Schulz

On Windows redirect mlv_dump output to a file and use a decent editor (Notepad++ or else) to look for "raw_info".
add

>>output.txt

to the command you want to execute

theBilalFakhouri

Quote from: Walter Schulz on January 07, 2018, 11:09:46 AM
On Windows redirect mlv_dump output to a file and use a decent editor (Notepad++ or else) to look for "raw_info".
add

>>output.txt

to the command you want to execute
Thanks!

dfort

@theBilalFakhouri -- Sorry, I didn't realize you were using Windows cmd that doesn't have less. If you want to take a quick look without having to output a file you should be able to pipe it through more.

mlv_dump -v 1736x1152.MLV | more

The screenshot you posted shows null blocks. Those are commonly seen at the end of a trimmed MLV. What file are you looking at?  I'm using your 12-bit sample files.

@bouncyball -- I have changed my philosophy from mapping as few focus pixels as possible to mapping every focus pixel that has ever revealed itself.

My first map files looked like this, which is the pattern we commonly see when new users ask what's up with the dots:



However, in certain situations more focus pixels were showing up:



When we looked at the 100D it showed an even larger area with focus pixels:



At first it looked like pattern didn't extend all the way to the edges of the full raw buffer but under certain conditions they did:



In zoom mode we can't determine what area of the sensor was used so the zoom mode maps need to cover every one of the 21 possible locations:



All of this was discovered on plain 14-bit files. In hindsight perhaps I should have assumed that the focus pixel patterns covered the entire sensor because that would have "future proofed" some of the map files.

With crop_rec we found two superimposed patterns and 8...12-bit lossless added a possibility of the focus pixels being offset by one pixel in certain areas. Both of these require 2-pass maps in order to fully zap the focus pixels.

MLVFS only looks at the focus pixel map's filename to determine which map to use so it is necessary to have duplicate maps but it is not able to distinguish between a crop_rec or an mv720 or a 14-bit and a 12-bit map file so we need to cover all possible focus pixel locations. The largest map file covers 206,400 possible locations. Is that excessive? The full raw buffer size is 1872x1060 so that's 1,984,320 before taking the out of bounds and cropped out areas so maybe 10% of the pixels are being processed? My understanding of how chroma smoothing works, all of the pixels are processed with the same (or similar) blending algorithm we're using to remove the focus pixels.

So is it worth making separate maps that cover different possible focus pixel positions or cover all possibilities? The tests I made with the "one size fits all" maps in MLVFS looked good to me so I made a pull request which @dmilligan merged.

Note that there are still situations where focus pixels might still show up, like in the extreme edges of the frame and when shooting Dual-ISO but these aren't issues that can be solved with the map files.

As far as file headers, we should be free to do anything we want with FPM files because they were invented here but requiring a header in PBM files in fpmutil is not because that header will get stripped out if the file is edited in Photoshop and probably most other image editing applications.

bouncyball

Quote from: dfort on January 07, 2018, 06:48:01 PM
So is it worth making separate maps that cover different possible focus pixel positions or cover all possibilities?
The more I'm thinking of pros and cons of interpolation of focus pixels, more I'm biased to the side of covering all image with unified maps as you suggest.

Quote from: dfort on January 07, 2018, 06:48:01 PM
As far as file headers, we should be free to do anything we want with FPM files because they were invented here but requiring a header in PBM files in fpmutil is not because that header will get stripped out if the file is edited in Photoshop and probably most other image editing applications.
Yes I'm talking about FPM header not PBM's. 'fpmutil' respects and violates nothing in the spec of Netpbm. Even comment line accounted.
It's just I'm using comment line for additional information like camera ID and crop_rec flag. If image is written by software like gimp or photoshop comment line does not contain the needed information and this was the reason it outputs error on console.

I changed this behavior today. PBMs are read in this order:

IF (specific comment options) load parameters and data from file
ELSE IF (comment line is absent entirely) try loading only resolution
ELSE IF (proper file name) parse it and derive all available info from it
ELSE IF (command line switches specified) use 'camera' (-c) and 'mode' (-m) values
ELSE show error message saying that required information gotta be given from cli


Now it converts any PBM I throw at it. Extreme example:

./fpmutil -n -c EOSM -m croprec "image_with_ugly_name_and_without_comment_line.pbm"

This produces .fpm with standard MLVFS file name consisting EOSM ID and croprec resolution and no header before actual pixel data coordinates.

As you can see in this example, a new '-n' (--no-header) switch is used which exports only data FPMs w/o header (for out of the box MLVFS compatibility).

LINK to the repository

dfort

I've got another suggestion for fpmutil and one for mlv_dump on steroids.

I'm auto detecting crop_rec by looking at the "Sampling" -- if we have an mv720 raw buffer with 5x3 sampling it is mv720 mode but if the sampling is 3x3 it is crop_rec. In the future we might get some more resolutions out of the crop_rec module so we should consider how to handle these. One thing that seems clear is that each camera has three distinct patterns, one for each of the binning/skipping methods that are currently being used: 1x1, 3x3 and 5x3 so making rawi_hdr.crop a boolean variable won't cover future crop_rec resolutions.

I'm using mlv_dump on steroids more often than the "official" version because of the added features, including fixing the focus pixels automatically. However, there are times when I want to look at the focus pixels and there doesn't seem to be an option to turn off that feature. Maybe do something like this?

--fpi <method>      focus pixel interpolation method: 0 (none), 1 (raw2dng), 2 (mlvfs),default is 1


I should backpedal on my statement to cover all possible focus pixel locations for all situations. At one point I was thinking that if we combine all of the map files we can come up with one "universal" map file that covers all of the video modes but that would probably be taking it too far. In MLVFS we currently have no way to choose between an mv720 and a crop_rec map file so I'm using the 2-pass crop_rec for both video modes. Same for mv1080crop 8...12-bit lossless compression, which also requires a 2-pass map file, verses uncompressed files. However, we should use separate map files when possible for other applications.

One more idea to throw on the wall to see if it sticks is when converting a FPM file to PBM, have an option to create separate image files for each pass. This shouldn't be too hard because of the way 2-pass FPM files are sorted.

[EDIT] I'm sorting 2-pass map before outputting a PBM file so this "combined" image file can no longer be converted back into a FPM file that works properly. I don't know if I'll do this:

fpm2pbm.sh -s <input.fpm> # creates a separate PBM file for each pass
pbm2fpm <file1.pbm> <file2.pbm> etc...


I'm accounting for possible additional passes to remove user defined pixels. How about calling these "Spotone" passes?

theBilalFakhouri

Quote from: dfort on January 07, 2018, 06:48:01 PM
... If you want to take a quick look without having to output a file you should be able to pipe it through more.
mlv_dump -v 1736x1152.MLV | more
...Those are commonly seen at the end of a trimmed MLV. What file are you looking at? ...
Thank you!
I shot today a video with x5 crop mode at 14bit lossless resolution 1920x818 (14bit uncompressed works perfectly.)
The pixel dots pattern was different compared with full 2520x1072 resolution so no map worked for it.
Make your tests:
https://drive.google.com/open?id=1WdY7djRxC9tJjnJvQw_gQ5er30t_joKF

When I was shooting tests at 2520x1072 14bit lossless with PIXEL SCANNER pattern (Maybe a complex image to compress)
ISO above 100, I get a lot of data corruption and ASSERT**.LOG files compared to more normal scenes. If these files helpful for developing please tell me.

dfort

That particular file has the focus pixels shifted 8-frames. I think it has happened before but I can't find the post.



zoom mode is a special case because we're not sure what part of the sensor it was recorded on. I thought we covered all possible locations but maybe not?

In other news, the fpm2pbm.sh and pbm2fpm.sh scrips got a big speed boost:

fpm2pbm.sh
time ./fpm2pbm-old.sh EOSM_mv1080crop_640x360.fpm
real 2m16.894s
user 1m32.244s
sys 0m35.004s

time ./fpm2pbm-new.sh EOSM_mv1080crop_640x360.fpm
real 0m30.764s
user 0m17.644s
sys 0m13.061s


pbm2fpm.sh
time ./pbm2fpm-old.sh EOSM_mv1080crop_640x360.pbm
real 1m20.062s
user 1m19.193s
sys 0m0.888s

time ./pbm2fpm-new.sh EOSM_mv1080crop_640x360.pbm
real 0m18.770s
user 0m16.950s
sys 0m2.140s

bouncyball

Quote from: dfort on January 07, 2018, 08:52:16 PM
I'm auto detecting crop_rec by looking at the "Sampling" -- if we have an mv720 raw buffer with 5x3 sampling it is mv720 mode but if the sampling is 3x3 it is crop_rec. In the future we might get some more resolutions out of the crop_rec module so we should consider how to handle these. One thing that seems clear is that each camera has three distinct patterns, one for each of the binning/skipping methods that are currently being used: 1x1, 3x3 and 5x3 so making rawi_hdr.crop a boolean variable won't cover future crop_rec resolutions.
Yes we already discussed using binning/skipping values as a crop_rec detection method (sampling) and I saw the script. And this is already implemented in sources on my HDD, not in repo.
Regarding rawi_hdr.crop: it is member of my so called shortcut structure introduced for not using 'raw_info' struct from 'raw.h'. It is uint32_t not boolean. standard C has no idea what boolean type is :).

Quote from: dfort on January 07, 2018, 08:52:16 PM
I'm using mlv_dump on steroids more often than the "official" version because of the added features, including fixing the focus pixels automatically. However, there are times when I want to look at the focus pixels and there doesn't seem to be an option to turn off that feature.
Yes why not. In MLV App you can turn off focus pixel correction entirely. A while back I wanted to introduce --nofixfp (like --nofixcp) in mlv dump. It just mlv_dump on steroids became low priority to me. But I'll add this switch to it.

Quote from: dfort on January 07, 2018, 08:52:16 PM
I should backpedal on my statement to cover all possible focus pixel locations for all situations.
Yes I've halted all my changes regarding your new pattern generators for a while to decide what to do: keep both or substitute older with newer, unified one. Or maybe separate generators by uncompressed and lossless category.

Quote from: dfort on January 07, 2018, 08:52:16 PM
One more idea to throw on the wall to see if it sticks is when converting a FPM file to PBM, have an option to create separate image files for each pass. This shouldn't be too hard because of the way 2-pass FPM files are sorted.
Yes unfortunately 2pass FPM saved as PBM can not be converted back to 2pass FPM because it is already sorted but we have to scan Y coordinate in FPM and when it is less than previous value this indicates the point where second pass starts.

I'm working on PBM data as with bit/pixel array, toggling needed bit/pixel in place. In the pbm save function have to be added support of saving more than one PBM file. Perhaps better way is to add the line numbers of the following passes to the FPM header. All this indeed will take time...

Quote from: dfort on January 07, 2018, 08:52:16 PM
I'm accounting for possible additional passes to remove user defined pixels. How about calling these "Spotone" passes?
Cool name :)

regards
bb

bouncyball

I updated mlv_dump on steroids and added '--no-fixfp' switch. LINK

Danne

Mlv_dump on steroids rules. Looking forward to see where this hard to fix focus pixels issue will land eventually.

bouncyball


dfort

Quite a day of discovery. Updated the script and the MLVFS fpm files to account for possible 8-pixel shifting in mv1080crop and zoom modes (1:1 sampling) on the EOSM/650D/700D cameras. Tested out great in MLVFS. I haven't done much testing with dcraw lately but maybe I should because it is showing some issues.

@theBilalFakhouri -- Try out these new map files.

@bouncyball -- Thanks for the new feature on mlv_dump on steroids. It really helps with testing! Also keeping an eye on your ml-tools commits.