-
Quite a bit has happened since starting this topic. If you want to cut to the chase and want the quickest way to eradicate focus pixels simply download MLV App and load in the latest focus pixel map files. This video shows how to install the map files into the app:
The rest of this is post for historical (hysterical?) purposes:
@dfort,
One possibility is mapping the focus pixels for each camera and then interpolating out those pixels (rather than just chroma-smoothing the entire image). This might fix the dual ISO / focus pixel issue, IDK. It should be faster than chroma-smoothing, and not have any artifacts. It's a lot of work though, you need to create a focus pixel map for each camera and video mode.
So here we go--
Affected cameras: EOSM, 100D, 600D, 650D, 700D, (and more?)
[EDIT: Pretty sure now that the 600D doesn't show focus pixels in raw video.]
Focus pixels, phase detectors, call them what you will when these dots show up on your raw video they are annoying. There have been attempts at clearing them up like the PinkDotRemoval tool (http://magiclantern.fm/forum/index.php?topic=6658.0) that has apparently been abandoned. Chroma smoothing helps but it affects the overall image and it doesn't work for dual iso footage.
I tried to shoot something that shows as many of these pixels as possible by shooting an out of focus white paper with a couple different colors of lights and processing the image with a heavy hand to show the pixels as clearly and dramatically as possible.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm6.staticflickr.com%2F5819%2F22203578608_436dd64a17_c.jpg&hash=57c8b53530e671d6f16bc877e07b25dc)
Got your attention? This is a frame from an mlv file shot on an EOSM in crop movie mode. It doesn't look quite as bad when shooting regular subjects but it is a problem.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm6.staticflickr.com%2F5765%2F22373110971_36823003be_z.jpg&hash=0f8b4934659bb94adf758ee185684d23)
It is possible to get an even clearer view of these pixels by running a frame through dcraw using the -d option which, according to the documentation:
-d
Show the raw data as a grayscale image with no interpolation.
Here is a closeup of that same frame at the top of this post.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm1.staticflickr.com%2F753%2F22205148509_36f2af4ca2_z.jpg&hash=3bcfbe6938b571d819831c6e0c48d964)
Once we know the coordinates of these pixels they can be zapped.
Method 1 - In Camera
The ideal way to deal with them would be in the camera. In fact there is a way that "bad pixels" can be mapped out and apparently it is common practice to map out these faulty pixels when the camera is manufactured or repaired--saves on reject sensors. Assuming that we can't access the code that maps out pixels in the Canon firmware, the developers over at the chdk project have figured out more than one way to do it: http://chdk.wikia.com/wiki/Badpixel_removal
I'm not skilled enough to take what they have done and apply it to Magic Lantern but maybe someone who has the knowledge can drop a few hints?
Method 2 - In Post
Yeah, we can fix it in post. In fact there are several programs that can deal with pixel problems. Pixel Fixer (http://www.pixelfixer.org/) is a dedicated tool, RawTherapee (http://rawtherapee.com/) has an option to deal with hot and dead pixels and an interesting Python wrapper for the LibRaw library called rawpy (http://pythonhosted.org/rawpy/api/rawpy.enhance.html) that creates a ".badpixels" file dcraw can use to zap out pixels. This might be a way to create the database of focus pixel maps dmilligan suggested at the top of this post.
I tried using dcraw to get rid of one of the focus pixels on a dual iso frame and found that one pass wouldn't do it. Maybe it is because the process is breaking down by running the file through cr2hdr first? My understanding is that the debayering process is averaging neighboring pixels and in dual iso it seems to "smear" the pixel vertically. In addition, some of these focus pixels have a secondary pixel diagonally across that also needs to be mapped.
Here is what I ended up doing to get rid of just one focus pixel--and the diagonal secondary pixel.
1247 121 0
1248 120 0
1248 121 0
1248 125 0
1248 126 0
1248 122 0
1248 123 0
1248 124 0
1248 126 0
1248 122 0
1248 120 0
1248 124 0
Note that I hit the diagonal pixel just once but had to work top and bottom to middle and hit the same pixels several times in different sequences. Think of it as stippling a brush to spot a print--in photography school I spent many hours spotting prints. (By the way, this a map of all the focus pixels for the 650D (https://dl.dropboxusercontent.com/u/22843507/MagicLantern/pixCoord_threshold2068.txt) that was used on the PinkDotRemover tool.)
Here is the result. I circled the area where the hot pixel was located and yes, it looked just like those other ones before running the dng through dcraw.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm1.staticflickr.com%2F739%2F22345535026_e44d0ec5a2_z.jpg&hash=5328e9faf891de3a56d947d9d1ae1b54)
So if this is the solution then what's the problem?
The problem is that I'd like to get some feedback that this would actually be practical. It looks like there would have to be a different focus pixel map file for every camera and every crop factor (raw image size) and all of this would have to be assembled in a way that users could handle. Danne is the master of assembling scripts that can automate complex processes so it is possible to piece together something with dcraw--but then again is that the best solution? Maybe it would be better to adapt dcraw's "badpixels" code to mlv_dump or cr2hdr or MLVFS?
Just asking for a little guidance here.
-
Practical? This is a stepping stone in History. Probably by far from becoming one of the most sought-after script of the decade, if not the century. No kidding here, Buddy!
This means it would make some of us users with heavy dependent on PDR like our $200-$700 Toys feel like $1-2K, if not more!
There may have been a few run-in's with certain threads in the past on this Forum if I can recall correctly on top of my head but this is by far...
Probably the 2nd Method is the most ideal, BEST, most practical option there is and looking forward to jump on board with this yet another adventurous project of yours, Daniel!
Two-Thumbs up... 8)
-
Practical? This is a stepping stone in History. Probably by far from becoming one of the most sought-after script of the decade, if not the century. No kidding here, Buddy!
I could use some cheering--things didn't work out that well today. I struggled with rawpy and managed to write out a list of pixels with this:
import rawpy.enhance
import numpy
paths = ['image_000000.dng', 'image_000001.dng', 'image_000002.dng']
bad_pixels = rawpy.enhance.find_bad_pixels(paths, find_hot=True, find_dead=True, confirm_ratio=0.9)
# rawpy.enhance.save_dcraw_bad_pixels(path, bad_pixels)
numpy.savetxt('pixels.txt', bad_pixels, '%5.0f')
Note that I never did get the "rawpy.enhance.save_dcraw_bad_pixels(path, bad_pixels)" to work. Lots of searching turned up that rawpy uses numpy arrays so I figured out how to save that to a file but the list was huge! Checking the dng's that I used it looked like there were lots of false positives. I probably could have mapped out the pixels manually in the time it took me to get this far today.
It seems that the first thing to do is to map out the focus pixels on a "regular" raw file and make sure that works and all the focus pixels get mapped before trying to figure out what is going on with dual iso. Right now I don't think that dcraw is the right tool for taking out focus pixels on dual iso files because the way I understand it the pixel is filled in with the average of the adjacent pixels. That would mean that it would be trying to average the value of pixels that were exposed at different iso values. The pixels to average need to be on the same x axis, in other words the pixels to the left and right of the focus pixel.
-
Note that I never did get the "rawpy.enhance.save_dcraw_bad_pixels(path, bad_pixels)" to work. Lots of searching turned up that rawpy uses numpy arrays so I figured out how to save that to a file but the list was huge! Checking the dng's that I used it looked like there were lots of false positives. I probably could have mapped out the pixels manually in the time it took me to get this far today.
I second that and definitely should be worth the effort to do this manually. Any chance I could help? I miss spending time on Photoshop. lol
It seems that the first thing to do is to map out the focus pixels on a "regular" raw file and make sure that works and all the focus pixels get mapped before trying to figure out what is going on with dual iso. Right now I don't think that dcraw is the right tool for taking out focus pixels on dual iso files because the way I understand it the pixel is filled in with the average of the adjacent pixels. That would mean that it would be trying to average the value of pixels that were exposed at different iso values. The pixels to average need to be on the same x axis, in other words the pixels to the left and right of the focus pixel.
Bummer about dcraw though it makes perfect sense. I still have a good feeling about this possibility one way or another.
-
I'm not sure if I have the exact same issue, but I have had my 7D record hot pixels in raw video. This of course happens in H.264 as well if not using the sensor cleaning trick for a while (I believe this is a common problem with the 7D - occurs with the film school I go to with their 7Ds, and less of a problem with my T3i). The trick is doing manual sensor cleaning with just the body cap on and pointing down for thirty seconds. But what's nice about raw video is when I do get hot pixels, I am able to remove them with "Remove Chromatic Aberration" in Adobe Camera Raw (or Lightroom probably). I am completely aware not everyone can afford these programs, especially with Adobe Creative Cloud, but it is possible to buy Lightroom 5 on disc I believe for a reasonable price.
I hope this is helpful. It seems you are trying to eliminate the problem in the first place, which is great, just figured I would inform you and the forum of how I remove the hot pixels. Hope all is well!
-
I'm not sure if I have the exact same issue, but I have had my 7D record hot pixels in raw video.
Not the same issues--you have "real" hot or dead pixels. We're talking about the focusing pixels that show up on certain models. That list doesn't include the 7D.
-
This morning I mapped the focus pixels for the EOSM 1280x720 crop mode MLV. I had to do it manually row by row and ended up with over 6,000 pixels mapped. It seems to be working for all the images that I have tried. Here's a frame from the camera before processing:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm1.staticflickr.com%2F641%2F22229569068_efd9aca809_c.jpg&hash=0b985f804cbc1fcb21aa8dc6deab9ff9)
Looks ok until you take a closer look:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm1.staticflickr.com%2F761%2F22391282306_7833f4709c_z.jpg&hash=820523998e56f6d44f4fed3d34dcce9f)
Running it through dcraw with the -P option pointing to focus_pixels_EOSM_1280X720_crop_mode.txt (https://bitbucket.org/daniel_fort/magic-lantern/downloads/focus_pixels_EOSM_1280X720_crop_mode.txt) takes out the focus pixels.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm1.staticflickr.com%2F624%2F22229303160_9803f7f106_z.jpg&hash=6e5a8786c4f41d563712af812ae29fb0)
Yeah, I lost some resolution when it was saved as a portable pixmap format (PPM) which is the default for dcraw.
So the next challenge is going to be removing pixels on dual iso.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm6.staticflickr.com%2F5745%2F22229569838_cfcf5c981a_b.jpg&hash=6c6fab87c58685f7962ddff998ec000b)
This was shot at the same time but the focus pixels can't be removed either before or after processing. I'm not sure that doing the "stippling" method from my original post is feasible or even desirable. Looking at the dcraw code maybe (?) there is a way to average the pixels to the left and right of the focus pixels (x axis) and not use pixels that are on a different line (y axis) because that would be averaging in different iso values.
Snippet from dcraw.c (https://www.cybercom.net/~dcoffin/dcraw/dcraw.c):
/*
Seach from the current directory up to the root looking for
a ".badpixels" file, and fix those pixels now.
*/
void CLASS bad_pixels (const char *cfname)
{
FILE *fp=0;
char *fname, *cp, line[128];
int len, time, row, col, r, c, rad, tot, n, fixed=0;
if (!filters) return;
if (cfname)
fp = fopen (cfname, "r");
else {
for (len=32 ; ; len *= 2) {
fname = (char *) malloc (len);
if (!fname) return;
if (getcwd (fname, len-16)) break;
free (fname);
if (errno != ERANGE) return;
}
#if defined(WIN32) || defined(DJGPP)
if (fname[1] == ':')
memmove (fname, fname+2, len-2);
for (cp=fname; *cp; cp++)
if (*cp == '\\') *cp = '/';
#endif
cp = fname + strlen(fname);
if (cp[-1] == '/') cp--;
while (*fname == '/') {
strcpy (cp, "/.badpixels");
if ((fp = fopen (fname, "r"))) break;
if (cp == fname) break;
while (*--cp != '/');
}
free (fname);
}
if (!fp) return;
while (fgets (line, 128, fp)) {
cp = strchr (line, '#');
if (cp) *cp = 0;
if (sscanf (line, "%d %d %d", &col, &row, &time) != 3) continue;
if ((unsigned) col >= width || (unsigned) row >= height) continue;
if (time > timestamp) continue;
for (tot=n=0, rad=1; rad < 3 && n==0; rad++)
for (r = row-rad; r <= row+rad; r++)
for (c = col-rad; c <= col+rad; c++)
if ((unsigned) r < height && (unsigned) c < width &&
(r != row || c != col) && fcol(r,c) == fcol(row,col)) {
tot += BAYER2(r,c);
n++;
}
BAYER2(row,col) = tot/n;
if (verbose) {
if (!fixed++)
fprintf (stderr,_("Fixed dead pixels at:"));
fprintf (stderr, " %d,%d", col, row);
}
}
if (fixed) fputc ('\n', stderr);
fclose (fp);
}
-
I would suggest mapping the focus pixels from a silent DNG taken in each video mode. That way the coordinates are relative to the full, entire raw buffer including OB areas. That way you can use the same map for anything taken in that video mode, and simply adjust it based on the resolution crop and horizontal and vertical offset (which are already recorded in the MLV file)
-
Thanks, I'll do that--once I figure out what you just said. Haven't done anything with silent DNG's yet.
The still photos don't show the extent of the problem so I thought I'd shoot something for dramatic effect. Believe me the video looked much worse before YouTube compressed it.
EOSM 1280x720 Dual ISO crop mode MLV. I opened and closed the aperture because the pixels change appearance depending on the color and brightness of the image. The pixels dance around on dual iso video so it looks like we're going to have to hit a moving target!
-
Looks like this is going to be a popular project!
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm1.staticflickr.com%2F566%2F21813009004_e409ec9a49_n.jpg&hash=4d52f28f53a0450c622890859c9c6379)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm6.staticflickr.com%2F5744%2F22409729696_286a6c751f_n.jpg&hash=f087be175b9cccf86c748d86a3bf0fe4)
I've got mixed emotions about that. :) :'( ;) :-* :D :-\ ;D :-X >:( :-[ :( :-[ :o ::) ???
Ok, seriously--It looks like it will be possible to eliminate the focus pixels in post. One thing that needs to be done is to map the focus pixels on the various cameras that are affected. I was able to do it for the EOSM at one resolution but it was a very tedious process doing it manually and prone to error. Any suggestions on how to do this more reliably and less painfully?
By the way, I got called out of retirement to fill in for a couple of months so I won't have as much time as I'd like to work on this. It doesn't mean I'm going to disappear -- I'm still going to put in a few hours here and there trying to figure out how to make these pixels disappear but I could use some help.
-
Don't let your sense of humor run dry... EVER!
TBH I didn't even know that either Tapatalk or ML has a like button similar to FB? lol
I'm off this weekend and would like to get my hands tied down to remapping the Focus Pixels from my SL1 -- can this be done in Photoshop, right?
If so, I'll reply to your recent PM about this very matter.
Otherwise we can just stand by and said for others to chime in better "suggestions" on how to speed up the process.
Thanks, D! [emoji1]
-
I got you a testversion of folder_ProRes444_lut with an included dead_pixel list feature.
-
So bad-ass, Danne!
Can't wait to try it.
-
Whoo Hoo--Danne's folder_ProRes444_lut test version is working!
Well, almost. So far it only works for one camera (EOSM) at one resolution (1280x720) not on dual_iso yet and we got some kinks to work out but the results are encouraging.
-
*Wo0o0o0oT!*
-
How did you manage to map all of those pixels? Amazing. Thanks for nailing down the correct path to the dcraw command. Works correct now so will fix so it works with previews and stuff. Of course you have to do manual pixel mapping but there won,t be 6000 dots on all sensors to map out ;).
-
Basically, I started with shooting a flat gray frame and use "dcraw -b [name of dng file]" to get the pixels to stand out. Then in Photoshop adjust the brightness and contrast and with the "Units & Rulers" preference set to Units, Rulers: Pixels and Type: Pixels start hunting for the focus pixels. You have to be zoomed in all the way to measure individual pixels. The pixels that repeat in a pattern are the focus pixels. They were always spaced 24 pixels apart on the EOSM so I made a spread sheet and worked it out row by row. I also found that there were some rows that went all the way across the frame and others that were only in the center so first I mapped the ones that went completely across the frame and made a "badpixel" file then mapped out the ones in the center. After I thought I had them all I tried running the "pixel fix" on various frames and found several rows that didn't show up on the file I was using. It is tricky to find all of them because their visibility depends on exposure, contrast, color, etc of the image. Some rows also seem to change intensity from one side of the frame to the other so it becomes sort of a game of hide and seek to hunt them all down. I might have mapped a few points that aren't really focus pixels, I'm not really sure but it doesn't seem to be hurting the final results.
Looks like another how-to article. First I want to make sure I map all of the pixels on the EOSM, right now I've only got the ones in the crop mode area.
Then using dmilligan's suggestion interpolate the focus pixels needed map the various image size options. That's still a lot of work to do.
Oh, and by the way--I'm calling them focus pixels but I'm not really sure what they are.
One more thing, even cameras that don't have the focus pixel issue might have hot, dead, whatever, pixels that need to be removed. Maybe even the old hair in the gate issue that we used to have in the film days might benefit from this.
-
@dmilligan -- it would be great if you could share some of your knowledge on the following. At least let me know if I'm going in the right direction.
I would suggest mapping the focus pixels from a silent DNG taken in each video mode. That way the coordinates are relative to the full, entire raw buffer including OB areas. That way you can use the same map for anything taken in that video mode, and simply adjust it based on the resolution crop and horizontal and vertical offset (which are already recorded in the MLV file)
My first thought was to shoot a full-resolution silent DNG and try to map all the focus pixels for the full sensor but the focus pixels didn't show up on any of the full-resolution frames that I shot. Maybe the firmware mapped them out? The focus pixels did show up at different resolutions but the spacing between them varies. I think that might be because of the line (and row?) skipping at various resolutions.
With Danne's help I was able to extract just the metadata from an MLV file using mlv_dump:
mlv_dump -v -m [mlv file] > meta.txt
So now I've got to figure out what this means:
Crop: 336x184
Pan: 330x181
Space: 4064
If I understood how line/row skipping works and was better at math maybe mapping all of the focus pixels then predicting where the focus pixels would land at various resolutions would be feasible. For now, it takes me less time shooting a few frames at each resolution then using Photoshop and a spreadsheet to map them out one row at a time for each resolution. The dcraw -d option also helps to hunt down the pixels.
The dancing focus pixels in dual_iso video has become a little less mysterious. Here's a closeup of some focus pixels before processing in cr2hdr.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm1.staticflickr.com%2F723%2F21904694364_6bfa57295f_z.jpg&hash=3d0650e34e3d0e5aaa05a386c938d196)
The horizontal lines aren't static, they appear to be marching up the frame thus giving the appearance that the focus pixels are jumping around. After processing in cr2hdr or MLVFS the pixels appear to be "smeared" vertically. See one of my early posts for an example of this. I think that dcraw is averaging the surrounding pixels when the "badpixel" file is being processed but this isn't working with dual_iso shots. It seems to me that only the pixels to the immediate left and right of the focus pixel should be used to fill in the "bad" focus pixel and this should probably be done before processing in cr2hdr. Does that seem right? Maybe run it though a version of dcraw that only averages by columns and not rows first and then feed the output into cr2hdr? Is that possible?
I posted a snippet of the dcraw source code but on closer examination it looks like it is the section that reads the ".badpixels" file but not the section that fills in the "bad" pixels. I'm not very experienced with code and not really sure where to look but the dcraw source code is published on the Dave Coffin's dcraw website (https://www.cybercom.net/~dcoffin/dcraw/dcraw.c). Any hints on where to look?
-
Hello Folks-
Well where do I start with this? I was intrigued by the idea that @dfort shared this concept that there may be a possibility that SLR/100D uses the same Focus Pixel mappings on their sesnors as the EOS-M as well as other Rebels... At least they are supposed to be somewhat similar.
So off I go to dig into this wonderful world of Focus Pixelation which feels like a War of Terror that it never ends. Or at least it seems so.
I shot a test footage from SL1 in 24p 1280x720p MLV in both Non-Crop & Crop-Mode to see how much of the Focus Pixels disappear after converting with @Danne's incredible app (cr2hdr-r/ProRes_Folder service workflow) that included @Dfort's Pixel_Fix tags which supposedly does the magic for you. Let's see if it does with this test footage I shot.
(Mind you I shot several Dual-ISO takes just for the heck of it because I knew those Focus Pixels wouldn't disappear from Dual-ISO MLV at least not yet atm) Here it is:
Here are the screenshots:
Pre:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm6.staticflickr.com%2F5660%2F22612602312_da168ecfc2_n.jpg&hash=7a21913eb175bad61fb9499aafb22d60) (https://flic.kr/p/AscuRE) (https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm6.staticflickr.com%2F5719%2F22626132315_9c8819fd58_n.jpg&hash=d98387bb6eaaafc4e42e92601b91ba18) (https://flic.kr/p/AtoQRz)
Post:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm1.staticflickr.com%2F665%2F22600128466_1384160d4d_n.jpg&hash=1e6890ccac63c676ca0681968d9306af) (https://flic.kr/p/Ar6yPm) (https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm1.staticflickr.com%2F710%2F22003357814_bc3a35b7ed_n.jpg&hash=8fdf31bfb346f40d944975152fb3a5bd) (https://flic.kr/p/zwmXGb)
Terror and Behold ... Where did those Focus Pixels go?
Simply Amazing.
It is definitely a step closer to perfection and I am happy with the progress!
Excellent work once again by both Daniel & Danne on their never-ending contributions ... Now I reckon that many of you are probably wondering if we could do the same for the other Rebels?
Well you can help us by providing us short samples of MLV's (1 second is plenty) in every possible resolution with both Non-crop and Crop-mode from your 600D, 650D, 700D, 750D, etc... It would definitely speed up the process!
Chime on in...
-
Is that a row of pixels on the very bottom of the before frame? Looks like that needs to be mapped.
Also see a few showing up in the highlights. Need to see if those pixels are in the list of if maybe they need a little extra treatment in post.
-
Yep. You got that right. Also noticed some in highlights as well and I think this can be fixed in post but not sure if the mapping needs adjustments to get these killed as well?
This was shot on a 6.5mm 3.5 lens, in case you were wondering.
-
I was hoping that all the cameras with the focus pixel issue could share the same map file but it looks like this isn't going to work. At least not for the 700D. Here's what I could pull out of a couple of different image settings:
700D non-crop mode 1728x1152
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm1.staticflickr.com%2F756%2F22108377514_3b888afd91_c.jpg&hash=5b83ada6987b639e0bf1d2db661e3662)
700D crop mode 1280x720
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm6.staticflickr.com%2F5664%2F22731147105_b7a7860b47_c.jpg&hash=0ca13c45360ac91f512ca6315f4d8d44)
This is going to be challenging just trying to find the pattern. Quite a difference from the EOSM focus pixel pattern from my first post.
-
I've been testing Danne's new MLPP workflow and he discovered that it works with the focus pixel map file on dual iso mlv files. Quite a surprise. I thought dual iso was causing issues with this.
Compare this with the same shot run without the focus pixel map file.
Yes, I know that the color and contrast don't match between these clips. I used different processes to produce the videos.
-
gr8 stuff.
-
That is great stuff dfort. Greatful for adding the dual iso fix (--no stripe fix) To those challenging dual iso files, as well.
-
Simply beautiful and yet @Danne's new groundbreaking MLP rocks my socks!!! [emoji108]
Thanks @dfort for sharing.
-
Maybe focus pixels aren't pixels at all? Getting into the nitty gritty of the image sensor and how a Bayer filter mosaic works it occurred to me that maybe some of the green tiles were used in the "HybridCMOS AF" system used in the cameras that exhibit the focus pixel issue. A quick Google search turned up this interesting discussion that took place a couple of years ago on an astro photography site. Interesting that someone found the focus pixels on the whole sensor.
Here is that image from a Canon 650D sensor:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm1.staticflickr.com%2F578%2F22809705629_f1d329dd44_o.jpg&hash=3e8b12a9417efc73117f5045c8271884)
The discussion is also interesting: Examining the 650D/T4i "Hybrid CMOS AF" pixels (http://www.cloudynights.com/topic/413422-examining-the-650dt4i-hybrid-cmos-af-pixels/)
Here's the "Reader's Digest" version of that discussion:
While producing a Master Bias frame in PixInsight I noticed something interesting in the usually unremarkable pixel rejection maps that the ImageIntegration process creates. I seem to have inadvertently found the locations of all the "HybridCMOS AF" focus detection pixels! They showed up in the High Pixel Rejection Map.
Looking closely I think they have done something really smart here that takes advantage of one of the "weaknesses" of bayer matrix colour imaging.
First a full frame view. I have aggressively stretched the image to make the focus points obvious. JPEG compression made things rather blurry but you get the general idea here.
Now if we zoom WAY in (400%) we start to see interesting things.
Notice all the detect areas are hollow. Now why would that be....
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm1.staticflickr.com%2F657%2F23177670985_f4b26aecd3_o.png&hash=bc034fbbeec6a9a75fa835dee7c95093)
Finally we get to what I think is the clever part. All the previous images I had been working with the RAW bayer matrix data. If we now do a de-bayer run on this and convert it to colour we find.... It's all green!
This makes a lot of sense to me. With the RGB bayer matrix what we ACTUALLY have is RGGB. The green channel has 2x the pixels devoted to it as red or blue. So - if you MUST sacrifice any data at all from the sensor elusively using Green as the sacrificial pixel set is extremely logical. Green pixels are the only ones we have "spares" of!
BTW, the de-bayer method blurred these a bit to look like dots, but of course as you can see in the raw bayer matrix data that it is an artifact of debayer not physically true.
It is not at all. What Canon calls "Hybrid CMOS AF" - was introduced on he T4i/650D and is also included on the new T5i/700D. The new SL1/100D has a new variant called Hybrid CMOS AF II that seems (from marketing descriptions) to be the same thing but with focus-sense pixels across 80% of the sensor area instead of only the central area.
In all other Canon DSLRs the autofocus sensors are only found in the reflex viewfinder part of the camera. This means that focus can only be achieved when the mirror is down. The cameras with Live-view can use a software method to do a slower focus metric but it is not as fast/useful as the reflex built in focus point and can only be done between exposures. The idea with these in-sensor focus points is that these cameras can do fast/accurate focus as well as continuously adjust focus while recording video.
(The 650D/700D/100D do also have the traditional AF methods in the reflex section in addition to the Hybrid CMOS AF method)
Actually I was thinking "oh look it is not as bad as we feared it might be!" in that we effectively have a 2x oversample of Green pixels vs 1x red and 1x blue - this means that in the AF areas we drop down to a 1x sample of green. The loss of spacial resolution in those specific areas is still there but given bayer matrix reconstruction blurring of spacial resolution anyway I again think "not all that bad really".
One note on the observation that all focus pixels were green. The focus pixels can show up in just about any color the most common in Magic Lantern raw video seems to be magenta (a.k.a. pink) thus the popular term, "pink dots." Of course green is a primary color so combining the surrounding red and blue in the Bayer pattern the focus pixels change colors.
-
Yikes so this whole time it's been RGGB as oppose to RGB especially for the cameras affected by the so called "Focus Pixels" via Hybrid CMOS AF method.
[emoji102]
Seriously I love Google so much and excellent find Daniel on these articles ... Totally worth the read!!!
-
I've been testing Danne's new MLP workflow and the focus pixel map file that I created for the EOSM in 1280x720 crop mode video is working very well. I created the list using some DNG's that clearly showed the focus pixels, Photoshop with the rulers set to pixels, a Google spreadsheet and lots of patience. Basically I ran the DNG through dcraw using the -D or the -d option to make a grayscale image with no interpolation and added the -j and -t 0 options as suggested in the dcraw man page to prevent any stretching or rotation of the image.
dcraw -D -j -t 0 *.dng
This is what a potential focus pixel looks like in Photoshop. I emphasis potential because there are many "hot" pixels that show up but the focus pixels are arranged in a pattern. With the EOSM the focus pixels line up horizontally 24 pixels apart from one another. There are also short rows and long rows and once that was apparent it was easy to setup a spreadsheet to map an entire row at a time.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm6.staticflickr.com%2F5712%2F23458716453_b904fa4162.jpg&hash=5e39042325d4b8db07e31f7e52f9ebdf)
That particular pixel is at x=15 and y=12 so those coordinates would be entered into a dcraw formatted ".badpixels" file along with an optional date that the pixel was mapped. The text file looks like this:
# EOSM crop mode 1280x720
# Full width pixel rows
#
23 121 0
47 121 0
71 121 0
95 121 0
119 121 0
143 121 0
167 121 0
...
According to the dcraw man page:
List of your camera's dead pixels, so that dcraw can interpolate around them. Each line specifies the column, row, and UNIX time of death for one pixel. For example:
962 91 1028350000 # died between August 1 and 4, 2002
1285 1067 0 # don't know when this pixel died
These coordinates are before any stretching or rotation, so use dcraw -j -t 0 to locate dead pixels.
Of course looking at a text file with over 6,000 coordinates takes some abstract thinking to visualize what the pixel map actually looks like so I wrote this short shell script that turns the pixel map into an image file.
#!/bin/bash
# view_pix_map.sh
#
# Create a Portable Bit Map file from a dcraw ".badpixels"
# or an MLP "dead_pixel_list.txt" formatted file.
#
# Requires Imagemagick, bash and sed
#
# Usage view_pix_map.sh [size] [file]
# example: view_pix_map.sh 1280x720 dead_pixel_list.txt
#
# This reads the file, ignoring any blank lines or comments or lines
# that start with -P (MLP dcraw command), extracts the xy coordinates,
# discards the "UNIX time of death for one pixel" field (required by dcraw)
# and creates a Portable Bit Map graphic file showing the location of the
# mapped focus pixels. This .pbm file can be opened with Photoshop, Gimp, etc.
# it can also be opened with a text editor.
#
# 2016 Daniel A. Fort
# This is free and unencumbered software released into the public domain.
SIZE=$1
convert -size $SIZE -colorspace gray -compress none -depth 8 xc:white $2.pbm
sed -e 's/[[:space:]]*#.*// ; s/[[:space:]]*-P.*// ; /^[[:space:]]*$/d' $2 |
while read x y t; do
mogrify -colorspace gray -compress none -depth 8 -fill black -draw "point $x,$y" $2.pbm
done
Here is the EOSM focus pixel map file as an image file:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1547%2F24089384442_f775ba7a91.jpg&hash=ac525ab4e96f960653b9030061ad75f5)
As you can see it looks quite similar to the image posted in Reply #26. In this case it is in crop mode video so it is just the center of the sensor without any line skipping. There are lots of focus pixels packed into that area!
I chose the Portable Bitmap Format because it is a very simple file format that can be read with a text editor. The mapped focus pixels are the 1's.
P1
1280 720
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
...
It should be relatively simple to write a script that can create a focus pixel map that can be used with dcraw from this .pbm file. In addition, the .pbm file can be edited in Photoshop, interactively adding any missed pixels and erasing any excess. The only problem is that Photoshop saves .pbm files in binary format but it can be easily changed back to text format with Imagemagick:
mogrify -colorspace gray -compress none -depth 8 [filename]
So far I've only got the center area of the EOSM sensor mapped. I'm not sure if there is any shifting when the image size is changed in the camera and haven't yet determined how line skipping in non-crop video works so there is still lots to do on this project.
Of course others have already solved the focus pixel issue, most notably AWPStar with his MLVProducer for Windows. He has shared his method which is quite novel because he maps the coordinates from the center of the image. However I'm not looking to write a complete application, just making up some reliable pixel map files that work with dcraw and demystify the dots issue that affects some camera platforms.
-
Excellent progress and Indeed it is quite nice that @AWPStar was able to coordinate the map for FP from the center as oppose to the top left corner of the image file.
Because I think that this could simplify the entire post workflow and probably not have to worry about which resolution we shot in since it'll just start mapping out for FP from the center on out ... If that makes sense?
Meaning we can then shoot in either crop-mode or non crop-mode ... whatever suits the situation best for us while shooting on set. At least that's what I'm thinking atm.
-
Very nice work Dan.
-
I don't know why, but sometimes pointing from the center is not working and coordinates should be shifted. And i still don't know how to get right position in all cases.
Probably i should use cropPos and PanPos somehow.
And probably it is (PanPos-CropPos)
or not
-
I don't know why, but sometimes pointing from the center is not working and coordinates should be shifted. And i still don't know how to get right position in all cases.
Probably i should use cropPos and PanPos somehow.
Don't know what cropPos and PanPos is but I also found out that raw video isn't always centered in relation to the full sensor. Running exiftool on a CR2 file will give lots of useful information:
Sensor Width : 5280
Sensor Height : 3528
Sensor Left Border : 84
Sensor Top Border : 64
Sensor Right Border : 5267
Sensor Bottom Border : 3519
Unfortunately this doesn't make it into the MLV metadata and of course RAW has no metadata at all so we're left guessing how to center the pixel map.
Now that I can see what a dcraw ".badpixels" map looks like as an image file I searched around and collected several "dot removal" tools and made image files of their pixel maps. Note that other tools can also remove the focus pixels but some of them use chroma smoothing or other methods that don't require a pixel map.
Let's start with one of the earliest tools, foorgol's PinkDotRemover tool 650D (http://www.magiclantern.fm/forum/index.php?topic=6658.0):
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1661%2F24191162316_21f19939fe.jpg&hash=064375607ad3d8c86b12bf05be68d044)
I was surprised by how "dirty" the image appears. According to the author he used a script to extract the position of the dots. It looks to me that many of the coordinates are mapping random noise. What seems strange to me is the pattern looks more like what I've seen on a 700D rather than a 650D yet the tool was made to work with 650D, EOSM and "Crop Mode" though I only found one pixel map file in the tool so it must be doing some sort of scaling. The pixel map covers a 1280x720 area.
Next up, maxotics unfinished FocusPixelFixer (https://bitbucket.org/maxotics/focuspixelfixer/src/016f599a8c708fd0762bfac5cd13a15bbe3ef7ff)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1676%2F23921667400_f961da255b.jpg&hash=4a07529aa6631c35fc159535af923545)
Quite a difference. maxotics put a lot of effort into the "pink dot" issue and it appears that he has come up with a more orderly pattern. This is also 1280x720. I'm not sure if it covers both crop and non-crop but it seems that he has done most of his research using EOSM crop mode video so I would assume that this pattern is for crop mode.
The newest program that has dot removal, MLVProducer for Windows (http://www.magiclantern.fm/forum/index.php?topic=15271)
AWPStar created not one but four different pixel maps. These are for two different focus pixel patterns in crop and non-crop mode. His pixel maps are different than the others because his x y coordinates are zeroed out at the center instead of the upper left corner like the others.
Let's look at his crop mode version, dot32_crop:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1716%2F24134665201_2aca7fe8c6.jpg&hash=990f289f6331c7f45e906174ed5276c1)
This is very similar to the one from the focuspixelfixer pattern but in a slightly larger 1792x1008 size. Note that the images I'm posting have been resized to fit into the forum's guidelines. If you want to examine the full sized images you can find them on my flickr photostream (https://www.flickr.com/photos/dan_fort/).
The non-cropped version looks quite a bit different, dot32:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1707%2F23589034184_8d3e07acd6.jpg&hash=00f85b7c690beb6e76a94e5b32c51504)
Compare this to the crop version number 2 which is different than the other three because this one I calculated to be 1280x720 while the others are 1792x1008. This one is also the closest in appearance to the one that I created for the EOSM crop mode: dot32_crop2
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1551%2F23849414909_731087a0ea.jpg&hash=658cf708ae318e2df76c08933fb5ef6f)
And the non-crop version number 2: dot32_2
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1522%2F24109197062_a653abe601.jpg&hash=de3206eb31809a3c95ea4ee1be3573ff)
Something that is interesting is the number of coordinates that were mapped.
| PinkDotRemover tool 650D: | 7,747 |
| FocusPixelFixer | 10,248 |
| MLVProducer for Windows dot32_crop | 14,081 |
| MLVProducer for Windows dot32 | 25,860 |
| MLVProducer for Windows dot32_crop2 | 4,696 |
| MLVProducer for Windows dot32_2 | 12,930 |
My version for the 1280x720 EOSM crop mode has 6,120 mapped coordinates. So are more mapped coordinates better? I don't think so. I believe that only confirmed focus pixels should be mapped. I didn't use a script or a formulae to determine the position of each focus pixel, though I assumed that once I found a few points they would fall into a pattern. The most clear image that I found to verify my work was the one I posted on Reply #26 (http://magiclantern.fm/forum/index.php?topic=16054.msg157641#msg157641) of this topic. I superimposed my pixel map with that image in Photoshop and at first things didn't line up very well until I found out that the original was a "FIT" file and those files have the 0,0 x,y coordinates on the lower left corner instead of the upper left corner like most graphic image files. Once I flipped the image things lined up much better. In fact I discovered that a few lines that I assumed were long rows were in fact short rows.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1478%2F24219229485_436de6801f.jpg&hash=bc3cbca94c822d9909d1a4705c8b1421)
Now things are starting to make sense, at least for the 100D,* 650D, EOSM I should be able to create a full frame pixel map file. I've still got to figure out how line skipping works in non-crop mode. Apparently it uses every third line but my question is, which ones are the third lines?
*[EDIT: Closer examination shows that the SL-1/100D doesn't match the EOSM/650D.]
-
Each video frame in the MLV file gives you the position of the frame relative to the original raw buffer the frame was taken from (panPosX, panPosY). The raw buffer is always the same for each video mode, when you choose a resolution, you are just selecting thr size of the rectangle from which data is copied.
The position of the crop rectangle is not necessarily always centered (in fact you can move it around even while recording, so it can even vary from frame to frame, and this is why every frame has the panPos).
Therefore what you really need to do is make one map from for each video mode of the entire raw buffer. The easiest way to get the full raw buffer for a particular video mode is to use regular silent picture function. Then when you go to process a frame, shift your full map based on panPos.
(I feel like I already mentioned this is what needed to be done)
@AWPStar
Please consider releasing your source code. You have already been asked, and you beat around the bush offering excuses like the code is not ready and cleaned up enough to be released. This is a terrible reason. No code is ever perfect, and if you continue to use this as an excuse, you will never get to a point where you will release it. So just go ahead and do it. This is an open community and the ML project itself is open. We do the work we and share it freely because we expect others to do likewise.
-
@AWPStar
Please consider releasing your source code. You have already been asked, and you beat around the bush offering excuses like the code is not ready and cleaned up enough to be released. This is a terrible reason. No code is ever perfect, and if you continue to use this as an excuse, you will never get to a point where you will release it. So just go ahead and do it. This is an open community and the ML project itself is open. We do the work we and share it freely because we expect others to do likewise.
+1 and plus it would take us all into a new world of Magic sooner rather than later!
-
dmilligan
https://sourceforge.net/projects/mlvproducer/files/src/ (https://sourceforge.net/projects/mlvproducer/files/src/)
-
yay! thanks
-
Thank you, AWPStar! [emoji482]
-
Great AWP!
http://www.gnu.org/philosophy/pragmatic.html
-
My network connection went down last night so I couldn't answer some of the comments right away but the good part was I had fewer interruptions and was able to create a full sized focus pixel map for EOSM, 650D--Yay!
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1505%2F23610232483_6f69ea994d.jpg&hash=dc11d2088033ff4bb22cb9b370b256b1)
@dmilligan - You did mention back in Reply #7 (http://magiclantern.fm/forum/index.php?topic=16054.msg156006#msg156006)
I would suggest mapping the focus pixels from a silent DNG taken in each video mode. That way the coordinates are relative to the full, entire raw buffer including OB areas. That way you can use the same map for anything taken in that video mode, and simply adjust it based on the resolution crop and horizontal and vertical offset (which are already recorded in the MLV file)
I wasn't sure I understood it back then but it is starting to become more clear now that you brought up the (panPosX, panPosY). I wasn't sure where to look for it but mlv_dump -m -v [mvl_file] gave me this for a 1280x720 shot:
Block: VIDF
Offset: 0x62f16260
Size: 1616896
Time: 43283.330000 ms
Frame: #1027
Crop: 336x184
Pan: 330x181
Space: 4064
Now how do you read the Crop and Pan?
I want to add that AWPStar pointed out his SourceForge project page a while back. He even transcoded his binary data files to ASCII so I could examine his pixel map files. All of this took place out in the open on his MLVProducer for Windows (http://magiclantern.fm/forum/index.php?topic=15271.msg156663#msg156663) topic. AWPStar certainly isn't hiding any secrets!
-
Now that we've got these pixel map files that can be viewed and edited in a graphics editing program like Photoshop and Gimp the image file needs to be converted back to a dcraw ".badpixels" text file in order to be useful. This turned out a bit more complicated than the bash script to write the image file but it is working. Note that these scripts take a long time to run but it is much better than doing it manually.
Here's the script that will take a plain text formatted Portable Bit Map (.pbm) file and will use all the black pixels to create a dcraw formatted ".badpixels" file.
#!/bin/bash
# pbm2badpixels.sh
#
# Create a dcraw ".badpixels" formatted file from
# a Portable Bit Map file.
#
# The output file can be used with dcraw and MLP
# to remove the focus pixels from MLV and RAW video
# shot on Canon 100D, 650D, 700D and EOSM cameras.
#
# Requires bash sed and file
#
# Usage pbm2badpixels.sh [file.pbm]
# example: pbm2badpixels.sh EOSM_dead_pixels.pbm
#
# This reads the file, ignoring any blank lines or comments or lines
# that start with P1 (PBM "magic number"), extracts the image size,
# and creates a ".badpixels" formatted text file showing the location of the
# mapped focus pixels. This .txt file can be opened with a text editor
# for further refinement.
#
# Photoshop and perhaps other image editing programs save bitmap files
# in something other than plain text files. Imagemagick can be used to
# change these files into P1 plain text files using:
#
# mogrify -colorspace gray -compress none -depth 8 [filename]
#
# 2016 Daniel A. Fort
# This is free and unencumbered software released into the public domain.
# First check that a file to process was specified by the user.
if [ -z "$1" ]; then
cat << EndOfMessage
pbm2badpixels.sh
Usage: pbm2badpixels.sh [file.pbm]
EndOfMessage
exit
fi
# Next check that the specified file is the correct format for this script.
if [[ $(file -b $1) != "Netpbm PBM image text" ]]; then
echo "ERROR: Wrong Filetype"
exit
fi
# Output file named the same as input with the .txt file extension
output=$1".txt"
# Remove the "magic number" comments and spaces leaving just the raw pbm data
# and load that into an array.
pbm_data=($(sed -e 's/[[:space:]]*#.*// ; s/[[:space:]]*P1.*// ; /^[[:space:]]*$/d' $1 | tr " " "\n"))
# The file starts out with the width and height of the image file.
width=${pbm_data[0]}
height=${pbm_data[1]}
echo '#' $output >> $output
echo '# pbm2badpixels.sh generated' >> $output
echo '# dcraw ".badpixels" format file' >> $output
echo '# image size =' $width"x"$height >> $output
echo >> $output
# Adjust for where the pixel information starts and ends.
first_pixel=2 # first pixel at upper left - 0,0 position
last_pixel=$(($width * $height + 1)) # last pixel at lower right - width,height position
# set the counter to the first pixel
i=$first_pixel
# Loop through the pixels and write out the focus pixel locations
for (( y=0; y<=(($height -1)); y++ )); do
for (( x=0; x<=(($width -1)); x++ )); do
if [[ ${pbm_data[$i]} == 1 ]]; then
echo -e "$x \t $y \t 0" >> $output
fi
i=$[$i+1]
done
done
Using both of the scripts that I posted on this topic I was able to take my EOSM 1280x720 crop-mode focus pixel map file, turn it into a Portable Bit Map file, refine it in Photoshop and turn it back into a plain text file that can be used with dcraw (directly or through MLP). My original file had 6,120 focus pixels mapped but once it was cleaned up the amount of mapped pixels went down to 5,972. Not a big difference but I'm pretty sure that this file hits all the focus pixels without hitting any areas that aren't focus pixels. I ran some test footage through it and it looks great.
Of course there's lots to do. Each image size and aspect ratio will need their own file, line skipping needs to be worked out and then there is the issue with being able to pan the image area when in crop mode. Lots to do but at least with these scripts I'll be able to see what is going on instead of just working with lists of numbers.
-
This is great work Dan. Could be used to automate the process in MLP?
-
10q @dfort
-
Hell Yeah Dan ... Way to go on this yet another achievement of yours!!!
-
This is great work Dan. Could be used to automate the process in MLP?
Thanks. That's what I'm aiming at. I doubt that it will ever by completely automatic especially for the original raw video format because it doesn't have the necessary metadata to find the crop and pan of the image. I also have yet to figure out how to read the crop and pan information in the mlv metadata. Another thing I haven't done yet is to look more closely into video that was shot without crop-mode turned on.
One workflow problem I have is that the focus pixels are removed when the dng files are converted into non-raw image files like tiff, ProRes, etc. The focus pixels remain in the original mlv and the dng files and that would be a problem on a raw workflow using DaVinci Resolve or Adobe Camera Raw.
Of course the ideal solution would be to do it in camera. I mentioned that the CHDK (http://chdk.wikia.com/wiki/CHDK) (Canon Hack Development Kit) project was able to do it but I don't have the skills to implement their work in Magic Lantern.
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fvignette2.wikia.nocookie.net%2Fchdk%2Fimages%2F2%2F2e%2FRaw_menu_2.png%2Frevision%2Flatest%3Fcb%3D20130628234601&hash=4849ae91ac6c27111cf1a35611b15b37)
The CHDK Manual bad pixel removal tool allows the removal of defective pixels from each image as it is taken. While Canon firmware will automatically fix bad pixels that were found when the camera was manufactured, this CHDK feature will also remove "hot" or "defective" pixels which are not known to the Canon firmware (e.g. pixels that became defective during the camera lifetime). This feature affects both the JPG image and RAW image.
This is not on a menu, but is a feature enabled by putting a special "badpixel" file into the /CHDK/ folder on your SD card.
Line structure of this file:
x1,y1
x2,y2
and so on
Here {xn,yn} are the coordinates of bad pixels in RAW file format
Once you have generated a file with the list of all the "bad" pixels for your camera, CHDK can remove them automatically with the [Average] or [RAWConv] option selected. CHDK looks for the files badpixel and badpixel.txt in the /CHDK folder; this is a plain text file with coordinates of the bad pixels in the raw image, with one x,y pair per line. If both files are present, pixels listed in each file will be patched. Only the first 8kb of each file will be used.
[Off] with this setting no Bad pixel removal processing takes place.
[Average] with this setting CHDK calculates the color for the bad pixel based on its four neighbor pixels with a simple average calculation and then interpolates - bad neighbor pixels will be ignored in this calculation.
[RawConv] setting means - intended for use with post processing raw converter software to remove the bad pixels later in the workflow. With this setting CHDK just sets the bad pixel to the value 0 (zero), without any other calculation or modification. Most RAW-capable apps. will detect this and apply their own algorithms. This option is ignored in DNG mode (in DNG mode bad pixels are always averaged by CHDK).
Quote from senior developer ewavr - ‘You can compare bad pixel removal quality in both modes, IMO, 'RAWConv' mode is preferred, because CHDK interpolation is very unsophisticated’.
Note: With DNG 1.1 format enabled, bad pixels identified by badpixel.bin are always removed - (interpolated / averaged) by CHDK. This does not affect the 'Bad pixel removal' option, which also fixes user specified pixels.
There is also a topic on the CHDK forum discussing this but there hasn't been any activity on it since 2009 :-X
http://chdk.setepontos.com/index.php?topic=3098
-
Dude I knew it .... I remember this discussion re: CHDK and felt that this will one day be somehow implemented into our toys by ML and this is starting to become more of a reality than some fantasy at least it does to me.
I'm with RAWConv mode all the way baby!!!
-
Of course the ideal solution would be to do it in camera. I mentioned that the CHDK (http://chdk.wikia.com/wiki/CHDK) (Canon Hack Development Kit) project was able to do it but I don't have the skills to implement their work in Magic Lantern.
Fixing a couple hundred bad pixels on one still frame and fixing several thousand, 24/30 times per second, are two very different tasks. It seems unlikely to me that the ARM CPU could handle it. There might be some hardware processing somewhere that does it, but that would require extensive reverse engineering to find.
-
Fixing a couple hundred bad pixels on one still frame and fixing several thousand, 24/30 times per second, are two very different tasks. It seems unlikely to me that the ARM CPU could handle it. There might be some hardware processing somewhere that does it, but that would require extensive reverse engineering to find.
More on the order of around 6,000 pixels. Yeah, I only mention the CHDK project because it seems they found a way to access the "badpixel" removal system in some of the point and shoot cameras. There must be something already in the Canon firmware that deals with it because the CR2, JPEG and H.264 files don't show the focus pixels. They don't show up on the Live View screen either. I also hooked up an external monitor via HDMI and didn't see any dots. They only show up on the raw or mlv files.
In any case, I've heard the mantra--if it can be easily done in post there's no point doing it in the camera. Of course "easily done" is a rather subjective statement. I'm looking at these lines of code in mlv_dump trying to figure out how to work with crop and pan:
/* restore VIDF header */
mlv_vidf_hdr_t *hdr = slots[capture_slot].ptr;
mlv_set_type((mlv_hdr_t *)hdr, "VIDF");
mlv_set_timestamp((mlv_hdr_t *)hdr, mlv_start_timestamp);
hdr->blockSize = slots[capture_slot].blockSize;
hdr->frameSpace = slots[capture_slot].frameSpace;
/* frame number in file is off by one. nobody needs to know we skipped the first frame */
hdr->frameNumber = frame_count - 1;
hdr->cropPosX = (skip_x + 7) & ~7;
hdr->cropPosY = (skip_y + 7) & ~7;
hdr->panPosX = skip_x;
hdr->panPosY = skip_y;
void* ptr = (void*)((int32_t)hdr + sizeof(mlv_vidf_hdr_t) + hdr->frameSpace);
void* fullSizeBuffer = fullsize_buffers[(fullsize_buffer_pos+1) % 2];
:(
-
Lots of interesting things are going on in other parts of the Magic Lantern project but I'm still looking at dots and feeling confused. I hope the ML community doesn't mind me posting these lab notes and please jump in if feel you have something to contribute.
It is interesting having access to the 5D3 and checking out the differences between it and the ML menus on the EOSM. Kind of polar opposite cameras but also similar in many ways.
So on most cameras including the 5D3 the way you get into Movie crop mode is by using the magnifying glass button but the EOSM, 650D, 700D has a CROP_MODE_HACK feature that will do a digital zoom that apparently is a standard feature on the 600D. This "hack" will do a digital zoom for movie mode but not for stills. In 1920x1080 H.264 it is a 3x crop but in mlv and raw it depends on the image size selected. The EOSM maxes out at 1792 horizontal resolution in crop mode and that pretty much matches the 5x crop that you get with the magnifying glass zoom in. Now here's the interesting part, you can put the EOSM into crop mode video using the magnifying glass button on the touch screen. In fact you can go even further and do a 10x crop with the magnifying glass method and you can record video in that setting. Now will it actually give you a 10x digital zoom? Nope, it is recorded in 5x crop.
I also shot silent picture stills. I tried zooming in panning around the image shooting stills in both full resolution and "Simple" settings. In full resolution although it seemed like I was shooting stills in crop mode, the mlv wasn't cropped. With the "Simple" setting the panned shots were garbled up. In any case, shooting digital zoom stills doesn't seem to be a supported workflow so I'll skip that for now.
If the CROP_MODE_HACK centers the image then things will be much easier but according to dmilligan and AWPStar the crop and pan information is in the mlv file so I decided to take a good look at it.
As I was doing these tests I found that there are limits and then there are limits. What I mean by this is that you can only shoot a maximum of 1792 wide by 1026 high, it just won't go beyond that. Well, you might say that's a pretty good resolution but if you take it to the limit you can only record a second or so of video. Basically whatever fits in the buffer because it can't write to the SD card fast enough. I tried both mlv with and without sound and raw and the practical limit seems to be 1280x720 for crop mode video. If you really want to push it you can get a few seconds at 1792x1008.
While on the subject of width and height, there are lots of aspect ratios to choose from ranging from 5:1 (ridiculously wide) to 1:2 (cell phone vertical like) but at the extremes only the smaller image sizes are practical because you quickly hit the width or height resolution limits. Play around with the settings and you'll see what I mean.
Let's look at some test results. The goal is to come up with a master focus pixel map file and adjust it according to the crop and pan information that is embedded in every mlv frame. That means that it should be possible to deal with the focus pixels even if the crop area is panned in the middle of a shot.
First, lets take a look at the various images sizes available for the 16x9 aspect ratio:
640x360
Crop: 656x368
Pan: 650x361
960x540
Crop: 496x272
Pan: 490x271
1280x720
Crop: 336x184
Pan: 330x181
1600x900
Crop: 176x96
Pan: 170x91
1792x1008
Crop: 80x40
Pan: 74x37
The crop and pan should be in relationship with the full buffer. I'm not sure if I have been successful in determining the size of the "entire raw buffer including OB areas" but I did shoot full resolution silent mlv and dng and came up with some interesting information.
DNG - Photoshop = 5208x3477
DNG - exiftool:
Image Width : 5280
Image Height : 3529
Default Crop Origin : 0 0
Default Crop Size : 5208 3477
Active Area : 52 72 3529 5280
mlv_dump:
Crop: 0x0
Pan: 0x0
I also shot still silent in "Simple" mode and came up with a very different resolution:
DNG - Photoshop = 1734x693
DNG - exiftool:
Image Width : 1808
Image Height : 727
Default Crop Origin : 0 0
Default Crop Size : 1734 693
Active Area : 28 74 721 1808
mlv_dump:
Crop: 0x0
Pan: 0x0
Another test I ran was to pan all around to see how the pan numbers change but to my surprise so did the crop information. This is using the 1280x720 image size starting on the center then going clockwise from the top:
5x Zoom crop center
Crop: 592x336
Pan: 585x335
5x Zoom crop top
Crop: 592x376
Pan: 585x375
5x Zoom crop top right
Crop: 968x376
Pan: 961x375
5x Zoom crop right
Crop: 968x336
Pan: 961x335
5x Zoom crop bottom right
Crop: 968x296
Pan: 961x293
5x Zoom crop bottom
Crop: 592x296
Pan: 585x293
5x Zoom crop bottom left
Crop: 216x296
Pan: 209x293
5x Zoom crop left
Crop: 216x336
Pan: 209x335
5x Zoom crop top left
Crop: 216x376
Pan: 209x375
There is a pattern emerging but I'm still trying to figure out how to use this information.
According to @dmilligan:
I would suggest mapping the focus pixels from a silent DNG taken in each video mode. That way the coordinates are relative to the full, entire raw buffer including OB areas. That way you can use the same map for anything taken in that video mode, and simply adjust it based on the resolution crop and horizontal and vertical offset (which are already recorded in the MLV file)
In the process of running today's tests I found out that a full resolution silent picture doesn't show the focus pixels. A "Simple" silent picture does but it skips lines and so many of the focus pixels in crop mode video are missing because crop mode video doesn't skip any lines.
One final test--a CR2 file has even more metadata than MLV so maybe that shows more accurate information about the sensor size and possibly the raw buffer?
CR2
Photoshop = 5184x3456
exiftool
Image Width : 5184
Image Height : 3456
Canon Image Width : 5184
Canon Image Height : 3456
AF Image Width : 5184
AF Image Height : 3456
AF Area Widths : 516 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
AF Area Heights : 688 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
AF Area X Positions : 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
AF Area Y Positions : 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Aspect Ratio : 3:2
Cropped Image Width : 5184
Cropped Image Height : 3456
Cropped Image Left : 0
Cropped Image Top : 0
Sensor Width : 5280
Sensor Height : 3528
Sensor Left Border : 84
Sensor Top Border : 64
Sensor Right Border : 5267
Sensor Bottom Border : 3519
That's about enough information for today. I did shoot tests using all of the possible mlv aspect ratios and resolutions. Then of course there is raw which has even more resolution choices. MLV has 10 possible resolution choices for each aspect ratio while raw has 21 choices. That's 18 aspect ratio choices x 21 resolution choices = 378 per camera model. Making a focus pixel map for every possible aspect ratio, resolution and camera combination would be madness!
-
One more observation from today's test.
The mlv that was shot at 1280x720 using the magnifying glass crop mode method yielded a different crop and pan result from the crop mode video hack method. Perhaps the raw buffer is a different size between these two methods?
CROP_MODE_HACK 1280x720
Crop: 336x184
Pan: 330x181
5x Zoom crop center 1280x720
Crop: 592x336
Pan: 585x335
-
Don't worry about 'Crop' it's just 'Pan' rounded to the nearest 8x8 block (IDK why this is needed, it seems totally useless to me). What you need to focus on is 'Pan' it is the x,y position of the top left corner of the recorded image, relative to the actual full LV raw buffer for the current video mode. You can get the size of the full raw buffer from the raw_info structure contained within the rawi block.
FRSP is irrelevant. That's a totally different raw buffer from the LV raw one. When I say full raw buffer, I don't mean the full sensor, just the original "LV raw" buffer the recorded image was taken from.
Maybe it'll make more sense if I explain.
(0,0)
____________________________________________
| |^
| ||
| _______________________________ ||
| |\ |^ ||
| | CropPos(x,Y) || ||
| | || ||
| | |Rec |Raw Buffer
| | |Height |Height
| | || ||
| | || ||
| | |V ||
| |__________________________ |_ ||
| |<---- Recorded Width ---->| ||
| |V
|_________________________________________|_
|<----------Raw Buffer Width------------->|
How raw video works:
1. We find the memory location and dimensions of a buffer (called the LV buffer) that is created and continuously filled with sensor data by the Canon firmware. The dimensions of this buffer vary based on the current video mode (and this is what is limiting your resolution choices). This is the outer square pictured above.
2. User selects some resolution that is no larger than this buffer (this is where your limits come from) and divisible by 32*.
3. If the selected resolution is smaller than the raw buffer, ML selects some 'pan' position that approximately centers the recorded area within the raw buffer (the user can move this position around while recording for a 'digital dolly' effect).
4. Recording started.
5. Each VSYNC (when each frame is ready), ML directs some specialized hardware called "EDMAC" to copy the rectangle of our choosing (the inner rectangle pictured above), to a memory area controlled by ML (and then eventually on to the CF/SD card interface).
How simple silent pictures work:
1. We find the memory location and dimensions of a buffer (called the LV buffer) that is created and continuously filled with sensor data by the Canon firmware. The dimensions of this buffer vary based on the current video mode.
2. We copy the whole thing to a file.
So if you map the focus pixels using a simple silent picture, then for a particular raw video frame (of the same video mode) the position of the focus pixel is simply:
FPraw(x)=FPsilent(x) - PanPosX
FPraw(y)=FPsilent(y) - PanPosY
Note that you may get negative values or values larger than W x H, this just means there's a focus pixels outside of the recorded frame, and you just need to skip it.
*the reason it must be divisible by 32 is that the EDMAC needs the image data to be aligned on 4-byte word boundaries. Since the image data is packed 14 bit and is stored in 'row major order' (row by row) in memory, many pixels' values start in the middle of a byte or word and we can't copy parts of bytes and copying in the middle of a word (even if on a byte boundary) is unreliable/inefficient due to the nature of the ARM CPU and EDMAC.
-
Wow what a fantastic post. Thank you so much!
I never knew that simple silent pictures varied depending on the video setting. I thought that the still picture and movie settings didn't affect one another. Looks like I've got a lot to learn.
So now it makes sense why I saw a difference between the two methods of shooting crop mode video on the EOS-M. Comparing the RAWI block between the two modes:
CROP_MODE_HACK 1280x720
Block: VIDF
Pan: 330x181
Block: RAWI
Res: 1280x720
height 1060
width 1872
active_area.y1 28
active_area.x1 74
active_area.y2 1054
active_area.x2 1872
5x Zoom crop center 1280x720
Block: VIDF
Pan: 585x335
Block: RAWI
Res: 1280x720
height 1108
width 2592
active_area.y1 28
active_area.x1 74
active_area.y2 1102
active_area.x2 2592
So far all of the CROP_MODE_HACK mlv files that I have checked have the same raw buffer size. I'll check all of them just to make sure but that would make things very simple because there is no way that I know of to do a "digital dolly" using the CROP_MODE_HACK so only one mlv frame needs to be checked for cropping. Working it out with the 5x digital zoom will be more work but it should be doable.
So if you map the focus pixels using a simple silent picture, then for a particular raw video frame (of the same video mode) the position of the focus pixel is simply:
FPraw(x)=FPsilent(x) - PanPosX
FPraw(y)=FPsilent(y) - PanPosY
I've got a problem with simple silent picture when checking crop mode video because it looks like simple silent picture does some line skipping and crop mode video doesn't skip lines. At least that's how I understand it, got to look into it some more.
-
More tests, more data to absorb, more lessons learned. I'll start off by addressing my own ignorance:
I've got a problem with simple silent picture when checking crop mode video because it looks like simple silent picture does some line skipping and crop mode video doesn't skip lines. At least that's how I understand it, got to look into it some more.
I was switching the camera to still photo mode to shoot a simple silent picture. Doing that put the camera out of crop mode so I got the vertically squashed/horizontally stretched image which turned out to be the raw buffer for non-cropped raw video. I thought I'd just concentrate on one video mode at a time but it looks like I'm now juggling three modes. The good news is that it looks like there are only three raw buffer sizes to deal with, one for each video mode. I still need to continue testing the various resolutions but so far only these three are showing up.
1798x1026 CROP_MODE_HACK
2518x1074 5x crop mode
1734x693 standard non-crop video mode
Those dimensions are actually in the RAWI Block but it isn't apparent right away. For example on a CROP_MODE_HACK mlv there is:
height 1060
width 1872
active_area.y1 28
active_area.x1 74
active_area.y2 1054
active_area.x2 1872
RAW BUFFER(x) = active_area.x2 - active_area.x1
RAW BUFFER(y) = active_area.y2 - active_area.y1
In the above example:
1798 = 1872 - 74
1026 = 1054 - 28
Which is exactly the dimensions I got from a simple silent picture in CROP_MODE_HACK. This makes sense (sort of) because the maximum possible resolution in this mode is 1792x1026--not that you can actually shoot raw video at this size because of the write speed limit. Now is this really the raw buffer size? I'm not too sure because the pan information in the VIDF would put it way off center. For example, for a 1280x720 image centered inside of a 1798x1026 frame the upper left CropPos(x,y) PanPos(x,y) should be 259,153 but the VIDF Block has it at 585x335 330x181.
[EDIT: Correction, we're talking about pan not crop and I copied the wrong numbers in this post.]
Now that I've got a good image file of the focus pixels I'm thinking about loading some of the test shots as layers in Photoshop and seeing just where the focus pixels line up. Maybe that will help make sense of these numbers.
-
PanPos(X,Y) could actually be the number of bytes from the top left, not the number of pixels. If this is the case, then just multiply by 8/14 and see if you get numbers that make sense/line up.
-
You mean like this?
PanPos(X)
echo 330*8/14 | bc
188
PanPos(Y)
echo 181*8/14 | bc
103
Closer but still far off center.
Using Photoshop I lined up the a simple silent still (a.k.a. raw buffer) with a crop hack movie frame using 16:9 and 1280x720 and came up with a PanPos(X,Y) of 262,152. That's with the images and the focus pixel pattern lined up so I'm pretty sure this is correct. Note that the Photoshop pixel ruler starts at 0 as does the graphic in Reply #50 (http://www.magiclantern.fm/forum/index.php?topic=16054.msg160249#msg160249) so I'm pretty sure this is the correct PanPos(X,Y).
Of course this assumes that the image size in Photoshop is the same as what mlv_dump is using and I'm not sure that's a correct assumption. I found some interesting information by running mlv_dump and exiftool on the raw buffer and the mlv video images that I lined up in Photoshop.
Simple Silent Picture (a.k.a. raw buffer)
Settings:
1280x720
16:9
Crop Hack
Photoshop = 1798 x 1026
mlv_dump
height 1060
width 1872
pitch 3276
frame_size 0x0034FCB0
bits_per_pixel 14
active_area.y1 28
active_area.x1 74
active_area.y2 1054
active_area.x2 1872
Crop: 0x0
Pan: 0x0
exiftool
Rows Per Strip : 1060
Default Crop Origin : 0 0
Default Crop Size : 1798 1026
Active Area : 28 74 1054 1872
Crop movie
Settings:
1280x720
16:9
Photoshop = 1280 x 720
mlv_dump
height 1060
width 1872
pitch 3276
frame_size 0x0034FCB0
bits_per_pixel 14
active_area.y1 28
active_area.x1 74
active_area.y2 1054
active_area.x2 1872
Crop: 336x184
Pan: 330x181
exiftool
Rows Per Strip : 720
Default Crop Origin : 0 0
Default Crop Size : 1280 720
Active Area : 0 0 720 1280
So maybe the raw buffer is actually:
(active_area.x2 + active_area.x1) x (active_area.y2 + active_area.y1)
(1872 + 74) x (1054 + 28)
1946 x 1082
That would put a 1280 x 720 image centered at:
(1946 - 1200) / 2 x (1082 - 720) / 2
(666) / 2 x (362) / 2
333 x 181
That's a lot closer to mlv_dump:
Pan: 330x181
-
From source code:
uint16_t cropPosX; /* specifies from which sensor row/col the video frame was copied (8x2 blocks) */
uint16_t cropPosY; /* (can be used to process dead/hot pixels) */
Why 8x2?
- when copying raw data, you can only do so starting from a multiple of 8 pixels (14 bytes). Why? see the definition of struct raw_pixblock.
- to keep things simple, all the code in ML assumes a Bayer order of RG;GB, so you should always skip an even number of lines
Therefore, for focus pixel correction, you are interested in these two offsets.
uint16_t panPosX; /* specifies the panning offset which is cropPos, but with higher resolution (1x1 blocks) */
uint16_t panPosY; /* (it's the frame area from sensor the user wants to see) */
These were meant to be used for digital dolly at pixel-level resolution; however, I don't remember this feature being actually implemented. You do not need them - in theory (there is a bug, see below).
Now, cropPosX/Y are actually skip_x/y rounded to multiples of 8:
hdr->cropPosX = (skip_x + 7) & ~7; /* same as (skip_x + 7) / 8 * 8, using integer math */
hdr->cropPosY = (skip_y + 7) & ~7;
hdr->panPosX = skip_x;
hdr->panPosY = skip_y;
How are those skip offsets used when copying the raw buffer?
edmac_copy_rectangle_cbr_start(ptr, raw_info.buffer, raw_info.pitch, (skip_x+7)/8*14, skip_y/2*2, res_x*14/8, 0, 0, res_x*14/8, res_y, &mlv_rec_dma_cbr_r, &mlv_rec_dma_cbr_w, NULL);
So, clearly they are pixels, not bytes, and they are the top-left corner of the image.
Notice cropPosY does not respect the specification. It should be: hdr->cropPosY = skip_y & ~1. That's a bug.
To work around for existing MLV files, assume cropPosY = panPosY & ~1. That should do the trick.
So maybe the raw buffer is actually:
(active_area.x2 + active_area.x1) x (active_area.y2 + active_area.y1)
Sorry, this is nonsense. Raw buffer size is raw_info.width x height. Active area is the part that contains useful image data. What's outside active area is optical black area. Pitch is width in bytes.
To get the entire raw buffer from a DNG, including OB area: dcraw -4 -E
To get the active area only: dcraw -4 -D
Hope this helps.
-
Thanks for this (https://bitbucket.org/hudson/magic-lantern/commits/2e5367a0aa815d5945c6ba6bae4ba01f773401f2).
So with 1280x720 crop mode dcraw -4 -E on the simple silent picture resulted in a 1872 x 1060 image (full raw buffer) and dcraw -4 -D resulted in a 1798 x 1026 image (active area).
Photoshop, mlv_dump and exiftool are all in agreement about this and I'll stop my nonsense of trying to change these dimensions. I'll continue focus pixel hunting tomorrow with the new build.
-
Sounds like an excellent plan and thanks @a1ex for pointing us into the right direction.
-
Today I took the silent picture showing the full raw buffer, not just the action area, using dcraw -4 -E as instructed by a1ex and put that into Photoshop then added another layer with a frame of 1280x720 crop hack mode video. I lined up a makeshift test chart, verifying that the focus pixels also line up and presto:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1532%2F24015422169_6f1a4b9e5b.jpg&hash=7d808a9de2e656cf4b9632c0178950ed)
Note that the full 1872 x 1060 raw buffer is showing the inactive area on the top and left of the image, so much for my assumption that the active area is centered on the full buffer.
Taking a close look at where the top left corner of the video frame lines up with the image of the full raw buffer:
336x180
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1617%2F24356999376_e3e01f0b00_o.png&hash=a90b65b084b4ca1983c207787c6d19d3)
This matches up perfectly with the Crop information reported by mlv_dump.
Crop: 336x180
Pan: 330x181
Note that this is after yesterday's commit by a1ex (https://bitbucket.org/hudson/magic-lantern/commits/2e5367a0aa815d5945c6ba6bae4ba01f773401f2). The numbers mlv_dump was reporting before the fix were:
Crop: 336x184
Pan: 330x181
Of course in order to fix the focus pixels it needs to be exact so it looks like everything is working properly now.
More work to do but finally moving forward. Thanks @a1ex and @dmilligan for the expert advice.
I do have a thought to throw out there. I'm not much of a coder and what I'm doing is writing bash scripts in order prototype something that would work with dcraw in order to eliminate the focus pixels but wouldn't it be best to zap the focus pixels in mlv_dump or MLVFS or whatever program is used to create DNG frames from the mlv original? That way the dot-free DNG frames can be used in DaVinci Resolve or Premiere or whatever the user's choice transcoding/editing/grading program. I believe that the old PinkDotRemover tool 650D (http://magiclantern.fm/forum/index.php?topic=6658.0) actually modified the original raw video files. Maybe that's something to consider?
-
I think it makes sense, and probably should be enabled by default for the cameras that need it.
The video mode can be probably identified by (uncropped) raw buffer size - hopefully there aren't video modes with identical sizes, but different maps.
Offtopic: one may also create an overlay in ML that displays only these focus pixels, as a small bitmap. There may be useful info inside, and could probably be used for focus confirmation, or as a manual focus hint (to tell you in which direction you should turn the focus ring, for example).
-
Holy Moly ... That's some serious work you got going on, Dan! :D
Offtopic: one may also create an overlay in ML that displays only these focus pixels, as a small bitmap. There may be useful info inside, and could probably be used for focus confirmation, or as a manual focus hint (to tell you in which direction you should turn the focus ring, for example).
+1
-
650D at your service. Direction advice needed.
-
650D at your service. Direction advice needed.
Fantastic Walter! For starters make sure you're on the latest Nightly with the mlv_rec fix in it. Then shoot a short 1280x720 mlv using crop-mode from the ML menu and run on that mlv file:
mlv_dump -m -v [filename.mlv]
Report back with the image dimensions from "Block: RAWI" and "Block: VIDF" from one of the frames. For example, I'm getting this on a 1280x720 crop mode video.
Block: RAWI
Offset: 0x00000034
Size: 180
Time: 212.350000 ms
Res: 1280x720
raw_info:
api_version 0x00000001
height 1060
width 1872
pitch 3276
frame_size 0x0034FCB0
bits_per_pixel 14
black_level 2047
white_level 15000
active_area.y1 28
active_area.x1 74
active_area.y2 1054
active_area.x2 1872
exposure_bias 0, 0
cfa_pattern 0x02010100
calibration_ill 1
Block: VIDF
Offset: 0x06c02280
Size: 1616864
Time: 3365.025000 ms
Frame: #0070
Crop: 336x180
Pan: 330x181
Space: 4032
The important stuff in "Block: RAWI" is Res: and the active area height width information and in "Block: VIDF" I'm looking for the Crop: and Pan:
[EDIT: I originally pointed out the wrong fields--oops!]
That would be great for starters. Later I might ask you for some other raw modes and resolutions. By the way, can you get into crop mode using either the ML menu or zoom in with the magnifying glass button on the 650D? That was a new trick for me, didn't know the EOSM could do that. The two crop mode variations have different raw buffer sizes.
The video mode can be probably identified by (uncropped) raw buffer size - hopefully there aren't video modes with identical sizes, but different maps.
That's one of the tests I want to run--shoot every resolution in every raw mode just to make sure.
Offtopic: one may also create an overlay in ML that displays only these focus pixels, as a small bitmap. There may be useful info inside, and could probably be used for focus confirmation, or as a manual focus hint (to tell you in which direction you should turn the focus ring, for example).
Those focus pixels should good for something. I never get a shot that shows all of the focus pixels in one frame, sometimes they don't even appear at all. Maybe there's some sort of a hidden message in there? Anyway, for now I'll be happy if I can get rid of them reliably--there are reports that sometimes they come back in areas where the highlights are clipped. There's another test to run.
-
Report back with the image dimensions from "Block: RAWI" and "Block: VIDF" from one of the frames.
Canon Setting 1920x1080@30. ML started with default settings -> MLV_REC.mo enabled -> RAW Video (MLV) resolution set to 1280x720.
First attempt: Crop mode by Zoom-In button
Block: RAWI
Offset: 0x00000034
Size: 180
Time: 164.857000 ms
Res: 1280x720
raw_info:
api_version 0x00000001
height 1108
width 2592
pitch 4536
frame_size 0x004CB060
bits_per_pixel 14
black_level 2047
white_level 15000
active_area.y1 28
active_area.x1 74
active_area.y2 1102
active_area.x2 2592
exposure_bias 0, 0
cfa_pattern 0x02010100
calibration_ill 1
Block: VIDF
Offset: 0x03476260
Size: 1616896
Time: 1646.340000 ms
Frame: #0034
Crop: 592x334
Pan: 585x335
Space: 4064
Second set: Zoom in using ML menu "Movie crop mode ON"
Block: RAWI
Offset: 0x00000034
Size: 180
Time: 192.627000 ms
Res: 1280x720
raw_info:
api_version 0x00000001
height 1060
width 1872
pitch 3276
frame_size 0x0034FCB0
bits_per_pixel 14
black_level 2047
white_level 15000
active_area.y1 28
active_area.x1 74
active_area.y2 1054
active_area.x2 1872
exposure_bias 0, 0
cfa_pattern 0x02010100
calibration_ill 1
Block: VIDF
Offset: 0x0140f260
Size: 1616896
Time: 1037.776000 ms
Frame: #0011
Crop: 336x180
Pan: 330x181
Space: 4064
-
Thanks Walter. Looks like the 650D and the EOSM match. That was expected. Could you run exiftool on one of your CR2 files and see what the full sensor size is? The EOSM is 5280x3528.
-
Sensor Width : 5280
Sensor Height : 3528
Sensor Left Border : 84
Sensor Top Border : 64
Sensor Right Border : 5267
Sensor Bottom Border : 3519
Original Image Width : 5184
Original Image Height : 3456
Image Size : 5184x3456
-
Perfect! Here's the EOSM:
Sensor Width : 5280
Sensor Height : 3528
Sensor Left Border : 84
Sensor Top Border : 64
Sensor Right Border : 5267
Sensor Bottom Border : 3519
Original Image Width : 5184
Original Image Height : 3456
Image Size : 5184x3456
Thanks Walter. I'll get back to testing on Monday and will let you know if there's anything else to check out but for now we can assume that both the 650D and EOSM will be the same. Now there's the 100D and 700D to check out too. Any other cameras with the focus pixel issue?
-
Any other cameras with the focus pixel issue?
70D, I suppose.
EDIT: Oops ... Dual-Pixel AF, I see.
-
Made some test exporting not-demosaicing picture taken with "simple" Silent Picture whit Eos M (using rawtherapee, and photoshop to view single color channel), and it seems that focus pixels involved are only the red ones, maybe some blue ones, but not green. I'm tryng to figure out if the influence on the blue ones is some sort of "ghost" due to my test condition.
image here: https://drive.google.com/open?id=0B2JqnkAmnNAZTEhjNkZrdHlYdTg
-
it seems that focus pixels involved are only the red ones, maybe some blue ones, but not green.
That's right - it is the reason chroma smoothing from mlv_dump fixes those pixels (it does median filtering on r-g and b-g, and keeps green channel unchanged).
-
Made some test exporting not-demosaicing picture taken with "simple" Silent Picture whit Eos M (using rawtherapee, and photoshop to view single color channel), and it seems that focus pixels involved are only the red ones, maybe some blue ones, but not green. I'm tryng to figure out if the influence on the blue ones is some sort of "ghost" due to my test condition.
Interesting discovery. Exactly how did you do this?
Looks like a focus pixel at 1076,372 and 1080,378. I haven't made a pixel map for non-crop on the EOSM yet so I can't verify it.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1504%2F24411806046_44a38977ec_o.png&hash=45810ae98283a2a5d1e1a2af8c940530)
That's right - it is the reason chroma smoothing from mlv_dump fixes those pixels (it does median filtering on r-g and b-g, and keeps green channel unchanged).
I've had cases where chroma smoothing didn't get rid of all of the focus pixels. It also doesn't seem to work well when removing focus pixels from dual_iso video. Are there any issues using chroma smoothing over the entire image besides longer processing time?
Something else that came up is that in order to get this working automatically the camera model needs to be read from the metadata. mlv_dump is having a problem with the 700D. Can anyone verify this?
mlv_dump -m -v [700D_mlv_file]
Block: IDNT
Offset: 0x000001ac
Size: 84
Time: 0.004000 ms
Camera Name: 'ERR:1 md:0x 0 ml:0'
Camera Serial: ''
Camera Model: 0x00000000
Hopefully this is just an old file and is fine on the latest nightly build.
Another issue is the dng files created by mlv_dump is showing the wrong information for "Unique Camera Model" on the EOSM and 100D:
exiftool [file.dng]
Camera Model Name : Canon EOS REBEL SL1
Unique Camera Model : Canon EOS 650D
Camera Model Name : Canon EOS M
Unique Camera Model : Canon EOS 650D
Note that this is only an issue when the dng's are created with mlv_dump. MLVFS dng's show the correct Unique Camera Model.
-
Chroma smoothing over the entire image means some loss in contrast and "real" resolution, imho, and wrong results in dual iso.
For my test, I take a SP in "simple" mode of my room wall; in rawtherapee I've selected "none" for demosaicing filter, than exported as png; in Photoshop, simply selected single channels and made considerations by eyes.
-
I have 100d and i can help with developing this. What I can do? :D
-
I have 100d and i can help with developing this. What I can do? :D
Great! Do you know how to use mlv_dump and exiftool? What country are you in--or more precisely what country did your camera come from?
For starters I'd like to find out if your camera is identified as "Canon EOS REBEL SL1" or Canon "EOS 100D" and if you can run the same tests that Walter ran that would be a good start. Of course the big deal is mapping all of the focus pixels so if you shoot a blank wall or paper and bracket your exposure you should find a few frames that clearly show the focus pixels. Set your camera to movie mode and shoot "simple silent pictures" at various video settings. The 100D is a fairly new port so there's no features listing--does it have the crop mode hack?
-
yes it features crop mode hack
-
How can I help?
-
So I tried to use the mlv_dump, but it got error.
C:\Users\Oswald\My Documents\new folder>mlv_dump -m -v [M17-2218.mlv]
MLV Dumper v1.0
-----------------
Mode of operation:
- Input MLV file: '[M17-2218.mlv]'
- Verbose messages
- Verify file structure
- Dump all block information
[ERROR] Failed to open file '[M17-2218.mlv]'
C:\Users\Oswald\My Documents\new folder>
And I have mlv_dump and M17-2218.mlv on the same folder and it dont work
but here is information of exiftool. :)
Sensor Width : 5280
Sensor Height : 3528
Sensor Left Border : 84
Sensor Top Border : 64
Sensor Right Border : 5267
Sensor Bottom Border : 3519
Original Image Width : 5184
Original Image Height : 3456
Image Size : 5184x3456
-
Try without brackets.
-
yes it features crop mode hack
Great!
How can I help?
I already mapped the focus pixels for the EOSM but it would be great if you could verify the accuracy using your method. I've got a full sized pixel map image but I've got to do some more work on it to make sure it lines up properly.
https://flic.kr/p/BYmBxR
I was thinking of making a "master" that covers the entire sensor. That way I can figure out the "crops" of the various video mode raw buffers. Problem is I'm getting different dimensions depending how I measure the sensor.
Running exiftool on a CR2:
Sensor Width : 5280
Sensor Height : 3528
A full resolution silent picture should use the entire sensor but exiftool reports no sensor size information and the image size as:
Image Width : 5280
Image Height : 3529
mlv_dump agrees with that:
Res: 5280x3529
But that same full resolution silent picture in Photoshop is 5208x3477
My master pixel map is 5208x3476 so it doesn't match any of these dimensions.
So sure, I could use some help figuring this stuff out.
-
So I tried to use the mlv_dump, but it got error.
C:\Users\Oswald\My Documents\new folder>mlv_dump -m -v [M17-2218.mlv]
You put brackets around your file name? Sorry, guess I didn't explain not to put in the brackets. Do it just like you did with exiftool.
-
yes, now I got it. :)
My camera is bought from germany. I'll do it tomorrow that I'll shoot against white wall. And also check out what was the camera name on mlv_dump.
Filmed with 3x crop mode (what was in the ml menu)
Block: RAWI
Offset: 0x00000034
Size: 180
Time: 197.058000 ms
Res: 1280x720
raw_info:
api_version 0x00000001
height 1060
width 1872
pitch 3276
frame_size 0x0034FCB0
bits_per_pixel 14
black_level 2045
white_level 15000
active_area.y1 28
active_area.x1 72
active_area.y2 1060
active_area.x2 1872
exposure_bias 0, 0
cfa_pattern 0x02010100
calibration_ill 1
Block: VIDF
Offset: 0x000002f0
Size: 1616752
Time: 562.979000 ms
Frame: #0000
Crop: 328x184
Pan: 328x184
Space: 3920
and here with zoom (5x i quess) button:
Block: RAWI
Offset: 0x00000034
Size: 180
Time: 167.789000 ms
Res: 1280x720
raw_info:
api_version 0x00000001
height 1108
width 2592
pitch 4536
frame_size 0x004CB060
bits_per_pixel 14
black_level 2046
white_level 15000
active_area.y1 28
active_area.x1 72
active_area.y2 1108
active_area.x2 2592
exposure_bias 0, 0
cfa_pattern 0x02010100
calibration_ill 1
Block: VIDF
Offset: 0x05e1f260
Size: 1616896
Time: 5187.933000 ms
Frame: #0139
Crop: 584x336
Pan: 584x335
Space: 4064
-
Ah ha--got the full resolution of the sensor. (Within 1 pixel of the y axis.)
Shoot a full resolution silent picture, either dng or mlv converted to dng with mlv_dump --dng, and run dcraw -4 -E on the dng. The resulting .pgm file will be the full size of the sensor. For the EOSM/650D it is 5280x3529. The extra pixels are on the top and left, just like on the simple silent picture showing the video raw buffer.
So going back to the exiftool report on the full res silient picture:
Default Crop Origin : 0 0
Default Crop Size : 5208 3477
Active Area : 52 72 3529 5280
5208 + 72 = 5280 width -- Check
The height is completely off:
3477 + 52 = 3549 height -- Wrong!
Taking a close look in Photoshop:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1689%2F24449974755_0bae7b58b8_o.png&hash=e144cb0265084dc95d824d358dc18be8)
It appears to confirm that 72 pixels are taken off the x axis at the left but 50 pixels are taken off the top. Yet the image measures a total of 3529 high so that's what we can probably assume to be true.
That brings up the issue of what is going on with CR2 files? Running dcraw -4 -E on a CR2 file results in:
Image Size : 5280x3528
Both exiftool and Photoshop agree on this size. So where is the row shaved off on CR2 files, the top or the bottom? The only way to find out for sure is through more tests but we are close, very close and if you think about it--it doesn't really matter for raw video but as long as we're examining the full sensor we might as well find out.
Canon 100D / SL1 and Canon 700D / T5i testers, are you following along? What does exiftool report as the full sensor size of a CR2 and the image size of a full resolution silent picture (for DNG files) or mlv_dump -m -v (for mlv files) ? Are you also seeing a 1 pixel discrepancy in the vertical resolution?
BTW, I know that I mix things up a bit to keep it interesting but:
Vertical resolution = Y axis = Height
Horizontal resolution = X axis = Width
-
100d crashes when taking full res picture. But i got it recording full res mlv to memory card.
Default Crop Origin : 0 0
Default Crop Size : 5208 3477
Active Area : 52 72 3529 5280
and
Image wight: 5280
Image height: 3529
and when using it to cr2 file it shows:
Canon image wight: 5184
Canon image height: 3456
And mlv_dump shows camera model as:
Block: IDNT
Offset: 0x000000e8
Size: 84
Time: 2339.034000 ms
Camera Name: 'Canon EOS 100D'
Camera Serial: '30B08C252'
Camera Model: 0x80000346
I have B model firmware on this camera. (I installed magic lantern for b models)
-
100d crashes when taking full res picture. But i got it recording full res mlv to memory card.
Image wight: 5280
Image height: 3529
Interesting, same size as the EOSM.
Walter, could you verify this on the 650D? Anyone able to check the 700D?
So the 100D crashes when shooting full resolution still pictures to DNG but it works with MLV?
And mlv_dump shows camera model as:
Block: IDNT
Offset: 0x000000e8
Size: 84
Time: 2339.034000 ms
Camera Name: 'Canon EOS 100D'
Camera Serial: '30B08C252'
Camera Model: 0x80000346
I have B model firmware on this camera. (I installed magic lantern for b models)
This is also very interesting. Here's the same mlv_dump information from DeafEyeJedi's camera, A model firmware.
Block: IDNT
Offset: 0x000001ac
Size: 84
Time: 0.004000 ms
Camera Name: 'Canon EOS REBEL SL1'
Camera Serial: '9CBC0568A'
Camera Model: 0x80000346
I looked up the cameras that we know have the focus pixel issue in the exiftool source code -- lib/Image/ExifTool/Canon.pm:
0x80000301 => 'EOS Rebel T4i / 650D / Kiss X6i',
0x80000326 => 'EOS Rebel T5i / 700D / Kiss X7i',
0x80000331 => 'EOS M',
0x80000346 => 'EOS Rebel SL1 / 100D / Kiss X7',
"Kiss X7" is what the EOS Rebel SL1 / 100D is called in Japan. Maybe this camera has yet another firmware version?
We're gathering some good information here. Looks like it is best to identify cameras with "Camera Model" instead of "Camera Name" in mlv_dump. It also looks like all the affected cameras have the same sized sensor. So far it appears that there are two different patterns, one for the EOSM/650D another for the 100D/700D.
-
Walter, could you verify this on the 650D?
I will try but need some clarification. What do you need? FRSP is not a problem but I have no idea what "full res mlv" means in this context.
-
3477 + 52 = 3549 height -- Wrong!
3477 + 52 = 3529
The one pixel off could be an ML "bug" (incorrectly auto-detecting the raw buffer height). It doesn't really make a whole lot of sense to me to have an odd height. Or perhaps the physical sensor actually does have an odd height and Canon code chops it off for CR2s. Or maybe there's an extra row at the top that is skipped because it's GBGBGB row. IDK.
Here's the relevant code in for computing image size and active area for photo raw in raw.c (the one for LV raw is just above it):
/* autodetect image size from EDMAC */
width = shamem_read(RAW_PHOTO_EDMAC + 8) * 8 / 14; /* size B */
height = shamem_read(RAW_PHOTO_EDMAC + 4) + 1; /* size N */
/* in photo mode, raw buffer size is from ~12 Mpix (1100D) to ~24 Mpix (5D3) */
/* (this EDMAC may be reused for something else, usually smaller, or with a different size encoding - refuse to run if this happens) */
if ((width & 0xFFFFE000) || (height & 0xFFFFE000) || (width*height < 10e6) || (width*height > 30e6))
{
dbg_printf("Photo raw size error\n");
return 0;
}
/**
* The RAW file has unused areas, called "optical black" (OB); we need to skip them.
*
* To check the skip offsets, load raw_diag.mo (from the CMOS/ADTG ISO research thread),
* select the "OB zones" option, and adjust the skip offsets until the picture looks like this:
* http://a1ex.magiclantern.fm/bleeding-edge/iso50/ob/ob-zones-5d3-6400.png
*
* Use even offsets only, otherwise the colors will be screwed up.
*/
#ifdef CONFIG_5D2
skip_left = 160;
skip_top = 52;
#endif
#ifdef CONFIG_5D3
skip_left = 138; /* this gives a tight fit */
skip_right = 2;
skip_top = 80; /* matches dcraw */
#endif
#ifdef CONFIG_500D
skip_left = 62;
skip_top = 24;
/* skip one line */
raw_info.buffer += width * 14/8;
height--;
#endif
#if defined(CONFIG_550D) || defined(CONFIG_60D) || defined(CONFIG_600D)
skip_left = 142;
skip_top = 52;
#endif
#ifdef CONFIG_1100D
skip_top = 16;
skip_left = 62;
raw_info.buffer += width * 14/8;
height--;
#endif
#ifdef CONFIG_6D
skip_left = 72;
skip_right = 0;
skip_top = 52;
#endif
#if defined(CONFIG_50D)
skip_left = 64;
skip_top = 54;
#endif
#if defined(CONFIG_650D) || defined(CONFIG_EOSM) || defined(CONFIG_700D) || defined(CONFIG_100D)
skip_left = 72;
skip_top = 52;
#endif
#ifdef CONFIG_7D /* very similar to 5D2 */
skip_left = 158;
skip_top = 50;
#endif
I'd recommend running raw_diag.mo as per the comment in the code. You can also play with the height to see if there's more below the last row or if it's just garbage (raw_info.buffer -= width * 14/8; will also add a row at the top, note that doing this will flip flop the RG/GB rows, so colors will be wrong).
Note that for LV raw sizes for EOSM (and others) there's a
#warning FIXME: are these values correct for 720p and crop modes?
I will try but need some clarification. What do you need? FRSP is not a problem but I have no idea what "full res mlv" means in this context.
I think it just means FRSP using the MLV format.
-
Thanks!
DNG 650D FRSP
Image Width : 5280
Image Height : 3529
Default Crop Size : 5208 3477
Active Area : 52 72 3529 5280
Image Size : 5280x3529
MLV 650D FRSP
Block: RAWI
Offset: 0x00000034
Size: 180
Time: 14.999000 ms
Res: 5280x3529
raw_info:
api_version 0x00000001
height 3529
width 5280
pitch 9240
frame_size 0x01F18ED8
bits_per_pixel 14
black_level 2043
white_level 12000
active_area.y1 52
active_area.x1 72
active_area.y2 3529
active_area.x2 5280
exposure_bias 0, 0
cfa_pattern 0x02010100
calibration_ill 1
-
@Walter_Schulz
Perfect--just I expected. So far the EOSM and 650D are the same.
@dmilligan
Thank you so much for the detailed explanation. Guess if I studied computers instead of photography in college it wouldn't be taking me so long to figure this stuff out.
The height is completely off:
3477 + 52 = 3549 height -- Wrong!
3477 + 52 = 3529
Ha!! Got to frame that to keep me humble.
For those trying to follow along:
To make a Silent Picture (doesn't actuate the shutter and thus makes no noise) first you need to scroll all the way down in the modules list:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1710%2F23839313493_0fe4b73aa2.jpg&hash=3f3381fed5277fb7234fcb0a65258fb2)
Do like the message says and reboot the camera to load the module. Then look under the still shooting menu for Silent Picture, select it and there are several options:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1483%2F24383675521_504dcb4c69.jpg&hash=87ceceb0ba35318079acfec3a0295c53)
The options we're using are Full-res and Simple using a File Format of DNG and MLV. Note that in order to shoot MLV you also need to turn on the mlv_rec module.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1655%2F24465800635_5e51f7fcce.jpg&hash=77fc7e406328f153c7975d18b20784f5)
Full-res Silent Picture is marked "Experimental" so expect some unexpected behavior like crashes. The Simple mode is proving very useful for examining the full raw buffer. Think of it as the area that the camera "sees" before it is cropped to size and saved as a video file. In order to save the "uncropped" raw video buffer for these tests you need to be in video mode (not picture mode) and do a shutter-half press to take a Silent Picture. If you choose MLV the frames will be saved to a single MLV file--that is until you change resolution then it will start a new file.
To find out more about Silent Picture check out these topics:
http://magiclantern.fm/forum/index.php?topic=12733.0
http://magiclantern.fm/forum/index.php?topic=12523.0
-
So the 100D crashes when shooting full resolution still pictures to DNG but it works with MLV?
Yes, Both crashes, but MLV is recorded to card, but the dng isn't. SO FRSP is not yet working properly..
Yesterday i tried to get focus pixels showing, but i didn't succeed with it. Is it correct that the focus pixels should see when shooting with silent picture mode, simple?
-
@Oswald please describe steps to reproduce on the 100D thread. I'll fix it afterwards (just haven't held the 100D in hands this year) for you to be able to contribute to topic.
-
Yesterday i tried to get focus pixels showing, but i didn't succeed with it. Is it correct that the focus pixels should see when shooting with silent picture mode, simple?
Ah yes, focus pixels never appear when you want them to appear. They should show up in mlv video, raw video and Simple Silent in mlv or dng. I have never seen them in CR2 or Full-Res Silent.
What has worked best for me is to shoot a flat smooth subject like a neutral colored wall or blank sheet of white paper. Bracket the exposure so that you get a range from light grey to dark grey. The focus pixels don't seem to appear in pure white or black though they show up in some clipped highlights as green dots. Focus might also make a difference though I'm not sure. I've had them appear with both manual and EF-M lenses but I haven't found them when shooting without a lens. Hey, I tried all sorts of tricks to get them to appear. Even when you do get an image with focus pixels there's a good chance that not all of them will be visible in one frame. The most complete example I've seen was from this post in an astrophotography forum: Examining the 650D/T4i "Hybrid CMOS AF" pixels (http://www.cloudynights.com/topic/413422-examining-the-650dt4i-hybrid-cmos-af-pixels/) I referred to it in this post: http://www.magiclantern.fm/forum/index.php?topic=16054.msg157641#msg157641
Happy focus pixel hunting!
-
@dfort
Here's a focus pixel fixer for MLVFS:
https://bitbucket.org/dmilligan/mlvfs/commits/a5f271f69d977aa589d56e76d90d143a47c7df92
(you'll need to compile yourself, should be easy, you can use gcc or clang, doesn't matter, let me know if you have issues)
You must provide the maps. Name them like
[camera_model]_[raw_info.width]x[raw_info.height].fpm and put them in the current directory (directory from where you will launch mlvfs).
So for example a map for EOSM, crop hack mode would be named: 80000331_1872x1060.fpm
The contents of the maps should be a text file like
1 2
3 4
5 6
where you have the x,y location of each bad pixel on a line. My example file above has 3 bad pixels at (1,2) (3,4) and (5,6). The positions should be relative to the full raw buffer for the particular video mode (in other words, cropPosX and cropPosY will be subtracted to find the position relative to the current frame).
You need a map for each camera and video mode you want to correct (some of them will be the same or similar). If you can compile all these maps (and make sure the pixel fix actually works -> I have not tested it), I will distribute them with MLVFS and release a build.
If you run into problems with it not working, send me an example file and map for it (doesn't have to be a full map, just a few pixels), so I can test it (since I have neither of these to test with).
P.S. if we get this working and create all the maps, we can also add it to mlv_dump.
-
Happy Days!
I was reading through the source code snippets and experimenting with raw_diag.mo but now that you've gone this far I'll concentrate on creating the focus pixel maps.
Compiling MLVFS was super easy--just ran ./build_installer.sh and everything went smoothly the first time. Tested my build and it is working. I'll start putting together some pixel maps and testing them tomorrow.
By the way, here's a comparison of the "OB zones" First is the posted 5D3 image and second the EOSM image I was able to produce:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1634%2F23859109724_ae1c9a9fc7_o.png&hash=4e5fa8ab393963ca166e7ffa591f6fd2)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1572%2F24119537929_36c8458e9b_o.png&hash=940208bfe49e583cba4972ca6097e89c)
-
@dmilligan
I'm starting with 1280x720 crop mode hack because that is the setting that I'm most familiar with and have had success with dcraw and the pixel maps I have created. I believe I followed all your instructions and double checked that the pixel map lines up but haven't gotten it working yet. Note that I tried space and tab delimited files and it didn't seem to make a difference. I posted some test files in my bitbucket download area: https://bitbucket.org/daniel_fort/magic-lantern/downloads
The test files are zipped up in 2016-01-20_MLVFS_focus_pixel_fixer_test.zip.
Here are the procedures that I used.
I setup the video to "Movie crop mode ON" and "RAW video (MLV) ON, 1280x720" then shot a test target that was simply a piece of graph paper that I marked up in order to line up the layers in Photoshop. Silent Picture set to Simple, DNG. I also shot some Silent Pictures with MLV and it doesn't seem to make a difference other than some extra processing.
In Photoshop I lined up my master focus pixel map with the 1280x720 focus pixel map that I've been using with dcraw along with a frame of the movie file and the Simple silent picture. Note that the Simple silent picture was run through dcraw -4 -E as instructed by a1ex in order to show the entire raw buffer area of 1872x1060.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1677%2F23879158584_8b4a30115d_z.jpg&hash=7d5254cbe8600bbeec6a14a8916ce9c9)
Once everything was lined up I trimmed the master focus pixel map to match the 1872x1060 raw buffer and exported it as a Portable Bit Map file. In order to have that file work with the script that I wrote it needed to be converted with Imagemagick: 'mogrify -colorspace gray -compress none -depth 8 [filename]' then run through a slightly modified script that I posted earlier.
pbm2_MLVFS_pixel_map.sh
#!/bin/bash
# pbm2_MLVFS_pixel_map.sh
#
# Create an MLVFS formatted focus pixel map file from
# a Portable Bit Map file.
#
# The output file can be used with MLVFS
# to remove the focus pixels from MLV video
# shot on Canon 100D, 650D, 700D and EOSM cameras.
#
# Requires bash sed and file
#
# Usage pbm2_MLVFS_pixel_map.sh [file.pbm]
# example: pbm2_MLVFS_pixel_map.sh EOSM_dead_pixels.pbm
#
# This reads the file, ignoring any blank lines or comments or lines
# that start with P1 (PBM "magic number"), extracts the image size,
# and creates an MLVFS formatted text file showing the location of the
# mapped focus pixels. This .txt file can be opened with a text editor
# for further refinement.
#
# Photoshop and perhaps other image editing programs save bitmap files
# in something other than plain text files. Imagemagick can be used to
# change these files into P1 plain text files using:
#
# mogrify -colorspace gray -compress none -depth 8 [filename]
#
# This is a very simple script and does not support filenames and
# paths with spaces. It also runs very slowly so give it time.
#
# 2016 Daniel A. Fort
# This is free and unencumbered software released into the public domain.
# First check that a file to process was specified by the user.
if [ -z "$1" ]; then
cat << EndOfMessage
pbm2_MLVFS_pixel_map.sh
Usage: pbm2_MLVFS_pixel_map.sh [file.pbm]
EndOfMessage
exit
fi
# Next check that the specified file is the correct format for this script.
if [[ $(file -b $1) != "Netpbm PBM image text" ]]; then
echo "ERROR: Wrong Filetype"
exit
fi
# Output file named the same as input with the .txt file extension
output=$1".txt"
# Remove the "magic number" comments and spaces leaving just the raw pbm data
# and load that into an array.
pbm_data=($(sed -e 's/[[:space:]]*#.*// ; s/[[:space:]]*P1.*// ; /^[[:space:]]*$/d' $1 | tr " " "\n"))
# The file starts out with the width and height of the image file.
width=${pbm_data[0]}
height=${pbm_data[1]}
# Adjust for where the pixel information starts and ends.
first_pixel=2 # first pixel at upper left - 0,0 position
last_pixel=$(($width * $height + 1)) # last pixel at lower right - width,height position
# set the counter to the first pixel
i=$first_pixel
# Loop through the pixels and write out the focus pixel locations
for (( y=0; y<=(($height -1)); y++ )); do
for (( x=0; x<=(($width -1)); x++ )); do
if [[ ${pbm_data[$i]} == 1 ]]; then
echo -e "$x \t $y" >> $output
fi
i=$[$i+1]
done
done
Here are some images from the test.
Raw buffer saved as a portable bit map file. Note that it includes the out of bounds area so it shows the entire 1872x1060 area and the focus pixels are clearly visible.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1603%2F23882171703_a2ed142503_z.jpg&hash=5f06961592f31682b7b326020e4d2d08)
This shows the focus pixels a little better.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1541%2F23880839594_d315c2a8f5_z.jpg&hash=0f9477e0794ef9d5a9637d5e1afd902b)
Now zoom in all the way and check where a focus pixel is located. The one on the upper left is at 611 139.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1631%2F24426736091_9294549aa4_z.jpg&hash=5d2e18ae27fc0c6e28e67a30400c2fd4)
Which is in the 80000331_1872x1060.fpm file:
1362 138
1386 138
611 139
635 139
659 139
683 139
Good thing I checked everything--turns out one pixel was out of place in my master focus pixel file. I'll post the master file it once we're sure everything is working.
BTW--DeafEyeJedi lent me his SL1/100D so I'll start working on mapping focus pixels on that camera and I got a T5i/700D coming next week so that should keep me out of trouble for a while.
-
:D
I, m buying an eos m.
Hats off to you dfort.
-
:D
I, m buying an eos m.
I've got four of them--might be giving them away if this doesn't get resolved.
Came up with a problem with the 5x crop. I thought I'd attack that one next because when it is centered it gives pretty much the same results as the crop mode hack. The raw buffer is a different size. Here is what it looks like centered:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1627%2F24510174125_196f414f85_z.jpg&hash=06fb11fb03d41a3bf0e3a8b99697e136)
According to mlv_dump -m -v :
Block: RAWI
Res: 1280x720
height 1108
width 2592
Block: VIDF
Crop: 592x334
Everything checks out but when I panned the magnification to the upper left this is what happened:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1674%2F24214615560_81b2f39b41_z.jpg&hash=74a13210f08af3f5b48739424b8ab999)
Yeah, I went off the chart but the raw buffer moved along with the video frame. The mlv_dump information matches up with what I'm seeing in Photoshop:
Block: RAWI
Res: 1280x720
height 1108
width 2592
Block: VIDF
Crop: 216x374
The numbers work as far as where in the raw buffer the video frame is located but is there anything in the mlv metadata that can be used to figure out the movement of the raw buffer relative to the entire sensor? I can make a focus pixel map for a centered image but it won't automatically track when the magnified section is moved around.
I've also got a question about non-crop a.k.a. 1:1 where line skipping is involved. I could work out a new master pixel map for that but isn't there a way to adapt the master pixel map that I made for the entire sensor? Does it skip every 3rd row and column? Of course I could start over and hunt them down in Photoshop but I might be able to write a script to adjust the master. I'm also less familiar with using non-crop on these cameras and unlike the raw video I shot on the 5D3 the EOSM makes a frame that is squashed vertically or stretched horizontally--take your pick.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1511%2F24402330522_87817bea02_z.jpg&hash=a5ccc26746c627df8be533bfc1e0c3d0)
Looks like fun times ahead!
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1518%2F24510624205_411bc2733f_z.jpg&hash=d198532b0183d47f7ed93a2ea7822278)
-
I believe I followed all your instructions and double checked that the pixel map lines up but haven't gotten it working yet.
You sure you put the pixel map in the right folder? It worked perfectly here:
Before
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fbitbucket.org%2Fdmilligan%2Fmlvfs%2Fdownloads%2FFP_Before.png&hash=112476e60213ec2bba3774e9d496098f)
After
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fbitbucket.org%2Fdmilligan%2Fmlvfs%2Fdownloads%2FFP_After.png&hash=818b6e573dddc7d429af23c07bcbc2f1)
(NOTE: I probably didn't use exactly the same develop settings, so one of these appears darker)
The pixel maps have to be in the current working directory (not necessarily the same as the directory containing mlvfs executable). CWD is whatever directory you are in when you run mlvfs (btw, I have no idea what the CWD would be for the automator workflow, so you probably need launch mlvfs from the command line). Maybe I should make the .fpm search directory a command line parameter ;)
-
Oops--thought it was as simple as dropping 80000331_1872x1060.fpm in the same directory as the MLV files.
I'm lost. [EDIT: Got it to mount but still don't know where to put the 80000331_1872x1060.fpm]
With pwd showing the home directory and a copy of 80000331_1872x1060.fpm in the home directory:
/Users/rosiefort/Library/Services/MLVFS.workflow/Contents/mlvfs /Users/rosiefort/Desktop/mount --mlv-dir=/Users/rosiefort/Desktop/2016-01-20_MLVFS_focus_pixel_fixer_test/mlv
Mounts the file system from the command line but the focus pixels are still there.
In any case, great that it is working! Did I hit all of them? Looks like there are still a few left on the far left side of your after picture but they aren't there in your before picture--strange.
-
Does it skip every 3rd row and column?
I think it skips every 3rd line, but averages 3 columns of the same color:
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fa1ex.magiclantern.fm%2Fbleeding-edge%2Flv-binning%2F5d2-lv-binning-cell.png&hash=2a879ac3e5dcaade20aaf307142bc79b)
http://magiclantern.fm/forum/index.php?topic=16516
(not 100% confirmed though; still looking for test data for cameras other than 5D3)
-
@alex
Posted tests for the EOSM and SL1 on the Pixel binning patterns in LiveView (http://magiclantern.fm/forum/index.php?topic=16516.msg160857#msg160857) topic.
@dmilligan
Where do I stick the 80000331_1872x1060.fpm file? Be nice! I'd like to run a test using various resolution and aspect ratio settings to see if it is working as expected. Also, any thoughts on what to do about the 5x crop mode? I was thinking about just doing it centered though that would be pretty much like the crop mode hack with a different raw buffer size.
I'm still trying to understand how the full frame line skipping pixel binning thing works but it looks like I'm not the only one. Guess I could stop thinking about it and re-map the focus pixels so we can get it working.
-
Where do I stick the 80000331_1872x1060.fpm file?
Updated the automator workflow:
https://bitbucket.org/dmilligan/mlvfs/commits/c1cadee9ab6d9bb20dacbe5a63d4ed213df901ab
so now you can just put the .fpm file inside here (where the mlvfs executable is):
~/Library/Services/MLVFS.workflow/Contents
and run mlvfs via the automator workflow like normal.
-
Great--works on my end now.
I see some pixels on the very edges that didn't get zapped so I'll see if they are on the pixel map. Then of course I'll test various resolutions and aspect ratios on the crop hack setting then move on to other modes and other cameras.
Thanks again!
-
I see some pixels on the very edges that didn't get zapped
Yeah, it skips pixels on the edge because there's not enough adjacent pixels to interpolate from (I was just sort of hoping there wouldn't be any that fall on the edge). I suppose it could just only use the pixels available. I'll make an update.
Interpolation is also very simple and only in the horizontal direction. It just averages the next pixel of the same color immediately to the left and right of the current pixel (technically, this is also the median since there are only 2 values :P). The reason we only do horizontal is for dual ISO (which I would be interested to see some examples of to see if it works now without artifacts, it should).
-
mlvp code for interpolation if needed.
i = y * sWidth + x
' Horizontal difference
rz1 = Abs(dstt[i - 1] - dstt[i + 1]) + Abs(dstt[i - 2] - dstt[i + 2]) + Abs(dstt[i - 1 - sWidth ] - dstt[i + 1 - sWidth]) + Abs(dstt[i - 1 + sWidth ] - dstt[i + 1 + sWidth])
' Vertical difference
rz2 = Abs(dstt[i - sWidth] - dstt[i + sWidth]) + Abs(dstt[i - w2] - dstt[i + w2]) + Abs(dstt[i - sWidth - 1 ] - dstt[i + sWidth - 1]) + Abs(dstt[i - sWidth + 1 ] - dstt[i + sWidth + 1])
If rz1 < rz2 Then
dstt = (dstt[i - 2] + dstt[i + 2]) shr 1 ' /2
Else
dstt = (dstt[i - w2] + dstt[i + w2]) shr 1 ' /2
End If
The reason we only do horizontal is for dual ISO
I think if count on difference in vertical and horizontal you'll not get artefacts.
Original (http://j20.imgup.net/disotest164e5.png)
Pixel removed (http://j01.imgup.net/disotest2a81a.png)
Processed (http://p87.imgup.net/disotest3bb2f.png)
-
Interesting. A while back I posted a snippet of dcraw code that shows how Dave Coffin handled it:
/*
Seach from the current directory up to the root looking for
a ".badpixels" file, and fix those pixels now.
*/
void CLASS bad_pixels (const char *cfname)
{
FILE *fp=0;
char *fname, *cp, line[128];
int len, time, row, col, r, c, rad, tot, n, fixed=0;
if (!filters) return;
if (cfname)
fp = fopen (cfname, "r");
else {
for (len=32 ; ; len *= 2) {
fname = (char *) malloc (len);
if (!fname) return;
if (getcwd (fname, len-16)) break;
free (fname);
if (errno != ERANGE) return;
}
#if defined(WIN32) || defined(DJGPP)
if (fname[1] == ':')
memmove (fname, fname+2, len-2);
for (cp=fname; *cp; cp++)
if (*cp == '\\') *cp = '/';
#endif
cp = fname + strlen(fname);
if (cp[-1] == '/') cp--;
while (*fname == '/') {
strcpy (cp, "/.badpixels");
if ((fp = fopen (fname, "r"))) break;
if (cp == fname) break;
while (*--cp != '/');
}
free (fname);
}
if (!fp) return;
while (fgets (line, 128, fp)) {
cp = strchr (line, '#');
if (cp) *cp = 0;
if (sscanf (line, "%d %d %d", &col, &row, &time) != 3) continue;
if ((unsigned) col >= width || (unsigned) row >= height) continue;
if (time > timestamp) continue;
for (tot=n=0, rad=1; rad < 3 && n==0; rad++)
for (r = row-rad; r <= row+rad; r++)
for (c = col-rad; c <= col+rad; c++)
if ((unsigned) r < height && (unsigned) c < width &&
(r != row || c != col) && fcol(r,c) == fcol(row,col)) {
tot += BAYER2(r,c);
n++;
}
BAYER2(row,col) = tot/n;
if (verbose) {
if (!fixed++)
fprintf (stderr,_("Fixed dead pixels at:"));
fprintf (stderr, " %d,%d", col, row);
}
}
if (fixed) fputc ('\n', stderr);
fclose (fp);
}
I was able to remove the focus pixels on dual_iso with dcraw as noted in this post: http://magiclantern.fm/forum/index.php?topic=16054.msg157589#msg157589
The issue with dcraw to remove focus pixels is that a different pixel map is needed for different image sizes, it doesn't track 5x crop video and as far as I know it doesn't save to DNG with the fixed pixels.
@AWPStar I hope this topic is benefitting your 'MLVProducer for Windows' development. As you can see I was able to make image files of your pixel maps and if you compare the one you have for crop mode on the EOSM and 650D you'll see there are a few stray pixels in your maps that don't hit any focus pixel and there are also a few focus pixels that you missed. I wanted to convert my master map to your format but it seems that if you want to use the crop information embedded in the mlv metadata perhaps you should consider changing your pixel maps so 0,0 is at the upper left instead of in the center. Maybe that way we can also share the same focus pixel map files.
Taking a closer look at this first test with MLVFS I put a DNG on a layer over the master focus pixel map file in Photoshop. Here is a closeup of the focus pixels that appear at the top edge.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1665%2F24173835189_12c4197db2_z.jpg&hash=c8dad37eb87e0479240014a1af0d85fb)
Adjusting the transparency to show the pixel map reveals that the pixels that were not removed are on the focus pixel map file. (Is that what .fpm stands for?)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1672%2F23913431634_3c132f279a_z.jpg&hash=2cd2705e90cd5d9072bcbb3abf851ad0)
I don't understand why these didn't get fixed because they are not on the very edge and from my understanding the interpolation is happening to the pixels on the immediate left and right.
Moving on to the left edge.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1643%2F24515419016_cb90842427_o.png&hash=52b5261e67cdf48cd2564076cf5d4c19)
Those pixels are on the map but also at the very edge of the frame so it makes sense why they aren't getting interpolated. However, on the right edge:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1668%2F24459277661_e9ac7e54c0_z.jpg&hash=92267bd22b41adaaabe91bc39ccb298e)
There should be enough there to interpolate these pixels. Finally, on the bottom of the frame:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1711%2F24173835269_ce4654e16c_z.jpg&hash=a4f505615dbfd8cc7683dc21281a849d)
No problem here because there is enough space between the focus pixels and the edge of the frame.
The reason we only do horizontal is for dual ISO (which I would be interested to see some examples of to see if it works now without artifacts, it should).
So let's look at a dual ISO example. First without the 80000331_1872x1060.fpm:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1540%2F24460418311_088b48f7e9_z.jpg&hash=bd6b5beb5b46d0ec714d0a582ce48660)
I pushed the ACR settings to exaggerate the HDR look and cranked the Vibrance all the way so the focus pixels pop.
And this is what it looks like with the 80000331_1872x1060.fpm file inside of the MLVFS.workflow/Contents:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1466%2F24542740305_d4a1d9c47c_z.jpg&hash=5718b767b02d0df362629bd1f75c311c)
I'd say it is working!
Ok, there are still a few focus pixels visible on the left and top edge if you really look for them but dmilligan will fix that on his next update. 8)
-
Another test. This time shooting every possible combination of image size and aspect ratio in crop mode hack mode to stress test the focus pixel fixer on MLVFS.
With the EOSM the maximum possible image size is 1792x1026. The camera can record only a few seconds at that resolution but there are ways to shoot the maximum width and the maximum height by adjusting the aspect ratio. Here are the extremes.
1792x358
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1522%2F24437016012_348bf99fc3_z.jpg&hash=44e9656c9c455c025f8c16e65e8f9ef8)
640x1026
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1458%2F24519124126_d32b6aba46_n.jpg&hash=84ee6a2c5547ebf6cc410fd9e52c0769)
I'm glad to report that the focus pixel map file in MLVFS is working on all combinations of image sizes and aspect ratios in crop hack mode. Moving on to other video modes and camera models.
-
I'll make an update.
https://bitbucket.org/dmilligan/mlvfs/commits/a387b44405bca3aa9528c976553af08a94e961e4
-
@dfort
perhaps you should consider changing your pixel maps so 0,0 is at the upper left instead of in the center.
I'm all for it! But in this case i need(somehow :)) to get Full sensor pixel maps for all cameras. And handle crop and high fps modes.
-
I'll make an update.
Damn you're good--and quick too!
The update worked perfectly on my end--no more stray pixels around the edges. Looks like we can check off CROP_MODE_HACK for the EOSM. This should also work for the 650D|REBEL T4i|Kiss X6i by renaming the EOS pixel map file (80000331_1872x1060.fpm) to the number that identifies the 650D (80000301_1872x1060.fpm).
5x crop mode is going to be an even greater challenge than the CROP_MODE_HACK. To be perfectly honest with you I didn't even know about 5x crop mode until I started using a 5D3 (http://magiclantern.fm/forum/index.php?topic=5441), I didn't know other cameras didn't have the crop mode hack and didn't know what digital dolly (http://magiclantern.fm/forum/index.php?topic=6857) was all about. There seems to be so much to ML that I don't know about. In any case, when 5x crop is centered the results should appear identical to the crop mode hack but the big difference between these modes is the ability to pan around the frame in 5x crop mode. The issue is that the raw buffer also moves around the frame.
I started out with the image centered and the size set to 1280 wide. Here's what it looks like with the full sensor pixel map overlaid with the raw buffer and the cropped video overlaid on top of that:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1486%2F23921220614_c53b13bf55.jpg&hash=dda83be5874f34f0e9b9e67a698e985b)
Taking a close look at the raw buffer notice that it almost covers all of the focus pixels on the EOSM except for a few on the far right side:
[EDIT: These images are taken from the CROP_MODE_HACK. The 5x crop mode raw buffer covers these pixels but there are some focus pixels to the bottom of 5x crop mode raw buffer that aren't covered. See Reply #115 (http://magiclantern.fm/forum/index.php?topic=16054.msg161064#msg161064) for more details.]
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1711%2F24523215826_3a3e1958f9.jpg&hash=97c5a0802f530915fde3775fb653e2ba)
That should give about the same results as the CROP_MODE_HACK set at 1280x720 but after recording a few seconds it stops because of skipped frames while the CROP_MODE_HACK can record until the card fills up. So why use 5x crop mode over CROP_MODE_HACK? Because you can pan around the image just like when you're checking focus using the 5x and 10x magnifier. When I first set this up I ended up running off the edge of my makeshift chart as noted in Reply #95 (http://magiclantern.fm/forum/index.php?topic=16054.msg160817#msg160817) of this topic.
So what to do about this? I've got some ideas which I'll save for my next post.
@dfortI'm all for it! But in this case i need(somehow :)) to get Full sensor pixel maps for all cameras. And handle crop and high fps modes.
Everything I'm doing is free for everyone to use. So far I'm not entirely sure the full sensor map is 100% accurate. That's because I'm getting slightly different sizes reported from a CR2 and a Full-res Silent Picture DNG. It is only 1-pixel different but of course that makes all the difference in the world in this case. The working focus pixel map file along with a test MLV file is posted on my bitbucket download area: https://bitbucket.org/daniel_fort/magic-lantern/downloads
-
Funny thing, all footages that i have with focus pixels have CameraModel=0.
Is it about old ml builds?
-
Stay Calm ... Foam Party!!!
I'm even more surprised by the 1 pixel differences ... Great work as always, D & D!
One day in the future there'll be those who've never heard of Focus Pixels ... That's how big of a deal this is.
This is HiSToRy to be made and you guys are part of this small step for all mankind!
-
Funny thing, all footages that i have with focus pixels have CameraModel=0.
Is it about old ml builds?
Maybe old builds, maybe it is raw instead of mlv? There was a recent commit that is making all of this work so I would suggest getting new samples. I can help you out with that. Give me a few days and I'll start posting some sample frames from various cameras and some more focus pixel maps.
One day in the future there'll be those who've never heard of Focus Pixels ...
That's like so much of the jobs that I have worked on--if you notice it that probably means that I made a mistake. (I worked in editing and post production.)
-
@dfort
I'm making little experiment.
i've made full frame pattern (https://www.dropbox.com/s/jc64oanzi0f5c69/ffpattern.png?dl=0) And i want try to scale it to another crop modes.
(edit)
it's not working.
-
EOSM was missing a stub for a while and didn't record camera model correctly.
-
@dfort
I'm making little experiment.
i've made full frame pattern (https://www.dropbox.com/s/jc64oanzi0f5c69/ffpattern.png?dl=0) And i want try to scale it to another crop modes.
(edit)
it's not working.
Here is a link to my master pixel map in PNG format. (https://farm2.staticflickr.com/1551/24538110646_178fc1e2d7_o.png) Note that I use simple Portable Bit Map files when working with it. It is still a work in progress so don't be surprised if I come up with a revised version of it. The main goal is to come up with a list with the coordinates of all of the focus pixels. It would be nice to have one full frame pattern that works with all cameras, image sizes and video modes but it doesn't work that way. Looking at your map it appears that you're trying to hit ever possible location where a focus pixel might show up by repeating the pattern over the entire sensor. It doesn't work like that. You are hitting lots of areas that don't need treatment and possibly degrading the image. In addition, you can't scale it. You might be able to line skip the pattern to come up with a different pattern but I haven't tried that yet.
It seems that the 'right' way to do this is to map each of the raw buffers which varies depending on the camera and video mode. The test files I posted (https://bitbucket.org/daniel_fort/magic-lantern/downloads) include the 80000331_1872x1060.fpm file. It should also work with the 650D by simply renaming the file to 80000301_1872x1060.fpm. So far that's the only pixel map that has been tested and appears to be working. If you've been following along you'll know this is for the EOSM in crop hack mode video. I believe there will only be 3 focus pixel maps for each camera: crop hack mode, 5x crop mode and full frame mode. The software should be able to handle all possible image sizes from those three maps.
-
@dmilligan
Is there a preference to using a space or a tab as a delimiter in the focus pixel map files ".fpm" ? The reason that I ask is because they both seem to work but maybe there's an advantage to one over the other?
Today I posted CROP_MODE_HACK and 5x crop mode focus pixel map files for 650D and EOSM cameras. You can find them in my bitbucket download area: https://bitbucket.org/daniel_fort/magic-lantern/downloads
Inside of "2016-01-24_FocusPixelMaps_650D_EOSM_CropModes" you'll find:
80000301_1872x1060.fpm
80000301_2592x1108.fpm
80000331_1872x1060.fpm
80000331_2592x1108.fpm
80000331 = EOSM (tested)
80000301 = 650D (untested)
1872x1060 = CROP_MODE_HACK (finished)
2592x1108 = 5x crop mode (WIP)
The reason I'm marking 5x crop mode as a work in progress is because it might not be quite ready yet. The focus pixel map works fine as long as the image is centered but may not work when panning around the screen.
This is the mode that I didn't even know existed before I started this topic. The biggest difference between 5x crop mode and CROP_MODE_HACK is that you can pan around in 5x crop mode. It is a little tricky with the EOSM because this camera is the only one in the group with the focus pixel issue that doesn't have a magnifier button.
First of all press the INFO. button until you get to a Canon menu that looks like this:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1635%2F24560922036_a9f8664257_n.jpg&hash=68e8e2e1a3f8e45de87610219fd01d21)
Note that the camera is in Movie mode and in this case RAW Video (MLV) ON, 1280x432 Aspect Ratio 16:9. Now if you're math abilities is greater than mine you'll immediately note that 128x432 isn't 16:9. That's because in full frame video mode there is some line skipping. I haven't started on full frame video mode yet so let's just file that for future reference.
Touch the magnifying glass icon and a couple of boxes will appear on the screen.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1582%2F24504805031_bb6a8bf162_n.jpg&hash=a7308249381f40dc330078379234e2f7)
The box in the middle shows the 5x crop area and the number "x1" over the solid white box on the lower right shows that you are viewing the full frame. Press the magnifying glass icon again and:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1635%2F24219325289_3c1e24e1d1_n.jpg&hash=de746bf78319fdf3ed4fb6d724c2c2a9)
At this point you can press the INFO. button and find your way back the the ML overlays but the Canon and ML menus will be clashing with each other and the ML crop overlay will be missing so it is best to line things up and perhaps make marks on your screen when using crop mode. In any case, if you press the movie record button the video will be recorded in 5x video crop mode. All the screen overlays will disappear except for a small movie camera icon with a MB/s indicator when recording, I believe this applies to all MLV and RAW video modes.
You can press the magnifier once more and the screen will show "x10" magnification but the video will still be recorded in 5x mode.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1713%2F24587073155_cfa6e449b3_n.jpg&hash=d380ccd1b7dfabc1970a421ee2a65a64)
The big difference between 5x crop mode and CROP_MODE_HACK is that you can pan around the entire sensor in 5x crop mode. Here I moved a bit to the left and up using the triangles at the edges of the screen.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1704%2F24219324979_1188a12133_n.jpg&hash=26d89805136ffd2269791a7b1c0cca16)
Now before you get all excited and plan on doing shots that smoothly pan around the entire sensor be warned that there are issues with 5x crop mode. One problem I encountered is when recording 1280x720 5x crop mode it stops recording after a few seconds due to skipped frames while in CROP_MODE_HACK it will record continuously (provided your card is fast enough of course). In addition, I've been shooting test after test in 5x crop mode and it seems that even though you can pan to various positions even while recording, the results don't always match what you see on the screen.
Here is what the raw buffer looks like when laid over the entire sensor.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1503%2F24217943129_3e1d048b21_z.jpg&hash=51c66a7319fb948b473d714d152a4f05)
The reason the full sensor image looks green is because I've been experimenting with the RawTherapee trick @otherman used to show the Bayer pattern. This gives one more confidence check that the layers are lined up properly but the focus pixels are harder to find.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1680%2F24290177560_4a908ac390_z.jpg&hash=3494e8cdf58acbca604fc7bc9348c0e0)
Switching over to the master focus pixel map it is obvious that a few focus pixels fall outside of the centered 5x raw buffer area.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1689%2F24503416881_17a8bcac45_z.jpg&hash=4abfb0dd12440d1beffd5b1e983ea4e8)
(Erratum: I showed a 5x crop mode raw buffer in Reply #108 (http://magiclantern.fm/forum/index.php?topic=16054.msg160922#msg160922) but it was actually a CROP_MODE_HACK raw buffer. The author apologizes for this error and hopes nobody was unduly harmed. :o)
The raw buffer doesn't cover the entire sensor yet in 5x crop mode you can pan around the entire sensor--how? The raw buffer also moves around the sensor. Here is a second raw buffer covering the upper left area of the sensor
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1709%2F24477371332_2aa77851db_z.jpg&hash=3660431a8adc073a3a2ccbaa8114fc3f)
Note that only the out of bounds area of the raw buffers overlap. So how many possible 5x raw buffer positions are there? How to handle the focus pixels all of these raw buffers? That's why the 5x crop mode focus pixel map file I posted is still a work in progress.
When I started on this project I thought that creating one master focus pixel map would cover all of the various video modes. Now I'm realizing that probably isn't possible unless there is a way to find the location of the raw buffer in relationship to the entire sensor.
An alternative is to find all the possible positions of the 5x crop mode raw buffer over the entire sensor and keep adding points to the focus pixel map file until all the possible focus pixels are covered. This isn't ideal because it will be interpolating pixels that aren't focus pixels but it will probably work.
Feedback anyone?
-
For the 5x crop mode, I'm afraid the metadata regarding raw frame position relative to full image is not saved. The position of the raw buffer changes as you pan around (notice there is some slowdown when you move the focus box in certain locations - that's when the raw buffer changes its position).
An approximate (!) position can be found with focus_box_get_raw_crop_offset. A more exact position can be probably obtained from CMOS registers, the math is probably not straightforward, and the trick will be camera-specific (the meaning of those registers is not the same on all cameras).
So I'd say we have two options here:
- just give up and use chroma smoothing in this mode
- use a heuristic (pan the bad pixel map a few pixels around until you find a good match) - difficult to implement and error-prone, but with a bit of robustness (clever math), one can make it work in most practical situations.
-
I should probably keep my mouth shut (so you can just disregard this in case I have missed some info), but can't you just reset crop position before filming in 5x crop mode. (the lens is more sharp in the middle anyway).
Alt. only support this feature at middle fixed crop mode...?
-
Well another option is to use CROP_MODE_HACK instead of 5x crop mode. That's probably what I would do because at the same 1280x720 image size recording is only limited by card size while with 5x crop mode it stops after a few seconds. Probably due to the much larger raw image buffer used by the 5x crop mode?
In any case, since I got myself so deep into this I thought I'd check to see what is going on by shooting Silent Simple frames while panning around the sensor. It turned out that there are "only" 12 possible positions for the raw buffer. Three across by four high. This screenshot of some raw buffers in Photoshop should make it clear.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1475%2F24510484991_4ab3b20762_z.jpg&hash=be9e64c3766e627260e91f878c0ec31c)
Just for the sake of experimenting I'm thinking about making a second focus pixel map for 5x crop mode that includes all the possible focus pixel positions. Doing an interpolation on a pixel that doesn't require it isn't ideal but it might work better for dual iso than chroma smoothing. As far as anything requiring some clever math---
The height is completely off:
3477 + 52 = 3549 height -- Wrong!
3477 + 52 = 3529
[EDIT: @poromaa I just thought up of a use for panning around in 5x crop mode video mode--it can give you some of the capabilities of a PC (perspective control) lens. For example shifting up instead of tilting the camera up can keep vertical lines straight when shooting architecture.]
-
Just for the sake of experimenting I'm thinking about making a second focus pixel map for 5x crop mode that includes all the possible focus pixel positions.
Indeed, this should work really well.
-
Agreed. And I just had an idea to help the "minimize the damage" for pixels that would be falsely accused of being focus pixels: don't change the value if it's already between the value of the pixel to the left and to the right. In fact, using this simple heuristic might identify given a set of focus pixels, which set is the correct one: set with the fewest false positives wins. Since there are only 12 possibilities, it should be easy to find a clear winner and quickly.
Can you send some samples from a couple different crop positions and corresponding maps.
-
This is just beautiful...
-
Can you send some samples from a couple different crop positions and corresponding maps.
Certainly. The centered 5x crop is the only one that I have finished and tested so here is the focus pixel map with a Simple DNG of my makeshift test chart.
https://bitbucket.org/daniel_fort/magic-lantern/downloads
Filename: 2016-01-25_5x_crop_centered.zip
Once the sun comes up I'll shoot a short MLV file with this setting.
I'm going to work on the focus pixel map that includes all possible 5x crop focus pixel positions today. It is still very early and I'm not quite awake yet but as soon as I have another 5x crop position ready I'll post it. My scripts run really slow and this is very tedious work but by the end of the day I hope to have the final 5x crop focus pixel map finished.
If anyone out there is following along and is thinking of trying this experiment--the trick is to do this on a very steady tripod and lock everything down. Even then an entire test has to be done at once because once you move the camera, let's say to change the card or battery, the alignment is shot and you have to start the test all over. I had to start over again several times, mostly because the results I was getting wasn't what I was expecting. The test that finally worked was to shoot literally hundreds of Simple Silent Pictures in DNG format starting from the top left corner scanning left to right like TV scan lines. I tried it using MLV format for the Simple Silent Pictures but every time it switched to another buffer position the image got corrupted. Also, panning around and shooting MLV video wasn't working as expected. Lots of unexpected things going on here but that's part of the process.
-
Ok--so I didn't even wait for the sun to come up to start testing. I leaned out my window and shot Dual ISO towards the sunrise with an 8mm fisheye lens. Here's a full frame Dual ISO CR2 just to get your bearings.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1499%2F24238403399_034eaf9acb_z.jpg&hash=8e890651c3117aed95e681826b9d7e5e)
This is 5x crop mode centered with the MLVFS 80000331_2592x1108.fpm in place.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1708%2F24523928181_f5d0a1eacf_z.jpg&hash=492f3050774ff61f09da07c5730fcdca)
And without the focus pixel map file:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1595%2F24523928691_c5a3d8fc08_z.jpg&hash=df7f25bc417a0d3786ea82aba8c1fde4)
Close up of the focus pixels for dramatic effect:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1571%2F24523927561_1562bebc52_z.jpg&hash=29ad240c7b9788860663f9b54aaa53bd)
Finally, moving the 5x focus box all the way to the top left corner:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1528%2F24238403329_24a04562f8_z.jpg&hash=995e499a43317e041ffdabec09840983)
This is so extreme that even without the focus pixel map file it misses all of the focus pixels because on the EOSM they are concentrated in the center of the sensor.
(the lens is more sharp in the middle anyway).
Alt. only support this feature at middle fixed crop mode...?
Yes, this lens isn't that sharp in the corners and for that matter not too sharp in the center when shooting wide open. I'll admit I was also a bit sloppy as I was hanging out the window in the dark. For all you gear heads out there I used a Rokinon 8mm T3.8 Cine UMC Fisheye CS II Lens with Nikon F Mount and Fotasy NIK-EOS M adapter.
I can see some creative possibilities using the corners of a fisheye lens for a sort of "Leaving Las Vegas" effect. There are certainly some other uses like the pseudo PC lens I mentioned earlier so let's support this feature no matter where on the sensor you put the 5x crop box.
I'll post the sample files when my script finishes processing the next focus pixel map file. (Slow running script and even slower Photoshop user.)
-
Ok--sample files and focus pixel map files from this morning's test can be downloaded from here:
https://bitbucket.org/daniel_fort/magic-lantern/downloads
2016-01-25_5x_crop_sample_files.zip
This has just the center and the top left corner MLV and .fpm files as shown in my previous post. Some interesting observations. The center raw buffer area map shows 8,622 positions and has almost all of the focus pixels that are on the sensor. The upper left raw buffer has only 390 positions and the 1280x720 active area doesn't contain any focus pixels.
I'm also starting to work with the SL1 / 100D. In order to map the entire sensor it seems logical to start with the 5x raw buffers and it looks like this camera has 21 possible raw buffers, three across and seven up. Makes me wonder if I should redo the EOSM test to make sure that it only has 12 possible raw buffers in 5x crop mode.
[EDIT: Just ran another test on the EOSM and this time also came up with 21 possible 5x raw buffers. Glad I caught it before merging all the focus pixels.]
-
[EDIT: Just ran another test on the EOSM and this time also came up with 21 possible 5x raw buffers. Glad I caught it before merging all the focus pixels.]
Good catch, Dan! [emoji106]
-
Updated set of focus pixel map files for the EOSM and 650D covering all crop modes are ready for testing.
https://bitbucket.org/daniel_fort/magic-lantern/downloads
2016-01-26_FocusPixelMaps_650D_EOSM_CropModes.zip (https://bitbucket.org/daniel_fort/magic-lantern/downloads/2016-01-26_FocusPixelMaps_650D_EOSM_CropModes.zip)
The 5x crop mode fpm now has the focus pixels mapped for all possible positions that can be panned around the sensor. It came out to 18,576 positions. Here's what it looks like:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1478%2F24258430119_af5ea4a68d_z.jpg&hash=598c56ee2beea819f0ec8501df716757)
Yeah, I know it is a bit difficult to view this on the forum so here's a link (https://flic.kr/p/CXCN8a) to a full resolution file.
A few people have asked how they could help out but only dmilligan and I have working on this. Well hopefully that's about to change. I posted a zipped version of MLVFS.workflow with the fpm files installed on my bitbucket download area for testers to try out. I'll be updating this as new MLVSF testing builds and focus pixel map files become available.
2016-01-26_MLVFS_test_build.zip (https://bitbucket.org/daniel_fort/magic-lantern/downloads/2016-01-26_MLVFS_test_build.zip)
So far only EOSM and 650D in CROP_MODE_HACK and 5x crop mode are supported.
Of course we'll also need a focus pixel map file for the non-crop 1:1 raw video mode. Starting over again and searching for focus pixels by eye is very tedious and error prone and I haven't come across any software that can do this automatically. I'm wondering--would it be possible to take the master focus pixel map file and delete two out of three rows and columns in order to allow for line skipping and/or pixel binning and come up with a map file for full frame mode? It seems like a relatively simple task to write a script that can do this.
-
Great work dfort. i could mess with this in MLP. I see the fpm files contains all the coordinates, now what would be needed to make it work in all modes you think? I could seek for pan pos numbers and apply that specific fpm maps file to those numbers?
-
I'm wondering--would it be possible to take the master focus pixel map file and delete two out of three rows and columns in order to allow for line skipping and/or pixel binning and come up with a map file for full frame mode?
Take all the values in the master focus pixel map file, divide them by 3 and see what happens (throw out the decimal point if you get a remainder). Might have to shift it around a little, but this should get you close, but with just some extra pixels that might be skipped (IDK if there are skipped focus pixels or not, it could be that lots of focus pixels get skipped, a few get skipped, or none get skipped => if none get skipped, you are finished).
If you're feeling adventurous, it should be fairly easy to just modify MLVFS source code to do this (in cs.c, just divide x and y values by three after reading them in load_focus_pixel_map function, also you can shift around by adding an offset). Hint: after dividing x and y by 3, print them back out (e.g. printf("%d %d", x, y); ) and you're new pixel map will be in MLVFS's standard out ;)
Then you can try excluding pixels from the map based on what their modulo 3 is. So for example: only include pixels whose coordinate values in the master pixel map is divisible by 3, or only include the values whose coordinate values has a remainder of 1 when divided by 3, etc. (Hint: in C you do modulus with the % operator).
Also, I have no idea how the apparent horizontal pixel binning (http://www.magiclantern.fm/forum/index.php?topic=16516.0) may effect focus pixels. So you might find that you only need to throw out values based on their y position modulo 3 (don't throw any out based on x position).
-
Looks like another reason to refine the master focus pixel map. Right now I'm using an oversized image an lining up the focus pixels that appear on the Simple Silent Pictures that show the full raw buffer. I think that the master should be the size of a Full Resolution Silent Picture run through 'dcraw -4 -E' in order to include the out of bounds area. That is the closest that I've gotten to match the full sensor size reported by exiftool on a CR2 file. The resulting "Mother of all Focus Pixel Map Files" can then be shrunk down using the formulas that dmilligan suggested to come up with a 1:1 mode focus pixel map file. Maybe this will help better understand how line skipping/pixel binning is being handled? Then again I could just hunt for focus pixels on a 1:1 mode raw buffer and move on to other projects.
Speaking of other projects, I do need to work on other stuff so it would be great if ML users could continue testing what has been done so far. One test for EOSM/650D users would be to shoot 5x crop mode video at various image sizes and pan around the sensor. With the latest focus pixel map files there should be no traces of focus pixels in either 5x crop mode or CROP_MODE_HACK. Of course for now this only works with MLV video and the testing builds of MLVFS.
@Danne - This should work fine with MLP. The MLVFS test build will only apply the focus pixel fix to the cameras that need it. Since you are diving into C programming you might check out dmilligan's notes and see if you can get that working. Of course you'll need the master file--I'll post it when I get it lined up with a FRSP.
@DeafEyeJedi - While I continue testing with your SL1 / 100D you could test the 5x crop mode with the EOSM. You've got my Ricoh 4.8mm and Rainbow 8-48mm zoom which are perfect for crop mode video. Of course if you pan around the sensor too far you'll run off the image circle of these lenses.
I'm getting quite an education using Magic Lantern on various camera models.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1645%2F24002969023_909ff2e651_n.jpg&hash=35144381d51cbe48ee9e3446dfe5ee82)
-
Looks like there is a potential problem with the 0x80000326 => 'EOS Rebel T5i / 700D / Kiss X7i' metadata.
mlv_dump -m -v
Block: IDNT
Camera Name: 'ERR:1 md:0x 0 ml:0'
Camera Serial: ''
Camera Model: 0x00000000
From MLV processed through MLVFS
exiftool run on the DNG
Make : Canon
Camera Model Name : Canon EOS REBEL T5i
Unique Camera Model : Canon EOS REBEL T5i
Camera Model Name : ERR:1 md:0x 0 ml:0
Unique Camera Model : ERR:1 md:0x 0 ml:0
exiftool run on a DNG Simple Silent Picture
Make : Canon
Camera Model Name : Canon EOS REBEL T5i
Unique Camera Model : Canon EOS REBEL T5i
exiftool run on a CR2 file
Make : Canon
Camera Model Name : Canon EOS REBEL T5i
Canon Image Type : Canon EOS REBEL T5i
Canon Model ID : EOS Rebel T5i / 700D / Kiss X7i
I searched the 700D issues on bitbucket but didn't find anything about this. It looks to me like a problem with mlv_rec.mo but don't know where to start hunting it down. Should I open an issue on bitbucket?
-
Yet another surprise with the EOS Rebel T5i / 700D / Kiss X7i, it shares the same focus pixel pattern as the EOSM and 650D. So simply renaming the focus pixel map files that have been posted to:
80000326_1872x1060.fpm
80000326_2592x1108.fpm
Should be all it takes to get this camera ready for testing except for the MLV bug mentioned in my previous post.
The 100D isn't going to be as easy. The focus pixel pattern looks completely different. The reviews on this camera noted that there are "focus points" across 80% of the sensor. Well, here's the proof. These are all the 21 5x crop mode raw buffers stacked up in Photoshop. I outlined the boundary of the focus pixels.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1560%2F24531109652_85223782a5_z.jpg&hash=b0c8072bb0eaa910896bea910aeb272c)
For the focus pixel maps that (should) work on the EOSM, 650D and 700D this image from an astrophotography forum (noted way back on Reply #26 (http://magiclantern.fm/forum/index.php?topic=16054.msg157641#msg157641))was the most detailed image I have ever seen of focus pixels. Any ideas how to get this level of detail on the 100D? It sure would make it easier to map all those points!
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm1.staticflickr.com%2F578%2F22809705629_53bd142a86_z.jpg&hash=7bf5efcfd33cb3501b454271c915de5b)
[EDIT: Re-reading the forum discussion where that image was posted it turns out that it is a "Master Bias Rejection Map" meant to remove unwanted noise in astrophotography. In other words, "badpixels" and in our case, focus pixels. I have downloaded a trial version of the software used to create that image, PixInsight (https://pixinsight.com/). From the tutorials it looks like you need to make a bunch of shots with the lens cap on. A while ago I bought some lens caps from B&H Photo because I'm always losing them and they sent me an email asking if I can post a product review. Maybe I can upload some of these shots along with my review? ;D]
-
I've been juggling three cameras, the EOSM, SL1/100D and T5i/700D. The most work right now is going to be the SL1/100D. My experiments with PixInsight (https://pixinsight.com/) haven't produced anything useable for locating the focus pixels. That's not to say anything bad about the software, it is incredible what it can do--for astro photography. The issue with the 700D is a show stopper for that camera. I tried naming the files with a 00000000 for the camera identifier but that didn't work. I filed a bug report (https://bitbucket.org/hudson/magic-lantern/issues/2460/700d-t5i-mlv-idnt-metadata-incorrect) but couldn't find where in the code things are getting mucked up.
For the SL1/100D and the 1:1 mode video for all the cameras I'm going to go back to hunting them down in Photoshop and logging them into a spreadsheet. Rather primitive but it might take less time to take a focus pixel census than to try and predict where they live. Once we have maps for full and crop mode we can speculate what sort of pixel binning/line skipping is going on inside the camera.
-
The issue with the 700D is a show stopper for that camera. I tried naming the files with a 00000000 for the camera identifier but that didn't work. I filed a bug report (https://bitbucket.org/hudson/magic-lantern/issues/2460/700d-t5i-mlv-idnt-metadata-incorrect) but couldn't find where in the code things are getting mucked up.
https://bitbucket.org/hudson/magic-lantern/pull-requests/684/mlv_rec-eliminate-propad_getpropertydata/diff
-
IT'S ALIVE!
That's a beautiful thing--works perfectly. We can check off 700D crop mode--once the commit gets into unified and the nightly builds of course.
mlv_dump
Block: IDNT
Camera Name: 'Canon EOS REBEL T5i'
Camera Serial: '775328882'
Camera Model: 0x80000326
exiftool
Camera Model Name : Canon EOS REBEL T5i
Software : MLVFS
Unique Camera Model : Canon EOS REBEL T5i
I've also got news on the 100D. Here's what an un-crop mode mlv looks like running on MLRawViewer.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1511%2F24032131713_2cec87086e_z.jpg&hash=0f81c6566ea14029ee19e8f9e5162e9f)
All that green area is where the focus pixels live. I then shot a white sheet of paper and was able to pull out a clean shot of the pixels with just a little bit of tweaking.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1700%2F24632726056_9cb616f032_z.jpg&hash=be1eb2763d640c5b560aeefc3b58a592)
That's clean enough to run through the pbm2_MLVFS_pixel_map.sh script. Tomorrow I'll see if it worked.
-
The script finished sooner than I thought it would. Tested out the 100D full frame video and it works.
Check off another focus pixel map file.
80000346_1808x1190.fpm
Still to do:
100D - 5x crop mode
100D - CROP_MODE_HACK
EOSM - Full Frame
700D - Full Frame
650D - Full Frame
I could use some help with the 650D since I don't have a test camera. There seems to be a difference between the way full frame raw video is handled between EOSM and the 700D. I suppose that the 650D works like the EOSM but would like confirmation. Could someone with a 650D please post a Simple Silent DNG with the move mode set to any full frame setting? In other words, no crop mode or 5x zoom. Thanks!
This is what 1280x720 (16:9) 700D full frame mlv looks like in MLRawViewer.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1525%2F24293404409_490f3a42fa_z.jpg&hash=0c6b3617b2d1367bf3ea1a43449e53b2)
And this is what EOSM 1280x432 (16:9) full frame mlv looks like.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1563%2F24034317463_26900c9786_z.jpg&hash=fe8d7d380bb13a5ca2cd18cdbf46e274)
It actually says in the ML menu at the bottom, "1280x432 Stretch by 1.67x to get 1280x720 (16:9) in post."
-
Thanks Walter for the 650D file. You got the focus pixels showing nicely that's going to help confirm everything is lining up.
Looks like I guessed right this time and the 650D 1:1 video mode is the same as the EOSM. Maybe this is the "squeeze" code?
Magic Lantern / modules / raw_rec / raw_rec.c
/* squeeze factor */
if (video_mode_resolution == 1 && lv_dispsize == 1 && is_movie_mode()) /* 720p, image squeezed */
{
/* assume the raw image should be 16:9 when de-squeezed */
//int correct_height = max_res_x * 9 / 16;
//int correct_height = max_res_x * 2 / 3; //TODO : FIX THIS, USE FOR NON-FULLFRAME SENSORS!
//squeeze_factor = (float)correct_height / max_res_y;
/* 720p mode uses 5x3 binning (5DMK3) or horizontal binning + vertical skipping (other cameras) */
squeeze_factor = 1.6666f; // 5.0/3.0
}
else squeeze_factor = 1.0f;
So is raw using 720p mode? That doesn't seem right because the maximum frame size is greater than 720p. On the EOSM/650D it is 1728x692 and on the 100D/700D it is 1728x1158. Then again it might be right because the maximum size for continuous mlv recording is 1280x720 but the image is cropped a bit. There is no cropping when switching between 1920x1080 and 1280x720 H.264.
Ok, back to dealing with the focus pixels.
The 100D not only has a different focus pixel pattern but it also seems to work differently. This morning I thought I'd try shooting out my window with the 100D in 1:1 mode--is this the proper term for non-cropped video mode? It looks pretty good for 1280x720 video.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1574%2F24645886136_b4d0ee74e0_z.jpg&hash=05a856595d660d9a5f65da9538655a0d)
Taking a closer look it seems that something is off. This is what it looks like with the 100D 1:1 without the focus pixel map installed in MLVFS.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1711%2F24578561521_cfee54b757_z.jpg&hash=b1b0793936eacf3538a36a3f400eb3f3)
And here is the same area with the focus pixel map installed. By the way, this focus pixel map is named 80000346_1808x1190.fpm.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1600%2F24376567940_57847e9114_z.jpg&hash=336ade2bebe1282cf30114fc30e2cde7)
So it looks like the focus pixel map file is working but since the focus pixels seem to be hugging high contrast areas like focus peaking dots and the pixel averaging is happening only in the x axis focus pixels are appearing in the edges of areas with strong vertical lines. Of course the same is happening with Dual ISO. I tried adding chroma smoothing but it didn't get rid of them.
I uploaded a short 100D MLV file and the 80000346_1808x1190.fpm file to my bitbucket download area.
https://bitbucket.org/daniel_fort/magic-lantern/downloads
2016-01-28_100D_non-crop_video_test.zip
Back to mapping focus pixels. The 100D is showing all of its focus pixels in 1:1 video mode but in 5x crop mode and CROP_MODE_HACK they aren't as easy to find. Of course that's what is needed to create a master focus pixel map file.
[Edit: Fun facts, the SL1/100D in 1:1 video mode has 69,385 focus pixels and that's with line skipping/pixel binning. In theory the full sensor should have about 3x that many. Yikes!]
-
Hang on, looks like I struck fool's gold. The ease with making the 100D 1:1 video focus pixel map was too good to be true.
When taking a Simple Silent Picture in regular video mode, not cropped and processing it through dcraw -4 -E like I've done for the other cameras gives this result.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1548%2F24586432161_2ff71dba88_z.jpg&hash=b297482e471c1e12f68b206f50f94065)
Shooting a white sheet of paper and adjusting the settings so those pixel pop makes it look like the only thing left to do is to save it in the proper PGM format and run it through the script. That's exactly what I did. Viola!
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1601%2F24384485540_9ce763d841_z.jpg&hash=d7e580ec21ee346d8fd7b6d61de6da83)
Strange it wasn't working in MLVFS--everything else was working perfectly. Taking a closer look at that same Simple Silent Picture that I took this morning run through dcraw without any options other than saving it as a TIFF resulted in this.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1482%2F24051905904_b807865f6a_z.jpg&hash=c71f7ff84cfe013d01e5bbe3fc041250)
Yuck, what a mess of green dots. Taking a closer look it seems that there are focus pixels outside of that green box.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1649%2F24653928396_27b5e8aa48_z.jpg&hash=2cfb7e273a0dfbb262950b1651e315fc)
I adjusted the brightness, contrast, saturation, etc. so we can focus on those focus pixels that popped up beyond the green box. So it looks like that pixel pattern that I was so sure were focus pixels turned out to be something else? Back to square one with this camera.
Any SL1/100D user out there want to chime in? @Oswald @otherman Anyone?
-
There must be a few ML users that are following this topic and have cameras affected by the focus pixel issue that would like to help with this project. Here's something I could use help with. It doesn't require programming skills and can be done on any computer platform--Windows users can do this in Cygwin (https://www.cygwin.com/).
I'm posting the focus pixel map files (.fpm) files in my bitbucket download area (https://bitbucket.org/daniel_fort/magic-lantern/downloads). Take the focus pixel map file for your camera and favorite MLV video mode and convert it to a Portable Bit Map file using the following script.
fpm2pbm.sh
#!/bin/bash
# fpm2pbm.sh
#
# Create a Portable Bit Map file from an
# MLVFS .fpm (focus pixel map file)
#
# Requires ImageMagick, bash and sed
#
# Usage fpm2pbm.sh [size] [file]
# example: fpm2pbm.sh 2592x1108 80000331_2592x1108.fpm
#
# This reads the file, extracts the xy coordinates
# and creates a Portable Bit Map graphic file showing the location of the
# mapped focus pixels. This .pbm file can be opened with Photoshop, Gimp, etc.
# it can also be opened with a text editor.
#
# It can be improved by extracting the size information from the file name.
#
# 2016 Daniel A. Fort
# This is free and unencumbered software released into the public domain.
SIZE=$1
convert -size $SIZE -colorspace gray -compress none -depth 8 xc:white $2.pbm
sed -e 's/[[:space:]]*#.*// ; s/[[:space:]]*-P.*// ; /^[[:space:]]*$/d' $2 |
while read x y t; do
mogrify -colorspace gray -compress none -depth 8 -fill black -draw "point $x,$y" $2.pbm
done
This script is a slight modification of an earlier script that I posted on this topic, view_pix_map.sh (http://magiclantern.fm/forum/index.php?topic=16054.msg159767#msg159767).
Next, shoot some Simple Silent DNG frames while in movie mode--a half-press of the shutter will save a Silent Picture. Run those DNG files through dcraw using these options:
dcraw -4 -E *.DNG
The resulting *.pbm (Portable Bit Map) files can be opened in Photoshop, Gimp or other graphics editing programs. Now carefully line up the DNG with the Portable Bit Map files of the image and the focus pixel map. You will need to adjust the exposure setting of the layers until you can clearly see the image and hopefully the focus pixels. Now either adjust the transparency (or opacity) of the layers or click the visibility on and off to see if the focus pixel map file is hitting all of the visible focus pixels.
These instructions may seem daunting at first but take it one step at a time and post questions if you get lost.
What happens if you find a focus pixel that doesn't match up with the map file? Make screenshots, upload sample files or if nothing else report the location of the focus pixel using x,y coordinates in pixels with the upper left corner of the focus pixel map file image set to 0,0.
If there isn't a .fpm file for your camera and format that means that I haven't been able to map it yet. I could especially use the help of SL1/100D users.
I do have another .fpm file ready, the 1:1 (non-crop) mode for the EOSM and 650D. Maybe these files can also help a1ex with Pixel binning patterns in LiveView (http://magiclantern.fm/forum/index.php?topic=16516.0)?
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1530%2F24391733510_08a1214d44_z.jpg&hash=b79c1e80b90e507f11b7ee329bc7406f)
Here's a list of the .fpm files ready for testing:
80000301_1808x727.fpm - T4i / 650D Full Frame (1:1) Movie Mode
80000301_1872x1060.fpm - T4i / 650D CROP_MODE_HACK
80000301_2592x1108.fpm - T4i / 650D 5x Crop Mode
80000326_1872x1060.fpm - T5i / 700D CROP_MODE_HACK
80000326_2592x1108.fpm - T5i / 700D 5x Crop Mode
80000331_1808x727.fpm - EOSM Full Frame (1:1) Movie Mode
80000331_1872x1060.fpm - EOSM CROP_MODE_HACK
80000331_2592x1108.fpm - EOSM 5x Crop Mode
80000346_1808x1190.fpm - SL1 / 100D Full Frame (1:1) Movie Mode
-
That is impressive list!
But there is another mode. 3x5.
-
That is impressive list!
But there is another mode. 3x5.
Another mode? What is 3x5?
I think we're using different terminology.
A while back we had a conversation on your MLV Producer for Windows (http://magiclantern.fm/forum/index.php?topic=15271.msg156512#msg156512) topic.
dot32.dat pattern for 3x3 scaled frames
dot32_2.dat for 3x5 scaled frames
dot32_crop.dat for crop mode
I took a look at your maps along with other applications that remove focus pixels on this post (http://magiclantern.fm/forum/index.php?topic=16054.msg159826#msg159826). From the looks of it what you are calling 3x5 may be the same as what I'm calling full frame or 1:1 mode. Maybe?
When I started this project I was wondering why it was such a problem getting rid of the focus pixels. Users were reporting that sometimes the dots re-appeared in certain lighting situations or that the pixels would sometimes shift unexpectedly. Re-examining the focus pixel maps from other applications it is obvious to me that they are missing some of the focus pixels and hitting areas where there are no focus pixels. In addition, none of these programs are using the MLV crop metadata--well, it was broken (https://bitbucket.org/daniel_fort/magic-lantern/commits/2e5367a0aa815d5945c6ba6bae4ba01f773401f2) until recently.
-
@dfort
I mean mode that makes frame by using every 3 horizontal pixel and every 5 vertical. And aspect ratio should be changed on post.
it is not full frame and definitely not 1:1 mode.
-
I mean mode that makes frame by using every 3 horizontal pixel and every 5 vertical. And aspect ratio should be changed on post.
it is not full frame and definitely not 1:1 mode.
I think you're referring to something I brought up in the beginning of Reply #136 (http://magiclantern.fm/forum/index.php?topic=16054.msg161281#msg161281):
/* 720p mode uses 5x3 binning (5DMK3) or horizontal binning + vertical skipping (other cameras) */
squeeze_factor = 1.6666f; // 5.0/3.0
I'm a bit confused by the way it is worded in the source code but so far the only cameras with the focus pixel issue that needs this "squeeze_factor" applied are the EOSM and 650D. As far as I've been able to figure out there are only three MLV/raw video modes.
- Regular "non-cropped" using horizontal binning and vertical line skipping to get as much of the sensor as possible. (This probably includes what you call 3x5 for the EOSM/650D)
- 5x crop mode using the magnifier button to do a digital zoom of the frame and optionally pan around the full sensor.
- CROP_MODE_HACK available from the ML menu and works with both MLV/raw video and 1920x1080 H.264 (but not 1280x720 H.264)
Please tell me if I'm leaving anything out.
-
Regular "non-cropped" using horizontal binning and vertical line skipping to get as much of the sensor as possible. (This probably includes what you call 3x5 for the EOSM/650D)
Regular "non-cropped" has scale factor 3x3. am i wrong?
I think it's better to show how it looks like.
3x3 (http://t32.imgup.net/scale3x3c563.png)
3x5 (http://m83.imgup.net/scale3x5dedb.png)
probably this mode turns on if frame rate is ~50-60 fps. But im not sure.
-
Any SL1/100D user out there want to chime in? @Oswald @otherman Anyone?
I am trying to test it. It is little bit tricky because I have windows. But I am trying to get it working.
I have recorded little bit raw footage, green dots are in shadows/highlight area.
But tomorrow I have time to play with 100d. ;)
-
probably this mode turns on if frame rate is ~50-60 fps. But im not sure.
You're right. I never noticed that before. Never even thought of shooting raw video with these cameras at 50-60 fps though it looks like it is possible at the smallest possible image size.
In addition, I am most familiar with the EOSM and that camera when set at 16:9 and 1280 in full frame it is always what you call 3x5 no matter if the Canon menu is set to 1920x1080 or 1280x720 (or 640x480 for that matter). The MLV menu always says that the frame size is 1280x432. Another interesting thing about the EOSM is that the MLV menu reports that the frame rate is 29.97 even when the Canon menu is set to 60p. (Stating the obvious it is 25 and 50 in PAL settings)
With the SL1/100D and T5i/700D set at the same 16:9 and 1280 when the Canon menu is set to 1920x1080 the MLV menu says that the frame size is 1280x720 but when the Canon menu is set to 1280x720 the MLV menu says that the frame size is 1280x432. It also reports that the frame rate is 59.95.
Thanks AWPStar. So it looks like I'm going to have to do some adjustments to the list and add some more .fpm files. Not that big of a deal because several cameras share the same focus pixel maps and the one that is different, the SL1/100D, isn't quite ready in any mode just yet.
So as you showed with your sample frames.
3x3 mode
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1510%2F24602316951_8fb42964aa_n.jpg&hash=c0d4ab507a8645164080824632cfca1d)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1525%2F24293404409_490f3a42fa_z.jpg&hash=0c6b3617b2d1367bf3ea1a43449e53b2)
3x5 mode -- though the source code calls it 5x3 so maybe we should follow that naming convention.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1488%2F24328157409_0d6abc78fd_n.jpg&hash=9635fe5447fffa410ca2d8432ac0497e)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1563%2F24034317463_26900c9786_z.jpg&hash=fe8d7d380bb13a5ca2cd18cdbf46e274)
I am trying to test it. It is little bit tricky because I have windows. But I am trying to get it working.
I have recorded little bit raw footage, green dots are in shadows/highlight area.
Great--make sure you look outside the green box for focus pixels. I'm guessing that the pattern probably extends to the entire sensor but only the ones in the green box exhibit that focus peaking behavior.
-
So, you have almost all maps? :)
I call it 3x5 because it is literally 3x5. Or 5/3. But definitely not 5x3 :D
-
There are basically 6 video modes, not all cameras have all of them. ML code refers to these as:
mv1080
mv720
mv640
mv1080crop
mv640crop
zoom
The last three of those are 1:1 (no pixel binning/skipping), mv1080 is 3x3 pixel binning skipping, mv720 is 3x5, and mv640 I don't know about.
5x zoom and 10x zoom use the same sensor setup, so they are considered the same mode. 10x zoom just zooms the display further.
The actual buffer sizes of these modes vary by camera and are not always exactly 1080, 720, etc (usually slightly smaller).
IIRC, the EOSM is unusual in that it is always in LV and that it doesn't switch to the selected video mode until you actually start recording. The rest of the time you are in the "EOSM default" video mode. I'm not sure about this, but I think this mode is actually mv720, but at 30fps.
For most cameras, mv720 is the 50/60 fps mode (though you can use FPS override to undercrank this mode back down to 30 or 24).
-
And I thought I was almost done.
Ok, so I maybe half-way done but at least we've got the right terminology. Too bad about the EOSM. The 1920x1080 H.264 video quality is on par with the other cameras but raw video quality sucks unless it is in crop mode. In which case it is probably even better than other cameras because you can use C-mount lenses on it. I'm wondering if there is any possible way to get mv1080 on the EOSM but I take it that has been tried before.
In any case another .fpm file is processing. Let's see if I got the terminology right this time:
80000301_1808x1190.fpm - T4i / 650D - mv1080
80000326_1808x1190.fpm - T5i / 700D - mv1080
Even though the EOSM shares the same focus pixel pattern it is getting left out of this party celebration.
-
Ok--I guess the testing procedure I posted in Reply #138 (http://magiclantern.fm/forum/index.php?topic=16054.msg161309#msg161309) was a bit complicated. As of now there are 0 downloads. So I'm going to make things simpler. I'm leaving those files on my bitbucket download area (https://bitbucket.org/daniel_fort/magic-lantern/downloads) but cleaning up the confusing mess and posting something that at least Mac users can work with right away--a testing version of MLVFS with all the focus pixel map files already installed. I haven't found a way to run both this test version along with the release version posted on the MLVFS - a FUSE based, "on the fly" MLV to CDNG converter (http://magiclantern.fm/forum/index.php?topic=13152.0) topic so if you want to see what it looks like without the focus pixel fixer you'll either have to swap out the testing version with the release version or if you know the old trick of "Control click -> Show Package Contents" simply take out the *.fpm files from the Contents. The MLVFS.workflow file is located in ~/Library/Services - that's the "Go -> Go to Folder..." you'll want to navigate your Finder to.
Here are the current video modes that are ready for testing:
80000301_1808x727.fpm - T4i / 650D - mv720
80000301_1808x1190.fpm - T4i / 650D - mv1080
80000301_1872x1060.fpm - T4i / 650D - mv1080crop
80000301_2592x1108.fpm - T4i / 650D - zoom
80000331_1808x727.fpm - EOSM - mv720
80000331_1872x1060.fpm - EOSM - mv1080crop
80000331_2592x1108.fpm - EOSM - zoom
80000326_1808x727.fpm - T5i / 700D - mv720
80000326_1808x1190.fpm - T5i / 700D - mv1080
80000326_1872x1060.fpm - T5i / 700D - mv1080crop
80000326_2592x1108.fpm - T5i / 700D - zoom
80000346_1808x1190.fpm - SL1 / 100D - mv1080
I'll do some more research on mv640 and mv640crop and come up with focus pixel map files for those modes.
In the meantime I've been looking further into the SL1 / 100D issues and it doesn't look as bad as I first thought. In fact some of it was probably my error. Here is another look at a makeshift chart shot in mv1080 mode.
The MLV file was converted to CDNG with MLVFS then the DNG file was converted to a TIFF with dcraw.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1647%2F24584066912_1487d4d746_z.jpg&hash=7ce47ad6d321ac3fb0ccfaace22ad26d)
The same frame only this time with the MLVFS test version and the 80000346_1808x1190.fpm - SL1 / 100D - mv1080 focus pixel map file in place.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1534%2F24584068572_3dea4e45bc_z.jpg&hash=ee24edcad3bbb737a9828be563b31a4e)
Of course one thing is getting a clean shot of a focus chart and another thing is getting an actual scene to look good. I found out that the map file was covering the right area and hitting only actual focus pixels. The issue with the pixels showing up on strong vertical areas isn't a problem exclusive to the SL1 / 100D but rather because only the immediate left and right (x-axis) pixels are being interpolated. This was done to accommodate Dual ISO. It shouldn't happen if the averaging happens also in the y-axis. Perhaps there is a way to identify if it was shot Dual ISO and process them differently than a regular MLV shot?
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1606%2F24675783996_4663c0fa17_z.jpg&hash=640cc98deb1b590ec51e4fd376780647)
-
I'm wondering if there is any possible way to get mv1080 on the EOSM but I take it that has been tried before.
It wasn't tried too hard ;)
Looking a bit in forum history, it seems like in the very first implementations of raw video, there was a (undocumented) way to do it: http://www.magiclantern.fm/forum/index.php?topic=5860 and http://www.magiclantern.fm/forum/index.php?topic=12022
Maybe I should revisit it, but that would be a different topic.
-
That is one cleaned up test chart dfort. Amazing what neighbouring pixels can do.
-
...I'm wondering if there is any possible way to get mv1080 on the EOSM but I take it that has been tried before.
It wasn't tried too hard ;)
...
Maybe I should revisit it, but that would be a different topic.
Getting the EOSM MLV performance on par with its DSLR cousins would be awesome. That's a topic I'd like to see! 8)
That is one cleaned up test chart dfort. Amazing what neighbouring pixels can do.
Speaking of cleaning up, I did some cleanup on the mv720 focus pixel map files and updated them on my bitbucket download area (https://bitbucket.org/daniel_fort/magic-lantern/downloads). It looks like a couple of people are playing around with the files so whoever you are you might want to get the latest update. I also just discovered that the mv1080 needs some cleanup work too.
Also some progress on the SL1/100D. Looks like the focus pixels make up a rather interesting pattern.
SL1 / 100D mv1080crop mode
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1488%2F24088530373_6b3f61d21f_z.jpg&hash=df728b8507140c5033890e971d390d65)
SL1 / 100D mv1080 mode
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1490%2F24419839300_1c27e457ac_z.jpg&hash=73266ebc6298a084cb1e866c43c415da)
Maybe the pattern is a clue to how line skipping works on this camera?
-
@dfort
I'm happy to help test Eos M focus pixels, but sadly I've hit a brick wall trying to output an image file from cygwin.
I seem unable to figure out the proper syntax for executing the script in cygwin. I already formatted it for linux line returns with notepad++, but after running:
$ /usr/bin/fpm2pbm.sh 2592x1108 80000326_2592x1108.fpm
sed: can't read 80000326_2592x1108.fpm: No such file or directory
I have both my fpm2pbm.sh & 8000326_2592x1008.fpm files located in the usr/bin/ folder. Pretty sure I'm doing something wrong, but I'm inexperienced with cygwin. Can you instruct me on how to execute the script properly?
-
@Steve_FR
Ok, let's work this out one step at a time.
I'm happy to help test Eos M focus pixels... 80000326_2592x1108.fpm
The files you need for the EOSM start with 80000331 though it doesn't really matter in this case because you're looking at the 5x zoom pixel map for the T5i/700D which happens to share the same pattern as the EOSM. What is unique about the EOSM is that it doesn't have mv1080 mode. Maybe someday but not now.
I already formatted it for linux line returns with notepad++
I take it you're referring to the fpm2pbm.sh script. That step shouldn't be necessary for the .fpm files because they were created on OS-X which uses the POSIX-compliant Line Feed (LF) character for line endings. POSIX stands for Portable Operating System Interface and that's what Cygwin gives you for Windows. BTW--I'm pretty sure you don't need to change line endings for the scripts or the map files to work.
...
$ /usr/bin/fpm2pbm.sh 2592x1108 80000326_2592x1108.fpm
sed: can't read 80000326_2592x1108.fpm: No such file or directory
I have both my fpm2pbm.sh & 8000326_2592x1008.fpm files located in the usr/bin/ folder....
You were probably in a different directory when you ran that command. In your case this should work:
/usr/bin/fpm2pbm.sh 2592x1108 /usr/bin/80000326_2592x1108.fpm
Let's do even better than that. You have the fpm2pbm.sh script in your default path (which includes /usr/bin) but it might not be executable. Run this command so you won't have to type in the path to the script.
chmod +x /usr/bin/fpm2pbm.sh
Also, it is better if you don't put the focus pixel map file in /usr/bin/ - that directory is reserved for program files. You can run it from whatever directory you happen to be in, most likely your home directory which is where you're at when you launch Cygwin. So assuming you want the EOSM pixel map the command is simply:
fpm2pbm.sh 2592x1108 80000331_2592x1108.fpm
Make sure you install ImageMagick via Cygwin for the script to run. Simply run the Cygwin setup program to install the ImageMagick package.
fpm2pbm.sh is a simple bash shell script and it runs very slowly. The script that goes the other direction, pbm2_MLVFS_pixel_map.sh, runs even slower--it is best to run a 2592x1108 file overnight, it takes that long. BTW--here's a tip when you have a long name to type in the terminal, simply type the first few characters and hit "tab" to auto complete the name as far as it can. If it doesn't auto complete hit the "tab" twice and it should show you all the possible files in the path.
One last tip, this is a work in progress so you should check to see if there is a newer version of the test files on my bitbucket download (https://bitbucket.org/daniel_fort/magic-lantern/downloads) area before you get started or we might be duplicating our efforts to refine these focus pixel maps.
Hope this helps.
-
@dfort
Thank you for the detailed instructions! They worked perfectly and the script executed. The .pbm file is building right now. Success!
I'm using the focus pixel file from your newest bitbucket 2016-01-31_Focus_Pixel_Map_Files_testing.zip archive.
I also corrected my other mistake (I am now using **331 instead of **326.)
-
@Steve_FR
Great. You may want to examine your camera's MLV files with mlv_dump (http://www.magiclantern.fm/modules/modules/mlv_dump.zip/mlv_dump.zip). If you run "mlv_dump -m -v *.DNG | less" you can scroll through the metadata using the arrow keys on your keyboard. The "Crop:" field tells you where to line up your image on the focus pixel map file. Make sure you're using the latest nightly build. There were a few recent updates that fixed a problem with the Crop metadata.
Speaking of updates, was an issue with the 700D (and possibly the 650D) that didn't save the camera metadata properly. It looks like the update hasn't made it into the nightly builds yet so I posted ML testing builds for those cameras in my bitbucket download area (https://bitbucket.org/daniel_fort/magic-lantern/downloads).
@Everybody
Focus Pixel Map files for all affected cameras and all MLV modes are now posted on my bitbucket download area (https://bitbucket.org/daniel_fort/magic-lantern/downloads). That includes the testing version of MLVFS for OS-X with the .fpm files already installed.
Here are some of the things you might find interesting.
The .fpm with the least amount of mapped focus pixels is for the mlv720 on the EOSM / 650D / 700D (they all share the same focus pixel pattern) at 2,940 while the largest file is for the 100D zoom mode at 151,122 mapped pixels. The 100D has a very aggressive focus pixel pattern on the sensor that fills most of the mlv modes.
Here's a list from what I found out about all the mlv modes for the cameras that show focus pixels on raw/mlv video files:
Mode Buffer Size Notes
mv1080 - 1808x1190 - not available on the EOSM.
mv720 - 1808x727 - needs to be stretched by 1.67x - up to 50/60 fps except on EOSM.
mv640 - 1808x1190 - basically the same as mv1080 mode.
mv1080crop - 1872x1060 - highest resolution and largest sustainable frame sizes on all cameras.
mv640crop - - not possible.
zoom - 2592x1108 - same 5x magnification as mv1080crop but can pan around sensor. Performance not as good as mv1080crop.
Some settings may seem impressive but just because you can set ML to record mlv/raw at frame sizes up to 1792x1026 and up to 60fps doesn't mean it is practical with these cameras -- you might get 1 sec. recording times at best. Scaling down the frame size and frame rate helps for sustained recording. Of course that also means you're not using the entire sensor. The highest quality and performance is mv1080crop. You can get the same image sizes along with the ability to pan around the sensor using zoom mode but recording times are limited in that mode probably because of the larger raw frame buffer size.
The only issue that I can see at the moment is that focus pixels still show up on vertical lines with high contrast.
@dmilligan
In the comments for mlvfs/mlvfs/cs.c:
//simply average the two horizontally neighboring pixels of the same color
//simple and fast, do we need something better?
//horizontal direction only b/c could be dual ISO
I also made that assumption but a while back I had good results using dcraw on a Dual ISO. My understanding is that dcraw is also including the neighboring vertical pixels when interpolating the "badpixels" values.
Here's what I posted back on Reply #22 (http://magiclantern.fm/forum/index.php?topic=16054.msg157589#msg157589):
I've been testing Danne's new MLPP workflow and he discovered that it works with the focus pixel map file on dual iso mlv files. Quite a surprise. I thought dual iso was causing issues with this.
Compare this with the same shot run without the focus pixel map file.
Yes, I know that the color and contrast don't match between these clips. I used different processes to produce the videos.
Maybe it is because 3 out of the 4 pixels are close enough in value or maybe dcraw is throwing out the outlier pixel?
-
So it basically sounds like the simple linear interpolation isn't going to be good enough. There are certainly numerous, more complex and robust ways of interpolation, I was just hoping to avoid needing to use them (main reason: speed). I will need some good example footage that clearly shows where the current method struggles (and some dual ISO too).
-
Compared the Eos M focus pixels from the 80000331_2592x1108 pbm file to the 5x silent image. Coverage looks excellent, and I haven't noticed any out of place pixels yet.
Looked at 80000331_1808x727 in full screen mode, coverage looks excellent as well.
After taking some sample silent pics in full screen mode (1808x727) on my Eos M, I'm noticing some strange extra pixels that appear to be vertical rows of focus pixels. The vertical pixel rows aren't on the 1808x727 pixel map, any idea if these are actual focus pixels or fixed pattern noise? I tried many different color backgrounds, but I have been unable to produce a completely clean example file yet.
It took a lot of contrast and color channel manipulation to spot them.
Small crop preview:
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fs23.postimg.org%2Fvp0fv8p8r%2FUntitled_1.png&hash=65e84d72d2e0522769eb9a6b43362aba)
Full size tiff uncropped (7mb, dirty example): https://drive.google.com/file/d/0B73VZPdpR0_hcnVobUhVb2Z0a0k/view?usp=sharing
-
I will need some good example footage that clearly shows where the current method struggles (and some dual ISO too).
Here are SL1/100D regular mv1080 and Dual ISO MLV. Sorry for the large size of the Dual ISO shot. I'll shoot some more samples when I get a chance. (The electricity was out since yesterday afternoon due to a big wind storm.)
https://www.dropbox.com/sh/un75uk8tym22fnn/AADyxww_M3PYCZLlQqI8AxV-a?dl=0
I'm noticing some strange extra pixels that appear to be vertical rows of focus pixels. The vertical pixel rows aren't on the 1808x727 pixel map, any idea if these are actual focus pixels or fixed pattern noise?
It sure looks like some of those are focus pixels. Could you upload the DNG?
-
Shot a test chart and uploaded it to that same dropbox shared folder.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1635%2F24127326604_38d4daf089_z.jpg&hash=173a1e9d96a06eceb3ac2ae5cb4c93a7)
I was hoping to be done shooting test charts for a while but they are good for showing problems. Note that the only focus pixels visible are on high contrast vertical lines.
100D mv1080
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1538%2F24459970160_b9da98c431_z.jpg&hash=0fc110363221342b531caa333574b66e)
100D mv1080 Dual ISO
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1588%2F24128655423_5e8989c271_z.jpg&hash=2fc2add6155e2d5551752c5a6f9f4a14)
-
@dfort
Can you send me this(not dual iso) mlv file? I want to compare interpolations.
-
@dfort
I took the simple silent image using simple .mlv, here is a link to the original .mlv file:
https://drive.google.com/file/d/0B73VZPdpR0_hMHJ6S0VOV0t2UE0/view?usp=sharing
I tried to reproduce the same lighting situation with a simple silent .dng. Link to here:
https://drive.google.com/file/d/0B73VZPdpR0_hdU81ODVRR3BpQlk/view?usp=sharing
The vertical columns on the left and right sides are visible in both files with some fiddling. Let me know if you need any other files.
-
@dfort
Can you send me this(not dual iso) mlv file? I want to compare interpolations.
I uploaded the files in this dropbox folder. The Dual ISO shots are clearly marked. (Keep reading, praise for your work is coming up.)
https://www.dropbox.com/sh/un75uk8tym22fnn/AADyxww_M3PYCZLlQqI8AxV-a?dl=0
The vertical columns on the left and right sides are visible in both files with some fiddling. Let me know if you need any other files.
Wow, your finding is very significant. Thanks for this!
First of all, here's what I was able to pull out of your MLV:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1642%2F24640822142_552876184e_z.jpg&hash=de3a21ed39c37086b85a477fef8fa959)
Now the same frame with the focus pixel map file in place.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1520%2F24390966089_dd13de3175_z.jpg&hash=d94cbbc59bad9496161ffe25505b86b4)
The pattern on your shot looks very much like a 700D sample that I had from a user when I first started this topic.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm1.staticflickr.com%2F756%2F22108377514_3b888afd91_z.jpg&hash=172bb250008d5508f9ba352be3b9eee8)
So much for trying to re-invent the wheel. Looks like foorgol (PinkDotRemover tool 650D (http://magiclantern.fm/forum/index.php?topic=6658.0)) and AWPStar (MLVProducer (http://magiclantern.fm/forum/index.php?topic=15271.0)) got it right. AWPStar's maps are by far the cleanest and most accurate. Mind sharing how you did it?
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1707%2F23589034184_8d3e07acd6_z.jpg&hash=5f1ffe046468bc342162ffccab014c2a)
I've got some more observations to share about the focus pixel patterns but first I need to get back to work updating the .fpm files for the EOSM / 650D / 700D. Thanks again Steve_FR!
-
i meant resolution measuring picture.
Mind sharing how you did it?
I didn't take pixels from the images. I generated them with some pattern.
Little bit later i will try to rebuild your maps.
-
i meant resolution measuring picture.
They are in that dropbox folder. If you want the chart itself, it is here:
http://www.graphics.cornell.edu/~westin/misc/res-chart.html
Little bit later i will try to rebuild your maps.
If you wait until I'm done rendering new focus pixel map files you'll find that they probably match yours very closely. Though as you can see I didn't plagiarize them.
-
MLVFS updated...
-
@dfort
I missed them. Thx!
@dmilligan
I think it can be better. i need more tests.
I'll let you know if I'll get the better results.
-
@AWPStar
Please let all of know if you can come up with better results, especially for Dual ISO.
@dmilligan
Huge improvement! I need to do some more Dual ISO testing but here are the results I'm getting with yesterday's update, commit 319d6e3.
SL1/100D mv1080
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1461%2F24478867630_05254f2321_z.jpg&hash=01c18edbface51eb72ca969f21ed4801)
SL1/100D mv1080 Dual ISO
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1557%2F24146248044_e13eb9e758_z.jpg&hash=fd95178a6fdb32321428ee2a395c4432)
SL1/100D mv1080 - before yesterday's update
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1606%2F24675783996_4663c0fa17_z.jpg&hash=640cc98deb1b590ec51e4fd376780647)
SL1/100D mv1080 - with yesterdays update - commit 319d6e3
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1564%2F24478868650_c938d55898_z.jpg&hash=550f162318332efedbfbcb2132ac6ca4)
There are still some focus pixels showing up on Dual ISO.
SL1/100D mv1080 Dual ISO - commit 319d6e3
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1535%2F24656585222_f776c2a2f3_z.jpg&hash=811afaefc93a14c5cf8db981d0a9de4e)
I was able to reproduce the issue with the focus pixels showing up across the entire frame. They start showing up in underexposed frames that have the exposure stetting stretched in Adobe Camera Raw. Here's that image Steve_FR made with the focus pixels fixed. Compare it to the same shot posted in Reply #163 (http://magiclantern.fm/forum/index.php?topic=16054.msg161506#msg161506). The image is stretched to the max so there's lots of noise but no dots.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1573%2F24484033550_64720e2391_z.jpg&hash=ba7aba2e378657d0a975e4e0f3d3e83d)
Steve_FR discovered that the best way to get these pixels to show is to shoot a dark frame. Here's a nice shot of the back of Steve_FR's lens cap.
EOSM black frame
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1680%2F24685948831_03ef5db8c1_z.jpg&hash=b2d073bfbd19cb787aa3fda0b3be4b50)
The revised fpm file seems to be working on those focus (?) pixels.
EOSM black frame with focus pixel map file
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1619%2F24685948231_d9924646dd_z.jpg&hash=36814318d266f937ef24fe69de3db765)
I'll post the updates for anyone wanting to run their own tests to my bitbucket download area (https://bitbucket.org/daniel_fort/magic-lantern/downloads). I'll also leave the previous versions up for a while in case anyone wants to do some regression tests. (a.k.a. before and after.)
[EDIT: I was wondering if maybe the SL1/100D has the same issue so I shot a black mlv and ran it with the focus pixel map in place and it looks like I've got more work to do on those focus pixel maps.]
100D mv1080 black frame with focus pixel fix
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1641%2F24779968205_26ae0677bc_z.jpg&hash=83770275779992bf0fcc0aed5bdc1bd0)
-
@dfort
Do you need some tool to work with maps? i can make standalone or integrate it in mvp.
-
Do you need some tool to work with maps?
Actually yes, there are several tools that I could use but I'm not sure how to describe them yet. Up until now all the progress has been on MLV but apparently there are still some situations where the original ML raw video may be preferred. I think that with the possible exception of zoom mode, it should be feasible to support raw with a little bit of user input. For example the user supplies the video mode (mv1080, mv720, mv1080crop, etc.) and by determining the frame size and looking up a list of crop values the focus pixel maps can be used with raw video. The thing is that there are more image size choices in raw than in MLV.
Hopefully the stuff I've been posting is helping developers improve their tools.
Part of the reason I started this topic was because of the problem removing focus pixels from dual iso footage and it is still an issue. I looked up various bad pixel interpolation techniques but the best material came from astronomy and it is way over my head. Way, way over. Kidding aside it seems to me that something that might work is taking the adjacent pixels in the x,y direction and averaging only the two closest matches. That means that it might take the two pixels to the immediate left and right or immediately above and below or if it turns out to be the best match, one +- horizontally and the second +- vertically. That just might work. One of the papers (http://www3.mpifr-bonn.mpg.de/lndrs/manual/ref_defect_pixel_interpolation.html) I looked at included the diagonal pixels but that might not be necessary.
Maybe processing that same area of the chart in various ways will help? All of these were from the same mv1080 Dual ISO MLV.
dual iso with fpm fix ACR
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1677%2F24693054941_f16d595205_z.jpg&hash=9aa82cace6f8454caf5a031ae0c17316)
unprocessed dual iso without fpm fix dcraw -D
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1602%2F24668744052_bbd9dfbba0_z.jpg&hash=c8abb3f0da3e0530e1c9bf84b4e862b6)
unprocessed dual iso with fpm fix dcraw -D
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1687%2F24159696443_fc75cee2e4_z.jpg&hash=656a714a8b69020378c07f7c11869034)
unprocessed dual iso with fpm fix dcraw -T
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1576%2F24159696353_5b0467f455_z.jpg&hash=90abae44352d378668c9fd34abc58243)
That green blip on top of the number 5 would not be there if the two pixels on x axis of the focus pixel were used for interpolation and the magenta in the number 7 caused by a diagonal pair of focus pixels should have been handled by one +x and one -y adjacent pixel on the lower left focus pixel and the horizontal pair on the upper right focus pixel. Is this making sense or have I been staring at focus pixels for too long?
-
As i understand, the problem is that at the contrast transitions, left and right pixels are different and algorithm should choose one of them.
Solution is:
1. Use threshold to not blend pixels.
2. Do something to make a chose.
I think we can compare difference at pixels above or below or both.
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fg81.imgup.net%2Fpx1aa68.png&hash=294a29fe2479b4348636c6a6bdd07ad3)(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fg03.imgup.net%2Fpx2ba43.png&hash=c60ef673841eecf4807facb82d74503e)
Or
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fn74.imgup.net%2Fpx343dc.png&hash=0c01d64fcec99c58b1a6208d7c2f3564)
OurPixel =((A1+A2)*B3)/(B1+B2)
-
A quick google search turned this up: http://harvestimaging.com/pubdocs/072_2003_SPIE_AdaptivePixelCor.pdf
As pointed out in that paper, the main issue is that red and blue pixels' neighbors of the same color are so far away (and if we are in a line skipping mode, they are even farther away).
This method sounds pretty robust, but it uses a rather large kernel (7x7), so it might be rather slow (esp. if the 100D really does have 100k focus pixels). It would also probably take me a while to understand their maths enough to actually implement it.
(NOTE: this method also has a means of correcting column defects, not just individual pixels, which might work as an alternate for the stripes correction, IDK)
A simpler possibility is to model the green channel (with a spline or something else) and then use that to interpolate the red or blue focus pixel. I should think the 8 nearest green pixels and the 4 nearest same color pixels would be enough for this (so we only need to read 12 pixels).
Dual ISO is still going to be a problem though. We might have to measure the exposure difference between line pairs and then offset the neighboring pixels that are at a different ISO by this much to allow them to still be used in the interpolation (problem with this: the pixels of the other ISO could be clipped or mostly noise).
One final possibility is to try and understand what the values of the focus pixels means, and then actually use their value in some way to help inform the interpolation. This could be helpful or not, IDK.
-
I also came across that paper but didn't bring it up because they were using a 7x7 area.
Something that got me thinking is how the Dual ISO issue seemed solved way back on Reply #22 (http://magiclantern.fm/forum/index.php?topic=16054.msg157589#msg157589) so I thought I'd take another look. It was done with Danne's MLP so I created a dcraw badpixels file for the 100D, ran it on the chart image and it looked good. The end product was a ProRes 444 QuickTime file so I tried it again with just dcraw and here is what a TIFF looks like.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1645%2F24703801431_80db5c12ae_z.jpg&hash=d1deb7e533e3c48ddbaeeeb155036159)
A closer look shows that the problem with the focus pixels is still there but much softer so it doesn't jump out. Perhaps this is because the default setting for cr2hdr that MLP is using has chroma smoothing turned on?
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1590%2F24703813871_ce053f9ebe_z.jpg&hash=8da038d40589e42268242a6bcb68c448)
For comparison here is what the DNG file created with MLP looks like before going through dcraw -P dead_pixel_list.txt.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1489%2F24501778230_f14e02a287_z.jpg&hash=8fac7f7638831ef7d0b90d7f0892568f)
BTW--new updates on the bitbucket download area (https://bitbucket.org/daniel_fort/magic-lantern/downloads). New 100D focus pixel map files.
-
new interpolation method, works better, I think, in most cases, esp dual ISO
https://bitbucket.org/dmilligan/mlvfs/commits/1bc6f7f
-
Yay!
SL1/100D Dual ISO with MLVFS commit 1bc6f7f and focus pixel map file
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1454%2F24179779693_4611cae5f6_z.jpg&hash=dad8163156d2c7f2d971b469365c3a4e)
No point posting non-Dual ISO shots because they look just as good as the previous build. Damn near perfect. My understanding is that no Bayer pattern demosaicing algorithm is perfect so toss in a bunch of focus pixels into the mix just to make it that much harder. The SL1/100D shows the most focus pixels and hard vertical lines seem to be the most difficult to tame.
SL1/100D Dual ISO with MLVFS commit 1bc6f7f and focus pixel map file
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1488%2F24511114270_c5c2dbc842_z.jpg&hash=33f47de02ba2af6d2cca4a0024815c4a)
I tried chroma smoothing again and this time cs5x5 seemed to give the best results.
SL1/100D Dual ISO with MLVFS commit 1bc6f7f and focus pixel map file Chroma Smoothing 5x5
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1706%2F24713157351_8defee8474_z.jpg&hash=b1e22df8ef0ffc965839a02b144fd05b)
The focus pixel problem is present in all Dual ISO shots I examined closely. Reviewing this shot I used before.
EOSM mv1080crop Dual ISO
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1540%2F24460418311_088b48f7e9.jpg&hash=af1608e91bd8c285e9ccf2833455e90e) (https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1466%2F24542740305_d4a1d9c47c.jpg&hash=cbf58a788775712038b528748906a45c)
Even with the latest MLVFS build and focus pixel map there are still a few stray focus pixels present.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1570%2F24780388846_37409e7de5_z.jpg&hash=d57768be13bdb40ef77c14a3a4ada5d5)
Now look very closely at the two green dots in the above shot and try to find them below.
Chroma Smooth 2x2
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1677%2F24511113310_f340b5944c_z.jpg&hash=3cc8935a7e119fdd9581441b2b6f3bcb)
Of course this is real pixel peeping. Once rendered to a movie file or posted online there should be no issues.
I'm hoping that this is the last chart I'm going to post in this topic.
Canon SL1/100D Dual ISO, MLVFS commit 1bc6f7f and focus pixel map
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1563%2F24180263413_bb2715f86a_z.jpg&hash=ee3b5b61d6e7563c3bc271033d8652b8)
@dmilligan
I'm not sure how you want to roll this out but of the four cameras we identified with focus pixels in raw video, three of the cameras share the same map files. I was going to suggest that if a camera has an id number within a certain range to use the same map but according to the exiftool code that isn't possible.
0x80000301 => 'EOS Rebel T4i / 650D / Kiss X6i',
0x80000302 => 'EOS 6D', #25
0x80000324 => 'EOS-1D C', # (NC)
0x80000325 => 'EOS 70D',
0x80000326 => 'EOS Rebel T5i / 700D / Kiss X7i',
0x80000327 => 'EOS Rebel T5 / 1200D / Kiss X70',
0x80000331 => 'EOS M',
0x80000355 => 'EOS M2',
0x80000346 => 'EOS Rebel SL1 / 100D / Kiss X7',
0x80000347 => 'EOS Rebel T6s / 760D / 8000D',
Oh yeah--updating the MLVFS test build in my bitbucket download area (https://bitbucket.org/daniel_fort/magic-lantern/downloads). I do encourage everyone with an EOSM, 650D, 700D or 100D that wants to work with MLV give it a try.
@AWPStar
Hope this helps your project too.
@Steve_FR
Thanks for finding those pixels that I overlooked.
-
Can't seem to let go of this topic--here's some more of my thoughts on how to deal with focus pixels.
So far it looks like there are a couple of ways to deal with them. One is the shotgun approach by applying chroma smoothing over the entire image and the other is targeting only the focus pixels themselves. Mapping out all the focus pixels that are on the sensor creates a rather large list of coordinates but the pixels make a simple repeating pattern that can be tiled, therefore saving lots of space and possibly speeding up processing.
The EOSM, 650D and 700D all share the same pattern. When seen in crop mode there is no line skipping.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1512%2F24842544375_7edbd139df_n.jpg&hash=423675bd3cefc7adbc6f7a1944f7f026)
The pattern on the 100D is denser but it is also a simple repeating pattern.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1719%2F24214373784_13871ea600_n.jpg&hash=d0b1927a734c6733d2f9e3661080f9da)
When the cameras shoot in mv1080 mode they all create the same pattern.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1638%2F24816316946_a6dba2a0c5_n.jpg&hash=ba3422bb0610c6ebd3458a733f91f1c6)
Likewise in mv720 mode.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1470%2F24214373764_d22ee0051a_n.jpg&hash=736435525962123a2aa3d441a0fbaae5)
As you can see you only need four basic patterns to cover all the cameras in every video mode. Actually if you are clever enough you could use the two full patterns and line skip them to create the other two patterns.
I've also been thinking about what causes these pixels to turn green or magenta. When processed through dcraw -d it is possible to get a clearer view of the focus pixels. Note how some turned white and others black.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1516%2F24748835451_0553ba7a56_z.jpg&hash=a99b40c6321c323e5debdf3bb9101f15)
Exactly how these focus pixels function isn't something that we explored in this topic but what we do know is that they seem to turn on and off. Sometimes they are invisible and I have yet to see an example that shows all of the pixels in a single shot. What is interesting is that it seems to depend on the color of the background and whether or not the pixel is next to a high contrast area giving the appearance something like focus peaking in a video monitor.
Back on Reply #69 (http://magiclantern.fm/forum/index.php?topic=16054.msg160512#msg160512) a1ex explained that the focus pixels are only on the red or blue areas of the Bayer pattern. If the pixel turns off it is filled in with the surrounding green pixels and if it lights up it has an absence of green thus turning magenta (a.k.a. "pink dots"). I'm only speculating but it seems to make sense.
Whatever the camera is doing in firmware to hide these focus pixels, it is doing a very good job. a1ex explained that there are other ways (http://magiclantern.fm/forum/index.php?topic=16054.msg161356#msg161356) to capture raw video. Perhaps one of these alternate methods can capture the data after it has gone through the focus pixel annihilation process? That would make this topic obsolete which would be a very good thing for EOSM/650D/700D/100D users who want to explore raw video.
I'm not done working on this. While MLVFS is great for Mac users, Windows and Linux users may be feeling left behind until other tools like mlv_dump can deal with focus pixels in the same way.
[EDIT: Well, MLV Producer does a good job and it will probably get even better.]
Right now I'm working on using the focus pixel map files we created for this topic along with the MLV metadata that shows how to crop the map file for each image size and creating files that can be used with dcraw. I'm also planning to include the maps for the first generation raw video which doesn't have the metadata but it wasn't too difficult to figure it out--well we'll see if it worked once the files are ready. I don't expect any developer to pick up on this side project but it does give users another tool to deal with focus pixels and it can be used with Mac, Windows and Linux.
-
While MLVFS is great for Mac users, Windows and Linux users may be feeling left behind
MLVFS works fine on Linux, and as of yesterday Windows (https://bitbucket.org/dmilligan/mlvfs/issues/7/dokany-support) too :)
a1ex explained that there are other ways (http://magiclantern.fm/forum/index.php?topic=16054.msg161356#msg161356) to capture raw video. Perhaps one of these alternate methods can capture the data after it has gone through the focus pixel annihilation process?
I wouldn't really say there are "other ways" to capture raw video, there are just other possible video modes. Capturing them is the same, you just have to figure out how to get Canon firmware to put the camera in those modes. The EOSM is unique b/c Canon firmware doesn't put it into mv1080 until you start recording (H.264). So if you want to record raw video on mv1080, you have to do it while also recording H.264 at the same time (ML prevents you from doing this now a days, but it was possible in some old versions when raw video just came out). That or do some reverse engineering to figure out how to put the camera in mv1080 without starting recording. All other cameras go into the video mode you selected in the Canon menu immediately, so actually mv1080 is pretty much the "default".
-
MLVFS works fine on Linux, and as of yesterday Windows (https://bitbucket.org/dmilligan/mlvfs/issues/7/dokany-support) too :)
Great news! Is the version with the focus pixel map files also working on Linux and Windows?
-
Right now I'm working on using the focus pixel map files we created for this topic along with the MLV metadata that shows how to crop the map file for each image size and creating files that can be used with dcraw.
8) Oh yes
MLVFS works fine on Linux, and as of yesterday Windows too :)
MLVFS fully cross platform now? Remarkable.
-
@dmilligan
[EDIT: Upon further checking it looks like I've got some more refinement on my script but this part of my post is still relevant.]
I've got a bit of a problem with my bash script to create dcraw badpixels files from the focus pixel map files. The crop metadata doesn't seem to be correct in some cases. I saw you had this in your code:
//there was a bug with cropPosX, so we'll round panPosX ouselves
int cropX = (frame_headers->vidf_hdr.panPosX + 7) & ~7;
int cropY = frame_headers->vidf_hdr.panPosY & ~1;
Looks like I need to read up on bitwise operators but isn't this something that should be corrected in the mlv_rec module?
-
Whack on the side of my head--figured it out.
So this change was done in mlv_rec by a1ex (https://bitbucket.org/hudson/magic-lantern/commits/2e5367a0aa815d5945c6ba6bae4ba01f773401f2) on January 13, 2016.
# hdr->cropPosX = (skip_x + 7) & ~7;
#- hdr->cropPosY = (skip_y + 7) & ~7;
#+ hdr->cropPosY = skip_y & ~1;
Of course that means that any MLV file shot before this change has the incorrect Crop information in the metadata. So MLVFS is taking the Pan and calculating the Crop. I was wondering why MLVFS pixel fix was working with my old files. Very clever programming on the part of dmilligan.
Nitpicky note -- the comment in MLVFS says there was a bug with cropPosX, it should read cropPosY. Super nitpicky, "ourselves" is misspelled.
My bash script was using the Crop information which works fine for new MLV files but I was a bit careless last night and checked an old MLV file, of course that didn't work. In any case I learned something about bitwise operators and now my script also works with older MLV files.
I'll make up a batch of focus pixel map files that can be used with dcraw. Here's the script I'll be using. Most of it is comments so I'll shut up now and encourage anyone interested in this stuff to read the source code. Most of it has already been discussed in this topic.
fpm2badpixels.sh
#!/bin/bash
# fpm2badpixels.sh
#
# Create a dcraw ".badpixels" formatted file from
# MLVFS .fpm (focus pixel map file)
# For removing focus pixels from Magic Lantern raw video
# files shot on on Canon 100D, 650D, 700D and EOSM cameras.
#
# Usage fpm2badpixels.sh [frame_size][pan][fpm_file]
# The frame size is the final size of the movie frame.
# The crop and pan information can be read from the mlv
# metadata using "mlv_dump -v -m [mlv_file]"
#
# example--for a 1280x720 mv1080 shot with a T5i / 700D:
# fpm2badpixels.sh 1280x720 296x247 80000326_1808x1190.fpm
#
# The output file in dcraw "badpixels" format is named the same
# as the input parameters. For the above example:
# 1280x720_pan296x247_80000326_1808x1190.fpm.txt
#
# The file can be renamed and comments added as per the
# dcraw documentation.
#
# You need to know the buffer size for the video mode you
# are using to pick the right .fpm file.
#
# Mode Buffer Size Notes
# mv1080 - 1808x1190 - not available on the EOSM.
# mv720 - 1808x727 - needs to be stretched by 1.67x
# vertically - up to 50/60 fps
# except on EOSM.
# mv640 - 1808x1190 - basically the same as mv1080 mode.
# mv1080crop - 1872x1060 - highest resolution and largest
# sustainable frame sizes on all cameras.
# mv640crop - - not possible.
# zoom - 2592x1108 - same 5x magnification as mv1080crop
# but can pan around sensor.
# Performance not as good as mv1080crop.
#
# Sorry but with zoom mode a new badpixels file needs to be made
# every time that the pan position changes.
#
# This is a very simple shell script and can be greatly improved.
# For example with MLV files it should be possible to extract
# the image size and crop information from the metadata and
# the focus pixel map file shouldn't be necessary because it is
# a simple pattern that can be built on the fly. In addition, raw
# video without metadata should be able to be supported with minimal
# user input about the video mode. The crop information should
# be able to be determined for most modes except zoom mode because
# of the multiple buffers and the frame can move around within the
# raw buffer.
#
# This script does not support filenames and paths with spaces
# and there is no error checking.
# Hey, what do you want for nothing.
#
# 2016 Daniel A. Fort
# This is free and unencumbered software released into the public domain.
SIZE=$1
PAN=$2
FPM=$3
output="$SIZE"_pan"$PAN"_"$FPM"
IFS='x'
read -r width height <<< "$SIZE"
read -r panPosX panPosY <<< "$PAN"
unset IFS
# We would rather work with the Crop information from the MLV file
# but there was a bug in the way that the Y axis was calculated.
# This was fixed by a1ex on commit 2e5367a but that means any
# MLV file shot on a nightly build prior to January 12, 2016
# has this bug. So let's round Pan instead of using Crop.
cropX=$((panPosX + 7 & ~7))
cropY=$((panPosY & ~1))
while read -r raw_buffer_x raw_buffer_y; do
((x = $raw_buffer_x - $cropX))
((y = $raw_buffer_y - $cropY))
if (( "$x" >= 0 )) && (( "$y" >= 0 )); then
if (( "$x" <= "$width" )) && (( "$y" <= "$height" )); then
echo -e "$x \t $y \t 0" >> $output
fi
fi
done < "$FPM"
-
Nitpicky note -- the comment in MLVFS says there was a bug with cropPosX, it should read cropPosY. Super nitpicky, "ourselves" is misspelled.
:P I misremembered which one (x or y) had the problem, I just knew what it was supposed to be.
-
[EDIT: I'm pulling the files because they aren't two sizes fits all. Even though the raw buffer pixel maps are the same size, the crops for different cameras are also different. This would mean a much larger inventory of dcraw "badpixels" files would be necessary to cover every camera, every MLV standard image size and every video mode.]
More goodies for experimenters and legacy raw format users. I uploaded a bunch of dcraw "badpixels" formatted files to my bitbucket downloads (https://bitbucket.org/daniel_fort/magic-lantern/downloads). The 2016-02-11_dcraw_badpixels.zip package contains all the standard sizes for the various MLV modes. I divided them into two groups. The A group is for the EOSM/650D/700D and the B group is for the 100D. These are the two focus pixel patterns that I have discovered so far.
How to use these files. Check out the dcraw documentation:
-P deadpixels.txt
Read the dead pixel list from this file instead of ".badpixels". See FILES for a description of the format.
FILES
:./.badpixels, ../.badpixels, ../../.badpixels, ...
List of your camera's dead pixels, so that dcraw can interpolate around them. Each line specifies the
column, row, and UNIX time of death for one pixel. For example:
962 91 1028350000 # died between August 1 and 4, 2002
1285 1067 0 # don't know when this pixel died
These coordinates are before any stretching or rotation, so use dcraw -j -t 0 to locate dead pixels.
So let's say you made your dng files with mlv_dump and want to create TIFF files without focus pixels. Here's an example:
dcraw -T -P A_mv720_1280x432_pan296x159.txt *.dng
These files can also be used with MLP. Simply take the appropriate file, rename it dead_pixel_list.txt and add that to the A_lut_hold. Then you'll have to add this to the beginning of the file:
-P ../../PRORES/A_lut_hold/dead_pixel_list.txt
-P ../A_lut_hold/dead_pixel_list.txt
-P A_lut_hold/dead_pixel_list.txt
As you can see the file doesn't have to be named dead_pixel_list.txt as long as these lines point to the right name. Danne has been adding features to MLP and focus pixel fixing is sure to get easier in the coming releases.
I used the fpm2badpixels.sh script from Reply #181 (http://magiclantern.fm/forum/index.php?topic=16054.msg162066#msg162066) and to make things easier I also created this simple script to create a batch of files:
#!/bin/bash
# This is a very simple hard coded script to generate
# dcraw "badpixels" formatted files from the .fpm files
# and a bunch of MLV files using the fpm2badpixels.sh
# script posted at:
# http://www.magiclantern.fm/forum/index.php?topic=16054.msg162066#msg162066
#
# Sorry but with zoom mode a new badpixels file needs to be made
# every time that the pan position changes.
# This script does not support filenames and paths with spaces
# and there is no error checking.
# Hey, what do you want for nothing.
#
# 2016 Daniel A. Fort
# This is free and unencumbered software released into the public domain.
# Change the path to match where your MLV files are located.
for MLV in /Users/rosiefort/Desktop/Humongous_test/zoom/*.MLV
do
# The script tells you what it is working on so you'll know
# that it is doing its job.
echo $MLV
# Using mlv_dump we look up the resolution and pan information.
size=`mlv_dump -m -v $MLV | grep Res | head -1 | sed -e 's/^[ Res:]*//'`
echo $size
pan=`mlv_dump -m -v $MLV | grep Pan | head -1 | sed -e 's/^[ Pan:]*//'`
echo $pan
# Run the fpm2badpixels.sh from the current directory and use
# the corresponding .fpm file also in the current directory.
#
# To figure out which .fpm to use for your camera:
#
# 80000301 => 'EOS Rebel T4i / 650D / Kiss X6i',
# 80000326 => 'EOS Rebel T5i / 700D / Kiss X7i',
# 80000331 => 'EOS M',
# 80000346 => 'EOS Rebel SL1 / 100D / Kiss X7',
#
# And for the video mode:
#
# Mode Buffer Size Notes
# mv1080 - 1808x1190 - not available on the EOSM.
# mv720 - 1808x727 - needs to be stretched by 1.67x
# vertically - up to 50/60 fps
# except on EOSM.
# mv640 - 1808x1190 - basically the same as mv1080 mode.
# mv1080crop - 1872x1060 - highest resolution and largest
# sustainable frame sizes on all cameras.
# mv640crop - - not possible.
# zoom - 2592x1108 - same 5x magnification as mv1080crop
# but can pan around sensor.
# Performance not as good as mv1080crop.
#
# I'm testing with both a 700D and 100D so I'll run two at a time.
#
./fpm2badpixels.sh $size $pan 80000326_2592x1108.fpm
echo ./fpm2badpixels.sh $size $pan 80000326_2592x1108.fpm
./fpm2badpixels.sh $size $pan 80000346_2592x1108.fpm
echo ./fpm2badpixels.sh $size $pan 80000346_2592x1108.fpm
done
If there's any resolution I missed you can make your own with these scripts. Note that the dcraw files will also work with legacy RAW video format but you will note that there are more resolution choices for raw than for MLV. To get these, and more, go to the RAW video (MLV) submenu, select Resolution and click the left and right thumbwheel. You can cycle through all the possible resolutions--there's a boatload!
A few notes about zoom mode. The files I created assumed the zoom was centered. Panning around the sensor requires something more sophisticated like MLVFS. Other programs may also support panning but only with MLV files because the original RAW format doesn't have the necessary Pan/Crop metadata. Another thing I didn't know about until I was creating these files was that with zoom mode the setting allows up to 2496x1074. However, these cameras can only record a few frames of video at that setting.
-
238,4 MB of mapped pixels :o.
Thank you for this. Just downloaded and will see what can be done in MLP and looking forward on further developments. Will try some of these maps with a friend,s 700D.
I could probably change the MLP script to grab the correct pixel map by matching the size through exiftool. Let,s see when I get around to it.
-
@Danne
You will need to use the 700D build I posted in my bitbucket download area (https://bitbucket.org/daniel_fort/magic-lantern/downloads) because the current nightly doesn't identify that camera properly in the MLV metadata. This commit (https://bitbucket.org/hudson/magic-lantern/commits/3a9c6deedafd66d093b60abc4cc31492f1d87d17) needs to get merged into unified for the nightlies to get updated.
Of course if you're going to do everything the hard way like I described it in my previous post you can get by without it.
-
Gosh this is all incredible stuff and even at this pace it'll most likely get fully implemented w ease by the end of Spring if not Summer.
Great work by Dan and every one else onboard whom have wholeheartedly contributed into this!
-
Well let's not pop the cork just yet. I was doing some tests and a bunch of the files I posted aren't working. I've been going over my scripts and can't find the problem. Better take a break and come back to it.
[EDIT: Found the problem. False assumption on my part that since the pixel maps for the raw buffers were the same the crops would be the same too. Wrong. Of course this means even more dcraw "badpixels" files should be created and I think things need to be scaled back a bit. The scripts are working fine though so experimenters can still roll their own.]
On an upbeat note I asked my brother to help me figure out how to calculate the pan positions just using the image size for each video mode and he was able to do it.
For mv1080 the minimum size is:
x=640 panPosX=616
y=128 panPosY=543
So the formula is:
panPosX=616-((x-640)/32)*16
panPosY=543-((y-128)/2)
He also came up with some error checking in case an invalid image size is entered. This is when having a brother that studied science is cool. I studied art. 8)
-
By the way--several people downloaded the package of dcraw "badpixels" files. So--all were made using the pan information from a 700D with the exception of the zoom mode which was done with an EOSM. Why? because the 700D was the easiest working with the Magic Lantern menu and I have a button to center the crop area for the EOSM.
Anyway, I'm taking down that big batch and will probably post something a little less ambitious later.
-
Here is a MLP testbuild which works with your fpm files through you skillfull scripting. Works with MLV files. It is integrated and automated so end user doesn,t have to worry about anything. Very fine work dfort. The test files looked super clean.
MLP_pixel_fix
https://drive.google.com/file/d/0B4tCJMlOYfirM2lhSHpXQUxOTU0/view?usp=sharing
-
Great! The goal is to get to the point where focus pixels are no longer an issue. In the meantime, here's something that is useful for more than just experimenters. This script will create a dcraw formatted "deadpixels" list from an MLV file. The raw buffer focus pixel map files need to be in the same working directory as the script but the MLV file can be anywhere in the filesystem. The simplest way to use it is to open a terminal in your working directory, type ./mlv2badpixels.sh and the path and file name of the MLV file. On many terminals you can simply drag and drop the MLV icon right into the terminal window. There are some options. -m adds the MLP headers and -o allows you to enter a custom file name. This still a fairly simple script so don't use spaces in your paths or file names. [EDIT: When I first wrote this I got MLP and MLV mixed up--corrected.]
This makes it so easy to create the files that I may just post all the standard MLV sizes--once I get around to it. Of course it can be better. I'm working on supporting panning in zoom mode. Right now it only works if the focus box is centered.
Oh, and I also found the way to center the zoom area. Simply press the SET button. This should work on all but the EOSM. The Trash button is supposed to do it too but of course it brings up the ML menu. On the EOSM a quick press of the Trash button centers the focus box and a longer press opens the ML menu--provided that's the way the Canon custom menus are configured.
Anyway, here's the script. Have fun!
#!/bin/bash
# mlv2badpixels.sh
#
# http://www.magiclantern.fm/forum/index.php?topic=16054
#
# Create a dcraw ".badpixels" formatted file from
# an MLV file (Magic Lantern raw video)
# For removing focus pixels from Magic Lantern raw video
# files shot on on Canon 100D, 650D, 700D and EOSM cameras.
#
# The focus pixel map files (.fpm) for the raw video buffer
# need to be in the same working path as this script.
#
# Mode Buffer Size Notes
# mv1080 - 1808x1190 - not available on the EOSM.
# mv720 - 1808x727 - needs to be stretched by 1.67x
# vertically - up to 50/60 fps
# except on EOSM.
# mv640 - 1808x1190 - basically the same as mv1080 mode.
# mv1080crop - 1872x1060 - highest resolution and largest
# sustainable frame sizes on all cameras.
# mv640crop - - not possible.
# zoom - 2592x1108 - same 5x magnification as mv1080crop
# but can pan around sensor.
# Performance not as good as mv1080crop.
#
# Sorry but with zoom mode a new badpixels file needs to be made
# every time that the pan position changes.
#
# This script does not support filenames and paths with spaces
# and there is very little error checking.
# Hey, what do you want for nothing.
#
# 2016 Daniel A. Fort
# This is free and unencumbered software released into the public domain.
usage()
{
cat << EOF
Usage: mlv2badpixels.sh [-hmv][-o output_file] <input_file>
Parameters:
-h Show this message
-m MLP format
-o filename Customize Filename
-v verbose
example:
./mlv2badpixels.sh -mv -o M21-1747_1_2015-12-06_0001_C0000.txt M21-1747.MLV
creates an MLP formatted file named M21-1747_1_2015-12-06_0001_C0000.txt
that can be used to remove the focus pixels on the dng files extracted from
M21-1747.MLV
This script will create a dcraw "badpixels" file
to remove focus pixels from an MLV file shot with
cameras that show their focus pixels in raw video.
EOS Rebel T4i / 650D / Kiss X6i
EOS Rebel T5i / 700D / Kiss X7i
EOS Rebel SL1 / 100D / Kiss X7
EOS M
EOF
}
MLP=
VERBOSE=
output=
while getopts “hmo:v” OPTION
do
case $OPTION in
h) usage
exit 0
;;
m) MLP=1
;;
o) output=$OPTARG
;;
v) VERBOSE=1
;;
:) echo "Error: -$OPTARG requires an argument"
usage
exit 1
;;
?) usage
exit 1
;;
esac
done
shift $((OPTIND - 1))
MLV=$1
if [[ -z $MLV ]]
then
usage
exit 1
fi
if [ $VERBOSE ]; then echo input file = $MLV; fi
camera_model=`mlv_dump -m -v $MLV | grep Camera"[[:space:]]"Model | head -1 | sed -e 's/^[ Camera \s Model: \s 0x]*//'`
resolution=`mlv_dump -m -v $MLV | grep Res | head -1 | sed -e 's/^[ Res:]*//'`
pan=`mlv_dump -m -v $MLV | grep Pan | head -1 | sed -e 's/^[ Pan:]*//'`
IFS='x'
read -r width height <<< "$resolution"
read -r panPosX panPosY <<< "$pan"
unset IFS
cropX=$((panPosX + 7 & ~7))
cropY=$((panPosY & ~1))
if [ $VERBOSE ]; then echo Resolution = $resolution; echo Pan = $pan; echo Crop = $cropX"x"$cropY; fi
raw_width=`mlv_dump -m -v $MLV | grep width | head -1 | sed -e 's/^[ width]*//'`
raw_height=`mlv_dump -m -v $MLV | grep height | head -1 | sed -e 's/^[ height]*//'`
raw_buffer=$raw_width"x"$raw_height
fpm=$camera_model"_"$raw_buffer".fpm"
if [ $VERBOSE ]; then echo Camera Model = $camera_model; fi
case $camera_model in
80000301)
camera=650D
;;
80000326)
camera=700D
;;
80000331)
camera=EOSM
;;
80000346)
camera=100D
;;
*) exit 2
;;
esac
if [ $VERBOSE ]; then echo Camera = $camera; fi
case $raw_buffer in
1808x1190)
video_mode=mv1080
;;
1808x727)
video_mode=mv720
;;
1872x1060)
video_mode=mv1080crop
;;
2592x1108)
video_mode=zoom
;;
*) exit 3
;;
esac
if [ $VERBOSE ]; then echo video mode = $video_mode; fi
if [ -z $output ]; then
output=$camera"_"$video_mode"_"$resolution".txt"
fi
if [ $VERBOSE ]; then echo output file = $output; fi
if test -f $output; then rm $output; fi
if [ $MLP ]; then
if [ $VERBOSE ]; then echo MLP formatted; fi
echo "-P ../../PRORES/A_lut_hold/$output" >> $output
echo "-P ../A_lut_hold/$output" >> $output
echo "-P A_lut_hold/$output" >> $output
echo "" >> $output
fi
while read -r raw_buffer_x raw_buffer_y; do
((x = $raw_buffer_x - $cropX))
((y = $raw_buffer_y - $cropY))
if (( "$x" >= 0 )) && (( "$y" >= 0 )); then
if (( "$x" <= "$width" )) && (( "$y" <= "$height" )); then
echo -e "$x \t $y \t 0" >> $output
fi
fi
done < "$fpm"
-
Nice progress. Did anything with the functioning of dots change from the first script? Does it work for all sizes in MLV that are not filmed in 3xzoom mode?
I like the MLP header -m option. However. For MLP to respect a pixel list file it has to be named dead_pixel_list.txt and be put in the A_lut_hold folder and first then it will cat the correct header path from the file.
Then again the experimental MLP version above automated a workflow around your script so that it converts and creates the pixel map for each specific MLV file from the affected cameras and puts it inside the folders of dng files without having to worry about it. This means there,s no need for the MLP header paths when using your script. I should erase them from your script in the MLP above to avoid confusion.
MLP will use the specific list created for each folder of dng files so you can also create prores files straight from the dng folder once it is processed from MLV to dng files. No need to reprocess the MLV file. The created file will be named dead_pixel_list.txt.
MLP calls you package of fmp files and the included mlv2badpixels.sh
loosely explained from MLP
In the mlv_raw.txt script
for FILE in `ls -tr -A1 *.MLV *.mlv 2>/dev/null`; do
pix=$(cat /tmp/magic_l_prores/list_dng) #cats the folder path
#Creating a pixel fix map
~/Library/Services/MLP.workflow/Contents/bmcc_ixml_luts/Pixel_Map_Files/mlv2badpixels.sh $pix/$FILE
doneThen moving to correct folder and renaming the file to dead_pixel_list.txt
mv ~/Library/Services/MLP.workflow/Contents/bmcc_ixml_luts/Pixel_Map_Files/*.txt ../${BASE}_1_"$date_01"/dead_pixel_list.txtthen a simple if, then, fi before transcoding
if ls "dead_pixel_list.txt"
then
dpf_a=$(echo -P "dead_pixel_list.txt")
fi
Processings scripts from MLP are
fold_prores.txt
fold_prores_02.txt
fold_prores_01.txt
process_01.txt
By the way. I added this to the path in your script so now you can simply drag the script to terminal instead of doing cd and ./ into the folder.
#!/bin/bash
workingDir=`dirname "$0"`
cd "${workingDir}"
Thanks for all your hard work on this.
-
Finally I'm on my way home from being on the road for this week ... Can't friggin wait to get my hands on a Mac and run this automagically script of yours! Thanks Danne for polishing up his fine ass script!
-
Nice progress. Did anything with the functioning of dots change from the first script? Does it work for all sizes in MLV that are not filmed in 3xzoom mode?
In the first script it didn't check if the file already exists so it would append the new list to the old list. Not very efficient. There were some other tweaks like taking all the information from the MLV file and naming the file by camera name, video mode and resolution. It works for zoom mode too but it doesn't support panning during the shot. That seems something that can be added.
By the way. I added this to the path in your script so now you can simply drag the script to terminal instead of doing cd and ./ into the folder.
The script needs to reference the raw buffer focus pixel map files. I'm going to have to experiment with your change to see how that works.
Of course it can be improved like supporting dynamic panning in a shot and adding an option to manually enter camera and video mode so that it can be used on RAW files or for MLV files that have corrupted metadata. I was thinking of putting up this script in bitbucket, sort of like a beginner's coding project.
-
How about simply grabbing the frame size from the RAW dng file and build a map file from that piece of information?
-
In addition the script needs information about which camera was used. Running exiftool on a dng created from a RAW file gives rather generic information.
Make : Canon
Camera Model Name : Canikon
-
Aha, but you can find out camera brand from the color matrix(courtesy Chmee)
Eos M starts with ColorMatrix1>0.6602 and I think 700D and 650D as well
-
Eos M starts with ColorMatrix1>0.6602
If the color matrix can identify the camera model (we already know it is a Canon camera) plus the video mode, it might be possible to code something that will automatically work with RAW files. Are you sure that you can determine the camera model from the color matrix? No two camera models have the same matrix?
I did a little more investigating and here's what I found on DNG's that were converted processed from a RAW original with raw2dng:
Unique Camera Model : Canon EOS M
Color Matrix 1 : 0.6985 -0.1611 -0.0397 -0.3596 1.0749 0.3295 -0.0349 0.1136 0.6512
Color Matrix 2 : 0.6602 -0.0841 -0.0939 -0.4472 1.2458 0.2247 -0.0975 0.2039 0.6148
So the EOSM starts with Color Matrix 1 0.6985 but hey--it looks like the camera is identified in RAW files. The image size is easy enough to get but what about the video mode?
More goodies for experimenters--I'm posting dcraw "badpixels" files for all of the MLV video modes and standard resolutions in my bitbucket download area (https://bitbucket.org/daniel_fort/magic-lantern/downloads).
(I saw someone jumped the gun and downloaded some EOSM files that were not quite complete. They should all be complete now.)
-
Well there is only one color matrix when using raw2dng. Then there is exiftool putting in stuff in post :). Actually I think dcraw uses the color matrix 2 as color matrix 1(numbers used in raw2dng). Think Chmee or Andy600 can explain this.
So to identify Color matrix 1 run raw2dng from command line and check the dng with exiftool or use this little script snippet and put it in a command file and run it next to a RAW file. The script uses the export path from MLP but I think you got that installed already.
#!/bin/bash
workingDir=`dirname "$0"`
cd "${workingDir}"
export PATH=~/Library/Services/MLP.workflow/Contents:$PATH
for FILE in `ls -A1 *.RAW *.raw 2>/dev/null`; do
#This little trick in the script makes raw2dng only print out only one dng from the file then exits
raw2dng "$FILE" | awk '{ print $1; exit }'
exiftool 000000.dng | awk '/Color Matrix 1/ { print $5; exit }' > color
mx=$(cat color)
#rename the dng file to the first color matrice number(cam identifier)
mv 000000.dng "$mx".dng
rm color
done
#will close and exit terminal
osascript -e 'tell application "Terminal" to close first window' & exit
exit 0
-
I created a bitbucket repository for prototyping, experimenting and general dealing with focus pixels in raw video.
https://bitbucket.org/daniel_fort/ml-focus-pixels
That way we can play with ideas for supporting legacy RAW video files, building the raw buffer focus pixel map files on the fly and whatever you want to try out and then share your results on this topic.
Thanks to Walter Schulz for the help with the 650D. I updated the dcraw files for that camera based on his feedback.
-
There should be a way to eliminate the focus pixel map files that are currently being used in MLVFS and MLP. Although the EOSM/650D/700D have different raw image size capabilities they share the same raw frame buffer sizes and pixel patterns. The 100D has a different pattern but upon closer examination it turns out that there are many similarities especially in the mv1080 and mv720 modes where line skipping modifies the focus pixels pattern to a point where they become essentially the same pattern.
Here's a screenshot of the upper left corner of the mv720 pattern for the EOSM/650D/700D. Let's call this group pattern A because these cameras all share the same pattern and we'll call the 100D pattern B because it is a newer camera than the ones in the first group.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1542%2F24731944109_e6d2563dcc_z.jpg&hash=a2079a875926b83c6e206712d7e03bab)
If you look closely you'll see that the pattern starts at position 72 horizontally, x axis and 290 through 465 in the vertical, y axis. The left 71 pixels is actually in the out of bounds area of the raw frame buffer so we can say that the pattern extends through the entire horizontal area of the picture area in all video modes. Vertically the difference between the patterns is simply that the B pattern occupies a larger area starting at 86 in the y axis and extending all the way to 669.
I wanted to see if it would be possible to create the mv720 raw buffer focus pixel map files with a simple bash script and this is what I came up with.
# focus pixel map files (.fpm)
# Proof of concept bash script
width=1808
height=727
output="output.txt"
pattern=A # pattern A for EOSM/650D/700D pattern B for 100D
if [ $pattern == "A" ]; then fp_start=290; fp_end=465
elif [ $pattern == "B" ]; then fp_start=86; fp_end=669
else exit 1
fi
for y in $(seq $((fp_start)) $((fp_end)) ); do
if (( (($y + 10)) % 12 == 0 )); then shift=2
elif (( (($y + 9)) % 12 == 0 )); then shift=3
elif (( (($y + 4)) % 12 == 0 )); then shift=6
elif (( (($y + 3)) % 12 == 0 )); then shift=7
else continue
fi
for x in $(seq 72 $((width)) ); do # 0-71 is out of bounds
if (( (($x + $shift)) % 8 == 0 )); then
echo -e "$x \t $y" >> "$output"
fi
done
done
This is just a first attempt of only one video mode and it could definitely be made better but the script does output a file that matches the current focus pixel map files. Using code instead of text files to map the focus pixels on the raw frame buffers would eliminate keeping an inventory of 15 text files that take up 6.2 MB of disk space.
A further refinement would be to bypass creating a map file of the full raw buffer and map only the cropped frame.
-
6.2mb isn,t much but if you can get it done without those files it is even better. Very interesting following this.
-
Wow! Would this not only save disk space but perhaps speed up the Automator workflow within MLP n MLVFS?
-
Hello, so... I've read through this discussion and understood non of its technical (kudos to everyone who builds this firmware). I've installed ML on my 700D, (yesterday) seen the pink dots and want to know how to remove them! what process do I need to follow to make these pesky dots go away? Thanks.
Russ
-
Try MLP
-
I prefer having separate files even if they are duplicates. It's the simplest and most flexible solution, that is also capable of handling any current (or future) scenario, without making any assumptions about similarities between the cameras and how they operate (we engineers have a saying: KISS (https://en.wikipedia.org/wiki/KISS_principle)). It's also a tiny pittance of memory that is required. Because they are so redundant and also plain text, they compress really well too, so the current download size of MLVFS is only 1.5MB even with all those files included, amongst other things.
If you really want to save space we could keep the files in binary format. Since you need only 2 16 bit integers (4 bytes) to store each coordinate, the pixel file of EOSM et al would be: ~24KB, and the 100D would be ~400KB. But this would make it more of a challenge to edit the files and easily use them for other things.
Even a small amount of code is typically more difficult to maintain than a large data file in the long run => I make a living b/c this statement is true ;)
ideas for supporting legacy RAW video files
This (http://www.magiclantern.fm/forum/index.php?topic=16650) should eliminate the need to support the legacy RAW format.
-
Hello, so... I've read through this discussion and understood non of its technical (kudos to everyone who builds this firmware). I've installed ML on my 700D, (yesterday) seen the pink dots and want to know how to remove them! what process do I need to follow to make these pesky dots go away? Thanks.
Russ
Hi Russ--
This is a "General Development Discussion" topic so it tends to get a bit more into the techie details than than most users really need or want to know. I also started out with a camera that has the "pink dots" issue and my first foray in Magic Lantern code was to compile raw2dng with chroma smoothing turned on so I could deal with those pesky dots.
To answer your question, MLVFS (http://magiclantern.fm/forum/index.php?topic=13152) and MLP (http://magiclantern.fm/forum/index.php?topic=13512) have both integrated the focus pixel map files that were developed in the course of this forum discussion. MLV Producer (http://magiclantern.fm/forum/index.php?topic=15271.0) for Windows has had pink dot removal before I even started this topic. PinkDotRemover tool 650D (http://magiclantern.fm/forum/index.php?topic=6658.0) is the Grand Daddy of pink dot removal tools but it is no longer maintained.
Even a small amount of code is typically more difficult to maintain than a large data file in the long run => I make a living b/c this statement is true ;)
Ha ha--how true. I guess that I should re-read "The Unix Philosophy (https://en.wikipedia.org/wiki/Unix_philosophy)" especially the part about storing data in flat text files. I'm not a programer but have been a weekend code hacker for a while and now that I'm retired I spent way too much time bashing out scripts.
I did find a maintenance issue with some of the raw buffer focus pixel map files. In the course of playing around with my scripts I discovered that some of the coordinates define areas that are beyond the edge of the frame. The reason was because one of my scripts would map a pixel at the width in resolution instead of width-1 because the coordinates should start at 0 instead of 1. Apparently it isn't causing any problems but I will fix them on my side.
By the way, you do know that none of this focus pixel fixing is going to work for 650D and 700D users until your mlv_rec fixes get into the nightly builds. I've been making comments on bitbucket (https://bitbucket.org/hudson/magic-lantern/commits/3a9c6deedafd66d093b60abc4cc31492f1d87d17) for a while. The 100D builds nikfreak has posted for testers is also missing that commit. I had to build it myself to test out that camera. So that leaves only the EOSM nightly builds as the only platform that currently works with MLVFS and MLP focus pixel annihilation.
This (http://www.magiclantern.fm/forum/index.php?topic=16650) should eliminate the need to support the legacy RAW format.
I'll get to testing mlv_lite and report on that topic but first back on this topic--more focus pixels!
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1555%2F24484251174_ffa68e762a_o.gif&hash=075eef85f6e3d97dbb0cabe0610a4427)
I was wondering how the two different focus pixel patterns ended up being the same pattern with line skipping and here's the answer. They are basically the same only the sensor on the 100D has a lot more of them. Not sure if the extra focus pixels become inactive when line skipping is active. In any case, it does make for some interesting observations that adds little value to users who just want to get rid of those pesky dots!
-
Ok--as promised I fixed the focus pixel map files that had problems. These should be replaced in the applications that are using these files:
80000346_1872x1060.fpm
80000346_2592x1108.fpm
These are the mv1080crop and zoom video modes for the Canon SL1/100D. Somehow I slipped on a some lines and a few pixel coodinates were missing. Didn't discover it until I ran diff between the files I had posted and the files generated by my experimental script. So I guess that little scripting exercise wasn't a waste of time after all. Just for fun I made a branch of the mlv2badpixels.sh script that doesn't require the focus pixel map files.
The bitbucket repository is located here: https://bitbucket.org/daniel_fort/ml-focus-pixels
For convenience the fpm files are located in the download area (https://bitbucket.org/daniel_fort/ml-focus-pixels/downloads) as a zip package.
I also removed all the focus pixel related files from my Magic Lantern fork repository. Those files have already served their purpose.
-
Nice! Updated the fpm files in MLP.
*Updated MLP to your latest script. Small modification to exclude an error. Sent you an issue report at bitbucket. Very nice going from 6.2MB to 6kb. Nice scripting.
-
Damn that's some serious digging if you were able to accomplish to go from 6.2MB to 6kb ... Way to go, Dan!
[emoji106]
-
Actually there was some sort of issue with bitbucket. The commits I made yesterday didn't take and so I had to push them out again this morning. Good thing I still had an extra copy of what I worked on. So actually the script went from 5.4 KB to 7.9 KB. The two focus pixel map files that were fixed yesterday also needed to get updated again this morning. These are the crop mode files for the SL1/100D and are 1 and 1.5 MB files. That's a lot of focus pixels--151,200 on the zoom mode map. That's too much for bitbucket to display or show a diff.
-
if someone can help me? i have a few shortmovies and i cant get rid of of those dots, but i think its my knowledges that is the problem. https://drive.google.com/file/d/0B4KcdNcQPavBaTNkMnBmTmFKbWs/view?usp=sharing
-
@Macke1
Can you specify what camera and what movie mode you filmed in? Crop mode, regular mode? 720 mode?
-
Seems you filmed with a 700D and in regular mv1080 mode. When I select (02) ,click the picture below, from the menu all three clips gets the correct pixel maps from dforts script. See here as well.
http://www.magiclantern.fm/forum/index.php?topic=13512.msg162922#msg162922
What is your problem?
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fs9.postimg.org%2Ftqa3fqlqz%2FScreen_Shot_2016_02_25_at_21_44_25.jpg&hash=9ef727b0bec935aa59cd9d5924d8b549) (http://postimg.org/image/tqa3fqlqz/)
-
@Macke1
I'm seeing the same non-issue as Danne.
Before focus pixel fixing
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1465%2F24895030179_9bbdaeb0c4_z.jpg&hash=6348b910a2dac49c1daa01785e1d0135)
After focus pixel fixing
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1573%2F24632078814_cde9f917b4_z.jpg&hash=dc4d0344c48d75aefee2245cf01b0d65)
I'll admit that I did have some problems at first but after trying a couple of times it worked. Maybe was is because the shots were very short or maybe it was because I was trying to do one at a time? In any case this is a new feature in MLP.
Speaking of new features, guess the cat is out of the bag (http://magiclantern.fm/forum/index.php?topic=13512.msg162922#msg162922). I have been experimenting with that formula my brother wrote to take an image size and figure out the pan information so my mlv2badpixels script could work with v1.0 raw video files--I can see dmilligan rolling his eyes.
Allowing for the user to enter information can also be useful if the original MLV file has incorrect metadata, like the camera missing issue that was recently resolved or if the MLV file is missing but the DNG files for that shot are available. Danne was able to pull out some of the information from raw files so the user just needs to provide the camera and video mode.
Of course like most things in Magic Lantern this is a work in progress. The script got a little ugly with the added features but it does seem to be working. You've got to be organized and make sure you don't mix up cameras and video modes. I've also been very careful about making sure that the 5x zoom box is perfectly centered when using zoom mode. However, I've been doing some tests and so far it seems to be working even when the zoom box is moved around. That makes no sense and requires more testing but if it is true it certainly is good news.
-
Digging your amazing work on these humpy dots with some quick test files shot from SL1 ... Really nice actually and Thanks @Danne for implementing his work into MLP (http://www.magiclantern.fm/forum/index.php?topic=13512.0MLP)! 8)
Pre: Post:
Dual-ISO's coming up next... :-X
-
@dfort ok, now i tried using 02 and it worked like a charm, will try more later this day!
-
@dfort,
Is it just me, or does the "after" picture look a bit blurry compared with the "before?"
-
They are different frames and the camera was shaky so it wasn't exactly a before and after test. I was just showing that the focus pixels are being removed.
Here's a better example, RAW video shot with SL-1/100D in mv1080 mode. This camera shows lots of focus pixels and all those colored focus pixels makes it look like there's a color shift.
As it comes out of the camera
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1497%2F24648177404_20bb0953bf_z.jpg&hash=7e0b61ca274132bd2791c6d48fbbfa86)
After focus pixel removal
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1588%2F24983205590_a2aeda2183_z.jpg&hash=f70acf134e05b8309f349a9fe27129a3)
Here's a blow up A+B comparison for dramatic effect. It seems like resolution isn't being affected. Note that there are still a few bright pixels in sharp high contrast areas. I'm not sure if those were caused by the "badpixels" interpolation in dcraw or the debayering algorithm. Most likely it is aliasing artifacts. Still, quite an improvement.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1521%2F25252552356_8ddcc558e2_o.gif&hash=57e28516edd6518d0ad3eafee0a18a97)
P.S. if we get this working and create all the maps, we can also add it to mlv_dump.
Is it working? :D
-
This is amazing.
-
Add this script into mlv_dump and it'll be golden!!!
-
Actually it is the focus maps and the pixel interpolating code that is needed for mlv_dump. The script I wrote is just for quick prototyping, though it is playing nicely with dcraw via MLP.
The script uses mlv_dump to extract the metadata from MLV files and exiftool for DNG files. There's nothing that I know of for RAW though as a test I did hack raw2dng (https://bitbucket.org/daniel_fort/magic-lantern/commits/cdceb3f58133003377a20a96ec4a23fb9794cda9) so it displays just the file information without creating the DNG files. I'd like to be able to figure out how to read the MLV header and RAW footer with a simple bash (or lua, python, perl) script so it won't require these dependencies--and to learn how to work with binary files too.
There's a lot more that can be done with focus pixels in post production. For example the Adobe DNG specifications (http://www.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec_1.4.0.0.pdf) has a FixBadPixelsConstant and a FixBadPixelsList (Opcode ID 4 and 5) in the documentation but I have not seen any examples of how this can be used.
-
Since this is a General Development topic maybe someone can help me figure out how to read MLV headers and RAW footers in order to find the information needed to deal with focus pixels?
Even though the raw format may be on its last legs I thought I'd start there because there's only one thing to find, image size. The footer of a raw file is fairly simple. Here's one that's easy to find because it has some blank space before the footer.
Running xxd on the raw file results in this:
49dffd0: ffff ffff ffff ffff ffff ffff ffff ffff ................
49dffe0: ffff ffff ffff ffff ffff ffff ffff ffff ................
49dfff0: ffff ffff ffff ffff ffff ffff ffff ffff ................
49e0000: 5241 574d 0005 d002 00a0 1800 3000 0000 RAWM........0...
49e0010: 0100 0000 a45d 0000 0300 0000 e84a 4fff .....].......JO.
49e0020: 0100 0000 0081 7906 a604 0000 1007 0000 ......y.........
49e0030: 5c0c 0000 a873 3900 0e00 0000 ff07 0000 \....s9.........
49e0040: 983a 0000 0000 0000 0000 0000 c806 0000 .:..............
49e0050: 8604 0000 1c00 0000 4800 0000 a204 0000 ........H.......
49e0060: 1007 0000 0000 0000 0000 0000 0001 0102 ................
49e0070: 0100 0000 ca19 0000 1027 0000 b7fc ffff .........'......
49e0080: 1027 0000 55fc ffff 1027 0000 88ee ffff .'..U....'......
49e0090: 1027 0000 aa30 0000 1027 0000 c708 0000 .'...0...'......
49e00a0: 1027 0000 31fc ffff 1027 0000 f707 0000 .'..1....'......
49e00b0: 1027 0000 0418 0000 1027 0000 4b04 0000 .'.......'..K...
The stuff on the right is probably meaningless in this situation. The best explanation I found how to read this is in a post about how to replace a file footer. It starts with a snippet from Magic Lantern / modules / lv_rec / lv_rec.h
#ifndef _lv_rec_h_
#define _lv_rec_h_
#include "raw.h"
/* file footer data */
typedef struct
{
unsigned char magic[4];
unsigned short xRes;
unsigned short yRes;
unsigned int frameSize;
unsigned int frameCount;
unsigned int frameSkip;
unsigned int sourceFpsx1000;
unsigned int reserved3;
unsigned int reserved4;
struct raw_info raw_info;
} lv_rec_file_footer_t;
#endif
where each char is 1 byte (2 hex characters), short is 2 bytes, and int is 4 bytes, that way you can crossreference the struct definition with the number's you are editing in hex.
the raw hex stores these numbers in "little endian" which means that if you want to obtain the real value of an integer from raw hex you would have to write it in reverse( raw hex: "52 41 57 4D 80 07 38 04 00 5F 37 00 F4 21 00 00 ..." is 00 00 21 F4 or just 0x21F4)
I used windows calculator in programmer mode to convert from hex to dec quickly (f4 f5)
I used footers from raw videos that were correct to calculate how many frames i had using the framesize field which is the size of a frame in bytes. I took the file size and divided by the frame size to get a rough rounded down frame count that i would translate into hex, into little endian and write into the raw hex in that location in the footer.
I can't quite follow that. Does he mean to start at the end of the file and go through it backwards and suddenly it will become clear? Is this like Leonardo DaVinci's writing that you have to hold up a mirror to read his notes?
Looking at the raw2dng source code is this how it it is done?
case 0x100: /* ImageWidth */
printf("width: %d\n", data);
raw_info.width = data;
break;
case 0x101: /* ImageLength */
printf("height: %d\n", data);
raw_info.height = data;
break;
I'm a bit lost. Can someone help me with this? Remember I'm just trying to do this with a simple shell script.
-
When it comes to MLV you better stick with g3gg0 directly because MLV is his brainchild.
And everyone involved in developing MLV converters should know, of course.
-
When it comes to MLV you better stick with g3gg0 directly because MLV is his brainchild.
Yes, of course, here's the MLV topic I've been trying to understand: http://magiclantern.fm/forum/index.php?topic=7122.0
It is sort of like g3gg0 wrote a novel and I'm still learning the alphabet.
-
To understand the RAW footer you need to look at the struct definitions. To write the data to the file we simply dump the struct's in memory representation to file.
typedef struct
{
unsigned char magic[4];
unsigned short xRes;
unsigned short yRes;
unsigned int frameSize;
unsigned int frameCount;
unsigned int frameSkip;
unsigned int sourceFpsx1000;
unsigned int reserved3;
unsigned int reserved4;
struct raw_info raw_info;
} lv_rec_file_footer_t;
Notice it references struct raw_info. So basically just insert raw_info struct at the end:
struct raw_info {
uint32_t api_version; // increase this when changing the structure
#if INTPTR_MAX == INT32_MAX // only works on 32-bit systems
void* buffer; // points to image data
#else
uint32_t do_not_use_this; // this can't work on 64-bit systems
#endif
int32_t height, width, pitch;
int32_t frame_size;
int32_t bits_per_pixel; // 14
int32_t black_level; // autodetected
int32_t white_level; // somewhere around 13000 - 16000, varies with camera, settings etc
// would be best to autodetect it, but we can't do this reliably yet
union // DNG JPEG info
{
struct
{
int32_t x, y; // DNG JPEG top left corner
int32_t width, height; // DNG JPEG size
} jpeg;
struct
{
int32_t origin[2];
int32_t size[2];
} crop;
};
union // DNG active sensor area (Y1, X1, Y2, X2)
{
struct
{
int32_t y1, x1, y2, x2;
} active_area;
int32_t dng_active_area[4];
};
int32_t exposure_bias[2]; // DNG Exposure Bias (idk what's that)
int32_t cfa_pattern; // stick to 0x02010100 (RGBG) if you can
int32_t calibration_illuminant1;
int32_t color_matrix1[18]; // DNG Color Matrix
int32_t dynamic_range; // EV x100, from analyzing black level and noise (very close to DxO)
};
Each field in the struct is stored one after another, and each field's size depends on the data type, so here's the sizes you need to know:
int, unsigned int, or int32_t: 4 bytes
short or unsigned short: 2 bytes
int32_t array[n]: 4 * n bytes
char array[n]: n bytes
a union is sort of like saying a particular memory location can have different names and data types that use up the same space. To simply things lets just leave out the unneeded extra interpretations, and merge these two definitions in the final resulting definition:
typedef struct
{
unsigned char magic[4];
unsigned short xRes;
unsigned short yRes;
unsigned int frameSize;
unsigned int frameCount;
unsigned int frameSkip;
unsigned int sourceFpsx1000;
unsigned int reserved3;
unsigned int reserved4;
uint32_t api_version; // increase this when changing the structure
uint32_t do_not_use_this;
int32_t height, width, pitch;
int32_t frame_size;
int32_t bits_per_pixel; // 14
int32_t black_level; // autodetected
int32_t white_level; // somewhere around 13000 - 16000, varies with camera, settings etc
int32_t jpeg.x;
int32_t jpeg.y; // DNG JPEG top left corner
int32_t jpeg.width;
int32_t jpeg.height;
int32_t active_area .y1;
int32_t active_area x1;
int32_t y2;
int32_t x2;
int32_t exposure_bias[2]; // DNG Exposure Bias (idk what's that)
int32_t cfa_pattern; // stick to 0x02010100 (RGBG) if you can
int32_t calibration_illuminant1;
int32_t color_matrix1[18]; // DNG Color Matrix
int32_t dynamic_range; // EV x100, from analyzing black level and noise (very close to DxO)
};
So, the first 4 bytes will be 'magic' (an identifier) and they are supposed to be set to "RAWM"
next is 2 bytes xRes, and 2 bytes yRes, 4 bytes frameCount, and so on and so fort.
It's important to note that the data is little endian, which means the bytes come in the backwards order that would probably make sense to you. The LSB is first. So a 4 byte int that appears in the file like: 0x010203040 that value is actually: 0x40302010 = 1076895760. Your hex editor probably has an option to display the little endian value of whatever you have selected.
MLV works the same way except the data starts at the beginning of the file. See mlv.h for the struct definitions.[/code]
-
Wow--thank you very much. Now I've something to chew on.
-
Looks like it wasn't all that difficult to read the raw footer in a shell script but it isn't nearly as elegant as dmilligan demonstrated in C. Maybe it is time to re-write mlv2badpixels in C?
raw_footer.sh
# The RAW file footer is 192 bytes long so we need to
# back up 192 bytes (hexadecimal c0) from the end of the file.
# The -s is the seek offset and the hex number is negative
# because we're seeking from the end of the file.
# The -g 1 option groups the output in single bytes to make
# it slightly more human readable.
printf "\nRAW footer:\n"
xxd -s -0xc0 -g 1 "$1"
# Note the 4-byte 'magic' RAWM at the start of the footer.
# We're interested in the frame size of the raw file
# and the two bytes following RAWM, 188 bytes (hex bc)
# from the end is the image width size in pixels.
# We want just those two bytes so we'll suppress everything
# else (-p) so grouping no longer can be applied.
printf "\nWidth in hexidecimal:\n"
xxd -s -0xbc -l 2 -p "$1"
# Print this out as a decimal number doesn't work because
# it is "little endian" and the bytes need to be read in
# reverse order. Note the "0x" to tell printf that this is
# a hex number.
printf "\nNormal hexadecimal to decimal conversion doesn't work on little endian encoding:\n"
printf "%d\n" 0x`xxd -s -0xbc -l 2 -p "$1"`
# The solution is to put the hex number in a variable and
# print it out in reverse order.
printf "\nConverted from little endian encoding:\n"
w=`xxd -s -0xbc -l 2 -p "$1"`
printf "${w:2:2}${w:0:2} \n"
# Finally print out the image width from the raw footer.
printf "\nImage width taken from the raw footer:\n"
printf "%d\n" 0x"${w:2:2}${w:0:2}"
# Take the next two bytes and do the same thing for the image height.
printf "\nImage height taken from the raw footer:\n"
h=`xxd -s -0xba -l 2 -p "$1"`
printf "%d\n" 0x"${h:2:2}${h:0:2}"
Here's a sample output:
./raw_footer.sh 700D_zoom.RAW
RAW footer:
3d90000: 52 41 57 4d 00 05 d0 02 00 a0 18 00 28 00 00 00 RAWM........(...
3d90010: 01 00 00 00 ac 5d 00 00 03 00 00 00 e8 4a 4f ff .....].......JO.
3d90020: 01 00 00 00 00 81 79 04 24 04 00 00 50 07 00 00 ......y.$...P...
3d90030: cc 0c 00 00 b0 fc 34 00 0e 00 00 00 ff 07 00 00 ......4.........
3d90040: 98 3a 00 00 00 00 00 00 00 00 00 00 08 07 00 00 .:..............
3d90050: 08 04 00 00 1c 00 00 00 48 00 00 00 24 04 00 00 ........H...$...
3d90060: 50 07 00 00 00 00 00 00 00 00 00 00 00 01 01 02 P...............
3d90070: 01 00 00 00 ca 19 00 00 10 27 00 00 b7 fc ff ff .........'......
3d90080: 10 27 00 00 55 fc ff ff 10 27 00 00 88 ee ff ff .'..U....'......
3d90090: 10 27 00 00 aa 30 00 00 10 27 00 00 c7 08 00 00 .'...0...'......
3d900a0: 10 27 00 00 31 fc ff ff 10 27 00 00 f7 07 00 00 .'..1....'......
3d900b0: 10 27 00 00 04 18 00 00 10 27 00 00 4b 04 00 00 .'.......'..K...
Width in hexidecimal:
0005
Normal hexadecimal to decimal conversion doesn't work on little endian encoding:
5
Converted from little endian encoding:
0500
Image width taken from the raw footer:
1280
Image height taken from the raw footer:
720
I'll update mlv2badpixels.sh so the user doesn't need to manually enter the image size for a v1.0 raw video file then I'll start reading some C programming tutorials.
Thanks again dmilligan!
-
Beautiful
-
Purely Magic....
-
Since it was possible to read the RAW footer with a bash script I thought I'd try it with an MLV header and it works. I'll update mlv2badpixels.sh (https://bitbucket.org/daniel_fort/ml-focus-pixels/overview) so that it isn't dependent on mlv_dump. It doesn't really improve the script and it adds a dependency on xxd but it is sort of a proof of concept that working with RAW and MLV files is possible with rather simple shell scripts. Of course C is still a better choice.
mlv_header.sh
# Some more command line tricks this time
# for reading MLV headers.
# Most of the information seems to be in
# a fixed offset location and is rather
# easy to find.
# Note that little-endian words need to
# be read right to left.
printf "resolution width: "
h=`xxd -s 0x44 -l 2 -p "$1"`
printf "%d\n" 0x"${h:2:2}${h:0:2}"
printf "resolution height: "
h=`xxd -s 0x46 -l 2 -p "$1"`
printf "%d\n" 0x"${h:2:2}${h:0:2}"
printf "raw buffer height: "
h=`xxd -s 0x50 -l 2 -p "$1"`
printf "%d\n" 0x"${h:2:2}${h:0:2}"
printf "raw buffer width: "
h=`xxd -s 0x54 -l 2 -p "$1"`
printf "%d\n" 0x"${h:2:2}${h:0:2}"
printf "camera information: "
h=`xxd -s 0x01dc -l 6 -p "$1"`
printf "%s\n" 0x"${h:6:2}${h:4:2}${h:2:2}${h:0:2}"
# However, the crop and pan is recorded for each
# frame so we need to find the 'magic' that
# identifies that block. We're not going to attempt
# to support dynamic panning for now so let's just
# grab the first frame.
# Find the first "VIDF" block, "56 49 44 46" in hex
# the 1!N; append the next line when not on the last
# line in case "VDIF" is split by a newline. Keep
# a large enough chunk to include where the panX
# and panY information is located.
# For example, here's a typical first VDIF match:
#
# 0000290: 56 49 44 46 f0 5d 0a 00 b4 39 07 00 00 00 00 00 VIDF.]...9......
# 00002a0: 00 00 00 00 f0 02 ee 01 e9 02 ee 01 b0 0f 00 00 ................
# ^^ ^^ ^^ ^^ This is what we need
k=`xxd -p "$1" | sed -n '1!N; /56494446/{p;n;p;q;}' | tr -d '\n' | grep -Eo -m 1 '56494446.{48}'`
# Next find the panX and cropX and print them
# to the terminal:
printf "panX: "
printf "%d\n" 0x"${k:50:2}${k:48:2}"
printf "panY: "
printf "%d\n" 0x"${k:54:2}${k:52:2}"
-
I cannot find a decent way of dealing with these damn focus pixels shot on a T5i/700D using a Mac and Adobe CC workflow. Most of the apps recommended on here and online are all PC based like pinkdotremover and I cannot get them to work on my Mac. Does anyone have solutions that doesn't involve coding or something that extreme? I keep reading something about using "chroma smoothing" to fix the issue but I find no info how to do this online either or what it actually is and I am an AV tech. Please, any Mac user out there with the similar issue, help me!
Thanks.
-
http://www.magiclantern.fm/forum/index.php?topic=13152
-
@Roberto_Mena
There are several tools that work on Mac. dmilligan pointed out his MLVFS (http://www.magiclantern.fm/forum/index.php?topic=13152) which has been working on Mac and Linux for quite a while and more recently on Windows. What is great about MLVFS is that the DNG files are created on the fly with the focus pixels removed. In fact if you would have started with MLVFS you might not even know there is an issue with focus pixels on your T5i/700D.
Another fine tool is MLP (http://magiclantern.fm/forum/index.php?topic=13512.0). It can work in conjunction with MLVFS or via mlv_dump which is included in the MLP.workflow package. Note that when using mlv_dump the DNG files will still have the focus pixels but they are removed in the MOV files.
Finally, though a Windows program MLV Producer (http://magiclantern.fm/forum/index.php?topic=15271.0) can run under WINE on the Mac and it too can remove the focus pixels.
I have also written some shell scripts that can be used with dcraw if you want to roll your own application. This script (https://bitbucket.org/daniel_fort/ml-focus-pixels) is also used by MLP.
-
Thanks for this new info! I will try it out!
-
I finally got MLVFS and FUSE to work and got rid of my focus pixel issue! Thank you soooo much dfort and dmilligan for all your help you guys are awesome! You can't imagine how much this means to me to finally get this working!
-
Hi guys. So my latest test attempts with MLVFS do get rid of the "focus pixel" issue associated with the T5i/700D but I am now getting different weird artifacts here and there. One of the artifacts looks like purple/pinkish dots that seem to sparkle in and out of the image. The other type of artifact looks like spots of my video are turning into old school video game graphics like old pixels that kind bleed into small spots of my video. Most of these artifacts are not on my image and they are not stable like the "focus pixel" issue I was getting before. Any ideas how to fix this and what settings should be set to 'On' with MLVFS (i.e. Bad Pixel Fix, Vertical Stripes Fix, etc.)? Thanks!
-
Sounds like aliasing (https://en.wikipedia.org/wiki/Aliasing). Upload a sample DNG, all processing options should be off.
-
I am certain its not aliasing because Im familiar with what that looks like. How big of a DNG sample size do you need? Like one DNG frame? A couple of seconds? Thanks in advance.
-
One DNG would be good for a start but even posting a jpeg here would help. Best of all would be a short MLV file that we can download and examine. That would take out the possibility that the issue was introduced in your post process.
-
I tested MLVFS with only one setting turned on which was Chroma Smoothing at 5x5 and it seems to have fixed the issues. I'll keep you updated. Thank you.
-
Hi, I have a 100D/SL1, and by default the raw video is unusable, with a pattern that resembles the one described earlier in this thread.
Thanks to your amazing work, using MLVFS, almost all of these evil dots are gone, and the footages are now so much better. However, I still have some dots left (the most obvious is two lines near up and down) are some artifacts. I wonder if I'm the only one, or if this is a problem that you are aware about.
Here is an example of a 1440×602 raw video, with various settings enabled. I don't know in if I was in mv1080, mv1080crop. The only modified settings is RAW VIDEO, 1440×602.
No fix at all : https://picload.org/image/rglowipw/nofix.png
With bad pixel on : https://picload.org/image/rglowipi/bad_on.png
Aggressive : https://picload.org/view/rglowiiw/agressive.png.html
Bad + 2×2 Chroma: https://picload.org/image/rglowipa/bad_chroma22.png
Bad + 3×3 Chroma: https://picload.org/image/rglowipl/bad_chroma33.png
Aggressive + 5×5 Chroma: https://picload.org/image/rglowipr/agressive_chroma55.png
-
I'm a little confused, with MLVFS the focus pixels should be gone even with nothing "turned on". If you have to turn on bad pixel fix or chroma smooth, then focus pixel fix is not working.
-
Oh! Too bad… Yes, in the provided "nofix.png" picture, nothing is turned on, using the mounted MLV.
I use a fresh Windows 10 Pro with Dokan 0.8.0 (redist version). I copied the directory containing all the unzipped files of "MLVFS_x86.zip" in "c:\Program Files (x86)". The user (me) executing mlvfs.exe is admin class, as it seemed to work, but cmd itself is "not run as admin". I run mlvfs.exe in this directory.
mlvfs.exe d:\mnt --mlv_dir=d:\mlv
Am I doing something that can cause the malfunction? What can I test that might help?
-
OK, solved! "Run as Admin" cmd is mandatory in order to get all the dots away with nothing turned on.
I still get some artifacts on the white cars and power cables I can't seem to get rid of, is there anything I can do about that?
https://picload.org/image/rglcpccd/ok.png
-
@ Welles, looks like aliasing. No perfect solution, but there have been a lot of suggestions for reducing or preventing it if you look around. Crop mode nearly eliminates it (but has its own drawbacks).
-
Crop mode does help with aliasing but this looks like an issue with interpolating values to fill in the focus pixels. Don't know if I got the terminology correct but basically if the area surrounding a focus pixel has fine detail and high contrast it is difficult to come up with a value that will hide the focus pixel. This is more of a problem with dual iso. Overall it looks pretty good.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7672%2F27137030795_dfd9845240_z.jpg&hash=8bb63fe08a996b3535b564d5ef068235)
Upon closer examination the problem becomes apparent.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7730%2F27137030895_377d6cfb5a_z.jpg&hash=a6ee6412b05ea8ad476ec0ca987a4e42)
Have you tried adding some chroma smoothing? Don't do the bad pixel fix, just chroma smoothing.
-
Someone compared FRSP vs LV raw crop frame?
If the hybrid AF is like a dual pixel, one photodiode is composed of two :
http://www.imaging-resource.com/ee_uploads/news/4486/hybrid-cmos-af-iii.jpg
There is a possibility that in the photo mode, data of halves focus pixel are averaged. What is the result, focus pixel works as normal pixel.
So maybe just change the ADTG/CMOS registry to get a image without focus pixels.
-
Have you tried adding some chroma smoothing? Don't do the bad pixel fix, just chroma smoothing.
This should help a bit. But this is to be expected when shooting Dual-ISO videos from our DSLR's.
-
Someone compared FRSP vs LV raw crop frame?
There are no focus pixel issues with FRSP.
There is a possibility that in the photo mode, data of halves focus pixel are averaged. What is the result, focus pixel works as normal pixel.
So maybe just change the ADTG/CMOS registry to get a image without focus pixels.
I've said several times that the ideal way to deal with focus pixels is to do it in camera but that's way over my head. Exactly how would you do it?
-
We need to know why FRSP is no focus pixels.
Maybe a moire test, but I'm not sure.
-
@Frank7D, dfort, DeafEyeJedi
Thank you for your analysis. Crop mode does help a lot, but indeed this mode can't be used all the time. I'll test chroma smoothing on the white car this week.
A comparison from my first raw video (posted in the "Share Your Videos" thread) is provided below, normal first, crop mode below. DNGs from MLVFS are imported directly in PP, with a basic grading using a LUT, only with exposure, highlights, shadows, blacks and whites.
There also a disturbing irisation on the rocks (last pic). To my untrained eyes it does not look like focus pixels nor aliasing. What do you think?
Full shot here:
https://picload.org/image/rgcrgirr/shot0001.png
200% zoom on the house:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fpicload.org%2Fimage%2Frgcrgpli%2Fsanstitre.png&hash=c3b6a86e53a149db13acc83eac8aa48d)
Full shot, crop mode:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fpicload.org%2Fimage%2Frgcrgirl%2Fshot0004.png&hash=ee885ab66e4958ad9db2edd3f2312169)
Strange rocks
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fpicload.org%2Fimage%2Frgcrgiri%2Fshot0005.png&hash=028216d1e915571f2e72e84cb408ea2f)
-
@dfort:
I've just tested, chroma smoothing do not do much about that artifacts on the white car it seems.
Nothing: https://picload.org/image/rgcaadrl/nofix.png
Chroma 3x3: https://picload.org/image/rgcaadgc/c33.png
Chroma 5x5: https://picload.org/image/rgcaadra/c55.png
-
@Welles-
Just to be sure ... You are actually 'Refreshing' the webpage of the Localhost while running MLVFS after each selective options has been changed, correct?
Because otherwise it won't take action if that's what you're referring to from earlier re: aliasing on the wires & artifacts on the white car.
Perhaps upload a original MLV (of that particular footage) for us to play with and see what we can get out of on our end.
-
@DeafEyeJedi
I did not refresh, because it seemed to me at first that changes are taking effect immediately, and apparently it's not :-[
So I did it again, here how:
I have a sequence opened in PP, direct import of the mounted MLV in PP, no transcoding.
I tested each setting (nothing, then chroma 3x3, then chroma 5x5) like this:
1. Closing PP,
2. Change the setting with refreshing the browser, loading the preview
3. Reopening PP, play a few seconds of video, export a PNG image (the same each time).
Here they are:
https://picload.org/image/rgcalapw/nofix.png
https://picload.org/image/rgcalapl/c33.png
https://picload.org/image/rgcalapi/c55.png
I'm trying to upload the MLV (do you the .IDX file too? idk if it is important), but I'm on a shitty connection right now.
-
No need .IDX as we'll obviously be able to generate that file on our own while running MLVFS.
-
OK, the MLV is uploaded here: http://dl.free.fr/en1sTPWzp
Oh, I've just noticed that they modified the site since the last time I put something there: you have to disable ad block and there's an annoying ad to watch before you can click the green button and download, I'm very sorry about that.
-
@DeafEyeJedi,
I uploaded the MLV here, it is hopefully more convenient:
https://filesender.renater.fr/?s=download&token=562051f3-665f-d279-e59f-d03c0c4a90a3
-
Thanks @Welles for the file. Based on my observations with a workflow that I use in DR12.5 Beta 3 which includes for me to use Cinelog-C as a flat log and then apply Cinelog-C_Rec709 FM on top along with a few color tweaks just to make it pop. Apparently you may or may not be right that it doesn't seem to help (or it may seem to) when applying varieties of fixes within MLVFS Localhost webpage. Here are the results in comparisons.
No Fix
Bad Pixel Fix
BPF + CS2x2
BPF + CS3x3
BPF + CS5x5
Aggressive
Agg + CS2x2
Agg + CS3x3
Agg + CS5x5
Since it's also coming from a 100D user in myself as well ... I believe this is what we can expect from a such tiny camera in terms of quality (from the sensor) that delivers Raw, sure it's not a 5D3 but hey I wouldn't know which camera you used if you hadn't told me in the first place.
-
@DeafEyeJedi,
Thanks so much for having a look at this! I guess I'm gonna go with crop mode as much as I can then.
The only drawback I see is about the limited focal length on the wide side: by using the value provided by ML in the movie menu (5.83 instead of the regular 1.94 with a resolution of 1440×602) my 18-55mm lens is the equivalent of a 105-320 in crop mode. Maybe someday I'll buy the 10-18mm IS STM.
On the good side, beside getting rid of the artifacts, being more in the center of the glass should provide better quality, or at least reduce drastically the vignette effect of the lens. And my 55-250 is the equivalent of a 320-1460mm!
-
Were you able to tell which one was slightly better than the other? I could.
Another trick is to try and shoot it as wide open as possible w the iris in order to try and defocus the wires or even should help with not showing as much aliasing and such.
Perhaps try scooping up a used wide angle lens as I've gotten myself a decent 6.5mm f3.5 from Opteka which was made for APS-C Sensors , however , it's not as sharp as I would like which then can help lessen the chances of aliasing/artifacts. Still a tad slow for me tho.
Also I've seen others get away with nice crop mode footage by using a 8mm f2.8 from either Samyang or Rokinon in which I plan on selling my crappy Opteka 6.5mm f3.5 for one of these.
Definitely could use the extra light for indoors and more quality in sharpness overall.
-
Were you able to tell which one was slightly better than the other? I could.
Yeah, from what I see, the more CS, the worst it is. I'd better not touch this. But I do not see any difference between NoFix, BadPixelFix and Agressive alone. Do you?
-
About two weeks ago greg wrote:
Someone compared FRSP vs LV raw crop frame?
If the hybrid AF is like a dual pixel, one photodiode is composed of two :
http://www.imaging-resource.com/ee_uploads/news/4486/hybrid-cmos-af-iii.jpg
There is a possibility that in the photo mode, data of halves focus pixel are averaged. What is the result, focus pixel works as normal pixel.
So maybe just change the ADTG/CMOS registry to get a image without focus pixels.
Has someone investigated in this direction?
-
Thanks dfort, dmilligan, A1ex, g3gg0 and many others!
I did implement it to mlvp. But, still don't know what to to with .RAW and 3x5 skipping frames.
dfort, special thanks to you for your patterns!
-
Hi @AWPStar
Glad that you were able to use some of this stuff. For .RAW you need to supply the camera and video mode information because that isn't available in the metadata. There is a separate pattern for 3x5 skipping frames (also known as mv720).
Check out the source, written in bash:
https://bitbucket.org/daniel_fort/ml-focus-pixels/src/28fe6f4a5eb75a166bfecbca5a5fdcbdb7cd4666/mlv2badpixels.sh
-
Fresh release with your patterns :) Thank you!
http://www.magiclantern.fm/forum/index.php?topic=15271.msg170632#msg170632 (http://www.magiclantern.fm/forum/index.php?topic=15271.msg170632#msg170632)
-
Looks like this (http://www.magiclantern.fm/forum/index.php?topic=16054.msg160371#msg160371) is coming back to haunt us:
The video mode can be probably identified by (uncropped) raw buffer size - hopefully there aren't video modes with identical sizes, but different maps.
@rbrune has come up with a way to use the crop_rec module to record a sort of an mv720 mode with a 3x3 pattern instead of a 3x5 pattern. This of course is good news because the frame doesn't need to be adjusted to correct the aspect ratio but it is going to cause an issue having two conflicting focus pixel maps. The focus map affected is 80000331_1808x727.fpm. The 80000331 is the EOSM id and 1808x727 is the full raw buffer size. @a1ex has an idea to deal with this (http://www.magiclantern.fm/forum/index.php?topic=16608.msg174084#msg174084). In the meantime I'm going to see if I can create a focus pixel map file for this new video mode.
-
O0o0hhh this is so petrifying and yet feels so good in a way that I could've never imagined especially with an EOSM -- Thanks to everyone on their contributions in opening this wonderful can of worms especially to @rbrune which we shall probably name the so called ghost after? :)
The best of luck goes to @dfort and please do not hesitate to ask for help, if any.
-
Ok--here we go. A new focus pixel map file for the experimental EOSM crop_rec module. This is how it was done.
* Set up a focus target and put the camera on a sturdy tripod.
* Turn on the mlv_rec, crop_rec and silent modules.
* Shoot a short MLV file of the target and without moving anything shoot a Simple Silent picture--keep the camera in movie mode.
* Create dng files of the MLV file using your favorite converter, you only need to save one dng file.
* Run "dcraw -4 -E" on the Simple Silent shot, this will give you a .pgm file of the entire raw buffer.
* First put the raw buffer still into Photoshop (or Gimp) and raise the exposure so you can see some detail--it will be in black and white.
* In Photoshop I changed the Image mode from Grayscale to RGB.
* Throw in the dng file that you pulled from the MLV onto a second layer.
* Line it up, but don't do it by eye--even one pixel misalignment will cause big problems. Instead use "mlv_dump -v" and note the Crop numbers. In this case it came out to:
Crop: 304x28
Pan: 298x28
Like I said, use the Crop to line up the layer:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc5.staticflickr.com%2F6%2F5799%2F30746791076_1c2f54acee_z.jpg&hash=232e9f188e10a003e44aad1a7d51ab7a)
* Turn the layer on and off to check that things are lined up properly.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc2.staticflickr.com%2F6%2F5807%2F30666326092_667b092b57_o.gif&hash=4c5bf676e54fca1ca2f3b09523401d8d)
* Ok, cool. Now zoom in at gawk at all those focus pixels you need to hit.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc8.staticflickr.com%2F6%2F5576%2F30694807311_3f9677f70d_z.jpg&hash=9485933a9fbb19335dc904c1ba167fec)
* After a lot of trial and error I found out that making a new white layer and using the pencil tool set to 1-pixel black works great. I also set the layer to Multiply as I'm working. Obviously once you discover the pattern you can replicate it over and over until the whole focus pixel area is covered. By experimenting we discovered that not all focus pixels appear at all times. We also discovered that the pattern is repeated across the entire horizontal axis of the sensor but it doesn't extend all the way to the top and bottom. You don't want to map areas that don't have focus pixels because it might soften the image a bit, it makes for huge map files and it is too much unnecessary work.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc7.staticflickr.com%2F6%2F5538%2F30746793486_b0faf15aa4_z.jpg&hash=8fc91429e0c9d8f8a65faa0a4e0276b3)
* Change the layer back to Normal and export it as a Portable Bit Map file.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc6.staticflickr.com%2F6%2F5681%2F30694808341_65fb10121f_z.jpg&hash=d17c068373ebfa52da7a19f8f69cbaef)
* I've written some bash scripts to convert the .pbm file to either a text file that can be used in dcraw (using the "dcraw -P deadpixels.txt" option) or a .fpm file that can be used in MLVFS. All of these tools and the map files I built are located in a bitbucket repository:
https://bitbucket.org/daniel_fort/ml-focus-pixels
After making map files for all the cameras that show focus pixels and all the raw video modes you would think that I could just knock out a new map file but it isn't that easy. Sometimes the focus pixels are subtle and sometimes they don't show up at all. Then there are situations where you're sure you hit all of them and then you shoot something that activates a whole slew of focus pixels that you missed. So if you find some that I missed--please let me know.
-
Some more focus pixel hunting on the EOSM crop_rec project turned up some new ones that I haven't seen before. Makes me wonder if the other fpm files have all the focus pixels mapped properly. Haven't heard any complaints so like they say, if it ain't broken don't fix it.
Here's what the crop_rec pattern looks like after hunting focus pixels on dark frames and user supplied MLV files.
Full raw buffer map
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc8.staticflickr.com%2F6%2F5682%2F30826868775_601fa381c5_z.jpg&hash=e09d48a3b3db5aa676e67a834a6e917d) (https://flic.kr/p/NY4L6v)
80000331_1808x727 (https://flic.kr/p/NY4L6v)
Here's a closeup of the pattern
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc7.staticflickr.com%2F6%2F5535%2F30195009694_48c0782285_z.jpg&hash=915b269e12aaf0e70c487afddfd3562b)
Somehow it doesn't seem right. More like some sort of alien hieroglyphics.
-
I've tried to diagnose it by looking at the focus pixel map from 1:1 crop mode. We have some idea about the 3x3 binning filter, but we don't know the exact offset, so there must be 9 possibilities.
After losing patience waiting for fpm2pbm, I wrote an equivalent script in octave:
function M = fpm2pbm(image_size, filename)
# fpm2pbm.m
#
# Create a Portable Bit Map file from an
# MLVFS .fpm (focus pixel map file)
#
# Based on fpm2pbm.sh, http://www.magiclantern.fm/forum/index.php?topic=16054.msg161309#msg161309
# Requires GNU octave.
#
# Usage fpm2pbm(image_size, filename)
# example:
# octave> fpm2pbm([1108 2592], '80000331_2592x1108.fpm');
# shell$ octave-cli --eval "fpm2pbm([1108 2592], '80000331_2592x1108.fpm');"
#
# This is free and unencumbered software released into the public domain.
M = ones(image_size);
d = dlmread(filename);
x = d(:,1); y = d(:,2);
i = sub2ind(image_size, y+1, x+1);
M(i) = 0;
imwrite(M, [filename '.pbm']);
end
Now, to get the 9 possible focus maps after 3x3 column binning / line skipping:
pkg load image
# column binning filter
f = [ 1 0 1 0 1 ] / 3;
# focus pixel map from 1:1 crop mode, EOS M (assuming centered focus box)
M = fpm2pbm([1108 2592], '80000331_2592x1108.fpm');
# 3x3 binned image (full-res version containing all the possible offsets)
# actual 3x3 binning will result in one of those: MB(i:3:end, j:3:end) for i=1:3, j=1:3)
MB = imfilter(M, f);
# save all 9 possibilities for starting offset
for i = 1:3
for j = 1:3
Mb = MB(i:3:end, j:3:end);
imwrite(double(Mb > 0.99), sprintf('map3x3+%d+%d.pbm', j, i));
end
end
This gives 9 nice patterns, nowhere near the above example. Go figure...
On the good side, the above script correctly finds the patterns from 1808x727 (5x3) and 1808x1190 (3x3), starting from 2592x1108 (1:1).
To get the 5x3 maps:
# save all 15 possibilities for starting offset
for i = 1:5
for j = 1:3
Mb = MB(i:5:end, j:3:end);
imwrite(double(Mb > 0.99), sprintf('map5x3+%d+%d.pbm', j, i));
end
end
(edit: fixed off-by-one error)
-
Stunning work as usual @a1ex!
-
After losing patience waiting for fpm2pbm, I wrote an equivalent script in octave:
Yeah, my pbm2fpm.sh and fpm2pbm.sh bash scripts are painfully slow. I was thinking about trying to optimize them but thought I was the only one using them and even I don't use them very often.
Octave? Hum. Never heard of it. I installed octave along with the image package you're using and figured out the very basics but those bits of code don't seem to add up to a fully functional script. I can't get it working. Believe me I really want to.
You used the zoom mode map file which could be a problem. That file takes into consideration all the possible positions that the raw buffer can move around the sensor so it is a combination of 21 layers (http://www.magiclantern.fm/forum/index.php?topic=16054.msg161210#msg161210) each representing a position of the raw buffer on the sensor. It would be better to use the mv1080crop map file--aka Movie crop mode which is 1872x1060. With that file you won't be looking at coordinates where no focus pixels have been sighted. Yeah, these focus pixels are elusive and I have never had a frame that showed all of the focus pixels at once.
One of my ideas when working on this project was to create a master map file of the entire sensor and pull the 3x3 and 3x5 cropped patterns like what you seem to be doing with your octave script. However, I don't have the coding chops for this and there was no way to figure out exactly what position the zoom mode raw buffer was at so I had to give up on that idea.
I wish I could see the patterns that you're looking at because it would help verify that we really do have all the focus pixels mapped. That last one for the crop_rec just doesn't look right. My hypothesis is that the 100D has the same pattern as the other cameras but it is covering a larger area of the sensor. It also seems to use those pixels differently, that's why they are more prominent than on the EOSM, 700D and 650D.
Having coordinates on the map files in places where there aren't any focus pixels might give us a hit in resolution but the trade off is possibly missing focus pixels. I'm thinking about combining the "normal" mv720 with rbrune's crop_rec mv720 like I did with the zoom map so that it will take care of both modes with the same fpm file. That is until there's a way differentiate these modes from the MLV metadata. No rush, EOSM crop_rec isn't exactly main stream at the moment.
-
First script: save it as fpm2pbm.m in the current directory, then copy the example command (you can run it either from octave prompt, or from the shell).
Second script: either save it as maps.m (or whatever), then type its name at the octave prompt (without extension, so just "maps"), or paste everything into the octave prompt.
My hypothesis about the mv1080 crop map: it might be some mix between the 3x3 map (the real focus pixel coordinates) and the 5x3 one (where Canon's FPN handling code is expecting to find the focus pixels).
We might be able to get a focus pixel map from a full-res silent picture. That one shouldn't be ambiguous, but we need to patch Canon code to change the raw type. I can look into it if there is a good reason for doing so.
-
@a1ex
I found out that octave scripts that end with an .m extension are for MATLAB compatibility so it seems that your fpm2pbm.m needs it but it doesn't work with the maps script. Anyway, got it working.
Checking out the EOSM focus pixel map files it looks like everything except the latest crop_rec is working as expected. The 100D map files completely blew away my hypothesis.
With the EOSM the full (1x1) pattern seems to be this:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc4.staticflickr.com%2F6%2F5711%2F30228968963_bcd4364245_z.jpg&hash=81e9d3fd0bbfe05fc9935b1ca243d662)
The possible 3x3 skip patterns according to your scripts are just two basic patterns. One looks like this:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc2.staticflickr.com%2F6%2F5591%2F30227791593_94bb719915_z.jpg&hash=e13c972a8263ef3211d5fde79db0e9a6)
And the other one is the one we're using for the mv1080crop and zoom raw video modes:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc8.staticflickr.com%2F6%2F5553%2F30227791343_e7b6e33cf0_z.jpg&hash=8905c0fe6a82d6c5224d03b9d882bbaa)
With 3x5 there are three basic patterns. This one:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc7.staticflickr.com%2F6%2F5749%2F30562156110_ec9b69a031_z.jpg&hash=eeb0be5a1469f1af20650c965b0ea05e)
I didn't make a mistake here, some possible 3x5 combinations actually miss all the focus pixels:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc4.staticflickr.com%2F6%2F5561%2F30227791843_fbd1de5c85_z.jpg&hash=9e6cc43524c3e6b00e05a02c1389c281)
And finally this one which is used in the mv720 focus pixel map file:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F6%2F5519%2F30826838016_d34739808b_z.jpg&hash=3c7af2d065963596e4081dd0ea454e4c)
So far everything seems to be falling into place. The EOSM patterns are also the same for the 650D and 700D.
There's only one more camera to check out, the 100D which has a 1x1 pattern that we believe looks like this. Note that if you overlay the EOSM/650D/700D pattern you'll see that it is a subset of this pattern:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc7.staticflickr.com%2F6%2F5592%2F30563354910_ea1824dc05_z.jpg&hash=7960ac40514749cd976be7c411c5257c)
Using the octave script the possible 3x3 patterns are:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc3.staticflickr.com%2F6%2F5754%2F30826838186_32d8d451d8_z.jpg&hash=60fede39de3c5101c8bbf1ef6749961d)
And:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc4.staticflickr.com%2F6%2F5606%2F30227792163_d34739808b_z.jpg&hash=395cee7189f4eb3bd10486cfaf87caaa)
Hum--that doesn't match up with what we ended up using for the focus pixel map files. The mv1080 map file pattern we're using for the 100D matches the other cameras. Ok, so how about the 3x5 patterns? This should also match the other cameras:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F6%2F5834%2F30562156640_eb5d6a61de_z.jpg&hash=9abacf8ba2a2e051bca8c55e228c7b6e)
And:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc4.staticflickr.com%2F6%2F5581%2F30227791923_d39e451240_z.jpg&hash=a82d0d92fa0a1cfdd0a14293d3aa0778)
What the.....????
My hypothesis about the mv1080 crop map: it might be some mix between the 3x3 map (the real focus pixel coordinates) and the 5x3 one (where Canon's FPN handling code is expecting to find the focus pixels).
Looking at what I came up with for the EOSM crop_rec pattern you're probably right. I've most certainly got some stray coordinates in there so how to figure this out so we can get the "real" pattern?
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc7.staticflickr.com%2F6%2F5535%2F30195009694_48c0782285_z.jpg&hash=915b269e12aaf0e70c487afddfd3562b)
We might be able to get a focus pixel map from a full-res silent picture. That one shouldn't be ambiguous, but we need to patch Canon code to change the raw type. I can look into it if there is a good reason for doing so.
That would be interesting but I'm not sure that would solve this puzzle. The focus pixels on the 100D are much easier to see so maybe giving that platform the crop_rec module might shed some light on this?
-
Absolutely stunning work @dfort and I am just in awe by staring at these patterned Black/White alienated-alike checker board especially on the last few ( seriously wth?! :o ) which is rather surprising to me!
... but I'm not sure that would solve this puzzle. The focus pixels on the 100D are much easier to see so maybe giving that platform the crop_rec module might shed some light on this?
That's true -- I notice that the SL1/100D does show the Focus Pixels in a certain way that makes it easier to see than the other affected ones.
@nikfreak -- is there a branch that I can use to test this with the new 10/12-bit options or is that already on your 'to-do list'?
-
@nikfreak -- is there a branch that I can use to test this with the new 10/12-bit options or is that already on your 'to-do list'?
Actually what we want is the crop_rec module for the 100D.
BTW--(off topic) what's the holdup with the 70D and 100D merge? Looks like those cameras are about as stable as the rest of the bunch.
-
I have been following this forum though i dont use 650d anymore. Btw waiting for the 70d merge
-
Let's try to make sense of the new information we got from looking at the 3x3 and 3x5 patterns from a1ex's octave script.
If we start with the 100D's 1x1 focus pixels by looking at the pixel maps for mv1080crop or zoom modes then run that through all the possible 3x3 combinations we get nothing that resembles what the focus pixel map that we know is working for that camera. In fact the mv1080 focus pixel maps for all cameras have the same pattern. If we overlay the map3x3+1+1 from the 650D/700D (and theoretically EOSM) mv1080 video mode with the same map3x3+1+1 from the 100D and look at the pixels that are common to all cameras something interesting shows up:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F6%2F5782%2F30280486504_bbb2cbbd5a.jpg&hash=c8000389076e6f9e5c8311fbd45aca38)
So here is a new hypothesis, not all focus pixels are activated when the shooting in mv1080 mode. Does this hold true for mv720 mode? The pattern that matches the working focus pixel map in this case turned out to be the map5x3+1+3 and overlaying the 100D with the other cameras' mv720 focus pixel map files shows that once again not all focus pixels are active.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc5.staticflickr.com%2F6%2F5614%2F30613826620_8902c5676e.jpg&hash=025b23f76d4f28d7e45b4d1fb2351f3d)
Ok--so what's my hypothesis on the weird map I pulled for the EOSM crop_rec?
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc7.staticflickr.com%2F6%2F5535%2F30195009694_48c0782285.jpg&hash=c5a69863a2ace6c36c34e4ad5120f8c6)
At first I suspected a rounding error on the frame size because we're using the same full raw buffer size for the crop_rec 3x3 mv720 as the original 3x5 mv720 so something might have been going on with the vertical line skipping. Nice try but not likely.
I believe that somehow I got some 3x5 frames mixed up with 3x3 when I was searching for all the focus pixels that I missed. Here's what it looks like if you overlay the 3x3 with the 3x5 focus pixel maps:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc5.staticflickr.com%2F6%2F5665%2F30798763052_49c0ac022e.jpg&hash=bde0106f19cc3f86dd25b3e58c466f27)
The mirror image might be a different combination of pixel maps but it does prove a point which is basically--my bad!
-
Was it my bad or perhaps an accidental discovery?
Overlaying both the focus pixel map file for the EOSM crop_rec and the "normal" mv720 full raw buffers gave an interesting pattern that was close to what I came up with when I was trying to map all of the focus pixels form a group of crop_rec MLV files. There must have been a regular mv720 file mixed in there because when I finally got the fpm file working it looked a lot like the pattern from laying the two map files over each other. I tried it again, simply by merging the two fpm files and it worked on all the files I tested. This is what the pattern looks like:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F6%2F5776%2F30648576800_e571bb9f6b_z.jpg&hash=93428543d04712a4253345ab0c1f9542) (https://flic.kr/p/NGiY6W)
80000331_1808x727 (https://flic.kr/p/NGiY6W)
You can click on that link if you want to take a closer look or even download a full sized version. Here's a section of that image:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc7.staticflickr.com%2F6%2F5634%2F22771730238_54e50c7d2b_z.jpg&hash=cf6cae7bdc19c2696ac86d3fa8133bd8)
This map file can handle both crop_rec and mlv_720 raw video files. Sure, it hits a lot of extra pixels for each video mode but it doesn't seem to affect the image much--at least not on the tests that I have done. In addition, it doesn't hit any pixels that aren't on either crop_rec or mv_720 modes. Since crop_rec and mv_720 have the same full raw buffer sizes and that's what we're using to identify the various video modes, this saves us from having to switch between two different focus pixel map files.
I actually got this idea as I was writing that previous post but I was having problems because I was converting between focus pixel map files and portable bit map files using the octave script that a1ex provided. It does work fast but the pixels are off by one in both the vertical and horizontal axis which of course renders the final fpm file useless. I wasn't sure how to fix the script but it didn't matter because all I really had to do was to merge the two working fpm files which are simply tab or whitespace delimited text files that look like this:
79 219
87 219
95 219
103 219
111 219
119 219
127 219
135 219
143 219
...
Technically you should be able to map a pixel that is located at 0 0 but it doesn't work in the octave script. Anyone have a solution for that? It would also be nice to have a script that converts from pbm to fpm that runs as fast as a2ex's octave script but honestly I think we've mapped all of the focus pixels for all the cameras with this issue. The newer cameras that are being ported to ML aren't showing their focus pixels on raw video files.
crop_rec is not in the nightly downloads yet but users are experimenting with it so it might be a good time to give the post processing applications developers a heads up that this is coming and that we've already got a focus pixel map file that can deal with it.
By the way, all the files I've been referring to are available on bitbucket:
https://bitbucket.org/daniel_fort/ml-focus-pixels
-
5x crop zoom mode on the 5D3 wasn't centered until this (https://bitbucket.org/hudson/magic-lantern/commits/c42732aaf809309e925554da8b4d7b3287ae9063) commit that was announced in this (http://www.magiclantern.fm/forum/index.php?topic=17021.msg174918#msg174918) forum post. Zoom mode is quite useful on the 5D3 because it can record at 2.5k resolution. @Danne even modified a lens (http://www.magiclantern.fm/forum/index.php?topic=12796.msg127357#msg127357) so it would be centered when using this mode.
That got me thinking about the zoom mode issues with focus pixels. The EOSM/100D/650D/700D cameras have the crop hack that was adapted from the 600D so zoom mode is like the ugly step child for these cameras but it does have a larger raw buffer so it can theoretically record larger image sizes. Of course you need to keep in mind the limits to writing to the SD card and the new 10bit 12bit option along with slowing down the frame rate helps with lowering the data rate. In any case, the problem is that in zoom mode you can pan around the sensor and if the image isn't perfectly centered when recording in raw 1.0 the focus pixels probably won't be removed. The reason for this is because raw 1.0 doesn't have the Crop metadata that MLV has in order to line up the focus pixel map files. The zoom mode map files are already a compromise because they are a combination of 21 possible raw buffer locations. So here's the idea, what if we make a new focus pixel map that has all the possible pan positions too? Panning moves aren't in pixel increments so the combined map file should still have enough "holes" in it to allow the image to come through. Now the question is if it will affect the image quality? This would only be used for raw 1.0 which is probably going to be replaced by raw 1.1 (MLV Lite) so it is more an exercise of how far can we go with these focus pixel maps rather than anything users would adapt. Still, if you come across a raw 1.0 file that was shot in zoom mode that wasn't centered exactly--this should still remove the focus pixels.
-
Let,s try it out. Gonna be a pretty big amount of coordinates in there but like you said. There should be enough neighbouring pixels to go by.
-
The purpose of this post is to hopefully grab the attention of anyone who is willing to help add automatic focus pixel removal to mlv_dump.
In order to make the work already done on focus pixel removal more accessible there will be some major changes in the ML Focus Pixels bitbucket repository (https://bitbucket.org/daniel_fort/ml-focus-pixels) in the next few days. There are several reasons for the reorganization. The scripts are maturing and doing more than their original functions, there are new focus pixel map files and combinations of old and new map files that we should keep organized as new video modes emerge and the code needs to be documented clearly so that the focus pixel cleanup algorithms can be implimented in other applications.
The goal is to make focus pixels a non-issue for users. The raw video processing applications should shield users from having to take any special steps to deal with it. MLVFS (http://www.magiclantern.fm/forum/index.php?topic=13152.0), MLVProducer (http://www.magiclantern.fm/forum/index.php?topic=15271.0), MLP (http://www.magiclantern.fm/forum/index.php?topic=13512.0) and cr2hdr.app (http://www.magiclantern.fm/forum/index.php?topic=15108.0) are using the focus pixel maps and scripts that were developed from the comments posted in this topic, most notably by dmilligan - Thanks David!
In order to deal with the focus pixels the first thing that needs to be done is to find them. This isn't as simple as it may seem at first because the focus pixels are only visible under certain conditions. One method that has proven effective for finding focus pixels is to shoot a "black frame" -- a common practice in astrophotograhy to sort out "hot pixels" from stars. Once enough coordinates are mapped a pattern becomes apparent and the location of the harder to find pixels can be predicted.
Once all the focus pixels are mapped the pattern created by the focus pixels can be turned into a formula. I experimented with this (http://www.magiclantern.fm/forum/index.php?topic=16054.msg162474#msg162474) and implimented it in a bash script that Danne is using in his Mac Automator apps. It takes an extra step to update a formula for a new raw video mode and takes more effort to maintain but the advantage is that you don't need to deal with a library of external map files.
-
I really like that focus pixel script of yours. Very nice to hear you are planning on improvements.
-
I just found out that the xxd trick to find the camera model isn't working with MLV Lite files. Everything else seems to be working. Maybe just go back to using
exiftool mlv_dump and not worry about it?
-
I put aside this topic for a while to concentrate on the 10bit 12bit raw video development when dmilligan pointed out that there might be a raw video stream that doesn't show focus pixels (http://www.magiclantern.fm/forum/index.php?topic=5601.msg175185#msg175185). It took some effort but at one point thought I found (http://www.magiclantern.fm/forum/index.php?topic=5601.msg175470#msg175470) it.
First of all -- thanks to dmilligan for having the patience to work with me on this.
NO MORE FOCUS PIXELS!
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc4.staticflickr.com%2F6%2F5607%2F30431903123_21e0ee762d_z.jpg&hash=189eec4e57cd0e4d30e5dfd792bc0325)
Here's what does the trick--only working on the EOSM at this time:
#ifdef CONFIG_EOSM
#define PREFERRED_RAW_TYPE 18
#endif
This is a before shot (using PREFERRED_RAW_TYPE 34)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F6%2F5522%2F31203782576_d42de1350a_z.jpg&hash=385cc3f60f31b8e81ca66557d1399618)
and here is what it looks like with PREFERRED_RAW_TYPE 18
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc6.staticflickr.com%2F6%2F5503%2F31125394261_7f266185aa_z.jpg&hash=0d3911ab8577cfa5c4f8f5eb11a65f49)
I'm still running some tests and haven't gotten it working with the crop_rec module yet but I wanted to share the news.
Alas, I wasn't the first one to try this out and the hot pixels that appeared in this video stream was even more difficult to deal with. The stream choices were limited to 64 but it seemed that the default for the 700D was 78 so I increased the limit to 128 and kept looking. A funny thing happened on a few of the video streams. There were more focus pixels. In fact at some of the settings the EOSM files looked identical to the 100D which if you've been following this topic has a different pattern than the 650D, 700D and EOSM which share the same pattern--or so I thought.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc3.staticflickr.com%2F6%2F5547%2F31075830970_4e0e688992_z.jpg&hash=89bc0ff90604c68b38f8b62e8549fd7a)
Here's a closer look:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc2.staticflickr.com%2F6%2F5617%2F30638207713_f2f4e007cb_z.jpg&hash=00715459178c8adb0bce4ca7272b3a30)
Normally the EOSM/700D/650D shows focus pixels over a horizontal strip while the 100D shows focus pixels over a much larger area of the sensor. Seems like there may be focus pixels over a larger portion that we first thought.
Weird part is that using the 100D focus pixel map file on this it took care of most of the pixels. Sure, it isn't perfect and it is extending beyond where the "normal" 100D focus pixels stop but it is an interesting discovery.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc4.staticflickr.com%2F6%2F5780%2F30638713043_4e0026f6c2_z.jpg&hash=ed08e2bc56ba07362a9b6e84b91449fe)
Movie crop mode also shows an interesting pattern that we haven't seen before:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F6%2F5672%2F31075833440_533e03cfac_z.jpg&hash=5802fac39e43c59ac3aacbf2f7dda9cc)
Ok, this is all very entertaining but is there any practical value in this? Maybe. When using the video stream that doesn't show focus pixels some hot pixels appear around high contrast boundaries.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc3.staticflickr.com%2F6%2F5459%2F30892543730_be804328eb_z.jpg&hash=fb70b32faff24f309e3efb25787df21a)
The "normal" focus pixel map file in MLVFS, MLP, cr2hdr.app and probably MLV Producer won't get rid of these pixels and chroma smoothing doesn't do it either but when using the focus pixel map file from the 100D they were gone! I haven't found a video stream that is 100% clear of focus pixels but perhaps there are video streams that can use a one size fits all focus pixel map file?
-
Just a reminder as it now could apply to a unifying scenario. I have seen you also discovered that those bunch of cameras use the same sizes for edmac raw slurp, too:
http://www.magiclantern.fm/forum/index.php?topic=16608.msg174241#msg174241
-
I'm good with a unifying scenario even if it means coming up with all new focus pixel map files.
The octave script a1ex made is super fast and I'd love to have another octave script that can take turn the pbm back to a fpm file but the script is off by 1 pixel on both the x and y axis. Can't figure out how to fix it.
The fpm file has this:
78 290
77 291
In Photoshop the pbm file has the dots at:
77 289
76 290
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc3.staticflickr.com%2F6%2F5476%2F31331787442_b3d6b76e85_m.jpg&hash=cf52d2f554280796893e04d5b0124730)
There isn't anything to adjust in the script but maybe there is a preference setting somewhere? It seems that instead of starting at 0,0 it starts at 1,1. If I put in a coordinate at 0,0 which should be a valid location, this error comes up:
error: index (0,_): subscripts must be either integers 1 to (2^31)-1 or logicals
error: unhandled execution exception -- eval failed
Here's the octave script again:
function M = fpm2pbm(image_size, filename)
# fpm2pbm.m
#
# Create a Portable Bit Map file from an
# MLVFS .fpm (focus pixel map file)
#
# Based on fpm2pbm.sh, http://www.magiclantern.fm/forum/index.php?topic=16054.msg161309#msg161309
# Requires GNU octave.
#
# Usage fpm2pbm(image_size, filename)
# example:
# octave> fpm2pbm([1108 2592], '80000331_2592x1108.fpm');
# shell$ octave-cli --eval "fpm2pbm([1108 2592], '80000331_2592x1108.fpm');"
#
# This is free and unencumbered software released into the public domain.
M = ones(image_size);
d = dlmread(filename);
x = d(:,1); y = d(:,2);
i = sub2ind(image_size, y, x);
M(i) = 0;
imwrite(M, [filename '.pbm']);
-
Just a reminder as it now could apply to a unifying scenario. I have seen you also discovered that those bunch of cameras use the same sizes for edmac raw slurp, too:
http://www.magiclantern.fm/forum/index.php?topic=16608.msg174241#msg174241
Lots of stuff going on at the same time. I'm suspecting that some of the recent discoveries the focus pixels that appear in ML raw video might have to do with a focus peaking feature that maybe Canon was looking into but wasn't implimented on the EOSM/100D/650D/700D. From my research focus peaking first came out on the M3. In any case this is all just speculation.
-
I laugh to think of all the time I spent on the EOS-M a couple of years ago. I blew many brain cells on the focus pixel problem. In case you haven't seen this, this is what I came up with at the end, a windows program that takes a known pattern (crop mode) and removes the pixels through debayering. The "parallelism" idea didn't work as either Alex or g3ggo predicted (frames get out of order). Anyway, there may be code in here that's of some use to you. I don't remember what I did, so probably couldn't answer any questions. Sor far, I find the stripe removal in mlrawviewer to be good enough for me.
https://bitbucket.org/maxotics/focuspixelfixer/src
I just looked at the code again. I think blood just came out of my ear :-X
-
Ha, It's so great to hear from you again @maxotics!
-
[...]but the script is off by 1 pixel on both the x and y axis.
Right, in octave, array indexing starts from 1. This should fix it:
i = sub2ind(image_size, y+1, x+1);
-
...there may be code in here that's of some use to you..
Oh yeah -- I looked at your code and even rendered your pixel map file in this post:
http://www.magiclantern.fm/forum/index.php?topic=16054.msg159826#msg159826
Right, in octave, array indexing starts from 1.
Figured it was that. Thanks for the fix!
-
Just discovered a nice program which seems useful to detect and coordinate focus pixels pretty quick. It,s called imageJ. I found it while searching for a dead pixel picker which would create coordinates when picking a dead pixel. This also works with the program with a plugin called point picker plugin. It depends on java runtime 6 but just follow the link when asked during installation and it should work after this.
ImageJ
https://imagej.nih.gov/ij/index.html
Installation instructions
https://imagej.nih.gov/ij/docs/install/osx.html
Point picker plugin
http://bigwww.epfl.ch/thevenaz/pointpicker/index.html#Download
I started playing with a dng file from my eosm, exported to tiff with dcraw. Changing some parameters in something called find Maxima seems to be effective. Once sweet spot is found you can send the coordinates out to a list and go from there. I couldn,t really get good use of it yet cause I don,t know quite buffer correlations and so on but will experiment some more when I can.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs29.postimg.org%2Fm8tzgb9nr%2FScreen_Shot_2017_01_08_at_15_52_10.png&hash=b659e2428652b97b01f228a2507ac2af)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs29.postimg.org%2Flusngppk7%2FScreen_Shot_2017_01_08_at_15_53_11.png&hash=cdfa76412e75ad56a8aa7ee9a5cd9080)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs29.postimg.org%2Feqau7oiav%2FScreen_Shot_2017_01_08_at_15_53_21.png&hash=ffb72bb48b236a154adff6d758862498)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs29.postimg.org%2Fvcsegrb8n%2FScreen_Shot_2017_01_08_at_15_53_29.png&hash=03194f9f3b8d2dca9dcf971d6a25d309)
-
ImageJ is pretty cool. I've used it for photo stitching. I'm curious why EOS-M Focus Pixels is still an issue when MLVProducer seems to deal with them very well, with additional filters to soften aliasing (moire). It even does 10bit. Is it because MLVProduder is PC, or is there something else I'm missing? Also, MLRawViewer removes focus pixels through "stripe" removal. Finally, I want to take very opportunity to rave about MLVProducer :D
-
It,s not an issue maybe cause of the work of dfort but it might be useful for fast isolating hot pixels with the point picker. And it seems way faster than imagemagick to find the focus pixels.
By the way. Mlvproducer is using Dfort,s very pixel lists in his program if I,m not mistaken. As well as MLP, cr2hdr.app and MLVFS :P
-
Thanks Danne. Yes, I knew that but you're SO RIGHT. I need to rave about dfort too! :) DFORT, consider yourself an ML hero in my book up there with Alex and g3ggo! (Whatever happened to 1%?). Wait, getting off track. I've been using and loving my EOS-M and I know I have DFORT to thank!
-
If you are going to be giving credit to anyone it should be dmilligan. He guided me through this whole process.
Interesting find there Danne. There are lots of ways to deal with focus pixels. The focus pixel map is just one way. It may be overkill at times because we're always filling in those coordinates even when the conditions that the frame was shot in doesn't trigger the focus pixel to appear.
By the way, I first got into compiling the Magic Lantern code just so I could turn on Chroma Smoothing in raw2dng to deal with focus pixels on the EOSM. I still turn on Chroma Smoothing when using MLVFS (http://www.magiclantern.fm/forum/index.php?topic=13152.0) to get rid of the few remaining focus pixels that sneak in at times.
-
I tried applying the added coordinates by making a list and running it through dcraw. I didn,t seem to do much. I went with the crop_rec.mo on my eosm. Maybe a bad choice?
Anyway. I went with dcraw -T -j -4 -W to create the tiff. Not with -E(where did you find that setting anyway? Not documented) cause it wouldn,t be effective finding the pixels.
In my thinking the very coordinates created from the tiff would work with dcraw when put into a list? Is raw buffer to be considered as well?
-
I tried using hot pixel fixes with dcraw and never got anywhere. As dfort pointed out, the pixels aren't predictable (at least I couldn't) and the ones that can fire take up almost the entire frame. if I remember correctly. For me, the problem is solved because our sensitivity (or lack of it) for color allows a significant amount of color-information removal to take place before we notice it--the whole idea behind 4:2:0 compression. At first, that answer I'm giving you was not good enough for me. I hated the idea of losing, or doubling up on data points like video compression. So even when Alex or G3ggo came out with chroma smoothing I resisted it.
Now it doesn't bother because my primary love of RAW video is the fact that it allows me to set each pixels 8-bit value (even if smoothed) from 14-bit sources. One of my biggest curiosities is why I can never get 8-bit video to have the same DR feel as RAW. Even with the EOS-M at sub 720p, the RAW has a more nuanced looked than, say, 4K from a GH4 or A7 (I've tried just about every consumer camera you can think of). How can that be? The only cameras that seem to approach the RAW "nuance" starts with the Canon C100 or Sony FS5, both close to $5,000. My guess, until someone with more knowledge can explain it to me, is that in-camera exposure is always a tiny bit off and once it writes to an 8-bit (or 24bit full color) space, even if 4K, there is no way to re-adjust the exposure without seriously degrading the DR feel.
If the above is right, everyone's effort in 10bit is game-changing to me. Initial tests I've done, and looking at other videos' shows that even 10bit RAW is vastly superior to any in camera video out there, even 4K. 10bit not only allows higher resolutions, it can also reduce file size. A focus pixel removed through crude chroma smoothing will probably not be visible to anyone who put a priority on DR. Video shot in 10bit RAW, I wager, will provide a feel that even 8K will not provide. Thoughts?
-
@maxotics
Quite a post. A bit off topic but I'm pretty much in agreement.
I have been very close to pulling the trigger on a C100 to replace one of my EOSM's. Ok, just kidding but if I were to get serious about the gear I own, the price of the original C100 is currently the same as a 5D mark iii ($2,499 at B&H). Just a step up is the C100 Mark II which is close to the same price as a 5D mark iv. The advantage of the C100 for me is that Magic Lantern isn't available for that camera so I would probably spend less time hacking and more time shooting. What is holding me back from getting one is that my friends who shoot on Arri cameras most of the time tend to look down on the low end Canon cine cameras.
I've shot a short using a 5D3 with MLV raw video. The workflow and data wrangling is a PITA compared to working with the MOV files straight out of the camera. Other ML users will probably disagree with me but the images from the C100 are about as good as ML raw. Of course you don't have as much control as you do with raw but I used to be a photographer when everything was shot on film (before Photoshop) and had to get it right in the camera.
Back on topic, the goal of this project was to eventually shield users from ever having to deal with focus pixels. MLVFS is doing a good job of that now. I haven't tried MLV Producer recently (I'm on a Mac) but have been hearing some good things about how it handles focus pixels. The big challenge will be getting this working in mlv_dump.
-
When treating a darkframe as a flatframe in mlv_dump you,ll get this revealing output.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs27.postimg.org%2F9lcxsqpgz%2FScreen_Shot_2017_02_02_at_15_15_24.png&hash=86133f1048011c17984b987514521ef9)
-
That reminds me of the image file I was able to pull off the focus pixel map from the PinkDotRemover tool 650D (http://www.magiclantern.fm/forum/index.php?topic=6658.0):
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F2%2F1661%2F24191162316_21f19939fe_z.jpg&hash=0a348b9f8f1d914c7a5366085c14cb8c)
I have never seen a single frame that displays all of the focus pixels so that may mean that using a previously saved dark frame as a focus pixel map might not work all of the time. It might not even work from the beginning to the end of a take because lighting, sensor temperature, etc. can change while the camera is rolling. My understanding from the astrophotograhy forum posts I've read is that a dark frame should be taken under the same conditions (especially sensor temperature) as the shot that you are going to be applying it to.
-
darkframe is working in shadows and the best output is achieved with chroma smooth 2x2. The more or less perfect result is to use the focus pixel maps as well as darkframe and cs2x2.
-
I am working on a method of removing focus pixels myself and I thought I post the pixel map for the Canon EOS M here, just in case there is anybody who can make use of it with the usual tools. I did not try myself. I have a method of extracting the focus pixel map from actual MLV files. All I need is a short recording with the highest possible resolution in fullframe and crop mode. If anybody can provide that for other cameras, I can extract the map and most it here.
https://www.dropbox.com/s/sdqopv87n6w9us6/EOS_M_FocusPixels.zip?dl=0 (https://www.dropbox.com/s/sdqopv87n6w9us6/EOS_M_FocusPixels.zip?dl=0)
If you make a recording, remove the lens and point the camera to the sky. Then expose the image so that you have 98% white on screen. The white should not clip.
-
How did you build them? Any code to check out?
-
I am extracting the red, green and blue channel from the raw_buffer and saving them as 8-bit grayscale tiff files. Then I use photoshop to change the colors. Pretty straight forward.
Here's the code. Don't know if that's of any use:
size_t halfW = _rawInfo.width/2;
size_t halfH = _rawInfo.height/2;
CGColorSpaceRef colorSpace = CGColorSpaceCreateWithName(kCGColorSpaceGenericGrayGamma2_2);
size_t bytesPerPixel = 1;
CGContextRef redBP = CGBitmapContextCreate(nil, halfW, halfH, 8, halfW*bytesPerPixel, colorSpace, kCGImageAlphaNone);
uint8_t* redPtr = CGBitmapContextGetData(redBP);
CGContextRef blueBP = CGBitmapContextCreate(nil, halfW, halfH, 8, halfW*bytesPerPixel, colorSpace, kCGImageAlphaNone);
uint8_t* bluePtr = CGBitmapContextGetData(blueBP);
CGContextRef greenBP = CGBitmapContextCreate(nil, halfW, _rawInfo.height, 8, halfW*bytesPerPixel, colorSpace, kCGImageAlphaNone);
uint8_t* greenPtr1 = CGBitmapContextGetData(greenBP);
uint8_t* greenPtr2 = greenPtr1 + halfW;
register int32_t x, y, yadj, xadj, gadj;
yadj = (_rawInfo.cfa_pattern == 0x01000201) ? 1 : 0;
xadj = (_rawInfo.cfa_pattern == 0x01020001) ? 1 : 0;
gadj = (_rawInfo.cfa_pattern == 0x02010100) ? 0 : 1;
for (y=0; y<_rawInfo.height; y+=2) {
for (x=0; x<_rawInfo.width; x+=2) {
int32_t r = get_raw_pixel(&_rawInfo, _rawBuffer, x+xadj, y+xadj);
int32_t g1 = get_raw_pixel(&_rawInfo, _rawBuffer, x+1-gadj, y);
int32_t g2 = get_raw_pixel(&_rawInfo, _rawBuffer, x+gadj, y+1);
int32_t b = get_raw_pixel(&_rawInfo, _rawBuffer, x+1-xadj, y+1-yadj);
uint8_t r8 = raw_to_8bit_linear(r, &_rawInfo);
*redPtr = r8;
uint8_t b8 = raw_to_8bit_linear(b, &_rawInfo);
*bluePtr = b8;
uint8_t g18 = raw_to_8bit_linear(g1, &_rawInfo);
*greenPtr1 = g18;
uint8_t g28 = raw_to_8bit_linear(g2, &_rawInfo);
*greenPtr2 = g28;
redPtr++;
bluePtr++;
greenPtr1++;
greenPtr2++;
}
greenPtr1 += halfW;
greenPtr2 += halfW;
}
CGImageRef redImageRef = CGBitmapContextCreateImage(redBP);
NSImage* redImage = [[NSImage alloc] initWithCGImage:redImageRef];
[[redImage TIFFRepresentation] writeToFile:@"~/raw_red.tiff" atomically:YES];
CGImageRef blueImageRef = CGBitmapContextCreateImage(blueBP);
NSImage* blueImage = [[NSImage alloc] initWithCGImage:blueImageRef];
[[blueImage TIFFRepresentation] writeToFile:@"~/raw_blue.tiff" atomically:YES];
CGImageRef greenImageRef = CGBitmapContextCreateImage(greenBP);
NSImage* greenImage = [[NSImage alloc] initWithCGImage:greenImageRef];
[[greenImage TIFFRepresentation] writeToFile:@"~/raw_green.tiff" atomically:YES];
So far I only found focus pixels in the red and blue channel.
-
Interesting stuff. I have never seen a single frame that displays all of the focus pixels. Could you show us what the tiff files look like? So you're using Photoshop to remove the focus pixels by changing their color?
If you skimmed through this topic you'll find out that there are basically just two different patterns. The EOSM/650D/700D all share the same pattern and the 100D is different. I suspect that the EOSM2 shares the same pattern as the 100D because it is using the same sensor. Of course there's no ML port for that camera so it is just theoretical.
By the way, since you have an EOSM have you tried the crop rec module with it? It was a challenge figuring out the focus pixels for that video mode.
-
Here are the red and blue tiffs:
R: https://www.dropbox.com/s/3430fvt3dufjlx3/1600x540_red.tiff?dl=0 (https://www.dropbox.com/s/3430fvt3dufjlx3/1600x540_red.tiff?dl=0)
B: https://www.dropbox.com/s/ffzkeucgwcgbcul/1600x540_blue.tiff?dl=0 (https://www.dropbox.com/s/ffzkeucgwcgbcul/1600x540_blue.tiff?dl=0)
and here's the corresponding MLV:
https://www.dropbox.com/s/ro5n385c29xtjxz/1600x540.MLV?dl=0 (https://www.dropbox.com/s/ro5n385c29xtjxz/1600x540.MLV?dl=0)
I found out that you basically need to record one map for fullframe mode and one for crop mode as the pixel layout seems to differ. The map for a particular resolution can be deferred from the map with the highest resolution as the frame is centered around the sensor center point. I'll implement a correction function in the next days that'll take the pixel map and interpolate the bad pixels. Should work.
-
Thanks for the files, let's take a look.
Here is a frame of the MLV with the vibrance and saturation pushed to show the focus pixels. Never mind the color it is most likely way off.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F4%2F3859%2F33218738255_f29582b342_z.jpg&hash=8f60cbcbf7d62f33a7b71190ef18b2f0)
From the looks of it this was shot in mv720 mode. The EXIF data shows that you used an EOSM.
Here is your red tiff--
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F1%2F610%2F33218736005_32d1113f0b_z.jpg&hash=43fd9738fecd8dfa8995e8a7fda25927)
and your blue tiff.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F1%2F649%2F33062132172_c24004377f_z.jpg&hash=13a4ecd7200b7b7a3acadc3420845cac)
The first thing I did was to put those two tiffs in Photoshop layers to see how they line up. It doesn't seem right because the pixels are adjacent to each other in the x-axis.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F1%2F667%2F33062173662_e038d7b62b_z.jpg&hash=12d5cb3709df4db14bab17d55f100ce9)
I don't know how that happened but if you take a close look at the focus pixels in your shot you'll see that they are offset diagonally.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F4%2F3724%2F33062131762_1c4f43f674_z.jpg&hash=72cdb62f175dfc10ed6edb29147c8b1c)
The pixel map that is working in MLP, cr2hdr.app, MLVFS and MLV Producer uses a pattern that hits all of the known focus pixels. Here it is superimposed over your image:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F4%2F3904%2F33090851191_eb2c8e7ba3_z.jpg&hash=a339d9ee2a36c47cc9f8fc45d3376517)
When your shot is run through one of the apps that can deal with the focus pixels using this map file, the focus pixels do not appear in the final image.
Another thing to point out is that your adjustments might work for that one shot as long as the lighting doesn't change. There are a lot of focus pixels that are not represented in your tiff files. I too had that cross shaped pattern for a while until a user showed me that there are focus pixels outside of that area (http://www.magiclantern.fm/forum/index.php?topic=16054.msg161436#msg161436). As far as I can determine the focus pixels on the EOSM/650D/700D have a pattern that looks like this:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F4%2F3845%2F33090853571_2d10d6f2ac_z.jpg&hash=5f5437b6be87056b46bf4e9e4608dc0b)
That map will cover all of the possible focus pixels that might appear in all lighting situations.
What would really help is a better way to blend the pixels. Your idea of changing the colors is interesting, are you interpolating just one color channel surrounding a focus pixel?
A problem with blending surrounding pixels comes up when there are high contrast sharp lines like in this shot from a 100D:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F8%2F7730%2F27137030895_377d6cfb5a_z.jpg&hash=cbbe7edd8b5a21745ede22e742394edf)
I'm not sure what will work for situations like this. What helps is to combine chroma smoothing along with the focus pixel map but even that isn't enough in some cases.
I found out that you basically need to record one map for fullframe mode and one for crop mode as the pixel layout seems to differ. The map for a particular resolution can be deferred from the map with the highest resolution as the frame is centered around the sensor center point.
If you read through this topic you'll find out that the maximum area is represented by the full raw buffer which you can capture using the silent picture module. MLV files have crop metadata which shows where on that full raw buffer the image is located. Don't assume that it is centered.
There are more than those two modes, though most EOSM users only use those two. There's mv1080 (not available on the EOSM unless you also record H.264), mv720, mv1080crop and zoom mode. The zoom mode is interesting because it uses a much larger full raw buffer and the image can move around the sensor. There is a "digital dolly" option that can actually move around the sensor dynamically so you need to look at the crop metadata for every frame. Oh, and with the experimental crop_rec module there are even more possibilities.
You are more than welcome to look through my focus pixel bitbucket project (https://bitbucket.org/daniel_fort/ml-focus-pixels). The fpm.sh script has some comments you might find useful. Improvements are always welcome.
By the way, @bouncyball -- if you read this message please push those changes that added the -n switch to the script.
-
...By the way, @bouncyball -- if you read this message please push those changes that added the -n switch to the script.
+1
-
...By the way, @bouncyball -- if you read this message please push those changes that added the -n switch to the script.
Done :)
-
Thanks!
That -n switch shows the uncropped raw buffer. Well, almost. It doesn't crop the upper left corner but it does crop the lower right. I'll look into it.
What it should do is to dynamically create files identical to the static map files used in MLVFS.
-
@dfort
Thank you for your summary. That was very helpful. I was able to extract all the focus pixels now using the silent module. The pixels for red and blue also now line up.
A problem with blending surrounding pixels comes up when there are high contrast sharp lines like in this shot from a 100D:
I have the same problem on my EOS M. It comes up when using Apple's Raw Engine and when using the Adobe one. Though I only tested 10 bit mode. Do you have this problem in 14 bit mode as well?
-
probably need chroma smoothing for all bit depths. then sharpen.
-
probably need chroma smoothing for all bit depths. then sharpen.
Could also be that it has something to do with fullframe mode vs crop mode. Does fullframe mode scale the channels in camera? That could also result in pixel components being off slightly. I noticed that crop mode does not result in a stretched image. I'll test that out tomorrow.
-
Full frame skips pixels in a 3x3 pattern for mv1080 and 3x5 in mv720 (default for EOSM) while the crop modes, mv1080crop and zoom, don't skip any pixels so they are outputting a 1x1 pattern but of course a smaller area of the sensor. There's some confusion with the crop_rec module because on the EOSM it uses the mv720 full raw buffer but with the 3x3 pattern of mv1080. Yes, it can get confusing.
-
That -n switch shows the uncropped raw buffer. Well, almost. It doesn't crop the upper left corner but it does crop the lower right. I'll look into it.
Yes, It needs more investigation at least changes for detecting new crop modes in a better way (adding info to mlv headers etc).
I did it for the reason that in my lately developed (development is not finished yet) version of mlv_dump I used MLVFS pixel correction code which does subtraction of crop(x,y), hence map generated by your script for 700D mlv sample, which I have, was off by that offset. So, I thought that modifying the map generating script itself was more appropriate way to achieve correct result. I did not think about right lower corner though :)
Regards
bb
-
In reality this means we dynamically build each pixel map file for each mlv file and apply mlvfs pixel mapping code to every mlv file rather than relying on static map files? Needed because of the +,-1 height crop issue which sometimes occur breaking the static maps?
-
I don't really understand what do you mean by saying "static" map. AFAIK MLVFS maps are absolute values relative to raw frame buffer x=0 and y=0, no crop/pan offset applied. To alter pixels of visible rectangle only, that offset subtracted by pixel correction code in MLVFS. Daniel's script does the same and that's the reason of -n switch requirement - to subtract offset only once not twice.
-
My focus pixel correction is working:
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi.share.pho.to%2Fa471fff5_l.jpeg&hash=8b6a1dd496c49fe3e410d3bb7806e098)
Thank you for your support.
-
@martinhering: really like your software, so sad it's not open source (yet?) :)
-
@martinhering: really like your software, so sad it's not open source (yet?) :)
I couldn,t agree more. This community is based around sharing sources. It,s what makes it growing.
-
I don't really understand what do you mean by saying "static" map. AFAIK MLVFS maps are absolute values relative to raw frame buffer x=0 and y=0, no crop/pan offset applied. To alter pixels of visible rectangle only, that offset subtracted by pixel correction code in MLVFS. Daniel's script does the same and that's the reason of -n switch requirement - to subtract offset only once not twice.
What I mean by "static" map is that it is a stand alone file that maps the location of all the focus pixels of a full raw buffer. When an application uses these "static" files it needs use the crop and image size metadata in order to create a new map file for the MLV file being processed. My bash script creates these map files dynamically so it doesn't wrangle a bunch of data files. The down side to this is that it creates the map files using a formula so the source code is harder to update, especially when new patterns come up like the ones from the crop_rec module.
I like your -n switch because with it you should be able to create a full raw buffer map file from the script. The issue is that it needs to use the size of the full raw buffer. The way you did it the map file starts at the beginning of the full raw buffer but is cut off.
dmilligan wrote an excellent post (http://www.magiclantern.fm/forum/index.php?topic=16054.msg160249#msg160249) on how this works. Using his illustration, here is what is happening:
(0,0)
____________________________________________
| |^
| ||
| _______________________________ ||
| |\ |^ ||
| | CropPos(x,Y) || ||
| | || ||
| | |Rec |Raw Buffer
| | |Height |Height
| | || ||
| | || ||
| | |V ||
| |___________________________|_ ||
| |<---- Recorded Width ---->| ||
| |V
|_________________________________________|_
|<----------Raw Buffer Width------------->|
The "static" maps use the Raw Buffer Width and Height and the apps using these map files crop it down to the Recorded Width and Height. What the -n switch is currently doing is this:
(0,0)
___________________________________
| | ^
| | |
| |___ |
| |^ |
| || |
| || |
| |Rec Raw Buffer
| |Height Height
| || |
| || |
| |V |
|_________________________________|_ |
|<---- Recorded Width ---->| |
V
<----------Raw Buffer Width------------->
From what I've seen from your latest mlv_dump that deals with focus pixels is that you are using external (shall we call them "static" ?) map files, one per MLV file. In most cases that's fine but if you have a shot that uses the "digital dolly" where the image area moves around the raw buffer during the shot it won't work. You should be checking the crop on every frame and if there's a change you need to adjust for that as you are creating the DNG frames.
One idea that I had to integrate these map files into mlv_dump was to use graphic image files of the full raw buffers in the source code and on compile time these are converted into large arrays. That way you don't need to wrangle external data files or use a formula to create the focus pixel pattern. If a new focus pixel pattern comes up, like with the crop_rec module, you can create a new image file for it in Photoshop or Gimp. I'm not sure how difficult this would be to code--I'm still struggling writing a "Hello World!" program in C.
-
Hey! Thank you for a long and comprehensive post :)
I followed this thread from the mid of 2016 and really appreciate the effort you put into solving of this quite complicated issue.
From what I've seen from your latest mlv_dump that deals with focus pixels is that you are using external (shall we call them "static" ?) map files, one per MLV file. In most cases that's fine but if you have a shot that uses the "digital dolly" where the image area moves around the raw buffer during the shot it won't work. You should be checking the crop on every frame and if there's a change you need to adjust for that as you are creating the DNG frames.
Not exactly.
The mlv_dump version you are referring to is my and Danne's previous experiment with ml-dng branch which is used in MLP right now. Yes, last change I made there was focus map loading. It uses cold pixel interpolation from raw2dng to wash out focus pixels and the map is not "static" but generated by 'fpm.sh' script as usual. Digital dolly shots are indeed broken here.
This time, my latest experiment (http://www.magiclantern.fm/forum/index.php?topic=18975.0=) is based on the MLVFS processing, hence uses "static" maps only and the algorithm is as following:
Code tries to load 'input_mlv_name.fpm', if it does not exist, next step is to load 'camera_id__raw_width__raw_height.fpm' like MLVFS does. If map loaded, code makes sure, whether raw data is dualiso or not (specified by switch), after that, accordingly, horizontal or around pixel interpolation is done, getting edge pixel cases into account (thanks to dmilligan). Digital dolly shots are ok in this case. All pixel coordinates are calculated dynamically for every frame according to panPosX,Y.
One idea that I had to integrate these map files into mlv_dump was to use graphic image files of the full raw buffers in the source code and on compile time these are converted into large arrays. That way you don't need to wrangle external data files or use a formula to create the focus pixel pattern. If a new focus pixel pattern comes up, like with the crop_rec module, you can create a new image file for it in Photoshop or Gimp. I'm not sure how difficult this would be to code--I'm still struggling writing a "Hello World!" program in C.
I really like this idea will think about it.
The best regards
bb
-
which is used in MLP right now.
cr2hdr.app
Digital dolly, anybody ever used that feature?
-
Digital dolly, anybody ever used that feature?
IDK, but it indeed sounds cool :D
-
That feature would probably be a bit more interesting in crop mode with automatic subject tracking :D
It is possible to move the crop window on the fly, though a bit tricky, as it breaks Canon's pattern noise correction. It can probably be corrected in software, and the tracking algorithm will be likely slow, but the hardware can do this.
-
automatic subject tracking
Lock on to a bird or person for instance. Possible, how?
-
For tracking: meanshift/camshift (http://www.docs.opencv.org/master/db/df8/tutorial_py_meanshift.html) or something similar.
For moving the sensor area: alter the image position registers used in crop_rec (in rough increments) and adjust our raw recording area for fine increments.
From crop_rec.c:
/* raw buffer centered in zoom mode */
case CROP_PRESET_CENTER_Z:
cmos_new[1] = PACK12(9+2,42+1); /* vertical (first|last) */
cmos_new[2] = 0x09E; /* horizontal offset (mask 0xFF0) */
break;
-
Would be great! :)
Can this be used to stabilize the image according to camera level sensor data? Well as far as sensor precision and CPU speed can get.
bb
-
Unfortunately, Canon's level sensor is not the best for this purpose.
http://www.magiclantern.fm/forum/index.php?topic=18473
http://www.magiclantern.fm/forum/index.php?topic=13229.0
-
Ah... understood, gyroscope's absent.
-
Since crop_rec came up, has anyone else noticed focus pixels on the EOSM (or my 700D crop_rec build) when shooting H.264 with the crop_rec module active?
-
Yes, always there on eosm. How will other raw streams affect H.264 crop_rec footage?
On a sidenote I noticed some focus pixels with the -n switch when using mlv_dump on steroids processing crop_rec material. Posted test pics in bouncyball,s thread.
-
...has anyone else noticed focus pixels on the EOSM (or my 700D crop_rec build) when shooting H.264 with the crop_rec module active?
Yes, I can confirm this w EOSM.
On a sidenote I noticed some focus pixels with the -n switch when using mlv_dump on steroids processing crop_rec material. Posted test pics in bouncyball,s thread.
Good catch, Danne!
-
Trying to figuring out the -n switch and why some focus pixels aren,t covered. Using the script without the -n switch I notice the first nine coordinates aren,t created like when running without it. Check these two examples.
https://bitbucket.org/Dannephoto/magic-lantern/downloads/pixelmaps.zip
I try to understand by checking the script but couldn,t figure out what is going on.
-
@dfort: would you mind to upload to your bitbucket download area all original raw buffer .PBM files made/used by you to generate .fpm maps for MLVFS?
-
Referring to these files?
https://bitbucket.org/daniel_fort/ml-focus-pixels/downloads/2016-02-18_raw_buffer_fpm_files.zip
Or here
https://bitbucket.org/daniel_fort/ml-focus-pixels/src/4a431cfc9748af34056cca702bd4f421cd54fbcd/focus_pixel_map_files/?at=default
-
@Danne:
Nope, they are text maps. I need bitmaps in the PBM format which where used to generate these FPMs.
I can generate PBMs from the existing fpms with script but I'm interesting in original files saved from graphical software.
-
Got it. Different time zone here and about to start a busy day but managed to add graphic files in two formats to the repository. PBM P1 (plain text) and PBM P4 which is more common in graphics programs. Either format should open fine for you. I'll explain more about these files later.
focus_pixel_image_files (https://bitbucket.org/daniel_fort/ml-focus-pixels/src/4ae95c03935d9ef657146f5df354f0bb12426e94/focus_pixel_image_files/?at=default)
-
@dfort: would you mind to upload to your bitbucket download area all original raw buffer .PBM files made/used by you to generate .fpm maps for MLVFS?
You got it. Just some notes about this. In my repository you'll find a couple of bash scripts that will convert between fpm and pbm named fpm2pbm.sh and pbm2fpm.sh. These run very slow but do the job. a1ex wrote an octave script named fpm2pbm.m that will convert from fpm to pbm much faster but it requires setting up octave along with a graphics library. The pbm format used in these scripts is known as P1 or "Plain PBM" and is about the simplest graphics image format possible but also very inefficient because it is saved as a plain text file. Most image file editing programs that can work with pbm files will save in a P4 binary format which is also very simple but the file size is considerably smaller. It is easy to convert between these formats with ImageMagick. I saved pbm files in both formats for you. So as you can see these focus pixel map files can be represented by either a text file with the coordinates of every known focus pixel or as an graphic image file.
There are only two sensors that are causing these focus pixels. The EOSM/650D/700D all use a sensor that creates one pattern which I called the "A" pattern in my script and the 100D creates a different pattern which I named "B".
Here's a look at the crop_rec pattern A image file:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F1%2F716%2F32513024163_10c7f34dcc_c.jpg&hash=7b7e6cc09777fdc1ddc32d6596d96dfb)
Note that the image is extracted from the full raw buffer and there is an area on the top and left of the full raw buffer that is always blank so I didn't map any focus pixels in that area.
One thing that I haven't experimented with yet is removing focus pixels on simple silent shots. There are no focus pixels with Full Resolution Silent Pictures but when shooting simple silent it captures the raw buffer used in movie capture and it does show focus pixels.
-
Here are the focus pixel maps, I extracted so far:
https://www.dropbox.com/s/e42a015uoeabbv4/Focus-Pixels.zip?dl=0
Seem to work great with silent pictures and any video I came across yet. I am also trying to detect which pixel map to apply automatically, which seems to work so far:
if (_idntInfo.cameraModel == kMLVCameraModelEOSM) {
FTRawImageFocusPixelsType type = kFTRawImageFocusPixelsTypeEOSM;
NSInteger rawBufWidth = (block.cropPosX > 0) ? 80 + rawInfo->width + (block.cropPosX-80)*2 : rawInfo->width;
if (rawBufWidth == 1808) {
type |= kFTRawImageFocusPixelsType1808x728;
}
else if (rawBufWidth == 1872) {
type |= kFTRawImageFocusPixelsType1872x1060;
}
[_rawImage fixFocusPixelsWithType:type withCropX:block.cropPosX: block.cropPosY];
}
else if (_idntInfo.cameraModel == kMLVCameraModel100D) {
FTRawImageFocusPixelsType type = kFTRawImageFocusPixelsType100D;
NSInteger rawBufWidth = (block.cropPosX > 0) ? 64 + rawInfo->width + (block.cropPosX-64)*2 : rawInfo->width;
if (rawBufWidth == 1808) {
type |= kFTRawImageFocusPixelsType1808x1190;
}
else if (rawBufWidth == 1872) {
type |= kFTRawImageFocusPixelsType1872x1060;
}
[_rawImage fixFocusPixelsWithType:type withCropX:block.cropPosX: block.cropPosY];
}
I have yet to test the 650D and 700D though.
-
You got it. Just some notes about this. In my repository you'll find a couple of bash scripts that will convert between fpm and pbm named fpm2pbm.sh and pbm2fpm.sh. These run very slow but do the job. a1ex wrote an
There are only two sensors that are causing these focus pixels. The EOSM/650D/700D all use a sensor that creates one pattern which I called the "A" pattern in my script and the 100D creates a different pattern which I named "B".
Thank you so much for files.
Yes I know all about PBM (raw/ascii) formats. I need P4, P1 is a huge overkill. Yesterday, I also used all your scripts there and experienced lovely time wasting moments of my life :P. a1ex's fpm2bpm.m works, but octave is a real canon from the days of civil war vs pneumatic pistol we actually need to do the job :).
I've found out about "A" and "B" patterns from your script as well. Love this script and the idea to generate the maps on the fly by known patterns.
Nice to have full frame buffer map images. GIMP is great to work with both formats of PBM.
Thanks again
bb
-
@martinhering: You are making real progress every day! :)
-
Does anybody know what actually causes these Focus Pixels? Is it the sensor or the processor? And why do only the models with the DIGIC 5 processor show these?
-
Not a CPU issue. Sensor has them all phisically on the surface. Canon firmware, while reading raw data from the sensor, does processing and according to some register values gives number of so called (processed/unprocessed) raw streams. In one stream there may be maped out all bad pixels, in another, vertical stripe correction applied (or both) and so on. There can/should be at least one stream with all focus pixels maped out but no one knows how to make it working at the moment :).
On affected cameras, for MLV/SilentPic recording ML uses LV raw buffer with the stream which contains those pixels in it, hence they have to be removed in post.
-
Right, what @bouncyball said.
For an illustration of this, check out this post (http://www.magiclantern.fm/forum/index.php?topic=16054.msg176032#msg176032) from a while back. Thought I found the Holy Grail but it turned out that the cure turned to be worse than the disease. (I sometimes throw around these technical terms when communicating with developers.) Ok, kidding aside there might a video stream that doesn't show these focus pixels but we haven't found it yet.
I took another look at the @martinhering focus pixel files and I'm still scratching my head at how these work because some of the maps have pixels next to each other on the horizontal axis, others are missing sections and still others have focus pixels mapped in areas that we know there can't be any focus pixels because that it is an inactive area of the raw buffer. There are also some video modes that are missing, like zoom mode which has a full raw buffer size of 2592x1108 and can move around the sensor like this:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F2%2F1475%2F24510484991_4ab3b20762_c.jpg&hash=b7eab00adb405e0acc6f4543a4a89104)
That caused all sorts of issues when I tried coming up with a focus pixel map file that would take care of all possible raw buffer locations. I ended up just stacking all 21 raw buffers in Photoshop. This might be overkill because if the raw buffer happens to be at the top or bottom of the sensor it might be in an area that doesn't have any focus pixels but since we don't know the position of the raw buffer on the sensor we need to account for all possible locations.
Another challenge is with the crop_rec module. On the EOSM (and my experimental version for the 700D) it uses the mv720 full raw buffer (1808x727) with a 3x3 pixel skipping pattern and the focus pixels that show up on both the mv720 and mv1080 video modes appear on the image. I thought that using a combo map would work for both the crop_rec and mv720 video modes but after extensive testing with Danne, we found out that it doesn't work that way and we always ended up with at least some focus pixels showing up on crop_rec footage like in this example:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F1%2F435%2F32408373032_073a99b365_z.jpg&hash=ab1f0d2247416b0ac1b43f89f99a912d)
Adding some Chroma Smoothing helps get rid of these in raw video but crop_rec H.264 footage also shows focus pixels and removing these in post production is even more challenging.
I have yet to test the 650D and 700D though.
I've got a 700D and can provide you with test footage for all the video modes. I believe I read somewhere that you needed a blank wall at a certain exposure, could you please post exactly how you would like the tests shot?
-
I'm still scratching my head at how these work because some of the maps have pixels next to each other on the horizontal axis, others are missing sections and still others have focus pixels mapped in areas that we know there can't be any focus pixels because that it is an inactive area of the raw buffer. There are also some video modes that are missing, like zoom mode which has a full raw buffer size of 2592x1108 and can move around the sensor
I actually bought a used EOS M on ebay and did some silent pictures, and yes, I mapped out some pixels in the inactive area, I will correct that. You're right. I only checked fullframe and crop mode yet, since I could not find the other modes. The settings are kinda convoluted. I'll check the crop_rec module though and see if I can find other circumstances that produce focus pixels pattern I am not taking care of yet. Were can I set up "Zoom Mode"? I could not find the freakin setting. Also dolly mode on the EOS M is really unusable. It uses buttons that are normally used for other stuff. So I guess that this is not something that is in wide spread use.
I ended up just stacking all 21 raw buffers in Photoshop. This might be overkill because if the raw buffer happens to be at the top or bottom of the sensor it might be in an area that doesn't have any focus pixels but since we don't know the position of the raw buffer on the sensor we need to account for all possible locations.
This does not sound correct. Is this just a thing with dolly mode?
I've got a 700D and can provide you with test footage for all the video modes. I believe I read somewhere that you needed a blank wall at a certain exposure, could you please post exactly how you would like the tests shot?
I think you said that 700D, 650D and EOS M experience the same patterns, so I would just need one file in every mode to confirm that. Already thinking of buying more camera on ebay :-) If you could provide the silent pictures, that would be awesome. BTW do you have a Mac? Can you test my app?
-
One more thing:
What would really help is a better way to blend the pixels. Your idea of changing the colors is interesting, are you interpolating just one color channel surrounding a focus pixel?
A problem with blending surrounding pixels comes up when there are high contrast sharp lines like in this shot from a 100D:
I'm not sure what will work for situations like this. What helps is to combine chroma smoothing along with the focus pixel map but even that isn't enough in some cases.
Yes, I am correcting focus pixels on the actually raw buffer itself and I am just interpolating surrounding pixels from the same channel. About the false colors in sharp lines, I read some stuff about it and I think that this is a problem with the demosaicing itself. Hence Amaze which reduces aliasing. Since I cant change the demosaicing method of the Apple RAW engine at the moment, I might be able to find a solution in post. Something like chroma smoothing in areas around edges. I think it should be possible to come up with some GPU code.
-
The way you get into the various modes are:
mv1080 - Canon menu at 1920x1080 -- raw buffer size is 1808x1190 - 24, 25, 29.97fps - EOSM is unique in that it won't go into this mode unless it is recording H.264 video (http://www.magiclantern.fm/forum/index.php?topic=16608.msg178438#msg178438).
mv720 - Canon menu at 1280x720 -- raw buffer size is 1808x727 - 50, 59.94fps - EOSM is unique because it defaults to 29.97fps.
mv1080crop - Canon menu at 1920x1080, ML Movie crop mode -- raw buffer size is 1872x1060 - 24, 25, 29.97fps
zoom - Canon menu at 1920x1080, press magnify button to get to 5x LiveView -- raw buffer size is 2592x1108 - 24, 25, 29.97fps. Tricky on the EOSM because the magnifying glass button is in one of the LiveView layouts. You can actually record at about 2496x1078 in this mode but limited by the SD card hardware.
crop_rec - Canon menu at 1280x720, crop_rec module (from experimental builds) set to 3x3 720p -- raw buffer size is 1808x727 - 50, 59.94fps - EOSM at 29.97fps.
I believe you wanted a blank white wall no clipping, right? Here are silent DNG's from the 700D:
https://www.dropbox.com/sh/fvjgwdy9yyeycic/AAAF44SlZ7S642WQGRZJX5Dya?dl=0
You might want to shoot a black frame on your EOSM and push it as far as you can in ACR. It will be noisy but you'll see a different focus pixel pattern.
I'm on a Mac and have taken a look at your program. I've been a bit too busy on several projects lately to run any tests on it.
[EDIT] I never had much luck with digital dolly on the EOSM. It seems to work best on the 5D3 though I haven't played around with it on that camera. As far as zoom mode and digital dolly, there are 21 raw buffer positions on the sensor and no indication of which one is active so it is a challenge figuring out where the focus pixels are located. What I did might be overkill but it seems to be working for the few people that figured out how to use zoom mode on the EOSM/100D/650D/700D.
-
Thanks for the info. I could get some footage on the EOS-M in zoom mode and and made some new pixel maps:
http://fs.footageformac.com/index.php?dir=focus_pixels/pix_maps
For the mode auto-detection I did this to cope with the crop rect in zoom mode:
FTRawImageFocusPixelsType type = kFTRawImageFocusPixelsTypeEOSM;
NSInteger rawBufWidth = (block.cropPosX > 0) ? 80 + raw_info->width + (block.cropPosX-80)*2 : raw_info->width;
NSInteger rawBufWidthZoomRecording = (block.cropPosX > 0) ? 128 + raw_info->width + (block.cropPosX)*2 : raw_info->width;
if (rawBufWidth == 1808) {
type |= kFTRawImageFocusPixelsType1808x728;
}
else if (rawBufWidth == 1872) {
type |= kFTRawImageFocusPixelsType1872x1060;
}
else if (rawBufWidth == 2592 || rawBufWidthZoomRecording == 2592) {
type |= kFTRawImageFocusPixelsType2592x1108;
}
Seems to work.
I could not check your silent pictures, since I don't have a DNG decoder yet. I always take the silent pictures in MLV format. Can you do some MLV silent pictures with your 700D in all modes? Thank you.
I could not get mv1080 mode working on EOS-M. Recording in H.264 doesn't make sense obviously. And silent pictures either return mv720 or mv1080crop mode buffers.
-
Those new pixel maps are not covering all the possible focus pixels. Believe me, I've been there (http://www.magiclantern.fm/forum/index.php?topic=16054.msg161436#msg161436).
I could not check your silent pictures, since I don't have a DNG decoder yet. I always take the silent pictures in MLV format. Can you do some MLV silent pictures with your 700D in all modes? Thank you.
Sure -- uploaded to that same dropbox folder. Get dcraw -- we're big on open source software here.
I could not get mv1080 mode working on EOS-M. Recording in H.264 doesn't make sense obviously. And silent pictures either return mv720 or mv1080crop mode buffers.
Who says this makes sense? You can shoot silent stills while recording H.264. It might help if you use this branch:
https://bitbucket.org/hudson/magic-lantern/branch/raw-h264-proxy
-
Those new pixel maps are not covering all the possible focus pixels. Believe me, I've been there.
OK, since it is hard to get a good shot that covers all focus pixels, I deal with that once I have footage that needs better pixel maps. So far, I am satisfied with the result.
Sure -- uploaded to that same dropbox folder.
These helped a lot. Thank you. I am using the same pixel maps for 650D, 700D and EOS-M now. Seems to work.
http://fs.footageformac.com/index.php?dir=focus_pixels/pix_maps (http://fs.footageformac.com/index.php?dir=focus_pixels/pix_maps)
The crop-rec module gives the same raw buffer size than mv720 mode, I think, so it's kinda hard to come up with an auto-detection. But since the module is currently experimental, I am postponing this issue as well.
-
OK, since it is hard to get a good shot that covers all focus pixels, I deal with that once I have footage that needs better pixel maps. So far, I am satisfied with the result.
At some point that is going to bite you. Here is what happened to me--I too was satisfied that I got all the focus pixels covered when @Steve_FR sent in this shot:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F2%2F1642%2F24640822142_552876184e_z.jpg&hash=34b53869b8f312124857aabcf27777d1)
Applying the focus pixel map that I was satisfied with at the time resulted in this:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F2%2F1520%2F24390966089_dd13de3175_z.jpg&hash=29c2f195e48f85da47de17632a06e92b)
After adjusting my focus pixel map the focus pixels were gone:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F2%2F1573%2F24484033550_64720e2391_z.jpg&hash=7a41c490a54edf62d09135ff0f0346f3)
Ok--this shot won't make it into the "Year in Pictures" special issue but note that it did remove the focus pixels without affecting the overall texture of the sensor noise. The brightness shift was because I was playing around with the settings to make sure I got the areas that weren't covered by the old map.
Separating focus pixels into red and blue maps is intriguing. Does it work in all cases? Reason I ask is because it seems that the pixels change colors depending on the lighting. Here's an example from the edge of a frame from the 100D. Note that this is also showing focus pixels outside of the area that you are covering with your maps.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F2%2F1649%2F24653928396_27b5e8aa48_z.jpg&hash=bf9381796378d1fa310053a6143fdac9)
I am using the same pixel maps for 650D, 700D and EOS-M now. Seems to work.
Yes, that's all covered in this topic and in my ML Focus Pixels bitbucket repository (https://bitbucket.org/daniel_fort/ml-focus-pixels). Theoretically the 100D shares the same pixel map as the EOSM2 because they use the same sensor but the M2 hasn't been ported to ML and probably never will because it was not a very popular camera and distribution was mostly limited to the Asian market. Ok, maybe that was too much irrelevant information--there are only two main focus pixel patterns we need to deal with, one I call the A pattern for the 650D/700D/EOSM and and the other I call the B pattern for the 100D and possibly EOSM2. However, there have been reports of other cameras showing focus pixels in certain situations. Then again maybe there is a way to "fix" these cameras with focus pixel issues by adjusting the ADTG/CMOS registry as suggested by @Greg in this post:
http://www.magiclantern.fm/forum/index.php?topic=16054.msg167428#msg167428
The crop-rec module gives the same raw buffer size than mv720 mode, I think, so it's kinda hard to come up with an auto-detection.
Yes, that's another subject that was discussed. This is especially important with the mv720 raw buffer. As you know the "normal" mv720 uses 3x5 line skipping or binning and the aspect ratio needs to be adjusted. The crop_rec module also uses the mv720 raw buffer but uses 3x3 line skipping or binning so it doesn't need the aspect ratio adjustment. We probably need to ask @g3gg0 to add a new structure to the MLV format specifications.
https://docs.google.com/spreadsheets/d/1ItNuLj34JlK6bZgiAJLrmG7p3iyCmCju4XYyiwjqJOM/edit#gid=1
Then of course the ML source code needs to be modified to use the new structure. Note that there is another new structure proposed to support long lens names for the Non-CPU lens info branch. More about that project here:
http://www.magiclantern.fm/forum/index.php?topic=18083.0
One last note -- have you looked into Dual ISO (http://www.magiclantern.fm/forum/index.php?topic=7139.0) support for your app? Fixing focus pixels on Dual ISO is especially tricky. Read through the Dealing with Focus Pixels in raw video topic to find out why.
-
Why not just use your fine maps dfort and all is fine straight away. GPL style of course. That's how they come to be in the first place anyway.
-
Why not just use your fine maps dfort and all is fine straight away.
Well it would be good if someone else validates my focus pixel map files. Separating them into blue and red map files is a novel idea and maybe it is a better way of dealing with focus pixels.
Of course the best solution of all is to eliminate them in camera and maybe there is some focus pixel free video stream somewhere that we haven't found yet. Funny thing is that we seem to have found a way to put focus pixels into H.264 via the crop_rec module so that might provide a clue?
-
Separating focus pixels into red and blue maps is intriguing. Does it work in all cases? Reason I ask is because it seems that the pixels change colors depending on the lighting.
Well I did not find any downsides yet. I also think that separating the channels for correction yields better image quality, since you are only repairing 1 pixel and not its surroundings. I played a bit with the interpolation matrix and I think an average of the 8 surrounding pixels is good enough, but you need to make sure that the surrounding pixels are not dead pixels. I just leave them out and average the remaining pixels.
Here's an example from the edge of a frame from the 100D. Note that this is also showing focus pixels outside of the area that you are covering with your maps.
Do you still have the MLV? The reason, why I use the MLV is, that I can separate the components and save a grayscale image for every component (I could also do this with DNGs, but I dont have a decoder yet). I can then use the "Threshold" tool in Photoshop to extract the bad pixels. Very simple. I save the BW image as an 8bit png with 2 colors (2kb max). Then I read in the pixel map and create a table with x and y values for red and blue (there seem to be no focus pixels in green) and iterate over the table and fix the pixels. This way I can always simply open the BW png in PS and map out new pixels on the go and never need to change the code. Very fast turnaround. I can also use component images of new MLVs and overlay the existing pixel map and map out new pixels this way. Very easy.
One last note -- have you looked into Dual ISO support for your app? Fixing focus pixels on Dual ISO is especially tricky. Read through the Dealing with Focus Pixels in raw video topic to find out why.
Not yet. I am also postponing that. I first need to give my app some more useful features like histogram and RGB parade, thumbnail grid, etc.
-
Do you still have the MLV?
No, I did the 100D tests a while ago with a borrowed camera and deleted most of the files to make space on my drive for other projects. You can very easily find many of the focus pixels that are not the pixel map files that you created by shooting a frame with the lens cap on. Then adjust the contrast, brightness, saturation and whatever else you've got on that dark and noisy frame and you should see something like this:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fc1.staticflickr.com%2F6%2F5664%2F22731147105_b7a7860b47_z.jpg&hash=8def116aa87a64e675c47abe3cb51953)
Like I said before, I have never seen all of the focus pixels show up on a single shot so if you created your map files from a single exposure you will eventually have someone come to you with a shot where your app didn't removed all of the focus pixels.
-
@dfort
You can very easily find many of the focus pixels that are not the pixel map files that you created by shooting a frame with the lens cap on.
I bought a used 700D of eBay and could verify this with some dark frames and I updated my pixel maps. Also had to update my auto-detection code for the 700D.
NSInteger rawBufWidth = 0;
NSInteger rawBufHeight = 0;
NSInteger rawBufWidthZoomRecording = 0;
switch (_idntInfo.cameraModel) {
case kMLVCameraModelEOSM:
type = kFTRawImageFocusPixelsTypeEOSM;
rawBufWidth = (block.cropPosX > 0) ? 80 + raw_info.width + (block.cropPosX-80)*2 : raw_info.width;
rawBufHeight = (block.cropPosY > 0) ? 10 + raw_info.height + (block.cropPosY-10)*2 : raw_info.height;
rawBufWidthZoomRecording = (block.cropPosX > 0) ? 128 + raw_info.width + (block.cropPosX)*2 : raw_info.width;
break;
case kMLVCameraModel650D:
type = kFTRawImageFocusPixelsType650D;
rawBufWidth = (block.cropPosX > 0) ? 64 + raw_info.width + (block.cropPosX-64)*2 : raw_info.width;
rawBufHeight = (block.cropPosY > 0) ? 26 + raw_info.height + (block.cropPosY-26)*2 : raw_info.height;
break;
case kMLVCameraModel700D:
type = kFTRawImageFocusPixelsType700D;
rawBufWidth = (block.cropPosX > 0) ? 64 + raw_info.width + (block.cropPosX-64)*2 : raw_info.width;
rawBufHeight = (block.cropPosY > 0) ? 26 + raw_info.height + (block.cropPosY-26)*2 : raw_info.height;
break;
case kMLVCameraModel100D:
type = kFTRawImageFocusPixelsType100D;
rawBufWidth = (block.cropPosX > 0) ? 64 + raw_info.width + (block.cropPosX-64)*2 : raw_info.width;
rawBufHeight = (block.cropPosY > 0) ? 26 + raw_info.height + (block.cropPosY-26)*2 : raw_info.height;
default:
break;
}
if (rawBufWidth == 1808) {
if (rawBufHeight > 1000) {
type |= kFTRawImageFocusPixelsType1808x1190;
} else {
type |= kFTRawImageFocusPixelsType1808x728;
}
}
else if (rawBufWidth == 1872) {
type |= kFTRawImageFocusPixelsType1872x1060;
}
else if (rawBufWidth == 2592 || rawBufWidthZoomRecording == 2592) {
type |= kFTRawImageFocusPixelsType2592x1108;
}
I could not verify yet, if it's really working on the 650D. I just hope so.
-
I could not verify yet, if it's really working on the 650D. I just hope so.
Looks like your testers are verifying it (http://www.magiclantern.fm/forum/index.php?topic=18949.msg181618#msg181618) for you. I also had the same experience, the code for EOSM/650D/700D is the same.
I examined files shot on various cameras at different ISO and exposures, scene brightness, etc. and eventually came up with my maps. There were a few holes here and there that I discovered when I wrote a function that would create the focus pixel pattern so eventually all of the possible focus pixel coordinates were mapped. Well, let's just say I'm pretty sure they are all mapped. Looks like you're doing the same thing so I don't understand why the EOSM is different from the 650D/700D/100D -- that doesn't match what I came up with.
-
Looks like you're doing the same thing so I don't understand why the EOSM is different from the 650D/700D/100D -- that doesn't match what I came up with.
The pixel maps as such are not different, but the crop rectangle logic is different. The various buffer modes are centered around a specific point and that point is differently offset. On the EOSM its 16px more to the right and 16 px more up. You only need this logic for reverse calculating the original raw buffer size for the auto-detection. You don't need it for the correction, as cropX/Y parameters are enough for that. Not saying that my maps are perfect yet. I am not confident yet that they are, but so far they are working. I'll update them as needed.
-
...You only need this logic for reverse calculating the original raw buffer size for the auto-detection...
Not sure I follow you on that. The original raw buffer size is included in the MLV RAWI block so why do you need to reverse calculate the size?
-
The focus pixel maps that we came up with on this topic were actually created by shooting several "simple silent" stills using the various video modes. The reason it can't be done with just one DNG is because there are a few different kinds of focus pixels and they become visible in specific lighting conditions. In other words, shooting a silent still with the lens cap on will show one pattern of focus pixels while shooting a medium grey target will show a different pattern. In addition, there are focus pixels that show up as different colors. @martinhering came up with a different method of dealing with these focus pixels using two different maps, one for red and a different one for blue.
-
Now how about that crop_rec focus pixel list for the 100D? Could it be added to fpm.sh?
I need some files. Turn on crop_rec, mlv_rec or mlv_lite and silent. Set the movie recording mode 1280x720 in the Canon menu and on the Magic Lantern menu to MLV and turn on crop_rec. Now leaving the camera on movie mode turn on silent and shoot some simple silent DNG frames. Check out the previous post for suggestions like shooting some varied scenes, dark, light, etc. Pick out a few frames that show lots of focus pixels and send them to me.
To make up the focus pixel maps I need the full raw buffer so sending me movie footage won't do it for me. The DNGs don't show the full raw buffer. For that I needed this tip:
To get the entire raw buffer from a DNG, including OB area: dcraw -4 -E
To get the active area only: dcraw -4 -D
Then comes the fun part, map all of the focus pixels and if a pattern emerges write an algorithm that can recreate that pattern.
-
Here are two files. Tell me if you need more.
https://bitbucket.org/Dannephoto/magic-lantern/downloads/100D_crop_rec_focus_pixel_DNG.zip
I specified:
dcraw -D -E -T INPUT.DNGSeems to work and we get a tiff instead of a pgm file. Output came out all black with -4 so skipped that.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs1.postimg.org%2F2t00gkocen%2FScreen_Shot_2017-10-05_at_11.04.47.png&hash=722a52e520ba392b4dd1198e6e779fea)
Ant farm :)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs1.postimg.org%2F4es1mjti0f%2FScreen_Shot_2017-10-05_at_11.06.45.png&hash=4a8b1ad4d905e516764b4d96a4712121)
-
I specified:
dcraw -D -E -T INPUT.DNGSeems to work and we get a tiff instead of a pgm file. Output came out all black with -4 so skipped that.
The problem with that is that your tiff file is a different size than a pgm file using dcraw -4 -E. This is what I need to work on the pixel map files:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4487%2F37257348780_bcb3d65171_c.jpg&hash=b954d39c2aa8836fc60389878afc8824) (https://flic.kr/p/YLiDLq)
This file is the size of the full raw buffer, 1808x727. The tiff you created trimmed off the out of bounds area and it is 1660x699. Yes, the pgm file looks black but if you adjust the brightness and contrast you can see the detail. Notice how it clearly shows the area that got cropped off.
The good news is that 100D crop_rec isn't showing two focus pixel patterns superimposed over each other like on the EOSM and 700D so this is going to be easier. Now all I need is more hours in a day to get through all my projects.
-
Ant farm :)
More like Fire Ants! :D
-
I see dfort.
dcraw -T -E -4 -b 20 INPUT.DNG
-
Ok--there's more than one way to do this.
I just checked the focus pixel map files for the 100D and it looks like the one for mv720 is a perfect match. Check out the image file labeled pattern_B_mv720_1808x727. The extra pixels on the sides are there because there are some focus pixels in that area that pop up in certain lighting conditions.
So--isn't your Switch app already working with the 100D in crop_rec?
-
Not really. I get this file from your script:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/allbadpixels.map
Tested against this file(turned off cold pixel fix):
https://bitbucket.org/Dannephoto/magic-lantern/downloads/100D_crop_rec.MLV
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs1.postimg.org%2F5949l7an1b%2FScreen_Shot_2017-10-05_at_23.10.08.png&hash=974a301c5ae6c2a606078c780f7605d6)
In your script fpm.sh
# These cameras don't have crop_rec yet. Remove from list once the crop_rec modules is available.
if [ "$camera" == "100D" ] || [ "$camera" == "650D" ]; then video_mode=mv720; fi
;;
1872x10**)
video_mode=mv1080crop
*Seems to work after all. Think my mlv_dump disables also map files when noo cold pixel fix is selected. Are the map files the same size crop_rec and non crop_rec?
-
Are the map files the same size crop_rec and non crop_rec?
mv720 and crop_rec 3x3 use the same full raw buffer size and in the case of the 100D it appears that they also share the same focus pixel map file.
So it is working? Can you show a before and after? I'm at work right now and can't test it.
-
Here is a test. Files developed with mlv_dump_on_steroids(fpmutil) inside Switch and then dcraw to tiff. Also checked agains fpm.sh
Testfiles:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/100D_mv1080p_M06-0739_samples.zip
https://bitbucket.org/Dannephoto/magic-lantern/downloads/100D_crop_rec_M06-0733_samples.zip
At grannys place(mv1080p, reference)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs1.postimg.org%2F1jpfliv2lr%2FScreen_Shot_2017-10-06_at_07.49.01.png&hash=393f29230c8ca3013aabb62fc4f6fdc7)
Checking the lamp for focus pixel residues. Seems fine.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs1.postimg.org%2F8xk4uafy0v%2FScreen_Shot_2017-10-06_at_07.45.54.png&hash=a00cb974425de550a61be671ffdbcd0a)
crop_rec(not sure what´s up with the black extra border. Didn´t show up in earlier build, another issue)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs1.postimg.org%2F6deahnf4jj%2FScreen_Shot_2017-10-06_at_07.47.44.png&hash=a94cb2045910728c927976a3ad4c1c53)
Check the texture residue
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs1.postimg.org%2F11zdwxsmfz%2FScreen_Shot_2017-10-06_at_07.48.26.png&hash=9c268f8dc24f81779c632e0c6bdc3fc6)
More texture
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs1.postimg.org%2F1qsngyfalb%2FScreen_Shot_2017-10-06_at_07.48.31.png&hash=49f2d6d3385988d1549dd7778c307c7d)
A little chroma smooth 2x2 solves the issue
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs1.postimg.org%2F43v9y5tqxb%2FScreen_Shot_2017-10-06_at_07.52.26.png&hash=1dacb5fc1a6933f3b54e0f36e089117c)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs1.postimg.org%2F3ws22q6qmn%2FScreen_Shot_2017-10-06_at_07.52.30.png&hash=bf4052a46f25aa6b3ffad285ca1d4884)
If all this is to be expected then I say it works. If any of the texture handling can be fixed in the script then there´s more work here :P
-
Those areas with residual focus pixels shouldn't be like that. The crop_rec 3x3 frames should be as clean as the mv1080. That black bar is an indicator that something isn't quite right:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4449%2F23677011828_88ffb8b5c0.jpg&hash=6c9ca94551f5520217cebba8691b258e) (https://flic.kr/p/C5fSHS)
You might also check out mv720 because it is using the same map file as crop_rec.
A while back you PM'd me a file from an EOSM that showed focus pixels. Turned out that it was shifted 8-pixels horizontally. In this case the focus pixel map is lining up so it will require some more digging.
Maybe something is going on in the crop_rec_4k branch? Lossless? Reduced bit rate? Something else?
-
Yes maybe. Could you check against this file if it works or not on alternate files and whatnot:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/100D_crop_rec_focus_pixel_DNG.zip
Too many variables for me to do any conclusions. You are the boss here.
-
Checked both of those files and the map file fits it perfectly.
-
Hi guys,
Based on your information and my experiments I've come up with the map which cleans Dann's samle MLV quite acceptably.
Here (http://nic.caucasus.net/fpm/) are sample maps.
All 3 maps are hybrid mv720+mv1080(shifted up by 30 pixels and cropped to mv720 size) maps.
1. 100D_crop_rec.pbm - is the graphical image to see how this map looks like.
2. 80000346_1808x726.fpm - is the standard map which made from this graphical image map.
3. 80000346_1808x726_dual_pass.fpm - this is more interesting one
3rd map coordinates are listed in this order: first mv720 and then mv1080, I called it dual pass map. Actually the interpolation process is linear but logically according to this map, interpolating code does mv720 interpolation first and then mv1080(shifted/cropped) interpolation.
Why I needed this: the focus pixels are so close to each other that interpolation of all pixels produces artifacts (artifacts which I've also seen on 700D Crop rec)
What does this mean: we have 2 available interpolation methods, older and simpler one from raw2dng code and later and more advanced, hence really better one, from cr2hdr/MLVFS. So when using raw2dng method there are some pinkish dot artifacts appear in the image. With MLVFS method there are none of these dots but if you interpolate all pixels in a raw according of their x,y, some horizontal very different color dot rows introduced all over the image.
With this dual pass map all artifacts are gone with better MLVFS interpolation method :)
All this can be tested with Danne's sample MLV.
Latest 'fpmutil' updated in the repository and can generate dual pass map for croprec 100D
regards
bb
Edit: This sample MLV from 100D has no shifted to any direction raw buffer. So all dfort's original calculations fit perfectly.
-
Very nice job Bouncyball. Will update and put it to real use in Switch. Will do some testing first and learn how to use it first.
This dual pass map should be used with eosm and 700D too right not only 100D?
-
Nice stuff, BB and Thanks for sharing your new dual pass map! :)
-
Actually this map is very simple. The coordinates go for full mv720 pattern first and then for mv1080 cropped to mv720 size and shifted up by 30 pixels. No special usage needed.
This kind of map is generated for 100D crop rec only. Have to look if this approach will help to use MLVFS averaging for 700D without artifacts.
Would be very interesting your opinion @Danne and @dfort. Please use both interpolation methods and compare with both maps.
-
As soon as I get some energy I'm on it.
-
Cool, thank you :)
-
All this makes sense. The reason we were able to get away with a single map file for the EOSM/700D crop_rec was because the sensor has fewer focusing points. The 100D and probably EOSM2 is jam packed with focus pixels. Here is a "theoretical" map for the 100D I made a while back using what we learned from the EOSM/700D:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4478%2F23682321558_ed0e241ccd_z.jpg&hash=8697c16408b663affc1eafdad8abb55c) (https://flic.kr/p/C5J67W)
and a closer look:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4497%2F23682325878_23c6f3a88f_z.jpg&hash=234a4a14bf61b58ab65d54e268d8ce55) (https://flic.kr/p/C5J7pq)
Reminds me of Space Invaders.
Since some of the focus pixels are adjacent to each other the 100D will probably clean up better using two passes as bouncyball discovered.
-
Reminds me of Space Invaders.
Yes :)
This (http://nic.caucasus.net/fpm/?dir=&download=100D_crop_rec.pbm) map also looks like you mentioned above.
-
Ok, 2 pass crop_rec mv720/mv1080 operation implemented into Switch. It works exactly as expected with perfect results. Great work Bouncyball :).
Commit:
https://bitbucket.org/Dannephoto/switch/commits/09b441918cae37cd55e016b7383d290fa30c4c07
Works in conjunction with (ms) mlv_dump_on_steroids in Switch. Still can´t get clean results with fpm.sh script so still need to understand how that can be done to work with (m) mlv_dump in Switch.
Not the nicest footage but still shows the effectiveness of fpmutil.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs1.postimg.org%2F98ou9fjn3z%2FScreen_Shot_2017-10-06_at_22.29.21.png&hash=b58da61a54e62eaa87407841ae4c1c3f)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs1.postimg.org%2F7v2b5e7xxr%2FScreen_Shot_2017-10-06_at_22.30.32.png&hash=5c7f0d96b054b676f90805152ea360c3)
-
This (http://nic.caucasus.net/fpm/?dir=&download=100D_crop_rec.pbm) map also looks like you mentioned above.
Tried your map file in MLVFS and came up with this:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4461%2F37543112211_15f714dc0a_z.jpg&hash=f04225fb0cfc250dca86efa2ff5ab02a) (https://flic.kr/p/Zcygh8)
Using the plain mv720 map file it looks pretty good but it has something Danne called "texture residue" all over it (click to see full sized):
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4514%2F37543114711_b342b821e8_z.jpg&hash=a7acfeee8040848e8d14d63513427b88) (https://flic.kr/p/Zcyh2e)
Finally, a little 2x2 chroma smooth and the MLVFS results look pretty good.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4504%2F37543117081_0a738c3791_z.jpg&hash=3c57c3b51f0f9b37ffdb727c10a10598) (https://flic.kr/p/ZcyhJ6)
I've had to use a little bit of chroma smoothing in several images using MLVFS so this isn't something that's unique to the 100D crop_rec.
-
Strange. I use the map and it comes out like this(totally clean without chroma smooth added):
Loading focus pixel map '80000346_1808x726.fpm'...(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs1.postimg.org%2F8vc0b459tb%2FScreen_Shot_2017-10-07_at_06.14.43.png&hash=d83db74140dad5603e362a7b5c24278e)
crop:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs1.postimg.org%2F8pb2m7803j%2FScreen_Shot_2017-10-07_at_06.24.33.png&hash=58ae82c405ca729960533c7de2be00ea)
Make sure you erase the top text row inside the file. Could be the problem. Noticed MLVFS(in Switch) was hanging otherwise.
#FPM 80000346 1808 726 1 -- fpmutil v0.5
78 86
86 86
...
-
I took bouncyball's image file (http://nic.caucasus.net/fpm/?dir=&download=100D_crop_rec.pbm) and converted it to a focus pixel map file:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4500%2F37543658941_d501f582c6_z.jpg&hash=53805601032ef97fd24d3dfdd7fa1042) (https://flic.kr/p/ZcB4Nv)
By the way, the "theoretical" map file I made a while back sort of works but why complicate things when the mv720 map is working fine--at least in MLVFS.
I'm still lost as to how you are doing that, "2 pass crop_rec mv720/mv1080 operation" - do you need a focus pixel map file that combines mv720 and mv1080 (which is how I made the "theoretical" map) or are you running a different pass with each map file?
Just for the record, it looks like crop_rec is recording a full raw buffer size of 1808x726 while regular mv720 shows a size of 1808x727. In MLVFS simply copy the map file that is in there and rename it to the crop_rec buffer size. Both files will be identical.
80000346_1808x726.fpm
80000346_1808x727.fpm
Yeah, not very efficient having two copies of the same file but that's the way MLVFS is set up. It does make it easier to experiment with new map files.
-
3rd map coordinates are listed in this order: first mv720 and then mv1080, I called it dual pass map. Actually the interpolation process is linear but logically according to this map, interpolating code does mv720 interpolation first and then mv1080(shifted/cropped) interpolation.
Seems all is done in one swoop but the it works the pixels in specified order from the file.
I took bouncyball's image file and converted it to a focus pixel map file:
Amazing there could even be a viewable picture in there.
-
Seems all is done in one swoop but the it works the pixels in specified order from the file.
Yes exactly. The map made from the graphical image will produce the ugly artifacts you see on the samples of dfort. But 2pass map has exactly the same coordinates but they are ordered differently in the fpm list and this works as expected. 1st goes all mv720 coordinated and then mv1080 (but cropped to mv720 size and shifted up by 30 pixels).
Honestly it's so simple! You could do this just by adding mv1080 coordinates at the end of the mv720 fpm (in this case you'll just end up with a lot of unused, out of the buffer, pixels because mv720 buffer is lot smaller then mv1080) and it would work except upper 30 pixel range.
The interpolating code takes one line at a time from fpm. It's only order of pixels matter and logically we get 2 passes.
example:
let's say mv720.fpm looks like this:
x y
-------
100 1
150 2
200 3
and mv1080.fpm look like this:
x y
-------
50 1
160 2
190 3
the map made from graphical image will be the fusion of both in this order:
x y
-------
50 1
100 1
150 2
160 2
190 3
200 3
and 2 pass map will be the fusion of both in this order:
x y
-------
100 1
150 2
200 3
50 1
160 2
190 3
This is the difference and it gives us logically 2 passes. Look at Y coordinates above.
regards
bb
-
Hehe, smart one.
-
And what's important - very simple :) and does not require any changes to the existing correcting algorithm.
-
Yeah, not very efficient having two copies of the same file but that's the way MLVFS is set up. It does make it easier to experiment with new map files.
Yes right and to bypass this in the 'MLV App' I integrated your pattern generators into the code :). Works on the fly like a charm. Internal generator can be overridden by placing the map to the binary directory.
-
In the fpm.sh:
instead of this line:
if [ $pattern == "B" ]; then mv720; fi # crop_rec not yet available on the 100D
insert this code and it should work for 100D crop_rec:
if [ $pattern == "B" ]; then
mv720
fp_start=89; fp_end=724; x_rep=8; y_rep=10
for j in $(seq $((fp_start)) $((fp_end)) ); do
if (( (($j + 0)) % y_rep == 0 )); then shift1=0; shift2=0
elif (( (($j + 1)) % y_rep == 0 )); then shift1=1; shift2=1
elif (( (($j + 5)) % y_rep == 0 )); then shift1=5; shift2=5
elif (( (($j + 6)) % y_rep == 0 )); then shift1=4; shift2=4
else continue
fi
output_row
done
fi
regards
bb
-
Thanks!
Committed that change (https://bitbucket.org/daniel_fort/ml-focus-pixels/commits/563b5167c187c538ca0d25e10644f3de38836125) in fpm.sh and will experiment with the 2 pass map file in MLVFS.
-
Although the bb 2-pass map looks pretty good in MLVFS without having to throw in chroma smoothing:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4514%2F37505495996_2269641c92_c.jpg&hash=d654e81a63a9d8946d8e555de4ac3164) (https://flic.kr/p/Z9ethG)
There are some residual focus pixels only at the top of the frame:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4496%2F36843453754_71e9a1cfb8_c.jpg&hash=b62de66126e07b71123a9b9f6d2ef2f8) (https://flic.kr/p/Y8Jkch)
I converted the map file for this image and it looks like this:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4477%2F36843465524_a50689db96_c.jpg&hash=91ece14ec3432a30233a348b0a704260) (https://flic.kr/p/Y8JoGd)
Danne and I worked hard on refining a crop_rec map for the EOSM and had problems getting it perfect. I have a feeling that using bouncyball's 2-pass map will help. Now getting back to those residual focus pixels, on the EOSM map the mv1080 focus pixels extends not only below but also above the mv720 focus pixels:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4459%2F36843672694_ba02cbfe85_c.jpg&hash=dd506e76249bb9ac22d3263a1e7632b2) (https://flic.kr/p/Y8Ksh7)
So it looks like if we add a few more rows at the top we'll be hitting those residual focus pixels at the top of the image.
[EDIT] After posting I noticed some people may be wondering about the blank area on the left. On a full raw buffer map it is "normal" to have a blank area on the top and left because this is out of bounds. In other words the image is never recorded to this area. Why? I don't know, maybe someone else can explain it.
-
There are some residual focus pixels only at the top of the frame:
Very strange because this map includes 30 pixel wide upper part of the image. What I mentioned In all my previous posts.
Here (http://nic.caucasus.net/fpm/?dir=&download=100D_crop_rec_frame_1.png) is the 16bit PNG sample file which is cleaned by this (http://nic.caucasus.net/fpm/?dir=&download=80000346_1808x726_dual_pass.fpm) map.
-
No pixel residues on my end:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs1.postimg.org%2F8ut09rt1j3%2FScreen_Shot_2017-10-07_at_22.01.12.png&hash=015c550d1c6ca91125f29f0ff653af88)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs1.postimg.org%2F91w857ejtb%2FScreen_Shot_2017-10-07_at_22.01.19.png&hash=c4df4ed8f712226043a461a66e4e85b0)
Are you using this map file? Erased top row?
http://nic.caucasus.net/fpm/?dir=&download=80000346_1808x726_dual_pass.fpm
-
this map includes 30 pixel wide upper part of the image. What I mentioned In all my previous posts.
Oops--I missed that when I tried to re-create your map. It does clean up nicely in MLVFS using your 2-pass map and doesn't need any chroma smoothing in MLVFS.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4472%2F23704425818_3c8664f333_c.jpg&hash=0b6f7dfdcb72de2e3ff35d36852bb869) (https://flic.kr/p/C7FnWS)
I'll commit this to the archives so I won't lose it. I'm having problems with some of my scripts that I use to convert fpm to pbm and back but once I get your map converted to an image file I'll save that too. Of course my scripts won't convert from an image file to a 2-pass map file so this will be a one way trip.
Just a few notes: Sometimes I get a crop_rec MLV that mlv_dump shows has a full raw buffer of 1808x726 and other times it is 1808x727. Danne took care of it by making my script accept a range of sizes but that isn't possible with MLVFS. Also, MLVFS only looks at the size of the full raw buffer to match up the map file so we can't have a separate maps for mv720 and crop_rec. I tried working around this by using the crop_rec map on mv720 but it caused problems (residual focus pixels) but with the 2-pass method maybe that will work? It isn't ideal to be hitting areas that don't have focus pixels but if it doesn't have an impact on the quality of the image or processing speed maybe that would be a way to get crop_rec working in MLVFS without having to swap out the map files?
-
Are you using this map file? Erased top row?
http://nic.caucasus.net/fpm/?dir=&download=80000346_1808x726_dual_pass.fpm
I was trying to make my own but missed the part about 30 pixels at the top. All good now.
Need some 100D mv720 clips to try my MLVFS theory--if you want to test just use bouncyball's crop_rec 2-pass map file on mv720 clips.
-
if it doesn't have an impact on the quality of the image or processing speed maybe that would be a way to get crop_rec working in MLVFS without having to swap out the map files?
+1
-
Committed bouncyball's 100D crop_rec map file to the ML Focus Pixels repository (https://bitbucket.org/daniel_fort/ml-focus-pixels/commits/b272f0050888ad2413829cc833c6756b782a225f).
Now for the instructions how to use this in MLVFS.
Grab that map file any way you want--here's a direct link to it (https://bitbucket.org/daniel_fort/ml-focus-pixels/raw/b272f0050888ad2413829cc833c6756b782a225f/focus_pixel_map_files/fpm_archived/100D_crop_rec_map_file/80000346_1808x726.fpm). Right click, save link. Make sure it is named 80000346_1808x726.fpm.
Now copy it to your MLVFS contents. On the Mac MLVFS is located in your home Library/Services. Right click (control click on Mac) and choose Show Package Contents. Copy the map file in there. Sometimes the raw buffer is 727, sometimes 726 so make a copy of that map file and rename it 80000346_1808x727.fpm. Yes, I know there's already a file in there by that name, save it somewhere safe because you might want to go back to it if this experiment doesn't work.
Ok now--100D owners, shoot some crop_rec and some mv720 footage and check for focus pixels. Please report back.
-
Oops--I missed that when I tried to re-create your map. It does clean up nicely in MLVFS using your 2-pass map and doesn't need any chroma smoothing in MLVFS.
Nice to hear. I was sure that you gone the way I did by yourself and just missed some details :).
I'm having problems with some of my scripts that I use to convert fpm to pbm and back but once I get your map converted to an image file I'll save that too.
Give a try to my 'fpmutil' (https://bitbucket.org/bouncyball/mlv-tools) it converts fpm->pbm->fpm in no second. Run './fpmutil -h' to get detailed help with examples. Note: you have to compile it because existing mac binary not updated and does not include the latest 100D crop_rec changes updated (https://bitbucket.org/bouncyball/mlv-tools/downloads/fpmutil.macos).
Also I'm sure you noticed header in the FPM. Header format is:
Sig CamID W H Crop program vers
----------------------------------------
#FPM 80000346 1808 726 1 -- fpmutil v0.5
and it's needed to convert fpm->pbm
If you want to convert your FPM to PBM you just need to insert this header as the 1st line of FPM and fill with appropriate values after '#FPM' signature with white spaces in between.
I could implement this header to MLVFS and make file name format like this: CamID_Width_x_Height_croprec.fpm. Had no time to do this yet.
Edit: unfortunately detection of crop_rec by RAWC header is not implemented anywhere yet.
-
Wow, you took this topic to a higher level. I'm just now trying it out and it is lightning fast compared to my scripts. Have you considered submitting it to the contrib section of the official ML source?
-
Even better. Bouncyball put fpmutil code into mlv_dump_on_steroids already:
https://bitbucket.org/bouncyball/magic-lantern/commits/408e300eb5841a90479300a7289086eca3b3f15b?at=mlv_dump-on-steroids
-
@dfort
Glad you liked it :)
Now try the link Danne posted above. On the fly map generation is implemented to the mlv_dump itself. No more fpm files in in the mlv_dump directory needed.
Have you considered submitting it to the contrib section of the official ML source?
Nope, I have not yet.
-
The 2-pass crop_rec map file is working so well with the 100D that I decided to do the same with the EOSM/650D/700D crop_rec map. The changes made the code much simpler.
Need to do some more testing but it should solve the hassle of having to swap out map files in MLVFS when switching between mv720 and crop_rec footage.
-
Damn @bouncyball's at it again. Keep on rolling and please never stop bouncing around! ;D
To be able to get clean footage from cameras affected by Focus Pixels without the need of Chroma Smoothing is just godsend!
-
Keep on rolling and please never stop bouncing around! ;D
Well... I'm trying ;)
-
I was testing 2pass method on cameras other than 100D and discovered that 1st pass is not needed for 700D (only mv1080, no mv720). EOSM definitely needs it. What about 650D? I don't have the sample MLV. I guess it does not support crop_rec at all.
-
On its way...
https://bitbucket.org/daniel_fort/magic-lantern/pull-requests/6/crop-rec-4k-650d/diff#chg-modules/silent/lossless.c
-
Good. Do you have some samples?
-
...1st pass is not needed for 700D (only mv1080, no mv720). EOSM definitely needs it...
Interesting. Usually what works on on the EOSM works on the 700D and vice versa. We're close to getting the 650D working in crop_rec and my bet is that it behaves the same as the 700D. These are all of the "pattern A" cameras so once the tests are in we can make specific cases for the cameras that need the 2-pass treatment and those that don't need it--though with zoom mode I'm hitting areas that don't need it and it doesn't seem to hurt. Then there's the zoom mode on the 100D that doesn't show any focus pixels at all. Go figure.
-
On second thought--it makes sense that the EOSM needs an mv720 pixel map pass because that camera is "normally" in mv720 mode except when recording H.264.
-
I need more 700D (crop_rec/no crop_rec) samples. Can someone upload them?
-
The funny thing is I never possessed/borrowed the camera affected by focus pixels :)
-
Hi Bouncyball - I have a 300mb mlv from a 650D running dforts latest crop rec experiment crop_rec_4k.2017Oct11.650D104.zip - it's 2512x1072. 14bit lossless compressed.
How should I pass it over to you - Hightail?
-
Hi saulbass. Noticed you used Switch? You can shrink this file by creating a sample file package(zip). You do it from either (ms) mlv_dump_on_steroids or (m) mlv_dump menus:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs1.postimg.org%2F17tf4dgmnz%2FScreen_Shot_2017-10-12_at_19.31.22.png&hash=9488d0bc59abffbe0927e472ae444638)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fs1.postimg.org%2F17tf4dgey7%2FScreen_Shot_2017-10-12_at_19.31.43.png&hash=7e7fd79ae8b31cd8f03accd7c43a5e5c)
-
Danne - thanks for the pointer - really useful to know - cheers!
-
@saulbass - could you check if this is happening on the 650D?
https://bitbucket.org/hudson/magic-lantern/pull-requests/872/crop_rec_4k-for-650d104/diff#comment-47051072
-
Hi Bouncyball - I have a 300mb mlv from a 650D running dforts latest crop rec experiment crop_rec_4k.2017Oct11.650D104.zip - it's 2512x1072. 14bit lossless compressed.
How should I pass it over to you - Hightail?
Sorry about very late response. It does not matter, just upload it somewhere and give the link.
Thx
bb
-
Hello
I noticed today a small problem with 12/11-8bit lossless pixel dots (14bit lossless works fine)
The focus pixels only appears in Contrast edge image and in the whole image except flat dark or flat white parts
Used camera: Canon 700D with croprec4kmlvsnd.2017Oct23.700D115 build
ISO: 100
All was Shooted at 12bit lossless on mv1080 mode 1736x976
Examples with no focus map applied :
White out of focus:
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fpreview.ibb.co%2FeZ6EVR%2FWhite_00000.jpg&hash=5cb165867b53424557de0bb2560d1021) (http://ibb.co/mO3Hjm)
Dark image:
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fpreview.ibb.co%2Fjm7XH6%2FDark.jpg&hash=809e2a2d460bc594cfd199bd21f7c8be) (http://ibb.co/c5tTqR)
Contrast Edge image:
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fpreview.ibb.co%2FbAydqR%2FContrast_Edge.jpg&hash=3c3808cbf2a51f0f183ca075340ca69c) (http://ibb.co/jcUUx6)
And this with focus map applied: 80000326_1808x1190.fpm focus map (only the middle cleard)
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fpreview.ibb.co%2FkCAV4m%2FCleard_Contrast_Edge.jpg&hash=9da8718fb1d25ea51567e7ffe9d037a7) (http://ibb.co/nsU1AR)
Orginal MLV, DNG samples and Images here:
https://drive.google.com/open?id=1-TaNgr0ZTjQ_Ea4Q6tPjAZVTIW_7qSG0
If this problem discoverd before, Can you give me the Map for this pattern? I tried all the maps didn't work.
-
Wow, fantastic testing!
I've been going though your sample files and have tried a few different methods. The best results I've gotten so far is using mlv_dump_on_steroids (http://www.magiclantern.fm/forum/index.php?topic=18975.0) (via Switch (http://www.magiclantern.fm/forum/index.php?topic=15108.0)) and adding a 2x2 chroma smooth. It looks like you are using MLVFS (http://www.magiclantern.fm/forum/index.php?topic=13152.0) and it should work the same there but I didn't test it because I broke it on my system, need to look into that.
I made a tiff from the dng file using dcraw. This is tends to show focus pixels more than Resolve, Adobe Camera Raw and other raw converters. dcraw is also showing some artifacts that can be smoothed out with the other apps but I wanted to show a worse case scenario.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4589%2F24380853157_55f8c8ecc7_c.jpg&hash=1596db8fff7e944ea8235c353bc1d048) (https://flic.kr/p/D9sfec)
Click on the image to see it full sized, zoom in, download, pixel peep, whatever.
Note that the test pattern that you are using is really challenging the pixel removal. Hard contrasty edges will often show the focus pixels even when running through the map file and that's why I added some chroma smoothing.
Ok--so we know that this worked with your 14-bit file but the problem is with the 12-bit file, right? Where's the 12-bit MLV? From your examples it looks like you found focus pixels that are outside of the area where we know there are focus pixels. Not surprising because there are settings that can show strange patterns all over the image even on cameras that don't normally show focus pixels.
mlv_dump -v
raw_info:
api_version 0x00000001
height 1188
width 1808
pitch 3164
frame_size 0x0039674C
--> bits_per_pixel 14
black_level 2047
white_level 5586
active_area.y1 28
active_area.x1 72
active_area.y2 1185
active_area.x2 1808
exposure_bias 0, 0
cfa_pattern 0x02010100
calibration_ill 1
-
Ok--so we know that this worked with your 14-bit file but the problem is with the 12-bit file, right? Where's the 12-bit MLV?
Right here (explanation (http://www.magiclantern.fm/forum/index.php?topic=19300.msg185746;topicseen#msg185746)):
--> bits_per_pixel 14
black_level 2047
white_level 5586
Note the 8...12-bit lossless mode uses a different LV raw type (http://www.magiclantern.fm/forum/index.php?topic=16040.msg191421#msg191421). There might be other raw types that allow image darkening via digital gain (to "reduce" the bit depth - actually the range of useful levels - while keeping the "old" focus pixels) - no idea how to find them other than brute-forcing (or whether they exist).
-
Learn something new every day. I've been doing lots of testing with 10-bit and mlv_dump shows bits_per_pixel as 10.
@theBilalFakhouri - I can clearly see focus pixels in areas where they haven't appeared before so I'll do some more digging into this. I can make a fpm file that covers the entire frame, like it does on the 100D, but it will take me some time before I can get to it.
-
There are two flavors of 10-bit:
- uncompressed (http://www.magiclantern.fm/forum/index.php?topic=5601.msg166924#msg166924), obtained by telling the image processing hardware to simply trimming the lowest bits;
Canon's image input/output modules can be configured for 10,12,14 or 16-bit uncompressed Bayer data.
- compressed (http://www.magiclantern.fm/forum/index.php?topic=18443.msg181620#msg181620) - lossless, but we can't / don't know how to configure the encoder to output 10-bit (but 10-bit input works);
workaround: restricting the useful levels in a 14-bit container does the trick;
this is done by darkening the image using digital gain, operating on integer math.
-
I used both MLVProducer 3203v (a quicker way to see focus pixels, remove it and whatever)
and orginal MLV Dump & on steriods version. Playing with maps and comparing the results.
as a1ex said
yeah I am using MLVProducer too to see bit-depth by Numbers(black-white level maybe) very nice and fast for Couples MLV files with some darkening image issue(at 12 11-8bit lossless it hadn't solved yet)
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fimage.ibb.co%2FdWnGC6%2F14bitlossess_00000.jpg&hash=a602b906e1c59feb772f40f80fb4d25d)
This another MLV Samples at 14 12 11-8bit lossless to compare
https://drive.google.com/open?id=1rhTS1i4pbqi6lLceIT6Tp3B4Ci52k_WM
and it's looks same problem in mv720 with crop_rec
thank you hard workers!
-
Made a new 80000326_1808x1190.fpm (https://bitbucket.org/daniel_fort/ml-focus-pixels/raw/46f41b39e8a3387a23b69f5aa2ba4243bf0342aa/focus_pixel_map_files/80000326_1808x1190.fpm) file that should cover up those pixels that are showing up on 12-bit lossless files. Problem is, I'm not seeing an improvement when using MLVFS. Try it out on your system and let me know if it works for you.
The SourceTree app is showing this in the diff:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4592%2F27476729519_97bea25fba_z.jpg&hash=c0818a639ff1c59f089062be87f7d053) (https://flic.kr/p/HS2qpR)
I'll also prepare a new mv720 map file and take a look at the other video modes. If it works I'll also need to update the script and pass along this information to the MLV app developers.
-
It's the same problem no improvement yet; only the middle was cleared.
I am using mlv_dump_on_steroids same as MLVFS method.
Any reason why?
-
Because none of RAW data from 700D had focus pixels all over the image ;) until now.
@dfort is that mv720 mode?
-
@dfort is that mv720 mode?
That one is mv1080 mode, haven't gotten around to the other modes yet. I'm having problems getting it working with MLVFS.
It is easy enough to extend the pattern so that it covers the entire sensor. It shouldn't hurt the other video modes so we probably don't need to complicate things by checking to see if the MLV file was recorded in 12bit and applying a new map file.
-
@dfort:
Got that contrasty pattern MLV. It has raw "height 1189" and for MLVFS '.fpm' have to be named "80000326_1808x1189.fpm". Is there other reason why MLVFS does not work?
@Danne:
Btw this contrasty MLV from @theBilalFakhouri is the very good example why MLVFS interpolation method is better then RAW2DNG. Try yourself switching '--fpi' method and you'll see a big difference ;)
-
Still looks like shit over here ;) :D
-
@dfort:
Got that contrasty pattern MLV. It has raw "height 1189" and for MLVFS '.fpm' have to be named "80000326_1808x1189.fpm". Is there other reason why MLVFS does not work?
Thanks, keep forgetting about that because your app doesn't need it. Still having some problems but I'll have to work on it later.
-
Still looks like shit over here ;) :D
I did not say it cleans all up (because this is unusual case). I said see the difference where pixels are cleaned ;) (believe me it's HUGE, lot of uncleaned focus dots/artifacts in dark areas of this "cage" pattern with RAW2DNG)
-
Thanks, keep forgetting about that because your app doesn't need it. Still having some problems but I'll have to work on it later.
Good luck :)
-
I dig your work @theBilalFakhouri and best of luck to @dfort!
-
No solution yet but I'm getting closer. It looks like the focus pixel pattern we've been using on the 700D works fine for the middle section but extending that same pattern so that it covers the entire sensor didn't work. This might be a case similar to what we've seen on the crop_rec pattern for the EOSM.
In the process of trying to figure this out I found other issues so instead of working this out by myself I'm going to document the process I went through.
Got that contrasty pattern MLV. It has raw "height 1189" and for MLVFS '.fpm' have to be named "80000326_1808x1189.fpm". Is there other reason why MLVFS does not work?
According to the current mlv_dump from the crop_rec_4k branch the height is 1188:
Res: 1736x976
raw_info:
api_version 0x00000001
height 1188
width 1808
However, mlv_dump on steroids shows something different:
mlv_dump_on_steroids
Res: 1736x976
raw_info:
api_version 0x00000001
height 1189
width 1808
Under the guidance of @a1ex and @dmilligan I measured all the full raw buffer sizes a while back and mv1080 for the 700D measured 1808x1190. I assumed that the full buffer size never changes -- but apparently it does and now we've got three possible sizes? Luckily all of them seem to line up from the upper left (0x0) so we don't need to change the map files, just the names.
Since I'm testing with MLVFS which map file is it using? In order to find out let's run MLVFS from the command line. This is how it is done on the Mac:
In your home directory, create a folder called MLVFS_TEST
Place the mlvfs executable in this folder.
Place your fpm files in this folder
In your home directory create another folder called MLV_TEST
Place your MLV files in this folder.
In your home directory create another folder called MOUNT_TEST
Type EXACTLY this in the console
cd ~/MLVFS_TEST
./mlvfs ~/MOUNT_TEST --mlv-dir=~/MLV_TEST -f -o local -o volname="TEST" -o allow_other
It needed some action like opening one of the DNG files in Adobe Camera Raw and this showed up in the terminal:
Loading focus pixel map '80000326_1808x1189.fpm'...
Todo -- Shoot some simple silent DNG's using the various modes and measure the size of the full raw buffers. The DNG's need to processed through dcraw like this in order to include the out of bounds zones:
dcraw -4 -E *.dng
-or- [EDIT] This won't get you the full raw buffer size.
I specified:
dcraw -D -E -T INPUT.DNGSeems to work and we get a tiff instead of a pgm file. Output came out all black with -4 so skipped that.
Last time I tried measuring the full raw buffers I was getting different images sizes in Photoshop and exiftool so I'm still not sure what to believe. [EDIT] On my latest tests they are showing the same size.
Question -- shouldn't the full raw buffer height alway be an even number? Otherwise wouldn't that mess with the Bayer pattern? Maybe the raw buffer itself hasn't changed, just the way it was either rounded up or down to report it as an even number of pixels?
Ok--let's get back to that test shot @theBilalFakhouri sent. I lined up the DNG with the new map file I created and saw this:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4637%2F38582081234_3b786d630d_z.jpg&hash=ebc72e4eedec5e8697f8bdd637f46ef7) (https://flic.kr/p/21Mnfz9)
What we are seeing a close up of the area between the middle where the focus pixel map (the black pixels) lined up and the upper area where it didn't. Seems like the focus pixels are in a different position and possibly a different pattern. [EDIT] My bad. I didn't line up the map file properly when I extended it to cover the top and bottom areas.
How do I know that I lined up the focus pixel map properly? The offsets can be found using mlv_dump with the -v option:
Block: VIDF
Offset: 0x0018c400
Number: 14
Size: 1620480
Time: 175.773000 ms
Frame: #0001
Crop: 72x118
Pan: 72x118
Space: 32
The upper left corner of the DNG lines up with 72x118 on the full raw buffer image file. In this case the Crop and Pan values are the same but if they are different use the Crop. (or is it the Pan? ugh--that's why we should save our notes.) If anyone wants to help figure out new focus pixel map files, all the tools you should need are in my ML Focus Pixels Bitbucket repository (https://bitbucket.org/daniel_fort/ml-focus-pixels).
-
Heavy. Tip. You can run tests via Switch too and you'll see the focus pixel maps loading in terminal. Simply add the new maps into mlvfs folder inside Contents folder inside Switch.
Then run (ml) from main menu.
-
Another tip, fpm.sh has a -v (verbose) mode:
./fpm.sh -v Contrast\ Edge.MLV
Camera: 700D
Camera Model: 80000326
Resolution: 1736x976
Video Mode: mv1080
Full Raw Buffer Size: 1808x1188
Pan: 72x118
Crop: 72x118
Output File: 700D_mv1080_1736x976.txt
File Format: dcraw
Writing file
Finished
I've got mlv_dump from the crop_rec_4k branch installed as my system default. mlv_dump on steroids already has the focus pixels algorithm. :D
-
And I'm using fpmutil(bouncyball) as well :)
-
MLVFS interpolation method is better then RAW2DNG. Try yourself switching '--fpi' method and you'll see a big difference ;)
RAW2DNG interpolation method it's look like only soften the pixels dots didn't remove it :D In many different shots it wasn't good enough as MLFVS method.
Todo -- Shoot some simple silent DNG's using the various modes and measure the size of the full raw buffers.
I didn't have all this knowledge yet to do this things. So maybe I can shoot silent DNGs and upload it?
-
So maybe I can shoot silent DNGs and upload it?
The problem with shooting silent DNG's is that I don't think you can save a 12-bit lossless compressed simple silent still. That's what you are using, right?
Seems all the testing was done on 14bit/10bit and not enough attention to 12bit lossless. That should be the sweet spot for the 700D. It is what the Black Magic Pocket Cinema Camera is using, right?
In any case, I've got a 700D so I should be able to get this figured out--eventually. There's a lot of dots in thar!
-
So right, Good luck 8)
-
According to the current mlv_dump from the crop_rec_4k branch the height is 1188:
...
Loading focus pixel map '80000326_1808x1189.fpm'......
Question -- shouldn't the full raw buffer height alway be an even number? Otherwise wouldn't that mess with the Bayer pattern? Maybe the raw buffer itself hasn't changed, just the way it was either rounded up or down to report it as an even number of pixels?
It's bad because getting height of the raw buffer is so straightforward. This value physically stored in the RAWI header's 'raw_info' part. If mlv_dump from 4k branch reports 1188 then it just rounds it to the lower even number before printing log. I have to look at the code.
-
Found this piece of code in latest mlv_dump from 4k branch:
/* well, it appears to happen that MLVs with odd sizes were written, restrict that */
block_hdr.raw_info.height &= ~1;
block_hdr.yRes &= ~1;
Doing this is just wrong (misleading information) because the change only applies to the MLV exported from mlv_dump and to the log printed to the screen, original MLV indeed has original odd number. The correct way to "restrict" it, is implementing this restriction to mlv_rec/mlv_lite modules.
-
Thanks @bouncyball -- things are starting to clear up. About a year ago @Danne noticed that the full raw buffer heights have changed so he did this adjustment (https://bitbucket.org/daniel_fort/ml-focus-pixels/commits/d18ee7cf2610d2af440fce4a05357cd7726bb571) to the focus pixel script. Since mv1080 went from 1190 to 1189 or 1188 we're allowing for a huge range of possibilities in the script:
case $raw_buffer in
- 1808x1190)
+ 1808x11**)
Add a comment to this line
video_mode=mv1080
I've been shooting simple silent DNG's on the 700D and keep getting 1808x1189 as measured in exiftool and in Photoshop after processing with dcraw. (By the way, just found out that saving to a Portable Bit Map file with the -4 and -E options shows the full raw buffer size but saving to a tiff does not.)
@theBilalFakhouri -- Almost there! I cleaned up the top. Of course I've got to do the same for the bottom part.
-
The problem was introduced when trying to autodetect resolution from registers (https://bitbucket.org/hudson/magic-lantern/commits/38e78de08ad9632c9dbd2079cff03ccef8fa124b?at=unified), rather than hardcoding it for each video mode. The method did not give exact results; they were sometimes off by one. Previously, we took the resolution from EDMAC configurations, but these were harder to autodetect with EDMAC_RAW_SLURP - because in this case we had to set up the EDMAC at the right resolution, rather than taking it for granted.
Back then, having exact resolution didn't seem important; at least nobody noticed it when reviewing this PR (https://bitbucket.org/hudson/magic-lantern/pull-requests/778/).
Later, shortly after crop_rec_4k was announced for 5D3, this adjustment (https://bitbucket.org/hudson/magic-lantern/commits/29d86cc25b636cb9cb1bdd6507caef6b13aa3359) was required for 8...12-bit lossless modes - otherwise, only every other frame was correct. You'll see the issue right away if you comment out the height adjustment. However, if you reduce the autodetected value by 2 pixels or a bit more, all frames will still be correct. So, it's critical to have the EDMAC resolution not higher than what the sensor is set up to deliver (but apparently it can be a little lower).
Now, why did I not adjust all the other models in the same way?
Resolution autodetection is inconsistent - sometimes this method gives the EDMAC resolution + 1, other times - 1 and other times the exact value. Here's a script that scans the ROMs for possible resolutions (pattern matching):
scan_raw_res.py (http://a1ex.magiclantern.fm/bleeding-edge/raw/scan_raw_res.py) and results: raw_res.txt (http://a1ex.magiclantern.fm/bleeding-edge/raw/raw_res.txt).
Interpretation:
- some models give exact values (with exceptions - some hidden video modes are configured differently):
6D, 100D, EOSM2
- most models have EDMAC resolutions configured at the one from capture registers + 1 (vertically)
1100D, 1200D, 1300D, 600D, 60D, 650D, 700D, EOSM
- one model has EDMAC resolutions configured at the one from capture registers - 1 (vertically)
5D3
- not analyzed: old DIGIC 4 models with LVState.
So... rather than hardcoding resolutions in each video mode (as we did some time ago), looks like we still have to hardcode a resolution adjustment (besides column_factor, which probably shows how many columns are read out in parallel).
Notice the EDMAC (used for transferring 14-bit Bayer data into RAM) is configured sometimes at even resolutions and sometimes at odd resolutions - in Canon code.
-
Oh yeah, I remember that pull request. Probably wasn't looking that closely at the focus pixels at the time but I did do a pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/795/raw-fixes-eosm-fix/diff) for the EOSM on top of that pull request.
Couldn't resist running that script on the cameras affected by the focus pixel issue--assuming the EOSM2 also has the issue:
for f in */ROM1.BIN; do echo; echo $f; python scan_raw_res.py $f | sort | uniq; done
100D.101/ROM1.BIN
C0F06800: 1808 x 1190 (timer A = 520)
C0F06800: 1808 x 1190 (timer A = 640)
C0F06800: 1808 x 298 (timer A = 568)
C0F06800: 1808 x 727 (timer A = 528)
C0F06800: 1808 x 727 (timer A = 640)
C0F06800: 1872 x 1060 (timer A = 576)
C0F06800: 1872 x 1060 (timer A = 600)
C0F06800: 2592 x 1108 (timer A = 732)
C0F06804: 5280 x 3505
C0F06804: 5280 x 3529
EDMAC: 1808 x 1190
EDMAC: 1808 x 220
EDMAC: 1808 x 299
EDMAC: 1808 x 727
EDMAC: 2592 x 1108
650D.104/ROM1.BIN
C0F06800: 1808 x 1189 (timer A = 528)
C0F06800: 1808 x 1189 (timer A = 640)
C0F06800: 1808 x 726 (timer A = 528)
C0F06800: 1808 x 726 (timer A = 640)
C0F06800: 1872 x 1059 (timer A = 546)
C0F06800: 1872 x 1059 (timer A = 640)
C0F06800: 2592 x 1107 (timer A = 716)
C0F06804: 5280 x 3505
C0F06804: 5280 x 3529
EDMAC: 1808 x 1190
EDMAC: 1808 x 220
EDMAC: 1808 x 727
EDMAC: 2592 x 1108
700D.115/ROM1.BIN
C0F06800: 1808 x 1189 (timer A = 528)
C0F06800: 1808 x 1189 (timer A = 640)
C0F06800: 1808 x 726 (timer A = 528)
C0F06800: 1808 x 726 (timer A = 640)
C0F06800: 1872 x 1059 (timer A = 546)
C0F06800: 1872 x 1059 (timer A = 640)
C0F06800: 2592 x 1107 (timer A = 716)
C0F06804: 5280 x 3505
C0F06804: 5280 x 3529
EDMAC: 1808 x 1190
EDMAC: 1808 x 220
EDMAC: 1808 x 727
EDMAC: 2592 x 1108
EOSM.202/ROM1.BIN
C0F06800: 1808 x 1189 (timer A = 528)
C0F06800: 1808 x 1189 (timer A = 640)
C0F06800: 1808 x 726 (timer A = 528)
C0F06800: 1808 x 726 (timer A = 640)
C0F06800: 1872 x 1059 (timer A = 546)
C0F06800: 1872 x 1059 (timer A = 640)
C0F06800: 2592 x 1107 (timer A = 716)
C0F06804: 5280 x 3505
C0F06804: 5280 x 3529
EDMAC: 1808 x 1190
EDMAC: 1808 x 220
EDMAC: 1808 x 727
EDMAC: 2592 x 1108
EOSM2.103/ROM1.BIN
C0F06800: 1808 x 1190 (timer A = 520)
C0F06800: 1808 x 1190 (timer A = 640)
C0F06800: 1808 x 298 (timer A = 568)
C0F06800: 1808 x 416 (timer A = 522)
C0F06800: 1808 x 727 (timer A = 528)
C0F06800: 1808 x 727 (timer A = 640)
C0F06800: 1872 x 1060 (timer A = 576)
C0F06800: 1872 x 1060 (timer A = 600)
C0F06800: 2592 x 1108 (timer A = 732)
C0F06804: 5280 x 3505
C0F06804: 5280 x 3529
EDMAC: 1808 x 1190
EDMAC: 1808 x 220
EDMAC: 1808 x 417
EDMAC: 1808 x 727
EDMAC: 2592 x 1108
I looked at firmware updates on the EOSM AND EOSM2 and it didn't change the results, at least not on those updates. In any case, all of the simple silent stills I've been shooting on the 700D in mv1080 mode have been coming in at 1808x1189, same with MLV files as reported by mlv_dump on steroids and MLVFS. The "official" crop_rec_4k mlv_dump seems to be the only app that I tested that reports the height differently for the reason that bouncyball pointed out (http://www.magiclantern.fm/forum/index.php?topic=16054.msg195195#msg195195).
@theBilalFakhouri -- Here is a brand new focus pixel map file (https://bitbucket.org/daniel_fort/ml-focus-pixels/raw/745a0e0388cdd5bf550554c0a71e1e031bb7fa3e/focus_pixel_map_files/80000326_1808x1189.fpm) that will work on the 700D when recording 12-bit lossless compression in mv1080 mode. Test it out and report back. This is what your test file looks in MLVFS using the new map file--no chroma smoothing this time.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4641%2F27544241039_968e4cf7f3_z.jpg&hash=0c8f95c07bc420189bb57c4717ca85ea) (https://flic.kr/p/HXZrez)
To do -- figure out what to do with all of the other focus pixel map files. Maybe they should all be covering the entire sensor just in case? It doesn't seem to hurt the image. At least it looks better than having colored dots all over the place! [EDIT] We're pretty much doing that already on the 100D.
-
Amazing work on those 700D pixels dfort.
-
It also works on the one @papasapien posted on the MLVFS topic. (http://www.magiclantern.fm/forum/index.php?topic=13152.msg195150#msg195150)
Here's one with the map file that is in MLVFS -- well, it doesn't even use the map file. It is stretched like crazy so the focus pixels pop:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4598%2F39298350962_fa5ac23a7f_z.jpg&hash=073eceea286ffd599b518a36d4866e09) (https://flic.kr/p/22SEjBd)
and here's the new map that covers the entire sensor.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4644%2F27550376209_a2d1b53c32_z.jpg&hash=f26c8f079a3f73c64295827c4418b689) (https://flic.kr/p/HYwT1n)
By the way, on this MLV file the full raw buffer size is also 1808x1189 and mlv_dump (crop_rec_4k) reports it as 1808x1188 for the reasons explained a few posts back.
-
@dfort
Really nice work man! it worked!
But unfortunately there are more pixels we couldn't see it. look like this:
Why we couldn't see them?
Because the pixels dots were behind the scenes in the white and green lines in the pattern
the edge only visible for us we can't see in the white/dark.
So I made the pattern flickering and I moved the camera slowly around the pattern to catch them but still too hard to see I am trying another patterns to make the view easier.
Is this really matter? I mean the most pixels was cleaned perfectly why we care for few Matrix Pixels
Because they were in my face and it will be in your face next time and in the people's faces that time :D so let's work on it.
Maybe it takes some time for me!
EDIT: I found it! Uploading the samples!
EDIT 2 : I was wrong this cold/hot pixels. SO PERFECT JOB @dfort thank you!
-
Here's a tip for posting YouTube videos. Change the https: to http: and enclose in youtube tags like this:
[youtube]http://youtu.be/7N5NgRBUWU8[/youtube]
Those remaining dots don't look like focus pixels. They may be hot/cold/dead pixels or even just dust on the sensor. Try Googling for help on how to deal with those kind of problems. If nothing works, you can add the coordinates of those stubborn pixels to the focus pixel map file. You need to take into account the Pan information.
mlv_dump -v Contrast\ Edge.MLV | less
Block: VIDF
Offset: 0x00000600
Number: 13
Size: 1621504
Time: 133.415000 ms
Frame: #0000
Crop: 72x118
Pan: 72x118
Space: 32
So if you find a problem pixel at say x=100, y=100 put "172 218" in the map file. It shouldn't matter if you put it at the start, end or middle of the file.
-
@dfort
Have you updated your script to generate this map?
-
Have you updated your script to generate this map?
Not yet -- I need to consult with my brain trust first, you and @Danne, about this. Of course anyone else who has an opinion about this should chime in.
The more I think about this stuff the more I'm leaning towards modifying the map files and script to cover the entire active full raw buffer area with the focus pixel pattern. One reason is because it will cover the 12-bit issue that @theBilalFakhouri brought to our attention. Another reason, and why I don't think it should just apply to 12-bit files, is because if only the center of the image has the pattern applied it might cause issues if we start to compare the areas that received the pixel pattern fix to those areas that didn't.
We're already doing that on the 100D and it doesn't seem to be a problem so how about doing the same for the other cameras?
One thing that I'm not too sure about is the 2-pass crop_rec map file for the EOSM but even there, covering the entire sensor in two passes shouldn't be a problem.
Your thoughts?
-
I'm all for your modification thinking. It will not affect the overall picture it seems.
-
Well if it does affect the image it will be the overall image not just the middle section of it.
-
Since you get more than decent results out of 100D focus pixel footage I don't think neighbouring pixels will be visible(distorting) on the image? Let's do som tests?
-
@Danne -- Let's see if I have this figured out, the "pattern A" cameras EOSM/650D/700D are the ones that sometimes report a different full raw buffer vertical resolution while the "pattern B" cameras, 100D and possibly EOSM2, always report the same height. Though I've got an mv720 map file that is 1808x727 and a crop_rec map file that's 1808x726, go figure.
The "pattern A" cameras are a problem on MLVFS because it means having to double up on some of the map files. On the new map files I'm taking into account the top and left out of bounds areas but am running the focus pixel pattern all the way to the right and bottom edges of the full raw buffer. That means that there is a difference between the 1808x1189 and 1808x1190 map files. Well, maybe it doesn't need to be this precise but better err on the side of being too precise.
I want the script to be able to create that precise full raw buffer map file that can be used in MLVFS. While we're at it, we should be able to verify these values:
[EDIT] I'm already seeing some issues on the EOSM. skip_left should be 72 and skip bottom default should be 0. I do see an unused area on the bottom of mv720 but it is 3 white pixels, not black as in the other out of bounds areas. One more problem--simple silent isn't working with the crop_rec_4k branch so I can't look at crop_rec.
src/raw.c
#if defined(CONFIG_650D) || defined(CONFIG_EOSM) || defined(CONFIG_100D)
#warning FIXME: are these values correct for 720p and crop modes?
skip_top = 28;
skip_left = 74;
skip_right = 0;
skip_bottom = 6;
#endif
#ifdef CONFIG_700D
skip_top = 28;
skip_left = 72;
skip_right = 0;
skip_bottom = zoom ? 0 : mv1080crop ? 0 : 4;
#endif
I'm not too concerned about skip_right and skip_bottom for the focus pixel map files and script. Also, your heuristic code that takes into account differences in the raw buffer height is working fine with everything including the crop_rec_4k mlv_dump that rounds the height to an even number.
@bouncyball - I take it that all this also applies to fpmutil, right?
-
@bouncyball - I take it that all this also applies to fpmutil, right?
Yes it tolerates some vertical range and when exporting map with default name inserts correct/exact height of original MLV to this name.
Regarding interpolation of the unaffected areas of the image, I think the less we alter the image is better and this belief based on proof that in some very contrasty situations interpolation produces even more artifacts than there were in the beginning. Remember example with cage pattern MLV which been posted here.
Even 100D example from you: View from the window with pool, trees and buildings. On the left building there is a contrasty pattern on the windows and only MLVFS newer interpolation method handles it more or less correctly.
Another example 2pass maps (needed to prevent introducing post interpolation artifacts). They of course doing better job and not screwing up the whole image but who knows in what video mode and pattern and frequency of repeating this pattern could be introduced issues when adding support for all other now unsupported cameras and video modes.
regards
bb
Edit: quite a hard journey isn't it?, well... you started all this ;)
-
No easy way around these pixels ey :P. Yet.
-
Hey Magic Lantern world
This samples for each mod with another pattern: I called it PIXEL SCANNER pattern
https://drive.google.com/drive/folders/1X47f9foy6CdOiifndJSdFHyEcGWFDO2V
Because it's PERFECTLY shows hot/cold/dead Pixels maybe it will help for cameras have this issue heavily.
by doing script take this pixels out and save the pattern for each camera sensor for users then applying it to the raw video.
(Every camera will have it own hot/cold pixels and even if this the same cameras. from 700d to 700d will be difference.So maybe every user will have his own cold/pixel pattern if I am correct)
Tip: for mapping 12\11-8bit lossless Pixel Dots easily or to make sure all the pixels dots was mapped:
Shoot a raw video sample for each mod with new PIXEL SCANNER pattern (I played the pattern on 42 inch screen and I smoothed the focus a bit for decreases moire and show pixels dots a bit clearly). Zoom in the pattern if the screen was small.
Then process mlv file once for new map applied and once with no map applied. Import dngs sequences and put them above each other in any editing software now disable\enable the video on the top layer and see the difference before\after.
For red box only: I don't know if this artifacts or remaining pixels dots see below.
For artifacts problem here what I got: (This is 12bit lossless raw video)
New map applied:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fpreview.ibb.co%2FnGAyGb%2FNew_map_applied.jpg&hash=a4b1ee0a6acf362d679e7889ce0b7453) (https://ibb.co/iQg5wb)
I zoomed some areas:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fpreview.ibb.co%2FdNDw2G%2FImage_shows_artifacts_and_remain_dots_pixels.jpg&hash=a09a7a33f6f886e1fb9ac82ce085c828) (https://ibb.co/bRTUNG)
Note: In the right part of sky this was overexposed so no artifacts on blown whites. And on the left-bottom corner in the horizontal white lines I don't know if this artifacts or remaining pixels dots. Here the original mlv make a tests:
https://drive.google.com/open?id=1hGJ6b-NH66mVFqtFIhnYzeTne9f83dmW
*A question: what is the process does MLVFS doing to remove this pixels dots?
By restoring the original pixels or changing the pixels dots to the safe pixels around pixels dots?
Affected cameras: EOSM, 100D, 600D, 650D, 700D, (and more?)
Method 1 - In Camera
I am somehow looking for this method.
I have question:
Is the pixels dots removing via Canon h.264 encoder or .cr2 .jpeg compression ? or the mlv_lite\silent_pic re enable somehow these pixels but the canon code disable it?
Maybe the encoder, reason: Enable proxy with mlv_lite you will have Clean H.264 and raw video with dots pixels in the same live view the same settings.
How does Framing mode in mlv_lite preview settings shows the dots pixels?
I am trying to find answer. If we, maybe we will see something like this:
Affected cameras: EOSM, 100D, 600D, 650D, 700D, (and more?)
-
@theBilalFakhouri - Those 12-bit files that you posted are proving to be quite useful. I was a bit skeptical of your moving PIXEL SCANNER pattern at first but it does a good job of bringing out those focus pixels.
Like I said before some of what you are seeing may be hot/dead/cold pixels. I started this topic because I wasn't satisfied with the dot removal tools that were available so I experimented with the "badpixels" feature in dcraw. As the name implies, it is a way to deal with those "bad" pixels.
In the old days we would check the camera's film gate for dust and hair before moving onto the next shot. Well dust and hair can still land on the sensor but we aren't checking for it any more because it is easy to fix it in post. I often thought it would be a nice to be able to easily map out problem pixels.
Before we start adding new features let's deal with the issues you brought up with those 12-bit files. I tried a few times to come up with a map file that will work for both mv720 and crop_rec when using MLVFS. Looks like it is finally working thanks to the 2pass map file @bouncyball came up with--and on those 12-bit files no less!
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4645%2F38502687695_898e422472_z.jpg&hash=acc4d7ab8e6b490227fed953a94f0912) (https://flic.kr/p/21EmkD2)
I've got some more experimenting to do with the map files but I'll keep pushing them to my ml-focus-pixels repository (https://bitbucket.org/daniel_fort/ml-focus-pixels) if anyone wants to try them out. I'm also make changes to the fpm.sh script which can be used to create the MLVFS map files.
For red box only: I don't know if this artifacts or remaining pixels dots...
What area are you zoomed in on on that red box? It is very difficult to make it out on YouTube. Your 12-bit MLV's are starting to look pretty clean over here with the exception of a few "bad" pixels. Tests on my 700D aren't showing anything like that so maybe you need to check your sensor?
I zoomed some areas:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fpreview.ibb.co%2FdNDw2G%2FImage_shows_artifacts_and_remain_dots_pixels.jpg&hash=a09a7a33f6f886e1fb9ac82ce085c828) (https://ibb.co/bRTUNG)
Note: In the right part of sky this was overexposed so no artifacts on blown whites. And on the left-bottom corner in the horizontal white lines I don't know if this artifacts or remaining pixels dots. Here the original mlv make a tests:
(Uploading)
The way you edited that image is a bit disorienting but I can see some focus pixels in the sky that are a problem. Do you have a link to download it yet? If the MLV is too large you know that you can cut it down to just a few frames, right? [EDIT] Got it!
-
Did anyone notice something weird?
By changing bit-depth from 14bit lossless to 12bit lossless the whole pixels dots disappeared? (In whites & darks) and we get in very magic situation another new pattern in the edges only.
I don't know what the parameters had been changed to get this clean image, But what is the fu***** thing makes these pixels disappeared this way? White level? I don't know.
I really believe we are so close to forget pixels dots !
WHAT IS YOUR THOUGHTS MEN?
-
...we get in very magic situation another new pattern in the edges only....
You mean like this?
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4639%2F39383553601_e90deb523a_z.jpg&hash=84d381aca27daf7b3ce025f2aad95988) (https://flic.kr/p/231c1oe)
Nothing magical here, look back through the various posts on this topic and you'll see several examples like this. The above detail was done in MLVFS without the focus pixel map and processed in dcraw. The image needs some help with the blown out pink sky.
Here is what it looks like with the new pixel map in MLVFS, 2x2 chroma smooth and some playing around with Adobe Camera Raw.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4685%2F25513558888_cbf34519d0_z.jpg&hash=aa987091060699568a79ff9b6f56b249) (https://flic.kr/p/ESxEbm)
Go ahead, click on the image and pixel peep. It is about as clean as any other camera in mv1080 mode except the 5D3 which is still the best because of the pixel binning methods used on that camera.
[EDIT] Pixel peeping:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4646%2F38550663695_ddc11ddc97_z.jpg&hash=458231fbf01cb6e8769a6f74260009e8) (https://flic.kr/p/21JAeer)
I'm not seeing the artifacts you pointed out. How are you processing your DNG's?
Hey Magic Lantern world
*A question: what is the process does MLVFS doing to remove this pixels dots?
By restoring the original pixels or changing the pixels dots to the safe pixels around pixels dots?
That is best answered by the developer, @dmilligan. However, MLVFS is open source (https://bitbucket.org/dmilligan/mlvfs) and you can go through the code to see how it works. You can find the fix_focus_pixels function in cs.c (https://bitbucket.org/dmilligan/mlvfs/src/24ebdf591dba0a0431cf1fb674b3a8fcd2730b62/mlvfs/cs.c?at=master&fileviewer=file-view-default).
-
By changing bit-depth from 14bit lossless to 12bit lossless the whole pixels dots disappeared? (In whites & darks)
What do you mean by that? Changing 14bit->12bit? By camera settings or resample raw data manually? And how this related to disappearing of focus pixels?
Edit: In camera, changing lossless 14bits to 12/10bits just amplifies signal and white level lowers, the actual raw data remains 14bit. Manual resampling is a real bit depth conversion.
-
@bb: in ML settings, see #424 and follow the links.
Short answer: it's Canon's imaging pipeline fixing these pixels; this only happens in 8...12-bit lossless modes (they use raw type 0x12 DEFCORRE, other modes use 0x10 CCD aka the first step in the pipeline).
-
@bb: in ML settings, see #424 and follow the links.
ok
-
@a1ex: Ah, yes I know that, thank you :)
-
I'm lost because all this talking is around 12bit lossless images he posted earlier and they have tons of focus pixels and magically in latest message they disappeared :)
-
@theBilalFakhouri and anyone else who want's a somewhat less technical explanation, check out Reply #285 (http://www.magiclantern.fm/forum/index.php?topic=16054.msg176032#msg176032) of the Dealing with Focus Pixels in raw video topic and this pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/774/no-focus-pixels-on-eosm/diff) that I submitted and later declined.
Bascially, there are several choices of raw types. The one used in 8...12-bit lossless modes has at least a part of code that hides the focus pixels but they are still showing up on contrasty boundaries. We haven't been able to figure out how to eliminate the focus pixels completely in-camera but it does make sense that something like this is showing up because these camera models use contrast-detection autofocus. It looks a lot like the "focus peaking" feature to help focus manual lenses.
What is interesting is that up until now the focus pixel maps that we've been using haven't been able to eliminate these focus pixels but these new maps are working, at least in MLVFS. I'm testing in MLVFS because it is easy to swap out the map files in this app.
Will it work on other apps? I tried it on dcraw and it doesn't work even though the map file is lining up perfectly. This means that the app developers might have to re-work their focus pixel fixing code if they want to support 12-bit lossless on these cameras.
-
I'm lost because all this talking is around 12bit lossless images he posted earlier and they have tons of focus pixels and magically in latest message they disappeared :)
They didn't disappear, they just don't show up under certain conditions. We still need to map out their known locations.
I updated the script and most of the fpm files for MLVFS. Everything seems to be working properly for the 650D/700D/EOSM cameras even in 12-bit lossless except for the mv1080crop that @theBilalFakhouri supplied. Maybe it is one of those cases where everything has shifted?
Still needs testing on the 100D.
-
with the exception of a few "bad" pixels. Tests on my 700D aren't showing anything like that so maybe you need to check your sensor?
Yeah I checked the sensor today I saw some dots on it, So the pattern is show perfectly the dust on the sensor :D
I'm not seeing the artifacts you pointed out. How are you processing your DNG's?
I tried both mlv_dump_on_steriods and MLVFS with new map but without chroma smoothing. After applying 2x2 chroma smooth the image is perfect clean. I am using Davinci Resolve 14 also ACR for comparing results.
I don't like chroma smooth because it's produce some more aliasing as I know?
The image needs some help with the blown out pink sky.
Any solution for pink highlights?
Short answer: it's Canon's imaging pipeline fixing these pixels; this only happens in 8...12-bit lossless modes (they use raw type 0x12 DEFCORRE, other modes use 0x10 CCD aka the first step in the pipeline).
Looking Nice
-
Re white level:
http://www.magiclantern.fm/forum/index.php?topic=16040.msg191131#msg191131
http://www.magiclantern.fm/forum/index.php?topic=16040.msg190644#msg190644
https://bitbucket.org/hudson/magic-lantern/commits/98539896ebd056e0a2128658aacd73b757a78b08
https://bitbucket.org/hudson/magic-lantern/commits/ab792b7c5137fceee4016bf9ef88767d031d27c5
Sample DNG: max = 4932 at ISO 100, 1080p24. However, that's just 1 data point out of nearly 100.
For existing 12-bit lossless footage from 700D at ISO 100 1080p(24?) with pink highlights (possibly other cameras/modes, no guarantees):
exiftool *.dng -WhiteLevel=4931
-
Played around with it and couldn't fix the pink highlights on the DNG's using exiftool but it cleaned up nicely with mlv_dump:
mlv_dump --dng
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4596%2F24534045197_c8853e59a2.jpg&hash=81cbfd1a3146016a70ec7e5df286883b) (https://flic.kr/p/DnZoTg)
mlv_dump --dng --white-fix=4931
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4684%2F24534050167_b4b00954fc.jpg&hash=64f7a6779f1faf9c8cd4318ab66509c0) (https://flic.kr/p/DnZqmX)
Processed through "dcraw -T" so no color correction, highlight recovery, etc. Now if we can only combine this with the focus pixel fixing that works so well in MLVFS.
Still to do - Figure out why the new maps aren't working with mv1080crop 12-bit lossless and need to run some tests on the 100D.
-
One more change to the focus pixel script (https://bitbucket.org/daniel_fort/ml-focus-pixels/src/f5c24a4701bb2bd1683e3a37f75b8f2994ceb771/fpm.sh?at=default&fileviewer=file-view-default) -- it can now automatically determine crop_rec footage.
Another interesting thing turned up with the latest round of focus pixel map updates. The non-crop maps for all the cameras are now identical. Why so many files for MLVFS? Because that app needs a different file for each full raw buffer size from each of the supported cameras even thought the files themselves have the same contents.
-
Is detecting only 3x3 sampling from RAWC is sufficient? Croprec module for affected cameras always uses 3x3? What about 1x1, is that used in mv1080crop?
I never had any other canon cameras than 7D and 5D3 :)
-
Is detecting only 3x3 sampling from RAWC is sufficient? Croprec module for affected cameras always uses 3x3?
At this stage of development of the crop_rec module, yes. The 100D/650D/700D/EOSM cameras use only one of the crop_rec module options so it is rather easy to figure it out. If it uses the mv720 full raw buffer -- 5x3 sampling is mv720 and 3x3 sampling is crop_rec.
@rbrune first figured (http://www.magiclantern.fm/forum/index.php?topic=16608.msg173854#msg173854) out how to get 3x3 sampling on the mv720 mode raw buffer to get around the lack of mv1080 mode on the EOSM (http://www.magiclantern.fm/forum/index.php?topic=16608.0). At some point maybe @mk11174 or someone else will figure out how to get some of the other crop_rec options working on these cameras (proof of concept (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183286#msg183286)) and then it will be back to the drawing board on how to deal with those new focus pixels.
What about 1x1, is that used in mv1080crop?
Yes, 1x1 is also used in zoom mode. Note that zoom mode cleaned up nicely with the latest map files:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4692%2F27654768469_6f800c7859.jpg&hash=2d7d7d8fef37153c308b2b987baea1b9) (https://flic.kr/p/J8KVbV)
The problem is with mv1080crop. It works fine on 14-bit files but on 8...12-bit, yuck!
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4641%2F38723910484_ca6353f2e5.jpg&hash=cd725b49aba506ad09add3f4f5a08a0b) (https://flic.kr/p/21ZUat3)
I'm taking a close look at it but it is difficult to see exactly what is going on. Both of these modes are using the same focus pixel pattern.
BTW--I had to make one more change in the fpm.sh script. On older MLV files without the RAWC block the script was picking up the audio "samplingRate". Fixed using the whole word (-w) option in grep.
-
Different stream for 8-12bit recordings causing pixels in random places?
-
Turns out that the 12-bit lossless mv1080crop focus pixels also have a pattern to them and can be cleaned up just as well as all the other video modes.
First, how about a flashy video to demonstrate the new focus pixel maps. Note how well they work with focus pixels but doesn't affect the dust on the sensor.
This was done from the 12-bit lossless compression samples @theBilalFakhouri supplied. The video modes that use 3x3 and 5x3 sampling show the focus pixels the most while the 1:1 sampling, zoom and mv1080crop (a.k.a. Movie crop mode or CROP_MODE_HACK) show the least. The focus pixel map file for zoom mode is working fine with 8...12-bit lossless compression but the mv1080crop still had some leftover pixels that seemed to show up in random places--or are they random?
Here is a closeup of the mv1080crop 12-bit lossless sample after running it through the focus pixel map, not all the leftover pixels we missed.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4641%2F38723910484_ca6353f2e5.jpg&hash=cd725b49aba506ad09add3f4f5a08a0b) (https://flic.kr/p/21ZUat3)
Let's take a closer look. Note how these focus pixels pop out as either bright green or magenta dots. The area around these pixels are also affected because of the blending used in the app--dcraw in this case.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4689%2F27675761939_733196bbf8_n.jpg&hash=982f6e89ccbb49ccfae7c72c7d248f3f) (https://flic.kr/p/JaBvPi)
Laying an image file of the focus pixel map file it is clear that we are missing these pixels.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4731%2F39422366582_2d2a1a67e8.jpg&hash=633f0ce553ef2912856701c30c88de54) (https://flic.kr/p/234BW8y)
It is like this all over the image. So did the focus pixels somehow move one pixel to the right? I made another map file with all of the coordinates shifted one pixel to the right and that took care of these two but others showed up. How about combining both map files? This would give a pattern that would hit two adjacent pixels:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4597%2F38745357544_4f77bac3a6.jpg&hash=07c9e4625dedce959d3dc684439535cc) (https://flic.kr/p/222N5VW)
Of course if you are averaging the area around the focus pixel to hide it, this won't work. However, using the 2-pass method that @bouncyball came up with, it worked:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4732%2F27675762849_2c2aee61b2.jpg&hash=755fac89959574be6df083e89d98c7aa) (https://flic.kr/p/JaBw5Z)
I know that some users will note that this isn't perfect. I'm using "dcraw -T" to create tiff files from the DNG's and there is no chroma smoothing. It looks better in Adobe Camera Raw but I wanted to show a worst-case scenario.
So what is going on here? I can only speculate. Maybe this has something to do with the lossless compression? Why is it not an issue in zoom mode? I don't really know but mv1080crop is only available on the cameras that have the focus pixel issue so maybe it has something to do with FEATURE_CROP_MODE_HACK.
Still to do: Update the script and the 100D map files for mv1080crop mode. [EDIT] Done!
-
The new focus pixel map files are ready for testing in MLVFS. You can find them in the ML Focus Pixels repository (https://bitbucket.org/daniel_fort/ml-focus-pixels) under Source/focus_pixel_map_files (http://Source/focus_pixel_map_files).
The fpm.sh script (https://bitbucket.org/daniel_fort/ml-focus-pixels/src/ea76e774d49a9f808dba336f737a0f808c6a7503/fpm.sh?at=default&fileviewer=file-view-default) is also updated and ready for testing.
-
Thanks for your time Dan, really appreciate what you are doing. I'll take a look at your updated script today (hopefully) :)
-
A couple more updates. In the ML Focus Pixels (https://bitbucket.org/daniel_fort/ml-focus-pixels/src) repository there is a scripts_archive directory (https://bitbucket.org/daniel_fort/ml-focus-pixels/src/2ca84f3bcc47b39194449f5352479bcec4c3f75b/scripts_archive/?at=default) with some scripts that I have found useful. A couple of them, fpm2pbm.sh (https://bitbucket.org/daniel_fort/ml-focus-pixels/src/2ca84f3bcc47b39194449f5352479bcec4c3f75b/scripts_archive/fpm2pbm.sh?at=default&fileviewer=file-view-default) and pbm2fpm.sh (https://bitbucket.org/daniel_fort/ml-focus-pixels/src/2ca84f3bcc47b39194449f5352479bcec4c3f75b/scripts_archive/pbm2fpm.sh?at=default&fileviewer=file-view-default) have been particularly useful. The updates improve processing speed though they are still excruciatingly slow. The octave version created by @a1ex, fpm2pbm.m (https://bitbucket.org/daniel_fort/ml-focus-pixels/src/2ca84f3bcc47b39194449f5352479bcec4c3f75b/scripts_archive/fpm2pbm.m?at=default&fileviewer=file-view-default), is much faster but the latest Homebrew octave installation doesn't work with the it so I have fallen back to using my bash scripts.
I choose to work with the plain text version of the pbm (Portable Bit Map) image file format because the file format is very simple, yet I was using ImageMagick in my scripts. The updated scripts no longer require ImageMagic though if it is installed you can now convert pretty much any graphic file format to a focus pixel map file with pbm2fpm.sh.
The main script is still fpm.sh which can create focus pixel map files as needed. It is more of a prototyping script and not too many people will actually use it. @bouncyball converted it to C and enhanced it in his mlv-tools project (https://bitbucket.org/bouncyball/mlv-tools) and it is used in his mlv_dump on steroids (http://www.magiclantern.fm/forum/index.php?topic=18975.0). These tools are also used in Switch (http://www.magiclantern.fm/forum/index.php?topic=15108.0) and other MLV processing apps.
-
pbm2fpm.sh pbm2fpm.sh have been particularly useful. The updates improve processing speed though they are still excruciatingly slow. The octave version created by @a1ex, fpm2pbm.m, is much faster but the latest Homebrew octave installation doesn't work with the it so I have fallen back to using my bash scripts.
I choose to work with the plain text version of the pbm (Portable Bit Map) image file format because the file format is very simple, yet I was using ImageMagick in my scripts. The updated scripts no longer require ImageMagic though if it is installed you can now convert pretty much any graphic file format to a focus pixel map file with pbm2fpm.sh.
I thought for fpm->pbm->fpm conversion you use 'fpmutil' (it does conversion almost in no time, and if you just did not figure out the procedure of doing this I can help). There are some examples printed out in console when "-h" or "--help" command line switch is used.
Yes I know it exports P4 (binary) PBMs, but if you need I can add P1 (ascii) support (have not seen real reason for adding this before). Are you directly editing this ascii pbms in text editor?
Also I can add option to export fpm without a header to simplify things further. Because MLVFS does not support this header and you should delete it manually after every export if MLVFS is the target not mlv_dump on steroids.
Edit: I could add this fpm header support to MLVFS itself... The purpose of this FPM header is the need of knowing exact resolution for the backward conversion to PBM and 'fpmutil' gets this information from my humble header of those fpms ;)
bb
-
I still use my scripts even though they are slower than molasses in January partially because they are the Devil I know. Ok--enough with the idioms.
I did a little experiment on those specks that are showing up on the @theBilalFakhouri test files and this might be the beginning of a feature that developers may want to add to their MLP processing applications. I was going to hold off publishing it until I could optimize the speed of my scripts a bit more but maybe I should just dump the scripts and see if I can get fpmutil to do the tricks that the scripts can do.
We determined that the specks on the footage were not focus pixels but possibly dead/hot/cold/stuck whatever pixels or maybe just dust on the sensor. What if this happened on a shot that we want to use? Maybe there was a speck on the sensor that ruined an entire day's shoot? What if our camera has a few dead pixels? dcraw has a feature to deal with problem pixels. When I started this project I made dcraw "badpixels" files to map out the focus pixels and it worked quite well.
So how about adding the locations of these pixels to the map file? This can be easily done manually but what if you have a bunch of spots to remove and you're more of a visual person--like me, I studied photography not computer science.
This is what I used to spot my photograhs:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4739%2F39511497532_04c97d6e8f_n.jpg&hash=b07f1ac9de57fe8f698cc35c768c50c3) (https://flic.kr/p/23cuKE5)
Kidding aside, one of my college classmates was Russel Brown (http://www.photoshop.com/people/russell), a Photoshop pioneer and I've managed to keep up with technology. Enough of the name-dropping and on with the spotting exercise.
Here is one of the spots that the focus pixel map file didn't remove:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4591%2F39511160302_4b289c6496_n.jpg&hash=12978656f88491e676e13f4f244f5e96) (https://flic.kr/p/23ct2pL)
This isn't a Photoshop tutorial so I won't go into all of the details but I made a new layer and used the pencil tool set to one pixel to spot out that problem pixel.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4596%2F39511160612_62c8f0a4f7_n.jpg&hash=9bb168d824432af8c207ee41ba4f67ee) (https://flic.kr/p/23ct2v7)
After hitting all the problem areas I made the background white. Here's a closeup of one of the spots, notice the x,y coordinates in Photoshop:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4725%2F38831993594_90e435df96_n.jpg&hash=7b610757fd551e8ba3afd5b41a798007) (https://flic.kr/p/22as7NW)
If we add these coordinates to the appropriate focus pixel map file in MLVFS it won't work because the map file covers the full raw buffer and the image we used is cropped out of that buffer. It is easy enough to find out how much to offset the x,y coordinates with mlv_dump using the "-v" option. Tip, pipe it into less so you won't have to scroll back through the entire output.
mlv_dump -v 1736x1152.MLV | less
...
Block: RAWI
...
Res: 1736x1152
raw_info:
...
height 1189
width 1808
...
Block: IDNT
...
Camera Name: 'Canon EOS 700D'
...
Camera Model: 0x80000326
...
Block: VIDF
...
Pan: 72x30
...
There's all the information we need. MLVFS will use the map file named "80000326_1808x1189.fpm" and we need to offset our coordinates 72x30. Since I'm more comfortable with Photoshop than basic arithmetic --
Change the canvas size to the full raw buffer size:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4683%2F39511160712_56ba36734f.jpg&hash=f0101843dfa436dafb408ea9857a9a87) (https://flic.kr/p/23ct2wQ)
and offset the layer we created to "Pan: 72x30":
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4687%2F38831993404_906569466a_n.jpg&hash=c4dd1e19e27f3219166ca787d9109b04) (https://flic.kr/p/22as7KE)
So that same pixel we just looked at is now at the coordinates based on the full raw buffer:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4689%2F39511161902_65984889cc_n.jpg&hash=94e004f31228b5638e01d319fe502cef) (https://flic.kr/p/23ct2Tm)
Fill in the edges with white, save the file. The pbm2fpm.sh script needs a plain text pbm file which surprise, Photoshop doesn't do but if ImageMagick is installed the script can work with pretty much any image file format. Here is a section of the output, note the pixel at 310x290:
667 262
245 263
643 276
310 290
1744 302
1724 314
1706 556
Like I said the script is slower than... well let's just say very slow so I tried fpmutil to convert This is where I had a problem:
./fpmutil fixed_pixels.pbm -o fixed_pixels.fpm
Focus Pixel Map Utility v0.5
****************************
Error: 'fixed_pixels.pbm' header corrupted, map can not be converted
This was a basic (a.k.a. P4 or non-text) Portable Bit Map file exported from Photoshop. Apparently fpmutil requires a specific header in the the pbm file. Speaking of headers, I'm not using any in my fpm files and my scripts strip out any headers. I don't have anything against the use of headers but if the file doesn't have a header we should probably be taking it from the filename like MLVFS and fpm2pbm.sh are doing.
Back to the spotting exercise. Add the list of fixes to the appropriate map file, "80000326_1808x1189.fpm" in this case and save it in MLVFS.
Before:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4647%2F39511161132_7f8def1bc4_n.jpg&hash=23e2867dbe395c5826df87396dd05ec8) (https://flic.kr/p/23ct2E5)
After:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4596%2F38831993544_02ef7f84be_n.jpg&hash=11b3dc368a68f5aa164e1415f6f6e47e) (https://flic.kr/p/22as7N5)
Sure, I could have done it like this: (x + Pan_X) (y + Pan_Y) = (238 + 72) (260 + 30) = 310 290 and just added that to the map file with the same results but imagine adding a feature on an MLV processing app where you can spot out specks as easily as using Spotone.
Another idea is to add a feature in MLVFS where the problem pixels for your camera can be saved in a map file. Maybe using the camera's serial number in the map file name? That way the "dead" pixels will be fixed automatically, sort of the way it was designed in dcraw (https://www.cybercom.net/~dcoffin/dcraw/dcraw.1.html).
Yes I know it exports P4 (binary) PBMs, but if you need I can add P1 (ascii) support (have not seen real reason for adding this before). Are you directly editing this ascii pbms in text editor?
No real need to support Plain PBM. According to the file specifications (http://netpbm.sourceforge.net/doc/pbm.html):
Plain PBM actually came first, but even its inventor couldn't stand its recklessly squanderous use of resources after a while and switched to what we now know as the regular PBM format.
I'm going to continue to support Plain PBM in my scripts unless I can figure out how to write binary files in bash or learn C enough to modify fpmutil to do what I want. Then again maybe I'll just stick with shell scripts and say I'm a die hard follower of the Unix Philosophy (https://en.wikipedia.org/wiki/Unix_philosophy) where all data should be text streams and worse is better (https://en.wikipedia.org/wiki/Worse_is_better).
Edit: I could add this fpm header support to MLVFS itself... The purpose of this FPM header is the need of knowing exact resolution for the backward conversion to PBM and 'fpmutil' gets this information from my humble header of those fpms ;)
I like the idea of having a header because it could eliminate duplicate map files. I would suggest having fpmutil fall back on the filename if the FPM header is missing.
@dmilligan and I had a philosophical discussion about having duplicate fpm files (https://www.magiclantern.fm/forum/index.php?topic=16054.msg162525#msg162525) and code to generate map files on the fly (https://www.magiclantern.fm/forum/index.php?topic=16054.msg162474#msg162474). Maybe it is time to re-think this now that we've got 32 map files, most of them with duplicate data, taking up nearly 30 MB of space.
-
... because they are the Devil I know.
Yup, I know this is the main reason :)
Apparently fpmutil requires a specific header in the the pbm file.
Not that much but alongside with image resolution in the header's standard line it needs camera ID and crop flag info which should be written in the #comment line of the header.
In short fpm and pbm both have this info in the header: camID, Crop flag, Xres, Yres. And the maps can have whatever name you desire. However, if name is not specified, 'fpmutil' names maps according to MLVFS established standard.
FPM header
#FPM 80000326 1808 726 0 -- fpmutil v0.5
PBM header
P4
# 80000326 0 -- fpmutil v0.5
1808 726
I like the idea of having a header because it could eliminate duplicate map files. I would suggest having fpmutil fall back on the filename if the FPM header is missing.
Yes right, fall back is exactly what have to be implemented in addition to the option to export FPMs without header.
... code to generate ... Maybe it is time to re-think this now that we've got 32 map files, most of them with duplicate data, taking up nearly 30 MB of space.
On the fly map generation is implemented to mlv_dump on steroids and MLV App.
Selecting bad/whatever pixels by mouse and adding it to the map, easily can be implemented to the MLV App. And they could be added as a 2nd (or even 3rd) pass coordinates.
I did not add your new unified pattern generators to the 'fpmutil' for now because I have not decided yet, fully substitute old pattern generators with newer ones or leave both. As I said I don't like the idea of touching unaffected healthy pixels by the map.
BTW, we could detect 8-10,12bit lossless raw by the white level, if it's lower then 15000 (or 14000) then it is most likely amplified 14bit lossless (aka 8-10,12bit lossless).
Edit: changes pushed to the 'fpmutil' repository: LINK (https://bitbucket.org/bouncyball/mlv-tools/commits/7942151ac10e1133b54dcea546dbf447d01fe3b0)
regards
bb
-
Selecting bad/whatever pixels by mouse and adding it to the map, easily can be implemented to the MLV App. And they could be added as a 2nd (or even 3rd) pass coordinates.
Sounds good :)
-
Hey @dfort How did you show raw information?
It's look like this after applying mlv_dump -v
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fthumb.ibb.co%2FcxbwxG%2Fff.jpg&hash=714cac8b5631798cf3861f99deed6266) (http://ibb.co/cxbwxG)
mlv_dump -v
raw_info:
api_version 0x00000001
height 1188
width 1808
pitch 3164
frame_size 0x0039674C
--> bits_per_pixel 14
black_level 2047
white_level 5586
active_area.y1 28
active_area.x1 72
active_area.y2 1185
active_area.x2 1808
exposure_bias 0, 0
cfa_pattern 0x02010100
calibration_ill 1
-
On Windows redirect mlv_dump output to a file and use a decent editor (Notepad++ or else) to look for "raw_info".
add
>>output.txt
to the command you want to execute
-
On Windows redirect mlv_dump output to a file and use a decent editor (Notepad++ or else) to look for "raw_info".
add
>>output.txt
to the command you want to execute
Thanks!
-
@theBilalFakhouri -- Sorry, I didn't realize you were using Windows cmd that doesn't have less (https://en.wikipedia.org/wiki/Less_(Unix)). If you want to take a quick look without having to output a file you should be able to pipe it through more (https://en.wikipedia.org/wiki/More_(command)).
mlv_dump -v 1736x1152.MLV | more
The screenshot you posted shows null blocks. Those are commonly seen at the end of a trimmed MLV. What file are you looking at? I'm using your 12-bit sample files.
@bouncyball -- I have changed my philosophy from mapping as few focus pixels as possible to mapping every focus pixel that has ever revealed itself.
My first map files looked like this, which is the pattern we commonly see when new users ask what's up with the dots:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm1.staticflickr.com%2F578%2F22809705629_53bd142a86.jpg&hash=f3cd3fb2afddfd3330d4139d5d5fd756) (https://flic.kr/p/AKBGNH)
However, in certain situations more focus pixels were showing up:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm1.staticflickr.com%2F756%2F22108377514_3b888afd91.jpg&hash=e7254c189bdbbf186202a21d9fa58869) (https://flic.kr/p/zFDdoE)
When we looked at the 100D it showed an even larger area with focus pixels:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1511%2F24032131713_2cec87086e.jpg&hash=3fa616b6f0972065ed28bc72e06c7623) (https://flic.kr/p/CBCXuv)
At first it looked like pattern didn't extend all the way to the edges of the full raw buffer but under certain conditions they did:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1649%2F24653928396_27b5e8aa48.jpg&hash=4ae9bc03b4f139af768fa4604c924c8c) (https://flic.kr/p/DyzPYY)
In zoom mode we can't determine what area of the sensor was used so the zoom mode maps need to cover every one of the 21 possible locations:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1475%2F24510484991_4ab3b20762.jpg&hash=a1381a1c2369dcc55447f91533543f17) (https://flic.kr/p/DkUDfP)
All of this was discovered on plain 14-bit files. In hindsight perhaps I should have assumed that the focus pixel patterns covered the entire sensor because that would have "future proofed" some of the map files.
With crop_rec we found two superimposed patterns and 8...12-bit lossless added a possibility of the focus pixels being offset by one pixel in certain areas. Both of these require 2-pass maps in order to fully zap the focus pixels.
MLVFS only looks at the focus pixel map's filename to determine which map to use so it is necessary to have duplicate maps but it is not able to distinguish between a crop_rec or an mv720 or a 14-bit and a 12-bit map file so we need to cover all possible focus pixel locations. The largest map file covers 206,400 possible locations. Is that excessive? The full raw buffer size is 1872x1060 so that's 1,984,320 before taking the out of bounds and cropped out areas so maybe 10% of the pixels are being processed? My understanding of how chroma smoothing works, all of the pixels are processed with the same (or similar) blending algorithm we're using to remove the focus pixels.
So is it worth making separate maps that cover different possible focus pixel positions or cover all possibilities? The tests I made with the "one size fits all" maps in MLVFS looked good to me so I made a pull request which @dmilligan merged (https://bitbucket.org/daniel_fort/mlvfs/commits/3e54629ac9544aac6b5d692f27247335b118bf2a).
Note that there are still situations where focus pixels might still show up, like in the extreme edges of the frame and when shooting Dual-ISO but these aren't issues that can be solved with the map files.
As far as file headers, we should be free to do anything we want with FPM files because they were invented here but requiring a header in PBM files in fpmutil is not because that header will get stripped out if the file is edited in Photoshop and probably most other image editing applications.
-
So is it worth making separate maps that cover different possible focus pixel positions or cover all possibilities?
The more I'm thinking of pros and cons of interpolation of focus pixels, more I'm biased to the side of covering all image with unified maps as you suggest.
As far as file headers, we should be free to do anything we want with FPM files because they were invented here but requiring a header in PBM files in fpmutil is not because that header will get stripped out if the file is edited in Photoshop and probably most other image editing applications.
Yes I'm talking about FPM header not PBM's. 'fpmutil' respects and violates nothing in the spec of Netpbm. Even comment line accounted.
It's just I'm using comment line for additional information like camera ID and crop_rec flag. If image is written by software like gimp or photoshop comment line does not contain the needed information and this was the reason it outputs error on console.
I changed this behavior today. PBMs are read in this order:
IF (specific comment options) load parameters and data from file
ELSE IF (comment line is absent entirely) try loading only resolution
ELSE IF (proper file name) parse it and derive all available info from it
ELSE IF (command line switches specified) use 'camera' (-c) and 'mode' (-m) values
ELSE show error message saying that required information gotta be given from cli
Now it converts any PBM I throw at it. Extreme example:
./fpmutil -n -c EOSM -m croprec "image_with_ugly_name_and_without_comment_line.pbm"
This produces .fpm with standard MLVFS file name consisting EOSM ID and croprec resolution and no header before actual pixel data coordinates.
As you can see in this example, a new '-n' (--no-header) switch is used which exports only data FPMs w/o header (for out of the box MLVFS compatibility).
LINK (https://bitbucket.org/bouncyball/mlv-tools/commits/7942151ac10e1133b54dcea546dbf447d01fe3b0) to the repository
-
I've got another suggestion for fpmutil and one for mlv_dump on steroids.
I'm auto detecting crop_rec by looking at the "Sampling" -- if we have an mv720 raw buffer with 5x3 sampling it is mv720 mode but if the sampling is 3x3 it is crop_rec. In the future we might get some more resolutions out of the crop_rec module so we should consider how to handle these. One thing that seems clear is that each camera has three distinct patterns, one for each of the binning/skipping methods that are currently being used: 1x1, 3x3 and 5x3 so making rawi_hdr.crop a boolean variable won't cover future crop_rec resolutions.
I'm using mlv_dump on steroids more often than the "official" version because of the added features, including fixing the focus pixels automatically. However, there are times when I want to look at the focus pixels and there doesn't seem to be an option to turn off that feature. Maybe do something like this?
--fpi <method> focus pixel interpolation method: 0 (none), 1 (raw2dng), 2 (mlvfs),default is 1
I should backpedal on my statement to cover all possible focus pixel locations for all situations. At one point I was thinking that if we combine all of the map files we can come up with one "universal" map file that covers all of the video modes but that would probably be taking it too far. In MLVFS we currently have no way to choose between an mv720 and a crop_rec map file so I'm using the 2-pass crop_rec for both video modes. Same for mv1080crop 8...12-bit lossless compression, which also requires a 2-pass map file, verses uncompressed files. However, we should use separate map files when possible for other applications.
One more idea to throw on the wall to see if it sticks is when converting a FPM file to PBM, have an option to create separate image files for each pass. This shouldn't be too hard because of the way 2-pass FPM files are sorted.
[EDIT] I'm sorting 2-pass map before outputting a PBM file so this "combined" image file can no longer be converted back into a FPM file that works properly. I don't know if I'll do this:
fpm2pbm.sh -s <input.fpm> # creates a separate PBM file for each pass
pbm2fpm <file1.pbm> <file2.pbm> etc...
I'm accounting for possible additional passes to remove user defined pixels. How about calling these "Spotone" passes?
-
... If you want to take a quick look without having to output a file you should be able to pipe it through more (https://en.wikipedia.org/wiki/More_(command)).
mlv_dump -v 1736x1152.MLV | more...Those are commonly seen at the end of a trimmed MLV. What file are you looking at? ...
Thank you!
I shot today a video with x5 crop mode at 14bit lossless resolution 1920x818 (14bit uncompressed works perfectly.)
The pixel dots pattern was different compared with full 2520x1072 resolution so no map worked for it.
Make your tests:
https://drive.google.com/open?id=1WdY7djRxC9tJjnJvQw_gQ5er30t_joKF
When I was shooting tests at 2520x1072 14bit lossless with PIXEL SCANNER pattern (Maybe a complex image to compress)
ISO above 100, I get a lot of data corruption and ASSERT**.LOG files compared to more normal scenes. If these files helpful for developing please tell me.
-
That particular file has the focus pixels shifted 8-frames. I think it has happened before but I can't find the post.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4644%2F39541580182_1c2d8bd15c_o.gif&hash=edb570f7ae7a81ca48cd476e59b71ea7) (https://flic.kr/p/23f9Wbs)
zoom mode is a special case because we're not sure what part of the sensor it was recorded on. I thought we covered all possible locations but maybe not?
In other news, the fpm2pbm.sh and pbm2fpm.sh scrips got a big speed boost:
fpm2pbm.sh
time ./fpm2pbm-old.sh EOSM_mv1080crop_640x360.fpm
real 2m16.894s
user 1m32.244s
sys 0m35.004s
time ./fpm2pbm-new.sh EOSM_mv1080crop_640x360.fpm
real 0m30.764s
user 0m17.644s
sys 0m13.061s
pbm2fpm.sh
time ./pbm2fpm-old.sh EOSM_mv1080crop_640x360.pbm
real 1m20.062s
user 1m19.193s
sys 0m0.888s
time ./pbm2fpm-new.sh EOSM_mv1080crop_640x360.pbm
real 0m18.770s
user 0m16.950s
sys 0m2.140s
-
I'm auto detecting crop_rec by looking at the "Sampling" -- if we have an mv720 raw buffer with 5x3 sampling it is mv720 mode but if the sampling is 3x3 it is crop_rec. In the future we might get some more resolutions out of the crop_rec module so we should consider how to handle these. One thing that seems clear is that each camera has three distinct patterns, one for each of the binning/skipping methods that are currently being used: 1x1, 3x3 and 5x3 so making rawi_hdr.crop a boolean variable won't cover future crop_rec resolutions.
Yes we already discussed using binning/skipping values as a crop_rec detection method (sampling) and I saw the script. And this is already implemented in sources on my HDD, not in repo.
Regarding rawi_hdr.crop: it is member of my so called shortcut structure introduced for not using 'raw_info' struct from 'raw.h'. It is uint32_t not boolean. standard C has no idea what boolean type is :).
I'm using mlv_dump on steroids more often than the "official" version because of the added features, including fixing the focus pixels automatically. However, there are times when I want to look at the focus pixels and there doesn't seem to be an option to turn off that feature.
Yes why not. In MLV App you can turn off focus pixel correction entirely. A while back I wanted to introduce --nofixfp (like --nofixcp) in mlv dump. It just mlv_dump on steroids became low priority to me. But I'll add this switch to it.
I should backpedal on my statement to cover all possible focus pixel locations for all situations.
Yes I've halted all my changes regarding your new pattern generators for a while to decide what to do: keep both or substitute older with newer, unified one. Or maybe separate generators by uncompressed and lossless category.
One more idea to throw on the wall to see if it sticks is when converting a FPM file to PBM, have an option to create separate image files for each pass. This shouldn't be too hard because of the way 2-pass FPM files are sorted.
Yes unfortunately 2pass FPM saved as PBM can not be converted back to 2pass FPM because it is already sorted but we have to scan Y coordinate in FPM and when it is less than previous value this indicates the point where second pass starts.
I'm working on PBM data as with bit/pixel array, toggling needed bit/pixel in place. In the pbm save function have to be added support of saving more than one PBM file. Perhaps better way is to add the line numbers of the following passes to the FPM header. All this indeed will take time...
I'm accounting for possible additional passes to remove user defined pixels. How about calling these "Spotone" passes?
Cool name :)
regards
bb
-
I updated mlv_dump on steroids and added '--no-fixfp' switch. LINK (https://bitbucket.org/bouncyball/magic-lantern/commits/ceea3be1a2e6712f1ba1246000a98e94f4479e91)
-
Mlv_dump on steroids rules. Looking forward to see where this hard to fix focus pixels issue will land eventually.
-
@Danne: We'll see :)
-
Quite a day of discovery. Updated the script and the MLVFS fpm files to account for possible 8-pixel shifting in mv1080crop and zoom modes (1:1 sampling) on the EOSM/650D/700D cameras. Tested out great in MLVFS. I haven't done much testing with dcraw lately but maybe I should because it is showing some issues.
@theBilalFakhouri -- Try out these new map files (https://bitbucket.org/daniel_fort/ml-focus-pixels/src/62ad767961f59539ecc368d7c1c6943aaf7afc08/focus_pixel_map_files/?at=default).
@bouncyball -- Thanks for the new feature on mlv_dump on steroids. It really helps with testing! Also keeping an eye on your ml-tools commits.
-
Added latest mlv_dump on steroids along with the later fpm files into Switch for testing. Todo: add the actual switch into menu.
Is fpm.sh updated?
fpmutil status?
-
fpmutil status?
fpmutil is lagging, not updated with new pattern generators yet (so does mlv_dump os). Work is in progress, some other changes have to be done before adding latest generators.
-
Updated the script and the MLVFS fpm files to account for possible 8-pixel shifting in mv1080crop and zoom modes (1:1 sampling) on the EOSM/650D/700D cameras.
As I understand in zoom mode image area (hence current viewable raw buffer region) can be shifted by pressing some buttons on the camera (or because of digital dolly mode or because some lua script).
This can be done on purpose or accidentally in any direction by multiples of 8.
If this is the case, don't you think this is the reason of existance of those shifted buffer MLVs?
-
When you pan around in 5x zoom mode if you get close to the edge of the full raw buffer the buffer jumps to a different position on the sensor. I was able to find 21 possible positions and these are much larger jumps than 8-pixels. We can't determine when it makes the jump or the position of the raw buffer on the sensor so I included all the possible focus pixel locations in the zoom map file. I have referred to this image several times because it clearly shows how the buffer moves around the sensor:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1475%2F24510484991_4ab3b20762_n.jpg&hash=57ae9ae2c74377358358df657316f122) (https://flic.kr/p/DkUDfP)
@dmilligan (https://www.magiclantern.fm/forum/index.php?topic=16054.msg160249#msg160249) and @a1ex (https://www.magiclantern.fm/forum/index.php?topic=16054.msg160318#msg160318) posted some more technical explanations well worth reading. In fact now that I reviewed them I'm not sure why I included Crop in my script, only Pan is needed to line up the focus pixel maps. [EDIT] Strike that, reverse it -- Crop is needed.
There's another plot twist -- the 8-pixel offset is happening in both zoom and mv1080crop modes. Both the 14-bit lossless EOSM mv1080crop MLV file we looked at back in September and the 14-bit lossless 700D zoom MLV from @theBilalFakhouri have the 8-pixel offset.
I shot tests on my cameras in various settings, matching the image size and video modes of those sample files and came up with different crop and pan values. On mv1080crop mode they should match.
"Normal" EOSM mv1080crop 1600x640
Crop: 176x222
Pan: 172x223
"Broken" EOSM 14bit lossless mv1080crop 1600x640
Crop: 176x220
Pan: 170x220
I'm not sure what is going on but is it fixable? Here is the zoom mode test from @theBilalFakhouri:
Before
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4699%2F39596717591_3a56c244de_n.jpg&hash=697aebe762179253bc5fd4c5fe21aca5) (https://flic.kr/p/23k2wBr)
After
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4760%2F25726455688_6398e94230_n.jpg&hash=491f4b006cad958c733e62c410383bb4) (https://flic.kr/p/FcmNYQ)
The mv1080crop example has more color variations so let's use that to see what is going on.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4622%2F39597077851_c05f87cb52_z.jpg&hash=c5b49f67a4471f56a36b0b6749d8b8d0) (https://flic.kr/p/23k4nGP)
Closer view:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4762%2F38701305375_0cf6cb87cd.jpg&hash=52d98b9c0f971ca5a28b776f12c1611f) (https://flic.kr/p/21XUiKM)
If we take the current focus pixel pattern and repeat it every 8-pixels on the x axis it will cover those shifted focus pixels:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4677%2F27820263039_fee548f088.jpg&hash=f7d6e9f22004cba49cb4a773d76bda69) (https://flic.kr/p/Joo7Xz)
But wait! There's more! We need to make a second pass sifted one more pixel to cover the possibility of yet another issue (https://www.magiclantern.fm/forum/index.php?topic=16054.msg195489#msg195489).
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4662%2F38701308485_c2b4671693.jpg&hash=d6c661ab8ad34af7dfe96e3b72859e67) (https://flic.kr/p/21XUjFp)
This is getting pretty extreme but it doesn't seem to have any adverse effects on the image. Note that the noise pretty much matches what we see between the focus pixels.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4695%2F27820268869_be44c87737.jpg&hash=59dc43e657c09912532447b48e5dd3b5) (https://flic.kr/p/Joo9G6)
Ok--let's back up and see at how it looks after removing the focus pixels.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4673%2F39568353052_9bfc1b59a7_z.jpg&hash=de9237f38b8d893ecb7c8224feb9253e) (https://flic.kr/p/23hw9PE)
This is working fine in MLVFS but there are some left over focus pixels if we use the same map file with dcraw.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4609%2F39568079602_a5c8ea231a.jpg&hash=164054519a68fd1a205789d0ce81e679) (https://flic.kr/p/23huKx1)
It is subtle so here's an even closer look:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4765%2F27820518499_87d81cd53e.jpg&hash=373acbbb84cf8be8074fd94c2d8cab81) (https://flic.kr/p/JopqU4)
I have no idea why only those few rows of pixels are still showing up. The map file is definitely covering them.
-
It seems like a bug in panning mode.
I have no idea why only those few rows of pixels are still showing up. The map file is definitely covering them.
They could be secondary artifacts after interpolation of adjacent or nearly adjacent pixels. That is why we are using 2pass mode for some flooded by focus dots maps. 'dcraw' does it differently.
-
It seems like a bug in panning mode.
The terminology could get confusing. As you know the panning information we're talking about here is the position of the upper left corner of the image to the full raw buffer. When I wrote the script I wanted to cover all the possible ways it could be used including legacy RAW and DNG which don't have that metadata. What the script does is to assume the image is centered on the buffer:
./fpm.sh -vm mv1080crop EOSM_14bit_losless_broken_raw_buffer_000000.dng
Camera: EOSM
Camera Model: 80000331
Resolution: 1600x640
Video Mode: mv1080crop
Full Raw Buffer Size: 1872x10**
Pan: 170x221
Crop: 176x220
Output File: EOSM_mv1080crop_1600x640.txt
File Format: dcraw
Not exactly the same as the original MLV but surprisingly it worked.
They could be secondary artifacts after interpolation of adjacent or nearly adjacent pixels. That is why we are using 2pass mode for some flooded by focus dots maps. 'dcraw' does it differently.
That's exactly what is going on. It was probably happening for a while but we haven't noticed it until now. Let's take a really close look at the problem pixels.
No focus pixel map
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4760%2F39599548601_0497898bbb.jpg&hash=a818ff5461fcfc093e5c261c6d1cd94c) (https://flic.kr/p/23kh3aX)
With focus pixel map
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4606%2F38891334494_2965058a7b.jpg&hash=002d50c50ece3a7e0230300c1caf1bd0) (https://flic.kr/p/22fGfMU)
Left over focus pixel after processing in dcraw
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4663%2F38703571525_77dfef854f.jpg&hash=a636cdcea41a9487f2cc6e858f89cb82) (https://flic.kr/p/21Y6Vpk)
What threw me off was that the leftover pixels were not on the original but they were being covered by the map file
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4677%2F24731974677_fe1998ed39.jpg&hash=9fdfc08a8df87af77641acdcb4842964) (https://flic.kr/p/DFtQqv)
My theory is that dcraw is interpolating all the surrounding pixels including that pixel that doesn't need fixing. We can't just remove them from the map file because focus pixels can show up in these locations. Why is it happening on just that row and not on other adjacent pixels? I don't really know but I'll bet that if that row is processed on a second pass it will work in dcraw.
Of course we're already doing two passes on this so maybe it needs third and fourth passes? It is just a matter of how the coordinates are sorted on the map file.
-
Quick update -- doing a second pass on the row of pixels that dcraw left behind worked.
Before
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4611%2F25730635868_d2c522ce74.jpg&hash=a440d80ccbed2a896a64b0ff0903f024) (https://flic.kr/p/FcJeAU)
After
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4670%2F39601255131_37eb85cc87.jpg&hash=542e5d0815f2d0c245315b382116fca6) (https://flic.kr/p/23kqMsT)
This isn't needed on the MLVFS maps but they should probably be updated so everything is consistant.
Of course the more testing the better, especially on the 100D.
-
Looking good!
-
'fpmutil' updated:
1. supports new (latest) pattern generators from @fdort. Activated by '-l' or '--lossless' option or detected automatically if MLV file specified.
2. supports exporting of separate '.pbm' images for each pass automatically. Can be overridden by '-1' or '--one-pass-pbm' option to save 1 pass all in one (old style) PBM.
The long help is here (https://bitbucket.org/bouncyball/mlv-tools)
Commits are here (https://bitbucket.org/bouncyball/mlv-tools/commits/all)
For now, does not support loading back all separate pass PBM files to convert to one FPM again.
regards
bb
-
Nice B! Will take a look when I can. Too much work atm...
-
@Bouncyball.
Tried the latest fpmutil to see if I could get rid of the focus pixels on this file:
https://drive.google.com/open?id=1N7NOWzcbCA5Ltd_hD1ta8rCgFvCsNGez
Coming from here:
http://www.magiclantern.fm/forum/index.php?topic=16054.msg195331#msg195331
Result:
applied -l(lossless) and -m croprec -c 700D and get the following file:
#FPM 80000326 1808 726 1 2 -- fpmutil v0.7
I´m using the latest version of mlv_dump on steroids tested in Switch and outside. All pixels still there it seems.
Tested in MLVFS with dfort´s lates fpm files and all pixels are gone!
-
Thanks Danne, I'll take a look.
-
@Danne
Fixed. Now seems it's working as expected. Try please.
Additionally implemented new feature to specify more than one input '.pbm' files (passes) separated by whitespace on the command line. This will output the combined multipass '.fpm'.
bb
-
Works perfectly! Worked straight up in Switch without any further script tweaks. Tested on a bunch of files from *Bilal* and they were all cleaned:
1736x688.MLV
1736x688(Stretch1152).MLV
1736x1152.MLV
1800x1008.MLV
2520x1072.MLV
mv720.MLV
-
Cool :)
-
What is "outside"?
I'm using fpmutil to look at the multipass map files and I might be starting to understand a little bit of why these "phantom" focus pixels are showing up in dcraw. I'm not really familiar with the way dcraw works thought it is certainly no secret (https://www.cybercom.net/~dcoffin/dcraw/dcraw.c). It looks like the focus pixels (bad_pixels in dcraw) are fixed in the same order that we are creating the map files--starting from the top left (0,0 position) and reading left to right one line at a time. No big surprise there but are the fixes happening before or after debayering? I'm not entirely sure but isn't part of the process to blend surrounding pixels? If that's true then reading from left to right, top to bottom, the surrounding pixels are blended into this pixel:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4698%2F39602094162_1c88c76105.jpg&hash=2d2699e47ea11b018db7f0310e147006) (https://flic.kr/p/23kv5SW)
Thus leaving behind a "phantom" pixel that wasn't on the original image:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4663%2F38703571525_77dfef854f.jpg&hash=a636cdcea41a9487f2cc6e858f89cb82) (https://flic.kr/p/21Y6Vpk)
This can be resolved using a multi-pass map file. That pixel is processed after the row below it so the area sampled is not contaminated with pink. Makes sense but what about this pixel?
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4702%2F39602094312_5b18ae3be6.jpg&hash=d11a21fa94ac978fb347fc8e3dd20df1) (https://flic.kr/p/23kv5Vw)
Using a multi-pass map that top row is processed last so why isn't it picking up the pink? Is it because adjacent focus pixels are saved in complimentary colors that cancel each other out? IDK.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4619%2F39632053131_e82aebf3a1.jpg&hash=1fdef92c378eaffaa568e46772ac2f47) (https://flic.kr/p/23o9CCT)
The mystery remains unsolved.
I keep seeing spots before my eyes.
Have you seen a doctor?
No, just spots.
Continuing with the dcraw tests, there was still one last row of those "phantom" pixels near the bottom of the frame. On MLVFS @dmilligan is doing a special process on the focus pixels near the edge of the frame. I have been recommending adding some chroma smoothing to get those last few stubborn focus pixels. I couldn't find a chroma smoothing option in dcraw but it does have this:
-m number_of_passes
After interpolation, clean up color artifacts by repeatedly applying a 3x3 median filter to the R-G and B-G channels.
Running two passes (-m 2) cleaned up those phantom pixels. At one point we were running chroma smoothing to eliminate focus pixels so I was wondering if maybe this can be used the same way in dcraw. Just to see what would happen I ran it ten times (-m 10) and it worked! Well, almost. There were still focus pixels showing up on the edges of the frame. Some of those pixels weren't even in the original so let's just say they were very bright "phantom" pixels.
I'm getting a lot of mileage on this 14-bit lossless MLV with the shifted pixels. I've been throwing more and more coordinates into the map file wondering just how far we can take it before it breaks. This is what the 4-pass mv1080crop focus pixel map looks like:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4713%2F39632691921_8f14a779a7.jpg&hash=24b10436de4c549c7c0698fab70e1e51) (https://flic.kr/p/23ocUwv)
It doesn't seem to be hurting dcraw:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4719%2F38912457504_f988a17310.jpg&hash=2a86eba8e2e9f035ec0493e0af7a481d) (https://flic.kr/p/22hyvVJ)
MLVFS looks like it might have a problem with it.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4626%2F38912454784_e505053fc6.jpg&hash=d2b3c6ab22bbc71626741b1365c5ba05) (https://flic.kr/p/22hyv7Q)
So I thought I'd try the old single pass map file. Along the way I discovered something interesting--recall this:
"Normal" EOSM mv1080crop 1600x640
Crop: 176x222
Pan: 172x223
"Broken" EOSM 14bit lossless mv1080crop 1600x640
Crop: 176x220
Pan: 170x220
Shifting the old map file 2-pixels did the trick.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4744%2F39591593742_1b79f6f5b5.jpg&hash=0e66d0884f885db283430739e4278426) (https://flic.kr/p/23jzgtd)
However, the results look pretty much the same as the mega map file. Maybe even a little worse? I'm pretty sure I haven't mixed them up.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4741%2F39621150731_c1b99edf5c.jpg&hash=28497a2a3d9734d39a9d6248e5b625a8) (https://flic.kr/p/23nbKJt)
It seems logical that we should be mapping only the fewest locations where focus pixels are showing up. However, it doesn't seem to be harming the image if we map all possible locations. This is good news for the map files we're making for MLVFS because we don't need to swap out map files--I believe Switch is doing this for the crop_rec map files.
While I was working on this post @bouncyball has been pushing updates so the testing continues!
-
This thread is the shit! Great posts @dfort.
-
I think that in order to avoid contamination with unusually bright/dark or different color, in the interpolation code, all the surrounding pixels must be double checked and suspects gotta be excluded from averaging. This will definitely help on very contasty edges too.
Those algos already do some checking (btw doing this on raw (bayer) data is a bigger PITA than doing on debayered RGB image). I've dug in this code long time ago. Have to revisit it if I have more time... no promises though :P
-
When I started this topic I thought it would be challenging but at some point the focus pixel issues would be completely resolved. There seems to be no end in sight.
I've been running acid tests on the new focus pixel maps with all the problem MLV files I could find and they seem to be working fine with MLVFS. Here's an oldie that @DeafEyeJedi shot a while back.
100D mv1080 Dual ISO before focus pixel fixing
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4714%2F39636230121_bde02439dc.jpg&hash=cacf265a625273b87aa0e2b8f64bfc66) (https://flic.kr/p/23ow3iX)
No need to pixel peep to find those focus pixels! Looks pretty good after fixing the focus pixels.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4619%2F38927961404_f89c5d7eb4.jpg&hash=b5b5e29a6bbbe0faee60cacd7d4c9068) (https://flic.kr/p/22iVYGm)
However, I wouldn't recommend Dual ISO in movie files because of the aliasing issues. Check out the far bleachers and the grid lines on the field.
If you want HDR video check out the work @Danne is doing on the HDR script keep original framerate topic (http://www.magiclantern.fm/forum/index.php?topic=21454.0).
Hum--wonder if the Dual ISO aliasing can be smoothed out with the methods discussed in these topics:
Experiment - reducing aliasing in post using optical flow (http://www.magiclantern.fm/forum/index.php?topic=20999.0)
Reducing aliasing in post using enfuse/hugin align_image_stack (http://www.magiclantern.fm/forum/index.php?topic=21089.0)
-
I could need a little more infor about this @Bouncyball:
Additionally implemented new feature to specify more than one input '.pbm' files (passes) separated by whitespace on the command line. This will output the combined multipass '.fpm'.
How is fpmutil working atm? Multipasses as default? I see a command line option for selecting 1pass:
-1|--one-pass-pbm export multi pass '.fpm' as one pass '.pbm'
One more thing. As I'm doing tests I often encounter some bit corruption like pink worm strings in highlights.. Solution is always to select -b 14 for upsizing to 14bit(mlv_dump on steroids). Question is if this should default like in original mlv_dump? Or else some heavy recoding of other functions like vs, cs etc to account for 10/12bit files?
-
Splitting multi-pass map files into separate image files is great for debugging (dedotting?) problem files. Put each of the map files on separate Photoshop layers and you can figure which focus pixels each pass is hitting.
I've got another idea, one that goes all the way back to the first post on this topic:
Here is what I ended up doing to get rid of just one focus pixel--and the diagonal secondary pixel.
1247 121 0
1248 120 0
1248 121 0
1248 125 0
1248 126 0
1248 122 0
1248 123 0
1248 124 0
1248 126 0
1248 122 0
1248 120 0
1248 124 0
Note that I hit the diagonal pixel just once but had to work top and bottom to middle and hit the same pixels several times in different sequences. Think of it as stippling a brush to spot a print--in photography school I spent many hours spotting prints. (By the way, this a map of all the focus pixels for the 650D (https://dl.dropboxusercontent.com/u/22843507/MagicLantern/pixCoord_threshold2068.txt) that was used on the PinkDotRemover tool.) [EDIT] Broken Link
Here is the result. I circled the area where the hot pixel was located and yes, it looked just like those other ones before running the dng through dcraw.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm1.staticflickr.com%2F739%2F22345535026_e44d0ec5a2_z.jpg&hash=5328e9faf891de3a56d947d9d1ae1b54)
Hang in there, this is going to be a little abstract. When we used to spot prints using Spotone it wasn't just one hit of the brush on the print and you're done. You need to hit the same spot repeatedly in order to blend it in to the surrounding area and match the grain pattern. In other words you need to stipple out the spots. Let's say that we have a hot pixel that is ruining all of our footage. How about making a "Spotone" pass by simply adding those coordinates at the end of the fpm file? That last pass shouldn't be sorted and in effect it is doing many passes over the same area until the problem pixel disappears. This is a little hard to do manually but in a GUI each time you hit the brush on a pixel, it should build the map file in the order the pixels were selected. That would simulate the process of spotting a print.
Does this make sense?
In effect it might be doing what the dcraw -m option does but only in the areas that are needed in a predetermined order.
-
How is fpmutil working atm? Multipasses as default?
It concerns only PBM image map output. In some cases, like crop_rec, the map represents combined coordinates of mv720 and mv1080, we call it 2pass map. If output is FPM, exported file is one, if PBM by default will be exported as many PBM files as passes are. When just converting FPM to PBM by default each pass is one PBM. If '-1' specified there will be exported only one combined PBM. In case PBM to FPM convertion you can supply several input PBM files on the command line and they will be combined in one multipass map and exported as FPM.
One more thing. As I'm doing tests I often encounter some bit corruption like pink worm strings in highlights.. Solution is always to select -b 14 for upsizing to 14bit(mlv_dump on steroids). Question is if this should default like in original mlv_dump? Or else some heavy recoding of other functions like vs, cs etc to account for 10/12bit files?
Not every 10/12 bit footage suffers, but yes all processing algorithms (written mostly by a1ex, and modified by dmilligan) been optimized or even specifically written for 14bit raw (black, white, histogram, raw2ev, ev2raw). I modified them as far as I understand the idea of this particular processing method. Now, on 10/12bit stuff, it works better then original code, but it is definitely not the best option. Would be better if author(s) could help to tweak the code further :), or even write some newer, better one :P.
-
I've got another idea, one that goes all the way back to the first post on this topic:
...
wasn't just one hit of the brush on the print and you're done. You need to hit the same spot repeatedly in order to blend it in to the surrounding area and match the grain pattern.
...
area until the problem pixel disappears. This is a little hard to do manually but in a GUI each time you hit the brush on a pixel, it should build the map file in the order the pixels were selected. That would simulate the process of spotting a print.
Does this make sense?
If those annoying, only manually removable pixels are a few, then yes, this makes sense :)
Interpolating bad pixel and then adjacent pixels several times definitely will give pleasing result if some very bright/dark or very different color pixel will not contaminate the average.
bb
-
My first success at removing focus pixels was to use the chroma smoothing in raw2dng. In fact I setup my first ML development environment specifically to get this feature by uncommenting just this one line:
/* useful to clean pink dots, may also help with color aliasing, but it's best turned off if you don't have these problems */
//~ #define CHROMA_SMOOTH
I sometimes still use chroma smoothing to get rid of some stubborn focus pixels. I recently discovered a similar feature in dcraw with the '-m' option. So I thought I'd see how well these methods work without using the focus pixel map files and it looks like I found the same bug in mlv_dump, MLVFS and dcraw.
Let's start with dcraw. The you can look at the code on dcoffin's website (https://www.cybercom.net/~dcoffin/dcraw/dcraw.c) -- search for the median_filter() function to see how it works.
I ran these tests with mlv_dump and MLVFS by removing the focus pixel map files and only using the chroma smoothing options with the same results.
This is what it looks like using mlv_dump on steroids with the --no-fixfp option -- thanks for this @bouncyball
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4719%2F39669287331_769a219e1c.jpg&hash=3d5ee62bc1be99c1ba5e8073f4ede2b2) (https://flic.kr/p/23rrt4R)
Note that there's a row of focus pixels right on the bottom edge of the image. Now let's do two passes with the median filter '-m 2'
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4757%2F25797024828_a1e80420e6.jpg&hash=bfa0be12d8bf3ffc1810dac122a73988) (https://flic.kr/p/FiAuJ9)
Much better but you can still see them. I found that '-m 4' was the limit, anything more didn't help and I tried it all the way to '-m 100'
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4698%2F38771778105_b6a1ef054c.jpg&hash=131ff516f5d018b364888f926c1300e6) (https://flic.kr/p/2258uQR)
Note that the focus pixels on the bottom just won't go away.
Now let's try it again only this time using mlv_dump with some chroma smoothing. I got the same results using cs2x2, cs3x3 and cs5x5:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4702%2F39669287541_f49bdf4649.jpg&hash=3700c8c0ede7723fa2de6fcf517078a9) (https://flic.kr/p/23rrt8t)
With dcraw we could shave off one pixel from the bottom of the image but with mlv_dump we would have to remove the bottom five pixels. In addition, there is another problem with some focus pixels that were close to the side of the image.
cs2x2 and cs3x3 gave the same results:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4619%2F25797024488_8da2228b53.jpg&hash=ffa7936d5a685730e0bd7eb2566a09ee) (https://flic.kr/p/FiAuCh)
cs5x5 also showed a green focus pixel:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4654%2F39669287051_dd8f7e2ca6.jpg&hash=598bb2b7b5147e0c2c614cc7c8ea21c1) (https://flic.kr/p/23rrsZ2)
Is there anything that can be done to improve how chroma smoothing works on the edges of the frame? The chroma smoothing code is a bit over my head.
-
Hey @theBilalFakhouri - @bouncyball updated mlv_dump on steroids with the latest focus pixel maps so going back to your shot, here it is processed through Adobe Camera Raw maxing out on the highlight recovery, clarity, vibrance and saturation. That should bring out any remaining focus pixels.
mlv_dump --dng --white-fix=4931
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4652%2F27892361989_5372f10344.jpg&hash=827d6214746d3806ad43098b5d3c6be7) (https://flic.kr/p/JuKDsV)
There is an area that still shows some focus pixel artifacts:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4764%2F38773658735_4ae816c05c.jpg&hash=92343235e33433f90aa0f7ee00ab51c8) (https://flic.kr/p/225i8Tv)
I doubt they would be a problem unless you stretch the image as far as I have. Adding chroma smoothing eliminates it and I don't see any other issues introduced by chroma smoothing.
mlv_dump --dng --white-fix=4931 --cs3x3
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4658%2F38773654225_b3f31fdff2.jpg&hash=d72c2824bccdcaa8bb1af20b5f1257a8) (https://flic.kr/p/225i7xK)
Click on the images to see them full size.
I take it you didn't use a VAF filter on your 700D. There are a few post production tricks you can use to get rid of the aliasing.
Let's see what happens.
Hum--is there a problem with the youtube tags on the forum? Here's the link:
Yep, some aliasing issues but that has nothing to do with the focus pixels.
-
Amazing work, Dfort and Bouncyball! Congratulations! I can't wait to test the new MLV_Dump on the 100D.
Dfort, could you please post a download link to the latest focus pixel maps, so I can test with MLVFS? Do I just replace the old maps with the new ones in the MLVFS directory or there is more to it? Thank you.
-
The latest map files are in the ML Focus Pixels Bitbucket repository (https://bitbucket.org/daniel_fort/ml-focus-pixels/src/02873af5448795686684b921ade8ca830e9f6cfb/focus_pixel_map_files/?at=default). Just replace the ones that are in MLVFS with these.
-
Thanks a lot, Dfort. I will test as soon as I can.
-
MLV App also updated with:
1. Latest focus dots fix (kudos @dfort).
2. Correct aspect ratio according to RAWC block info.
-
Made some more changes on the fpm.sh script.
When in doubt whether to use mv720 or crop_rec map files, default to crop_rec because it will work with either video mode.
Using the -n option to create full raw buffer sized maps will also default to crop_rec because it is assumed that these map files will be used in MLVFS that can't can't yet differentiate between crop_rec and mv720.
I also looked into removing dependency of the Crop metadata but it turns out that it is necessary to line up the map files properly.
Still todo -- run that 1-pixel offset map for mv1080crop mode only if it is 8...12bit lossless compression. Not a high priority because running additional passes that aren't really needed doesn't seem to hurt the image. At least not much.
-
Did some testing on the 100D. The 1:1 video modes, mv1080crop and zoom, could also have the focus pixels offset 8-pixels to the right. Discussion on the 100D topic (https://www.magiclantern.fm/forum/index.php?topic=16040.msg196160;topicseen#msg196160).
Updated the map files for MLVFS (https://bitbucket.org/daniel_fort/ml-focus-pixels/src/827db7b4f4fc9a271684706300a0e26ea5ad1749/focus_pixel_map_files/?at=default), updated the MLVFS pull request (https://bitbucket.org/dmilligan/mlvfs/pull-requests/13/update-for-1-1-sampling-modes/commits) and updated the fpm.sh script (https://bitbucket.org/daniel_fort/ml-focus-pixels/src/827db7b4f4fc9a271684706300a0e26ea5ad1749/fpm.sh?at=default&fileviewer=file-view-default).
-
Dfort,
You are a genius! Amazing work:
https://www.magiclantern.fm/forum/index.php?topic=16040.new#new
-
Dfort,
Testing the latest January 20-th build for the 100D which allows handheld FRSPs with stabilized lenses, I noticed some focus pixels when shooting at Dual ISO 100/800. Here are the original shot (size reduced) and a 100% crop from the area where they are clearly seen:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fthumb.ibb.co%2FiRYveG%2F100_D_FRSP_DI100_800_Focus_pixels.jpg&hash=1caa551ba8bd4573a9d85c8d0beb88ac) (https://ibb.co/iRYveG)
This is the original image.
Focus pixel area (100% crop) -> Here the "Add image to post" does not embed the 100% crop. I don't know why. So please magnify the above image and you will see the focus pixels along the edges of the cupboard doors.
-
Are you referring to the colored area under the knobs?
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm5.staticflickr.com%2F4710%2F28045142019_4bc2f452e3.jpg&hash=d3729b4b8d460eada28bbc73399749e2) (https://flic.kr/p/JJfFDn)
These aren't focus pixels. You should get similar artifacts on the 7D (camera no focus pixels) as you do on the 100D (camera with focus pixels). That's why some ML users won't shoot without a VAF filter on their camera. Well, except for 5D3 users, note that you can't even get a VAF filter for the 5D3.
-
No, I am referring to the long vertical colored edges of the cupboard doors. They look like a string of red/magenta dots. Please have a look at the photo at 100% magnification and you should be able to see them. When I get back home, I will try to find the 100% crop from the original where these dots are clearly seen.
-
Ok. This is a Dual ISO image? I'm surprised it looks this good. My understanding of how MLVFS fills in the focus pixels for Dual ISO is that it only samples the neighboring pixels on the horizontal axis. This is because Dual ISO has a different ISO every 2-pixels vertically. That's why it is difficult to remove the focus pixels on high contrast vertical lines. Someone please correct me if I'm wrong on this.
-
Dfort,
Following your request, I have now prepared a complete (almost) set of clips of the Pixel Scanner pattern for you to test and see whether you have totally and once for all killed all focus pixels on the 100D or there are still a few left. Using the October 23-rd, 2017 build I shot the clips in the RAW-video mode as follows:
1) Uncropped (at 1736x976 resolution)
2) Movie Crop Mode (at 1736x976 resolution) and
3) 5x-magnification mode (at 2520x1080 resolution)
In all these three modes I shot a clip at 14, 12 and 10 bit uncompressed and 14, 12 and 11...8 bit losslessly compressed bit rates. Here are the download links for the original MLV files that will be active for the next 7 days:
For the 5x_Magnification_100D_FP.rar and the Movie_Crop_Mode_100D_FP.rar archives:
https://we.tl/Kr6ltVMayi
And for the Uncropped_100D_FP.rar archive:
https://we.tl/ttpzKYjXHy
Please note that some of the files (e. g. the ones shot at 10 and 12 bit uncompressed in the Movie Crop Mode) contain corrupt frames but there should still be enough sound frames for you to check for focus pixels. Shooting the moving test pattern was quite a challenge for the 100D! I had to repeat the shots several times to get a few OK.
I hope, this helps.
Happy birthday to you, Dfort and thank you so much for all the presents we received from you lately!
-
Ok. This is a Dual ISO image? I'm surprised it looks this good.
Oh, don't be surprised! This is the 100D - a hell of a camera! Even more so now, that A1ex has added working image stabilization to the silent module which allows for hand held low-light Dual ISO FRSP-pictures with amazingly clean results using stabilized lenses. Once the hiccup issue is fixed, we should be able to shoot even Dual ISO video with comparably clean results at 2520x1080 resolution! For now, the 100D is second best after the 5DMkIII for Magic Lantern work. Now you understand why I am so eager to get the 100D working properly.
-
Please note that some of the files (e. g. the ones shot at 10 and 12 bit uncompressed in the Movie Crop Mode) contain corrupt frames...
Add to that 14 bit uncompressed. That's strange, all of the uncompressed settings show corrupted frames in Movie Crop Mode. You should probably report that on the 100D topic. Maybe that "hiccup" build I made also fixes this issue?
I just processed everything into movie files and need to take a closer look but first impression is very good--at least as far as focus pixels are concerned.
...Now you understand why I am so eager to get the 100D working properly.
Maybe a little less eager on the 7D? Getting 10/12bit working on that camera is a tough nut to crack.
-
Absolutely! I can't wait for 7D to become Number 2!
By the way, you did a hell of of a job with the focus pixels on the 100D. Congratulations! Did you notice that also sound is working on the 8...11bit ant 12bit lossless?
-
I looked closely at the latest 100D tests and the new maps are working quite well when pixel peeping still frames but look what happens when you put things in motion:
Fixing the focus pixels actually seems to help a bit with some of the aliasing issues. I didn't expect that 14-bit lossless exaggerated the aliasing, maybe the camera moved slightly on that take? Not much seems to happen in the 1:1 modes in video but we are seeing the "hiccup" issue in 5x zoom mode. The 14-bit lossless take is a bit short, for obvious reasons, so I slowed it down 50%. Note some glitches in zoom mode but it is perfectly useable. Uncompressed mv1080crop (a.k.a. Movie Crop Mode) was unusable but the lossless footage was quite good--yet another unexpected surprise.
What isn't surprising is the moiré pattern in the 3x3 sampling mv1080 mode caused by the vertical line skipping. The 1:1 sampling modes show almost no moiré pattern or aliasing. The rolling shutter, evident by the slanted lines, is also less in zoom and Movie Crop Mode.
The "no focus pixel fix" footage was processed through mlv_dump and the "with focus pixel fix" was done with MLVFS. I was just trying different workflows in Danne's Switch app. That's part of the reason why there's a slight color shift between the the two. Another reason is because we're removing a bunch of green and magenta dots. In some modes it is predominantly green, in others magenta so even though these are complimentary colors they don't exactly cancel each other out.
Overall, quite an interesting test. I think the take away from this is to avoid shooting moving vertical lines if at all possible! Don't run that video too many times or you might get hypnotized.
-
Dfort,
All clips that I provided were recorded with the camera on a stable tripod. The only shake that could have occured would be when pressing the record button. This is the reason why I recorded at least 3-4s. of every clip so that you have enough useable frames.
I am happy to see that the test clips were helpfull to gain new knowledge on the focus pixel/aliasing behavior of the 100D. I have to say that the camera struggled a lot with this test - the high-contrast periodically moving pattern seemed to bug the camera and very often it would refuse to record the clip, so I had to repeat the recordings several times until I got one recorded, sometimes with lots of corrupt frames. This never hapened with normal scenes, except those shot at 12 and 10 bits uncompressed in the Movie Crop Mode. I don't know how to explain this behavior.
Anyway, you seem to be right by saying that the focus pixel fairy tale is a never ending story, not to mention Dual ISO! What I am very happy about though is the fact that under normal shooting conditions, the focus pixels do not seem to be a problem any more. And I thank you for that!
-
Found the two registers that's related to the lovely pixels dots:
ADTG[8196] N (dwSrFstAdtg1[2], Line count + 1) mv1080=0x12e x5=0x0
ADTG[8197] N (dwSrFstAdtg1[3]) mv1080=0x38c x5=0x0
1-You can make any pattern you want!
Mode=mv1080 | 14bit-lossless | No focus map applied |Original MLVs Here (https://drive.google.com/open?id=1tZlsKwVzH0BGgCLEFN-x50WA5vbH0WaN)
Normal
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fpreview.ibb.co%2FcE6K7c%2FOriginal.png&hash=abaa038ffc94034af24a0b9ab8d327ae) (https://ibb.co/idBmnc)
Tweaked registers (vertical pattern on the whole image) click and zoom in.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fpreview.ibb.co%2Fnd1YHc%2FCustom.png&hash=3da92863ff7c90788bf87c3b2556170a) (https://ibb.co/kD9yiH)
at 0x0 it's become very dim
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fpreview.ibb.co%2FjG2jOH%2F0x0.png&hash=8bacb58b4a76e3eec6ee24f3297de01e) (https://ibb.co/esxJiH)
2-You can shift it myabe. (I lost the settings).
In x5 mode maybe more focus pixels appears and I got corrupted frames while recording when I pushed the values up.
In mv720 mode it's really hard to see what is going on because adtg_gui makes the camera freezes in that mode.
In photo mode these registers disappeared.
Make try on your camera (set preview mode to Framing to see the changes directly).
@dfort
Affected cameras: EOSM, 100D, 600D, 650D, 700D, (and more?)
[EDIT: Pretty sure now that the 600D doesn't show focus pixels in raw video.]
How the 600D was affected in these pixels (same patterns as 700d?) and what is the way that solved it?
@a1ex or @dfort (anyone have 5D3)
Can you check what is the values on 5D3 ? and if can try 700D's values above in mv1080 mode ?
And what is these registers job?
-
How the 600D was affected in these pixels (same patterns as 700d?) and what is the way that solved it?
Actually, nothing was done. The 600D is a Digic 4 camera and only the EOSM/100D/650D/700D (Digic 5 cameras) show focus pixels, as far as we know. The other Digic 5 cameras don't show focus pixels. OK, maybe the EOSM2 because it is from the same family but that has yet to be seen.
Note that if you use the silent module and shoot a simple silent DNG while the camera is in movie mode at the settings that you are trying to test, the saved DNG should also show the focus pixels. Processed through dcraw using the -4 -E options and you'll be able to have a clear idea what needs to be done in order to remove the focus pixels. I know this is a lot to go through but if you read through this discussion you'll see how focus pixel "fixing" works.
You're more than welcome to play around with my script and focus pixel map files:
https://bitbucket.org/daniel_fort/ml-focus-pixels
or try out the more advanced mlv-tools by @bouncyball :
https://bitbucket.org/bouncyball/mlv-tools
-
Thanks @dfort
I am trying to find what are these pixels and why we see them in RAW video.
-
Seems like these mysterious focus pixels keep coming back to haunt me. Another user pointed out that the full raw buffer has changed and MLVFS is showing focus pixels again. That should be fixed with some of the recent commits to the crop_rec_4k branch. In addition, 1920x1080 zoom mode should once again be working on the 100D/650D/700D/EOSM.
These commits will also allow us to simplify the focus pixel map files but if we fix them now anything shot with the crop_rec_4k branch in the last few months might show focus pixels so this is just a note to say that we're aware of the issue and a reminder to once again revise the focus pixel map files.
-
4K and other resolutions are starting to become a reality on the crop sensor cameras including the ones with the focus pixel issue so it is time to start updating the focus pixel maps.
What we need are some frames that show focus pixels at these new crop_rec settings. In addition, we need to capture full raw buffer images of the various settings. The way to do this is to turn on the silent module and set it to simple silent. Next, set the camera to movie settings using the crop_rec settings you want to use. Now simply take some simple silent images by doing half-shutter presses. Make sure to keep the camera in movie mode for this. Upload the MLV and DNG files and post links on this discussion so we can start working on this.
-
This is a good idea I was thinking the same but it's too early for now because of some presets need to adjust the horizontal and vertical offset to make it centered in the right way (now it's close to center) and this will change the position of focus dots.
I will try to do this ASAP.
EDIT: Of course if this will not affect the progress ? I mean we can map the focus dots now and shift it later?
-
I'd like to automate the centering part (to use generic code for all models, rather than hardcoding values for every single preset), so the cropped position might change a bit. Just FYI.
-
Hello @dfort
Here (https://drive.google.com/open?id=1Y07OcPnUw2_WVPLlCp08x-J5rEvL-DaO)a files for 1x3 Binning with full sensor height 1736x3478 it should cover all the focus pixels in this mode. And Here (https://drive.google.com/open?id=1didmJ4k7HfpNOg7NceiusTJJ8AVJwo0G)Full-Resolution LiveView MLV it should cover all focus pixels in 1x1 Binning mode maybe we should calculate only the offsets in MLVApp and etc..
We don't have to capture it in Pixel Scanner pattern. We are using 14-bit lossless and the focus pixels is static, I tried to capture it in the pattern but it's didn't show it better than the above samples.
Can you please generate the focus pixels maps?
@a1ex
Something I didn't get it very well, why the Full-Resolution LiveView MLV file has size of 64 MB (EDIT: It's only 1 frame)? and also it's 10-bit uncompressed. When using silent picture in 14-bit uncompressed it gives Max size ~30 MB ::) . Of course when exporting the MLV to DNG file it gives ~21 MB, but why the MLV gives 3 times larger size?
Thanks!
-
Thanks, I'll look into it this weekend.
-
Oh yeah, this is going to take a while.
1x3 focus pixels
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1945%2F44386142535_a1e3a72c43.jpg&hash=9f2da8bfeda19ac9fcf05c67f2743cb7) (https://flic.kr/p/2aCfzMz)
1x1 focus pixels
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1976%2F43483357050_bbf981a886.jpg&hash=677944cc5969c6400e7d62416cc04861) (https://flic.kr/p/29ftz7A)
The pattern is pretty much as expected. When I first started on figuring out focus pixels I thought the way to do it was to map the full sensor (1x1) and figure out the other resolutions by the binning/skipping patterns. It doesn't quite work out that way in practice so this is almost like starting all over again.
Of course you caught me right when I'm in the middle of several time consuming projects so any help I could get on this would be greatly appreciated.
-
Some more strangeness with these new 1x3 and 1x1 files.
In order to create the focus pixel map file I need to make an image file that shows the entire raw buffer. This is done with dcraw -4 -E. The resulting black and white image should be the same size as the full raw buffer reported in mlv_dump -v.
1x3
Image size = 1736x3478
Full Raw Buffer size = 1808x3508
Block: RAWI
...
Res: 1736x3478
raw_info:
...
height 3508
width 1808
1x1
Image size = 5208x3472
Full Raw Buffer size = 5280x3508
Block: RAWI
...
Res: 5208x3472
raw_info:
...
height 3508
width 5280
However, the dcraw trick doesn't work with these files. I can still figure out the focus pixel coordinates using the Crop and/or Pan information from mlv_dump so all is not lost but thought it was worth posting these findings.
-
@theBilalFakhouri - Got a 1x3 MLVFS focus pixel map file that works with your 700D MLV samples:
https://bitbucket.org/daniel_fort/ml-focus-pixels/src/default/focus_pixel_map_files/80000326_1808x3508.fpm
Test it out if possible--all your files cleaned up perfectly. However, there may be some focus pixels that show up in certain lighting conditions so give it a good workout.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1920%2F45340955181_58159e518f.jpg&hash=51550f46332711ef33c9db00be5561e0) (https://flic.kr/p/2c5Cfjz)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1935%2F44618115474_3dec20b5c3.jpg&hash=f11f1190f128bd67ebba771cf7c63685) (https://flic.kr/p/2aYKvg1)
Made this up from scratch. Still need to work on the 1x1 file then translate these to the fpm bash script and bouncyball's mlv-tools (https://bitbucket.org/bouncyball/mlv-tools/src/default/) so the various mlv processing apps can use them.
If we can get these settings working on the 650D and EOSM this focus pixel map should work with those cameras. The 100D (and EOSM2) will need a different map file.
-
@dfort
Great work and well done! Now 1x3 files is cleaned perfectly even with Dual ISO files. The only small problem we have already knew it is the "height" value in "raw_info" or should I say the offsets. Using mlv_dump -v in 1736x2214 MLV file: (EDIT: of course the resolution was dropped in crop_rec not in mlv_lite, if it was from mlv_lite it will be no offsets)
Block: RAWI
...
Res: 1736x2214
raw_info:
...
height 2244
width 1808
By changing the 1x3 focus map name from 80000326_1808x3508.fpm to 80000326_1808x2244.fpm now the focus pixels will be cleaned nicely in 1736x2214 files. I have tried it in MLVApp by dropping the focus pixels map in the MLVApp.exe directory.
I think we should a trick for that like using the full focus pixels map (The original one) and makes MLVApp checking the height in raw_info and make a calculation for the offsets depending on MLV files info --> Hello @masc @bouncyball @Ilia3101 is this possible ? :D
Same thing with 1x1 Binning but I think this will affect the "width" value too.
Here are MLV files in 1736x2214 Normal and with Dual ISO: Download (https://drive.google.com/open?id=1oSjehj-ejF3--E3hTMXkhYGw3_5SODwa) (The internet speed is bad, will upload ASAP Edit: Uploaded).
However, there may be some focus pixels that show up in certain lighting conditions so give it a good workout.
Can you share a sample? I will make more tests in future.
If we can get these settings working on the 650D and EOSM this focus pixel map should work with those cameras. The 100D (and EOSM2) will need a different map file.
Yes it should be the same in 650D and EOS M.
If I can help in anything please tell me how ;D .
Great work!
-
So cool dfort.
Man, Bilal, we need this thingy on the 5D mark III and the 100D. I am so irregular in my registry quests so it probably takes me 5 years to accomplish(if ever) what you do in a month ;D.
-
@Danne
nikfreak has helped with 100D for first tries to increase resolution in x5 but he is offline for about 10 days. Can you help again @nikfreak :D ? The only problem with me because I don't have these cameras and still I didn't know why it stuck okay let's have more tries to find out why :D
I will try to look into 5D3 too I think we should start from current crop_rec.c it has 1x3 preset. @a1ex is there bleeding edge version for crop_rec for 5D3? :P
-
Can you share a sample? I will make more tests in future.
Not really, it is just that sometimes focus pixels pop up in places we haven't mapped. This map covers everything that shows up in the samples you provided.
As far as the 1808x3508 map working on the 1808x2244 raw buffer, that's lucky though not surprising. Looking forward to checking out that new file once you get it uploaded.
Got a few busy days coming up so don't hold your breath waiting for me to finish working on these new map files.
-
Okay I have updated the download link above. Take your time ;)
-
Man, Bilal, we need this thingy on the 5D mark III and the 100D.
+1 :D
-
@theBilalFakhouri - Here's a map file for the 700D, full resolution 1x1 sampling.
https://bitbucket.org/daniel_fort/ml-focus-pixels/src/default/focus_pixel_map_files/80000326_5280x3508.fpm
-
Hello Dfort and Bouncyball,
As I promissed a while ago, here are a few shortened MLV files shot on the 100D at the high-resolution cropped modes in 14-bit lossless at the following resolutions:
2520x1304
3096x1320
4096x2280
https://we.tl/t-6PWhiEQK3i
The files, intended for the preparation of new focus pixel maps, were shot with the 100D on a tripod at Normal and Dual ISO 100/800 modes - the latter of which providing very good results in term of low noise and high dynamic range on the 100D. Therefore, Dual ISO is of great practical importance and I personally use it very often.
I used Danne's build from June 30, 2018. From all 6 files in the above archive only the 2520x1304 Normal one seems to be free of focus pixels. The other ones, especially the Dual ISO files, need cleaning, therefore new focus pixel maps for MLVFS, MLVDump, MLVProducer and MLVApp are necessary. In the Dual ISO files, the focus pixels are most obvious on the vertical edges of high-contrast elements such as the cupboard doors.
Please let me know if the files are OK for you, if not I can shoot and provide additional ones. Just let me know at what modes/resolutions do you need them.
The above link will be active for 7 days.
-
@dfort
It's working perfectly as expected, Thanks!
-
From all 6 files in the above archive only the 2520x1304 Normal one seems to be free of focus pixels.
Maybe the application you are using can already handle this file. Here's how I check for focus pixels and create a map file. Note that no programming skills are necessary just a basic understanding on how to use the terminal. You'll also need mlv_dump installed on your system along with some common command line tools -- bash, sed, file, dcraw and Imagemagick.
First use mlv_dump to make some dng files. Use the --no-fixfp option to make sure mlv_dump doesn't attempt to fix the focus pixels. (Click on the screenshots if you want to see them at full resolution.)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1962%2F43680359140_ff62702a21.jpg&hash=ef8d626930865f647c1f606bc75c97ba) (https://flic.kr/p/29xTfYm)
Tip: You can drag the mlv file into the terminal window so you don't need to enter the path and file name. I'm working in the same directory as the mlv files to keep the illustrations simple.
If you open one of these dng files in Resolve or ACR you might have a hard time seeing the focus pixels because these applications tend to smooth them out. In order to get them to pop we'll use dcraw to convert one of these dng files to tiff.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1969%2F31624407408_bbfc810561.jpg&hash=d5d0771ef14d0c440af8f43a4ca0250c) (https://flic.kr/p/QbxmxA)
Open the tiff in Photoshop or Gimp. It should look fine when looking at the whole frame.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1919%2F30556997457_af4bac8c48.jpg&hash=31e7ee33dc40ae859bdf35d39757f337) (https://flic.kr/p/NydAKK)
However, zoom in and punch up the saturation and vibrance.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1974%2F30556998397_2c743bd687.jpg&hash=a3b5f277bf57e222d43c3e1de4ff53ed) (https://flic.kr/p/NydB2X)
It is rather easy to create a new layer and hit these focus pixels with the pencil tool but first we need to make some adjustments in order to line up the map file.
The way the apps work with the focus pixel map files is that they first look at the full image buffer size and determine where the image is located on that buffer using the crop and pan information. We can do the same thing in Photoshop.
We are going to adjust the canvas size (not the image size!) so we need to turn that background into a layer.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1972%2F44585051745_2e7c19d4c5.jpg&hash=ef7c0e673cfeec8aa55641f7dac568e9) (https://flic.kr/p/2aVQ3yH)
Next, make the canvas size the same size as the full raw buffer. Use this command to easily scroll through the mlv_dump -v output:
mlv_dump -v 100D_Normal_2520x1304.MLV | less
The full raw buffer size can be found in the RAWI block:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1961%2F43681875080_4648c6d7e9.jpg&hash=fab63e6be2b43f63ce6def5e83f7a6e8) (https://flic.kr/p/29y22Bf)
Adjust the canvas size so it is the same as the full raw buffer:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1978%2F43681948190_d672c07d52.jpg&hash=4b4819af32921c0b94ef795517f2c5ab) (https://flic.kr/p/29y2pkL)
Next, the image needs to be moved around on the canvas so that it matches what is in the mlv file. Once again, using mlv_dump -v:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1908%2F31626262828_30d5c697b0.jpg&hash=6b467de49bdd02175b7e0a3c63a34a7c) (https://flic.kr/p/QbGS6A)
We're interested in the Crop. In many cases the Pan will have the same values but not always. For example, on the 700D full resolution 1x1 file it came out like this:
Crop: 72x30
Pan: 72x31
Ok--back to the one we're working on, move the image so that it is lined up according to the Crop data. Zoom all the way in on the upper left corner of the canvas:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1960%2F30558932527_abab3c4ffe.jpg&hash=d5012a40809132b57ed348a57daf634b) (https://flic.kr/p/NyovZ2)
Now we can make a new layer and start hitting those focus pixels with the pencil tool. Make sure the tool is adjusted to 1 pixel and black.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1909%2F44585653615_74eca49350.jpg&hash=d3e656ee4d765e140e2559fc011be7c6) (https://flic.kr/p/2aVT8tM)
You'll quickly find out that there are lots of focus pixels and this is very tedious. You'll also discover that there is a pattern but if you try to copy this pattern to other areas of the image there might not be any focus pixels there or there may be focus pixels that aren't covered.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1927%2F30560041727_3e1b5fc5a7.jpg&hash=f3ed9c12024eec6ebefab6a82f0cded9) (https://flic.kr/p/NyucHa)
I have never seen an image that shows all of the possible focus pixels, however, by working over an area, looking for a pattern and extending it over the entire image area you will eventually cover all the focus pixels. The way I do this is by copying the layer then moving the copy to extend the pattern, merge the layers, make another copy -- rinse repeat until the entire image is covered.
You'll notice that with the settings that easily show focus pixels, like 14-bit lossless, the focus pixels are confined to a defined area. So why map out the entire sensor? We found out that settings that don't easily show focus pixels, like 8-bit lossless, focus pixels will appear in areas that are not confined to a defined area. They usually show up in high contrast boundaries. Since this is video and the image is constantly changing the focus pixels pop on and off and give a "sparkling" effect. Thus it is important to map out the entire image area.
Now make a new white opaque layer and place it under the layer with the focus pixels that you just made, zoom back a bit and you should see a definite pattern.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1942%2F43683341020_426dfe9907.jpg&hash=39ff8cfcfcab15381f60db74828cf3db) (https://flic.kr/p/29y9xo5)
You'll note that the focus pixel layer extends to areas where the image ends. This isn't a problem though I prefer to remove the extra focus pixels that are in the out of bounds area of the full raw buffer.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1975%2F43683435010_b44ebea608.jpg&hash=d2fdcd8320a4152a9d1e3d952e1f0a13) (https://flic.kr/p/29ya2jA)
Now save the focus pixels as a portable bit map file and name it with the camera model number and the full raw buffer size:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1943%2F44775187144_9d65d69284.jpg&hash=ee1e7997ee58475091cbde6a2ee23f78) (https://flic.kr/p/2bdCxcf)
There are a few scripts and applications that can turn this pbm file into a fpm that can be used in MLVFS. I'm going to use my pbm2fpm.sh [EDIT] I renamed it to img2fpm.sh (https://bitbucket.org/daniel_fort/ml-focus-pixels/src/default/img2fpm.sh) because if you have ImageMagick installed it can use any image file format -- not because it is the best or fastest but I wrote it and know how it works. Photoshop saves what are known as P4 portable bit map files which are binary files. This is a bit of a problem with bash so the script uses ImageMagick to convert it into a P1 ascii file. The script is rather slow though I have optimized it so it isn't as bad as it used to be. I prefer running it with the -v option to see the progress. [EDIT] The -v option is now the default. Use the -q (quiet mode) option if you are piping it through your own scripts.
Ok--here we go:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1939%2F30560717207_54853df3e0.jpg&hash=09309814939de68fe098d3f784fbe4fc) (https://flic.kr/p/NyxEvn)
Now take that 80000346_2592x1332.fpm we just created, drop it in with the other map files in MLVFS and let's check to see if it is working.
Starting with a pixel peeping view of the normal file without fixing the focus pixels:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1950%2F43684992290_9beff82eac.jpg&hash=3046334c54cf76cb6876ad50cb4cbc37) (https://flic.kr/p/29yi1ff)
Here is how it looks like in MLVFS using the map we just created:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1972%2F45502073571_158a1b9de3.jpg&hash=40ec667080ab3331a264de462ca67d51) (https://flic.kr/p/2cjS2cR)
Looks good. How about the dual iso example?
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1915%2F44776677914_aea3faa1cb.jpg&hash=5a1c8e7636a1596a256cd236c61056b6) (https://flic.kr/p/2bdLbmb)
Oh no! Did we somehow miss those focus pixels that are showing up? I put this frame into the Photoshop project and turned on the focus pixel map layer we just created:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1940%2F30561791837_22a9dc6ea1.jpg&hash=1c41e54141498ca9ab1843e206dd43b4) (https://flic.kr/p/NyDaXt)
Looks like those stray focus pixels that are showing up are mapped so what is going on? In order to fill in the focus pixels we need to average in the surrounding pixels. Because of the way dual iso works, the line above or below will probably be so different that averaging the area immediately above or below a focus pixel won't work so we can only average pixels on the horizontal axis. This means that when we hit a contrasty vertical line, there just isn't enough information to fill in the focus pixel. At least that's how I understand this works with MLVFS, other apps may be different.
The best we can do with MLVFS is to turn on some chroma smoothing:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1931%2F44776679664_785f4f35ee.jpg&hash=554fb23b9d9e0c80bbe210a7f0bd5d74) (https://flic.kr/p/2bdLbSm)
Yes, a tiny bit is still there and some users say that they can see some reduced resolution with chroma smoothing on (I can't see it) but that's the limitation we've got to work with until someone figures out something better.
-
Looks like those stray focus pixels that are showing up are mapped so what is going on? In order to fill in the focus pixels we need to average in the surrounding pixels. Because of the way dual iso works, the line above or below will probably be so different that averaging the area immediately above or below a focus pixel won't work so we can only average pixels on the horizontal axis. This means that when we hit a contrasty vertical line, there just isn't enough information to fill in the focus pixel. At least that's how I understand this works with MLVFS, other apps may be different.
I think in MLVApp the focus pixels cleared after applying Dual ISO in this case the image will look like normal MLV and it will be no problem. Not sure but depending in my tests @bouncyball is this right? So the focus pixels map is good but MLVFS way isn't that good.
-
I've got some more work to do in order to get this into the bash script so @bouncyball can update MLVApp. We might need to change from using the full raw buffer size to determine which map file to use to using the full raw buffer width along with the sampling. This should allow us to work with all the various new crop_rec resolutions.
-
Thank you so much, Dfort, for this comprehensive review of the focus pixel issue. I never thought it was so complicated. Congratulations to the smart way you found to generate the focus pixel maps!
-
I think in MLVApp the focus pixels cleared after applying Dual ISO in this case the image will look like normal MLV and it will be no problem.
It turns out that the example that I used already has a focus pixel map file that is in the latest MLV App so I thought I'd check that out.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1970%2F45487859782_bb36b6d700.jpg&hash=0ec422e6bf3c713c516ae8cddb670de2) (https://flic.kr/p/2ciBaWw)
I'd say that's a problem.
The download link should still be valid if anyone wants to try it out:
https://we.tl/t-6PWhiEQK3i
I never thought it was so complicated.
I tried to simplify it as much as possible. You don't need any coding skills to create a new focus pixel map file.
-
Bilal's bug looks interesting:
there is a bug in the generic code when using other PowerSaveTiming registers that are not necessary for 700D, keep ADTG 8172 8178 82b6 and comment out the others ADTG.
[...]
Also after commenting out these ADTG registers some new focus pixels showed in 1x3 Binning mode.
Look up the "others ADTG" on this graph, in particular, 8196 and 8197:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fa1ex.magiclantern.fm%2Fbleeding-edge%2Flv-resolution%2F700D.png&hash=f56287b1aafb4c16937fdaef9d6d51d3)
ADTG 8196 is set to 459 (0x12E NRZI).
ADTG 8197 is set to 759 (0x38C NRZI).
ADTG 82B6 is set to 1191 (0x6F4 NRZI).
ADTG 8172/87 are set to 1193 (0x6FD NRZI).
Notice the first two are roughly symmetrical around the middle of the frame. If you consider the 28px top bar, they become perfectly symmetrical:
(1192+28) / 2 - 460 = 150
(1192+28) / 2 - 760 = -150
Hypothesis: what if these are related to focus dots?
Test images (silent pictures from 1080p24, default LiveView configuration):
fp-dark.dng (https://a1ex.magiclantern.fm/bleeding-edge/700D/fp-dark.dng)
fp-gray.dng (https://a1ex.magiclantern.fm/bleeding-edge/700D/fp-gray.dng)
Setting ADTG 8197 to 596 (close to half of the frame), i.e. dialing 0x37E in adtg_gui, gives the following silent pictures:
fp-dark-8197-596-nrzi37e.dng (https://a1ex.magiclantern.fm/bleeding-edge/700D/fp-dark-8197-596-nrzi37e.dng)
fp-gray-8197-596-nrzi37e.dng (https://a1ex.magiclantern.fm/bleeding-edge/700D/fp-gray-8197-596-nrzi37e.dng)
To my untrained eye, the focus pixels are completely gone from the dark frame in the bottom half!
On the gray frame, they are still present, but apparently somewhat weaker...
Exact half of the frame should be at 610 (dial 0x353 in adtg_gui); not tested.
Somehow I've got a CR2 with focus pixels, too: fp.cr2 (https://a1ex.magiclantern.fm/bleeding-edge/700D/fp.cr2)
-
I have played with these two registers a while ago (https://www.magiclantern.fm/forum/index.php?topic=16054.msg197741#msg197741) and found you can create patterns as you want :P and other things related with focus pixels.
-
Tried tweaking ADTG 8196 and ADTG 8197 but seems it doesn´t do anything to the 100D image in regards to focus pixels. I view through "framing" mode which reveals all pixels in live view window.
Maybe the tweaking shouldn´t be randow so if any ideas/directions are very welcome.
-
Somehow I've got a CR2 with focus pixels, too: fp.cr2 (https://a1ex.magiclantern.fm/bleeding-edge/700D/fp.cr2)
Interesting:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1965%2F30676655437_48c9855b9f_z.jpg&hash=32cbad7bde3fd2e288251fa7824f2afe) (https://flic.kr/p/NJMSUn)
Looks familiar (https://www.magiclantern.fm/forum/index.php?topic=16054.msg157641#msg157641).
I have found that the focus pixel pattern extends the full width of the sensor though they are visible only under certain lighting conditions. Some settings, like reduced bit depth lossless compression, will show focus pixels the full width and height of the sensor but they only show up on high contrast boundaries. That's why I've been extending my focus pixel maps over the full sensor.
If you can find a setting that makes focus pixels vanish, EOSM, EOSM2, 700D, 650D users will greatly appreciate it!
-
So, these pixels have a slightly different response even in the CR2?
To capture that CR2:
- enable adtg_gui in photo mode
- take a picture
- lock all ADTG registers
- take another picture
Narrowed down to ADTG[8900]: setting it to 0 in photo mode reveals that pattern.
Another trick: in 1080p, CMOS[8] is set to 0x400 by default; clearing this bit makes the pattern less noticeable.
-
Another trick: in 1080p, CMOS[8] is set to 0x400 by default; clearing this bit makes the pattern less noticeable.
Wonder if something like that is happening with the reduced bit rate lossless compression settings. The focus pixels seem absent at first but upon closer inspection they show up on high contrast boundaries all over the image area--not just the areas where the focus pixels readily show up in 14-bit images.
I'm a bit behind on my homework but it seems that it should be possible to reduce the number of focus pixel map files if we allow the map file to cover the maximum possible height of the sensor. That's 3508 for 1x sampling and 1168 (theoretically, haven't tested) for 3x sampling. Remember these are full raw buffer sizes, the actual image size will be somewhat less.
-
Wonder if something like that is happening with the reduced bit rate lossless compression settings. The focus pixels seem absent at first but upon closer inspection they show up on high contrast boundaries all over the image area--not just the areas where the focus pixels readily show up in 14-bit images.
With reduced bit rate, we are using a different raw type (https://www.magiclantern.fm/forum/index.php?topic=18393.0) (i.e. not the "minimally processed" Bayer stream we are used to). Just double-checked and had a little surprise - raw type 0x12 (what we use for reduced bit depth) is actually "lv_af_raw" (known to show this kind of dots even on the old 5D2). This is also labeled as DEFCORRE or DEFOE in some firmwares.
Best guess: the focus pixels must be corrected - to some extent - by Canon firmware. Are these dots always the same place? Maybe they are just the result of imperfect interpolation done by Canon?
-
Are these dots always the same place?
The dots always follow a defined pattern but that pattern has shifted horizontally on some builds. I believe that commit 5052fab (https://bitbucket.org/hudson/magic-lantern/commits/5052fabbc8d21446d99cd743b904c90243082720) fixed that so I should probably simplify several of my map files. I tried to have them cover all of the builds including the buggy ones but some of my maps are now huge and need multiple passes to clean up all of the possible the focus pixel locations.
-
@dfort as discussed in the EOS M thread - below is a link to two PNGs showing focus pixels for the EOS M shooting in 2.5k crop mode (5x zoom) at 2520x1308, in 9-bit lossless raw.
Both files are the same, one is in colour and the other I've switched to b&w in case that's helpful? I had to enhance and combine images together in order to see all the focus pixels (it's very tough to see all of them at once, as certain focus pixels become invisible depending on which colour temperature and other colour settings you set):
https://drive.google.com/drive/folders/1WGDLbFGz-j1UjHnjmigQNqnw28GtpGjL?usp=sharing
-
Very nice:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1962%2F32376045258_488d47ea48.jpg&hash=ecc86a8ba4524e34ef7f5af19b7f8034) (https://flic.kr/p/RjXGhS)
I'll need a short MLV file to work on the focus pixel map file. Though one thing you can do is to check the size of the full raw buffer:
mlv_dump -v [file.MLV]
For example (just for illustration, this isn't a 5x zoom file):
Block: RAWI
Offset: 0x00000034
Number: 1
Size: 180
Time: 0.783000 ms
Res: 1736x1120 <-- image resolution
raw_info:
api_version 0x00000001
height 1150 <-- full raw buffer height
width 1808 <-- full raw buffer width
A quick test to see if the current map file is working is to rename the '80000331_2592x1108.fpm' file to the size of your full raw buffer and try it in MLVFS.
-
...showing focus pixels for the EOS M shooting in 2.5k crop mode (5x zoom) at 2520x1308, in 9-bit lossless raw...
Never did get an MLV of that format to work on focus pixels but Danne just sent me a bunch of new crop_rec samples to work on. Included is one shot at 2520x1418 which is probably using the same size full raw buffer as the 2520x1308 MLV.
I'm also doing a big cleanup so there will be lots of changes.
The fpm files will no longer skip over the out of bounds area. It didn't save much space and with these new crop_rec samples the out of bounds areas seem to be not as fixed as they used to be. In addition, it is easier to examine something that has a pattern over the entire area instead of just a portion. We already learned that the focus pixel can show up outside of the "usual" center strip or cross shaped area. Finally, if you are going to process the image, do it to the entire image area to keep it even.
The repository has image files to make it easy to see what the patterns look like. These files were in Portable Bit Map version P1 which are about the most basic image graphics file format possible. The reason I'm using these is because you don't need a special library or dependency to work with these files, they are actually ASCII text files. Anyway, I switched over to PNG format to save space on the repository. The scripts still use PBM but rely on ImageMagic to do the file conversion.
Several of the map files were simplified because with this commit (https://bitbucket.org/hudson/magic-lantern/commits/5052fabbc8d21446d99cd743b904c90243082720) we no longer need to worry about focus pixel slippage. Well, at least we don't need to worry too much. I'm seeing some difference between the patterns of the full resolution vs. the 1:1 sampling. Shouldn't the patterns be the same?
The way I test out new resolutions is by creating layers in Photoshop and manually map out the focus pixels then use a script to convert it to a map file that can be used in MLVFS. Once it works in MLVFS I modify a script that can create the map files "on the fly" and the algorithm can be used in other applications.
One issue that came up with MLVFS is that it looks up which map file to use by the full raw buffer size. When dmilligan coded it that way he was aware that there might be a problem if two settings use the same buffer size but require different map files. That's what happened with the "basic" crop_rec settings so we had to create two-pass map files to work with MLVFS. I'm still working on updating these two-pass map files but the ones that are currently there are working fine.
To use these map files go to the ML Focus Pixels (https://bitbucket.org/daniel_fort/ml-focus-pixels/src/default/) repository, grab the contents of the focus_pixel_map_files and put these files into your working copy of MLVFS. Have fun!
-
Hello Dfort and thank you so much for your continued efforts on the focus pixel maps.
I just briefly tested with a 100D file shot at 2544x1432 resolution. Unfortunately, there are still focus pixels on it. Here is a frame grab:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.ibb.co%2Fkc4WMBM%2F100-D-2544x1432.jpg&hash=15f52558bffa9b66f5bb053bc9c22d93) (https://ibb.co/kc4WMBM)
What I did was: I downloaded the file called:
daniel_fort-ml-focus-pixels-9e3d2758530c
and extracted all 52 .fpm files into my MLVFSx86 directory after removing the old fpm files. Then I used MLVFS fo open the 100D original MLV file in Resolve and noticed the focus pixels. The above frame grab was just taken from the virtual DNG directory, opened in ACR and saved as a JPEG. The focus pixels are clearly seen between the two yellow buildings after magnification.
-
And here is a MLVApp shortened version of the original MLV file:
https://we.tl/t-d47exONMAE
It was shot at 10-bits lossless. Please let me know if you may need more 100D files.
-
That resolution wasn't included in the batch Danne sent me. 10-bit lossless is hard to work with when dealing with focus pixels. Could you shoot something in that resolution using 14-bit lossless? What works best for me is if you can shoot a blank wall, somewhat underexposed. That usually helps bring out the focus pixels.
I'm currently away from home so my ML time is a little sporadic but I do plan to cover all these new resolutions as soon as I can.
-
I sent files for eosm...
-
Could you shoot something in that resolution using 14-bit lossless? What works best for me is if you can shoot a blank wall, somewhat underexposed. That usually helps bring out the focus pixels.
Sure I could but 14-bit lossless is usable only at low cropped resolutions and does not really make much sense. At the resolution of 2544x1432 (16:9) and 14-bit lossless you get less than 1 sec. recording time, while at 10-bit lossless recording is continuous at 18 to 20 fps for that same resolution. The other resolution of practical interest is 2320x1306 (also 16:9). At this resolution and 10-bit lossless recording is continuous at 23 and even 24 fps if slightly underexposed.
In my opinion, if one really wishes to take advantage of the high-res crop modes, 10-bit lossless is the way to go on the 100D and the EOS-M. Quality degradation at this bit rate is negligible compared to 12 and 14 bits lossless but continuous or long enough recording at reasonable frame rates is really a game changer on these small cameras.
-
Even Dual ISO provides excellent results in these high cropped resolutions, greatly improving shadow noise. So, further investigations on the focus pixel problem in these modes is well worth the efforts.
-
What I did was: I downloaded the file called:
daniel_fort-ml-focus-pixels-9e3d2758530c
Huh? There is no such file in the repository. What you are pointing to is a commit that isn't even the one containing the fpm files.
I'm not arguing over what bit depth or compressed/uncompressed setting is best to use. All we're interested in here is to get those focus pixels to show up the best we can in order to make sure we can map all of them. What I was asking for is something that is known to work quite nicely for focus pixel hunting and it only requires one frame so don't worry about continuous recording.
Here is a frame from one of the EOSM files that Danne sent me. Notice that even without pixel peeping it is obvious there is some sort of pattern on the image:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7847%2F46349523694_ece685cb70.jpg&hash=89094f19f5b9be00eb2cf5979591b318) (https://flic.kr/p/2dBKqAS)
I pushed the color and contrast to the max for the focus pixels to pop out so when zooming in to look at the pattern it is very obvious.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7802%2F32136939647_928f488eaa.jpg&hash=7f6958c69e158310a44695590d96541b) (https://flic.kr/p/QXQdwk)
Now let's look at the example that you sent:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7824%2F47026609062_a8a6725687.jpg&hash=e99707d479aa11a7a56239ec7abe6214) (https://flic.kr/p/2eDzEuw)
My first impression was, where are the focus pixels? On first glance they were hard to see, even when zoomed in. I had problems finding focus pixels on reduced bit depth lossless compressed files and I didn't have much time today to fiddle around with this so I asked for something that would be easier to work with. A while back I posted a tutorial on exactly how to create a fpm file for MLVFS (https://www.magiclantern.fm/forum/index.php?topic=16054.msg207474#msg207474)--did you bother reading it? With just a little bit of initiative and absolutely no coding you could have created the file that you need for this.
What file do you need? Let's check your MLV file with mlv_dump, I cut out all the irrelevant stuff:
mlv_dump -v 100D_2544x1432_Focus_Pixels.MLV
...
Block: RAWI
...
Res: 2544x1432
raw_info:
...
height 1460
width 3072
...
Block: RAWC
...
sampling 1x1 (read every line, read every column)
Block: IDNT
...
Camera Name: 'Canon EOS 100D'
...
Camera Model: 0x80000346
...
Block: VIDF
...
Crop: 72x28
Pan: 72x28
...
Your image size is 2544x1432 but MLVFS looks up the full raw buffer size which is 3072x1460. You shot this with a 100D but MLVFS looks for the Canon code which is a hex number: 0x80000346. So what MLVFS is looking for is a file named: 80000346_3072x1460.fpm. Not in your MLVFS package? Not in my repository? Then instead of complaining about focus pixels, make a map file and share it with the ML community! Someone else finds it doesn't work for them? Great--they can complain to you because every line in the repository is annotated with your name, for example:
hg annotate -u 80000346_2592x1332.fpm
dan: 6 0
dan: 18 0
dan: 30 0
dan: 42 0
dan: 54 0
dan: 66 0
dan: 78 0
...
You get the idea.
This is the first sample I've gotten from Danne's new crop_rec resolutions for the 100D. I'm sure there will be lots of others so how about we let it simmer for a while before making the next batch of map files but in the meantime, let's see if there's anything we can do with this sample.
First of all, push the image to make the focus pixels stand out. I'm using mlv_dump to make the DNG files using the --no-fixfp opton to make sure it doesn't attempt to fix the focus pixels and Adobe Camera Raw but you can also do this with MLV App or MLVFS or just about any app that can work with MLV files.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7882%2F47026611562_c15cf40a7a.jpg&hash=5904bf54638298a0e6c9151c65c46749) (https://flic.kr/p/2eDzFeC)
Don't critique the color grading, we're only looking for focus pixels and there they are!
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7898%2F47026614802_cc393997ac.jpg&hash=ad8b376a1cc070af97bc8395d60991b4) (https://flic.kr/p/2eDzGcu)
This file was shot with 1x1 sampling so if we look at one of the 1:1 sampling files for the 100D it should match that pattern, right?
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7863%2F40114344333_65f7444f22.jpg&hash=7e0ffc6a69998442cfafca617e5fb080) (https://flic.kr/p/247Lv1r)
Doesn't look right, does it? What is going on? I was trying to combine the map files for every sample I could get. This was fixed rather recently but I left the map files like this for a while until I get new samples like I just got for the EOSM:
Several of the map files were simplified because with this commit (https://bitbucket.org/hudson/magic-lantern/commits/5052fabbc8d21446d99cd743b904c90243082720) we no longer need to worry about focus pixel slippage.
Looks like this pattern is a better match:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7891%2F47079169241_971cdabe24.jpg&hash=0d17fc98e44152ff0499446504ab8b74) (https://flic.kr/p/2eJe3Nv)
Anyway, it is getting late. I spent too much time explaining and too little time working on the new map file -- sort of how you spent too much time writing about the virtues of reduced bit depth and lossless compression and too little time reading up on how to deal with focus pixels.
To be continued...
-
Huh? There is no such file in the repository. What you are pointing to is a commit that isn't even the one containing the fpm files.
Dfort,
This is not true! The file called:
daniel_fort-ml-focus-pixels-9e3d2758530c.zip
that I downloaded from your download area, is in my download directory. It is a ZIP file, 12.4 MB of size. Inside, there is a directory called:
focus_pixel_map_files
which contains all 52 .fpm files that I used.
Please excuse my ignorance regarding the focus pixel issue. I do read a lot, but coming from a completely different background, I find it quite hard to understand what you guys are writing. And yes, I read your tutorial several times and got the idea. I even tried to create a fpm file but, unfortunately, did not come very far. And this is not a question of initiative. It's just that my knowledge and experience in the area are quite limited. Sorry about that. Nevertheless, by sharing my findings on this forum, I try to help as best as I can.
Anyway, I shot a plain surface with the 100D, as you requested. I did that at the maximum 3000x1432, 2520x1418 and 4056x2554 resolutions - all found in the 100D crop menu. These MLV files were filmed at all lossless bit depths available: 14, 12, 10, 9 and 8. I will shorten and rename the files and if you need them, I can send them to you later today. Please let me know if you would like me to do that.
-
For Dfort and anyone else who wishes to give it a try and create focus pixel maps for the 100D, here are a few test MLV files of a plain textured surface:
https://we.tl/t-QZNKv04pKQ
They were shot slightly underexposed at the 3 maximum resolutions and 5 bit depths described in my previous post #587. Please let me know if these files work for you and if yes, I will shoot and provide test files at all remaining modes in the crop menu.
-
...The file called:
daniel_fort-ml-focus-pixels-9e3d2758530c.zip
that I downloaded from your download area, is in my download directory...
Ah--my bad. Didn't know that downloading the entire repository from Bitbucket names the zip file like that. Pretty clever actually. That hex number at the end tells you the latest changeset included in that download.
A much better way to download the repository is to use the revision control system that we're using on Magic Lantern which is called Mercurial (https://www.mercurial-scm.org/) or hg for short. This you should know, it is the symbol for the the metal mercury from the periodic table of elements. Note that hg is a command line application--I take it you are using command line applications, right?
Did you know that you can run MLVFS from the command line? Why would you want to do that? So you can see which map file, if any, it is using. You can read more about it here (https://www.magiclantern.fm/forum/index.php?topic=13152.msg166584#msg166584) and on dmilligan's repository (https://bitbucket.org/dmilligan/mlvfs/src/master/). By the way, this is a GIT repository which is somewhat different from hg but very similar.
...I do read a lot, but coming from a completely different background, I find it quite hard to understand what you guys are writing...
Open source projects like this welcomes input from users from all sorts of backgrounds. My understanding is that you come from a science background so you should have an advantage over most users. I studied photography, Danne is a guitar instructor. Do you think we understand this stuff? Still, not knowing what we're doing isn't keeping us from diving into the deep end.
...Anyway, I shot a plain surface with the 100D, as you requested. I did that at the maximum 3000x1432, 2520x1418 and 4056x2554 resolutions - all found in the 100D crop menu. These MLV files were filmed at all lossless bit depths available: 14, 12, 10, 9 and 8....
You are working too hard and aren't following the instructions I gave you--
...Could you shoot something in that resolution using 14-bit lossless? What works best for me is if you can shoot a blank wall, somewhat underexposed. That usually helps bring out the focus pixels.
I'm currently away from home so my ML time is a little sporadic but I do plan to cover all these new resolutions as soon as I can.
Sure, I know you prefer to work with other bit depths but--
...All we're interested in here is to get those focus pixels to show up the best we can in order to make sure we can map all of them. What I was asking for is something that is known to work quite nicely for focus pixel hunting and it only requires one frame so don't worry about continuous recording....
What would really help is to shoot every crop_rec resolution possible. I'd also like to get the basic ones, mv720, mv1080, mv1080crop and zoom modes for the 100D using a recent build because I'd like to see if we can simplify the map files. Make sure to label them so I know what we're working with. Here's what Danne sent me for the EOSM:
Rosie-Forts-Computer:focus_pixel_MLV_all_presets_eosm rosiefort$ gls -v
01_anamorphic.MLV '014_4K 4038x2558_2351.MLV'
02_anamorphic_2351.MLV '015_4K 4038x2558_169.MLV'
03_anamorphic_169.MLV '016_3K 3032x1436.MLV'
'04_mv720p 1736x694 50fps.MLV' '017_3K 3032x1436_2351.MLV'
'05_mv720p 1736x694 50fps_2351.MLV' '018_3K 3032x1436_169.MLV'
'06_mv720p 1736x694 50fps_169.MLV' '019_2.5K 2520x1418.MLV'
'07_mv1080p 1736x976 46:48fps.MLV' '019_2.5K 2520x1418.wav'
'08_mv1080p 1736x976 46:48fps_2351.MLV' '019_2.5K 2520x1418_000002.dng'
'09_mv1080p 1736x976 46:48fps_169.MLV' '019_2.5K 2520x1418_000003.dng'
'010_mv1080p 1736x1158.MLV' '019_2.5K 2520x1418_000004.dng'
'011_mv1080p 1736x1158_2351.MLV' '019_2.5K 2520x1418_000005.dng'
'012_mv1080p 1736x1158_169.MLV' '020_2.5K 2520x1418_169.MLV'
'013_4K 4038x2558.MLV' '020_2.5K 2520x1418_2351.MLV'
Oops--this just in:
Rosie-Forts-Computer:WORK rosiefort$ gls -v
2520x1418_8bitLL.MLV 3000x1432_8bitLL.MLV 4056x2554_8bitLL.MLV
2520x1418_9bitLL.MLV 3000x1432_9bitLL.MLV 4056x2554_9bitLL.MLV
2520x1418_10bitLL.MLV 3000x1432_10bitLL.MLV 4056x2554_10bitLL.MLV
2520x1418_12bitLL.MLV 3000x1432_12bitLL.MLV 4056x2554_12bitLL.MLV
2520x1418_14bitLL.MLV 3000x1432_14bitLL.MLV 4056x2554_14bitLL.MLV
There are only three resolutions here. According to Danne's latest:
static const char * crop_choices_100d[] = {
"OFF",
"2.5K 2520x1418",
"3K 3000x1432",
"4K 3x1 24fps",
"5K 3x1 24fps",
"4K 4056x2552",
"2K 2520x1080p",
"mv1080p_mv720p mode",
"3x3 720p",
"1x3 binning",
"3x crop mode",
To be continued...
-
...here are a few test MLV files of a plain textured surface:
Textured surface? That just makes it harder to see the focus pixels. What I asked for was:
...What works best for me is if you can shoot a blank wall...
Check what Danne's samples look like compared to yours:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7830%2F32142177697_a04f92fb8f.jpg&hash=be92342edf37ea6914f4e87eb86a13f8) (https://flic.kr/p/QYi4Bx)
Also note that these latest samples don't include the settings you used for the first shot that you sent me. How about I start with that one while you sort out all the resolutions on the 100D. Note that this is something that I would do myself but I don't have access to a 100D at the moment.
To be continued...
-
New 100D focus pixel map files for the samples submitted by IDA_ML are now ready in the repository (https://bitbucket.org/daniel_fort/ml-focus-pixels). These were all 1:1 sampling and the same pattern were used on all of the samples. A couple of the samples shared the same map files. That's why there are only three new map files for the four different settings.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7862%2F33215099268_0963139a49.jpg&hash=657beb160b9093de8eb7cc97444afa2f) (https://flic.kr/p/SB74Pj)
Note that if you click on the image to get to my flickr account and take a very close look you'll notice a pattern on some areas that should be smooth. That's just the way MLVFS combined with ACR looks. Once these maps are working with MLV App we should be able to get even better results.
-
Beautifully done!
-
Nice work, Dfort. The image really looks clean and free of focus pixels.
Please note that shooting a "blank wall" as you suggested, is not a good idea at all. There are countless blank walls out there but very few of them make the focus pixels pop. I shot several walls, including white and dark ones and none of them worked. Finally, I went through our home library and after shooting test files of several dozens of book covers of different colors and texture, I finally found an old book that did a very good job. Here are a few test files shot at different resolutions in the 3k crop mode of the 100D using that book target:
https://we.tl/t-tnfeKJNJLn
The files were shot slightly underexposed and at 14-bit lossless, as you requested but in my experience, lower bit rates (e.g. 10bits LL) make the focus pixels pop up a little bit better. This is the reason why I shot my previous set of files at all 5 bit depths for you to check and see which one works best for you. If you open the above files in MLVApp, set exposure to -4, saturation and dark strength to 100%, dark range to 80% and play a bit with the curves, all focus pixels will be clearly visible on a dark background.
-
Marvelous work, @dfort! :)
-
When I borrowed a 100D from @DeafEyeJedi and got my first look at the focus pixels from that camera I thought this was quite a different animal. The area that the focus pixels covered and the pattern was quite different from the EOSM and 700D that I was just familiar with. However, now that I've seen lots of samples and some of the bugs in ML that caused the focus pixels to randomly change positions have been addressed, it is looking like all of these cameras that show focus pixels in raw video work pretty much the same. In some settings the focus pixels show up in plain flat colored areas while other settings seem almost focus pixel free until you take a close look at high contrast boundaries.
Please note that although it may seem that I'm only having a discussion with @IDA_ML, if anyone with a 100D would like to help out please do!
Please note that shooting a "blank wall" as you suggested, is not a good idea at all.
My preference is to work with something that doesn't interfere with the pattern. Very early on in one of my first posts of this topic I shared this image:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm6.staticflickr.com%2F5819%2F22203578608_436dd64a17.jpg&hash=8227f0d62df7ebbf5f6a0248fc168b17) (https://flic.kr/p/zQ49mQ)
That's a flat white wall shot at 14-bit no compression -- we didn't even have reduced bit or lossless compression back then. Creating an image like this is still possible so why isn't it a good idea? It certainly is easier to work with than a heavily textured image like the ones you are shooting:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7919%2F40131013353_6923c15550.jpg&hash=7c09213491b42939b5a5f7ead71d255d) (https://flic.kr/p/249eW8k)
...in my experience, lower bit rates (e.g. 10bits LL) make the focus pixels pop up a little bit better....
My experience is mostly with the 700D and EOSM and those cameras show the focus pixels only on high contrast boundaries when shooting with the reduced bit depth compressed settings. In fact it was nearly impossible to figure out where the focus pixels were until @theBilalFakhouri came up with a moving pattern that made it easy to locate the focus pixels and map them out.
The reason this happens is because with reduced bit depth we're using a different "raw type" @a1ex can explain it better:
With reduced bit rate, we are using a different raw type (https://www.magiclantern.fm/forum/index.php?topic=18393.0) (i.e. not the "minimally processed" Bayer stream we are used to). Just double-checked and had a little surprise - raw type 0x12 (what we use for reduced bit depth) is actually "lv_af_raw" (known to show this kind of dots even on the old 5D2). This is also labeled as DEFCORRE or DEFOE in some firmwares.
The 100D might be different but from your samples I don't see any advantage of using reduced bit depth settings for mapping focus pixels.
Now let's take a look at your latest samples. It looks like we already have a focus pixel map for these files. In fact all of these samples have the same full raw buffer size so they are using the same map file:
for f in *.MLV; do mlv_dump -v "$f" | grep 'Input MLV file\|height\|width'; done
- Input MLV file: '100D_3k_2288x1288_14bitLL_1.MLV'
height 1460
width 3072
- Input MLV file: '100D_3k_2304x1296_14bitLL_1.MLV'
height 1460
width 3072
- Input MLV file: '100D_3k_2320x1306_14bitLL_1.MLV'
height 1460
width 3072
- Input MLV file: '100D_3k_2336x1314_14bitLL_1.MLV'
height 1460
width 3072
- Input MLV file: '100D_3k_2352x1324_14bitLL_1.MLV'
height 1460
width 3072
- Input MLV file: '100D_3k_2512x1414_14bitLL_1.MLV'
height 1460
width 3072
- Input MLV file: '100D_3k_2528x1422_14bitLL_1.MLV'
height 1460
width 3072
- Input MLV file: '100D_3k_2544x1432_14bitLL_1.MLV'
height 1460
width 3072
- Input MLV file: '100D_3k_2560x1432_14bitLL_1.MLV'
height 1460
width 3072
- Input MLV file: '100D_3k_2576x1432_14bitLL_1.MLV'
height 1460
width 3072
- Input MLV file: '100D_3k_3000x1432_14bitLL_1.MLV'
height 1460
width 3072
Looks to me like these are all different image sizes of the same crop_rec setting. In fact the 80000346_3072x1460.fpm map file that is in my repository is working fine on these samples.
I might not have made it clear but what I'm looking for is a sample from each of these settings if you are using Danne's bleeding edge build:
static const char * crop_choices_100d[] = {
"OFF",
"2.5K 2520x1418",
"3K 3000x1432",
"4K 3x1 24fps",
"5K 3x1 24fps",
"4K 4056x2552",
"2K 2520x1080p",
"mv1080p_mv720p mode",
"3x3 720p",
"1x3 binning",
"3x crop mode",
I'd also like to get samples of the old standards, mv1080, mv720, mv1080crop and zoom. Finally, using one of the crop_rec_4k builds from the experiments download page (https://builds.magiclantern.fm/experiments.html) I'd like a sample for the "basic" 3x3 sampling using the mv720 buffer. The reason is because I'd like to see if I can simplify some of these map files. There was a fix (https://bitbucket.org/hudson/magic-lantern/commits/5052fabbc8d21446d99cd743b904c90243082720) rather recently so I need new samples to work with.
You don't need to give me all the various bit depths or compression settings. You don't even need to shoot different image sizes or aspect ratios. What we have learned so far is that the pattern doesn't change though the area where the pixels appear does change somewhat. That's part of the reason why I'm now mapping the entire raw buffer, including the out of bounds areas because those areas seem to be changing with some of these new crop_rec settings.
By the way, the reason we can get away with using one focus pixel map file per setting when changing the image size is because the crop/pan position is telling us what part of the full raw buffer is being used. Going back to those latest samples, the crop and pan metadata (https://www.magiclantern.fm/forum/index.php?topic=16054.msg160249#msg160249) tells you where the upper left corner of the image is in relationship with the full raw buffer:
for f in *.MLV; do mlv_dump -v "$f" | grep 'Input MLV file\|Crop\|Pan'; done
- Input MLV file: '100D_3k_2288x1288_14bitLL_1.MLV'
Crop: 104x48
Pan: 100x49
- Input MLV file: '100D_3k_2304x1296_14bitLL_1.MLV'
Crop: 96x44
Pan: 92x45
- Input MLV file: '100D_3k_2320x1306_14bitLL_1.MLV'
Crop: 88x40
Pan: 84x40
- Input MLV file: '100D_3k_2336x1314_14bitLL_1.MLV'
Crop: 80x36
Pan: 76x36
- Input MLV file: '100D_3k_2352x1324_14bitLL_1.MLV'
Crop: 72x30
Pan: 72x31
- Input MLV file: '100D_3k_2512x1414_14bitLL_1.MLV'
Crop: 72x28
Pan: 72x28
- Input MLV file: '100D_3k_2528x1422_14bitLL_1.MLV'
Crop: 72x28
Pan: 72x28
- Input MLV file: '100D_3k_2544x1432_14bitLL_1.MLV'
Crop: 72x28
Pan: 72x28
- Input MLV file: '100D_3k_2560x1432_14bitLL_1.MLV'
Crop: 72x28
Pan: 72x28
- Input MLV file: '100D_3k_2576x1432_14bitLL_1.MLV'
Crop: 72x28
Pan: 72x28
- Input MLV file: '100D_3k_3000x1432_14bitLL_1.MLV'
Crop: 72x28
Pan: 72x28
Interesting how those last several settings share the same crop/pan settings. This means they all start at the same point in the raw buffer but the lower right point is at a different location. If anyone cares to follow my tutorial on creating focus pixel map files (https://www.magiclantern.fm/forum/index.php?topic=16054.msg207474#msg207474) you will get a better understanding of how this works.
Finally--wouldn't it be great if focus pixels magically just vanished? There are some users and developers here on the ML forum working on that. Let's hope they succeed!
-
Finally--wouldn't it be great if focus pixels magically just vanished?
We might have gone as far as we can go, at least for now. Danne has come up with a setting that is almost focus pixel free on the EOSM:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7808%2F33228471908_5820f339d7.jpg&hash=928e468a22ada519d8aea9c07dea9d52) (https://flic.kr/p/SChB35)
Looks pretty good but if you look closely, zoom in and adjust vibrance, you'll see there are still some stubborn pixels there:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7908%2F47103711501_e564eb3310.jpg&hash=30f6be3f0f8c32d2aadb9858a72305cf) (https://flic.kr/p/2eLoQmV)
So did we find an intermediate stage where the focus pixels are processed in camera or maybe this is as far as it goes? I have a theory that perhaps the rest of the focus pixel removal might be done when the image is resized and compressed to H.264. Shooting an H.264 and comparing it to a raw image the compression artifacts blur out the image so just as an experiment what happens if we process this image into H.264?
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7827%2F46189890295_7f0352bb10.jpg&hash=f8403e301d8e8f4fec488b78c2b69a12) (https://flic.kr/p/2dnDg9T)
Looking at the full frame it is hard to tell the difference but zooming in:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7905%2F33228522768_415700a7df.jpg&hash=6abb86a7861883986af2c67f84aefbd1) (https://flic.kr/p/SChS9Y)
Pushing the vibrance way up they are still barely visible. Compressing it just a little bit more will blur them out. This image has gone from MLV to TIFF (via dcraw) to H.264 (via Adobe Media Encoder) and back to TIFF to upload on flickr. There may be some additional chroma smoothing or other processing going on in camera but you get the idea, maybe the in camera process to remove the focus pixels is not quite perfect because the resize and compression processes will blur out whatever is left over.
Speaking of imperfect processes, I've been a bit grumpy with a few of you and have not been very tactful with some of my posts. Please accept my apologies if I have offended anyone. On the other hand, please remember that we're all free to work on whatever we feel like. Not following instructions and posting over and over with what you want done is just going to push that item further down the developers' todo list.
-
Good theory. We could also rerun some of the raw_types/streams. Maybe this time an alternate one will work better.
-
Dfort,
I am sorry to see that the test files that I provided did not meet your expectations. I thought that something like this:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.ibb.co%2F4pmdWT9%2F100-D-Focus-Pixels-Book-Target.png&hash=71ceb4c1c66371ffc9159d86e103af3e) (https://ibb.co/4pmdWT9)
reveals the focus pixels quite well.
Here is a new set of focus pixel test files shot according to your instructions - this time of a plain white untextured surface at 14-bits uncompressed:
https://we.tl/t-SXg5P97HMO
The files were shot with Danne's January 25-th build which is not the latest but was stable enough for the shots. They cover all normal and cropped modes currently available on the 100D. I hope, I didn't miss any. If I did, please let me know which ones they are and I will prepare them for you. Also, please let me know if the above files work for you.
Over the weekend, I will prepare more test files using the older builds as you suggested. It would be helpful if you provide download links to the specific builds that you want me to use for the preparation of those files.
-
The new crop_rec settings are in active development so we're trying to hit a moving target. What I'm interested in seeing is if there are a limited number of full raw buffers and if once we make a map file for a full raw buffer will it work on a different setting that uses the same sized raw buffer.
Ok--let's see what we got:
100D_Files_for_FPM_15_02_2019 rosiefort$ for f in *.MLV; do mlv_dump -v "$f" | grep 'Input MLV file\|height\|width'; done
- Input MLV file: '100D_1736x1184_14bit_1x3_binning.MLV'
height 1217
width 1808
- Input MLV file: '100D_1736x576_14bit_60fps.MLV'
height 726
width 1808
- Input MLV file: '100D_1736x720_14bit_60fps.MLV'
height 726
width 1808
- Input MLV file: '100D_1736x976_14bit_3x_crop_mode.MLV'
height 1189
width 1808
- Input MLV file: '100D_1736x976_14bit_Normal.MLV'
height 1189
width 1808
- Input MLV file: '100D_1736x976_14bit_mv1080p_mv720p_mode.MLV'
height 1189
width 1808
- Input MLV file: '100D_1800x1008_14bit_Movie_Crop_Mode.MLV'
height 1059
width 1872
- Input MLV file: '100D_2520x1076_14bit_Normal_5xMag.MLV'
height 1107
width 2592
- Input MLV file: '100D_2520x1080_14bit_2k.MLV'
height 1108
width 2592
- Input MLV file: '100D_2520x1416_14bit_2k5.MLV'
height 1447
width 2592
- Input MLV file: '100D_3000x1432_14bit_3k.MLV'
height 1460
width 3072
- Input MLV file: '100D_4000x750_14bit_4k.MLV'
height 779
width 4072
- Input MLV file: '100D_5008x708_14bit_5k.MLV'
height 738
width 5080
These are the current 100D map files in the repository:
ls 80000346*
80000346_1808x1189.fpm 80000346_1872x1059.fpm 80000346_2592x1332.fpm
80000346_1808x1190.fpm 80000346_1872x1060.fpm 80000346_2592x1447.fpm
80000346_1808x726.fpm 80000346_2592x1107.fpm 80000346_3072x1460.fpm
80000346_1808x727.fpm 80000346_2592x1108.fpm 80000346_4128x2582.fpm
On first glance it looks like the map files are working so we only have to deal with a few new settings. Great!
Like the design in the middle of these samples, it actually helps show the pixels without getting in the way.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7905%2F32164024667_3d8a735b8d_n.jpg&hash=e5b82d4227fe46b9363237bc2babba9d) (https://flic.kr/p/R1e2Xr)
@Danne -- The other theory is that the focus pixels are mapped out somewhere in the firmware and we haven't found it yet. My understanding is that sensors often have defective pixels and instead of rejecting the chips the dead pixels can be mapped out. Your EOSM has some problem pixels and I've got an idea to take care of that.
Just not yet--got to run off now and take care of other business.
-
Went through the new samples from IDA_ML and as expected several of them work out of the box with the map files that are already in the repository. There were a few new resolutions that will require new map files but one in particular turned out to be a problem. It has the same full raw buffer size as standard mv1080 (3x3 sampling) but this one is 1x1 sampling. In addition, the focus pixels looked like they were in a different pattern than other 1x1 settings so I started with this one:
100D_1736x976_14bit_3x_crop_mode
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7868%2F46385419644_3f897155f5.jpg&hash=d37f8351f22c70e529b68cb07bc12d4d) (https://flic.kr/p/2dEVpdy)
This is after I put in about 3 hours into it starting over from scratch several times. No matter what I did, the focus pixels were still there. Once I thought I had it but upon closer inspection they were still there. In addition, processing the DNG files through ACR and dcraw gave completely different looking focus pixel patterns.
So I put it aside and tried another challenging file -- Danne's cat picture using his latest EOSM setting (https://www.magiclantern.fm/forum/index.php?topic=9741.msg212082#msg212082). This one also has a different pattern so I had to start from scratch. Looks pretty good from here:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7911%2F47056372282_7ca1184726.jpg&hash=a958b29678c0a2b7bf2417e4631930fb) (https://flic.kr/p/2eGdd4u)
This is what the pattern looks like:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7919%2F46194852345_2c385a2039.jpg&hash=5d1f25dbbf5d00e3f28f6b92cb0f6884) (https://flic.kr/p/2do5Gct)
The focus pixels are quite subtle, this is without any focus pixel fixing:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7878%2F47056373062_fb909eab0e.jpg&hash=4cde982dcec2334a8aa145ff71960072) (https://flic.kr/p/2eGddhW)
However, upon closer inspection with the new focus pixel map file it looks like there are still some very faint focus pixels still left in the image:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7832%2F46194853465_9be2babac1.jpg&hash=7b7e4e57a0ad481b952fd408d68e413d) (https://flic.kr/p/2do5GwM)
Hum, should have read my horoscope and hid under the covers today.
Feb 15, 2019 - Some unsettling news from far away could have you stressed out, Aquarius. This might affect your work if you aren't careful. Your mind may stray back and forth from work to the news and its implications. Stay focused. All signs are that whatever you've heard is likely to be far less troublesome than it appears right now. Don't fly off the handle until you know the facts.
Oh well, maybe tomorrow the stars will be in alignment. :P
-
Dfort,
I am about to shoot the other test samples that you requested as follows:
"I'd also like to get samples of the old standards, mv1080, mv720, mv1080crop and zoom. Finally, using one of the crop_rec_4k builds from the experiments download page I'd like a sample for the "basic" 3x3 sampling using the mv720 buffer. The reason is because I'd like to see if I can simplify some of these map files. There was a fix rather recently so I need new samples to work with."
I have already downloaded from the Experimental page the build called:
magiclantern-crop_rec_4k.2018Jul22.100D101
"for the "basic" 3x3 sampling using the mv720 buffer" but I am not sure which build works with the "old standards, mv1080, mv720, mv1080crop and zoom". I don't want to send you wrong samples, so please link the appropriate build to me that will provide the samples that you need.
Thanks.
-
To make sure you are using the "old standards" for the 100D, download the build from here:
https://builds.magiclantern.fm/100D-101.html
Shoot with the Canon menu set to 1280x720--that's ml720, then switch to 1920x1080--that's mv1080, press the magnifying glass button to get to 5x--that's zoom mode and finally turn on Movie crop mode--that's mv1080crop.
All of these settings should work with every app but what I'm looking to do is to see just how much we can simplify the focus pixel map files.
For the mv720 with 3x3 sampling, I believe that can be found in the crop_rec_4k build on the experiments downloads page. (https://builds.magiclantern.fm/experiments.html) Canon menu on 1280x720, crop_rec on and that should be the only crop_rec setting on that branch for the 100D.
-
Thanks for the clear and detailed instructions, Dfort.
Here you go:
https://we.tl/t-tEk7H1t90w
These are 5 more test samples shot with the two old builds that you suggested. Please let me know if you need more samples.
Very innovative work on the focus pixel issue, Danne and Dfort. The FPs on the cat image are barely seen. Congratulations!
-
Much better start today--all of the old standard settings are still working with the current map files. Note that with mv720 (5x3 sampling) the aliasing is severe and those green artifacts won't go away even with chroma smoothing turned on. This might be the result of extreme aliasing and not necessarily caused by focus pixels.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7874%2F40148284683_7feed76ecd.jpg&hash=fc2edff60ad07043783bd8c716823407) (https://flic.kr/p/24aLshT)
Speaking of chroma smoothing, one of my recommendations to getting rid of stubborn focus pixels that can't be mapped out is to turn on the chroma smoothing option. Let's make those remaining focus pixels on Danne's cat pop as much as possible:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7909%2F46199083575_9c276d4a59.jpg&hash=1a6cbe83bb3629516dcc364c2e3e1192) (https://flic.kr/p/2dosnZH)
Now turn on 2x2 chroma smoothing:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7899%2F46199084575_4647f7f3d6.jpg&hash=85d9d709294a9e8d4068299cd5377495) (https://flic.kr/p/2dosohX)
Looks like the chroma smoothing recommendation still applies.
I'd like to see if we can simplify some of the map files that have gotten very complicated. However, before possibly messing up something that is working, I'll finish up with the resolutions that still need map files.
In the meantime, enjoy this picture of Danne's cat--without focus pixels. Feel free to download the file and pixel peep.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7877%2F47060957342_b80a3efdef.jpg&hash=2cb16de583fc89f6be6b1fcdeffe65a2) (https://flic.kr/p/2eGBH3f)
-
@Danne - tell me more about this resolution:
100D_1736x976_14bit_3x_crop_mode.MLV
Block: RAWI
Offset: 0x00000034
Number: 1
Size: 180
Time: 0.788000 ms
Res: 1736x976
raw_info:
api_version 0x00000001
height 1189
width 1808
pitch 3164
frame_size 0x0039674C
bits_per_pixel 14
black_level 2047
white_level 16200
active_area.y1 28
active_area.x1 72
active_area.y2 1189
active_area.x2 1808
exposure_bias 0, 0
cfa_pattern 0x02010100
calibration_ill 1
Block: RAWC
Offset: 0x000000e8
Number: 2
Size: 32
Time: 0.796000 ms
raw_capture_info:
sensor res 5184x3456
sensor crop 1.62 (APS-C)
sampling 1x1 (read every line, read every column)
The metadata says that it is using 1x1 sampling but there are two focus pixel patterns that overlap. This will require a two pass map file like we're using for the EOSM basic crop_rec - mv720 raw buffer with 3x3 sampling.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7826%2F47061141192_4a7fb9fa75.jpg&hash=9f1b549bf87c8cd823e1b7f18b4ddc5f) (https://flic.kr/p/2eGCDG5)
Finally figured this out by using dcraw to debayer the image. This is why all my attempts to fix it with a one-pass map file failed.
-
Finally got that problem file cleaned up:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7827%2F46210872655_63f33810f8.jpg&hash=fc4fded699083d5c88a02e84b8c44ac8) (https://flic.kr/p/2dpuNtH)
It needed a pass of both a 3x3 sampling pattern and a 1:1 sampling pattern -- how can that happen?
80000346_1808x1189_1.png
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7904%2F46401308254_2aebf73f6a.jpg&hash=5e5211eb488eaa81913a40fe3660da16) (https://flic.kr/p/2dGjQm7)
80000346_1808x1189_2.png
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7857%2F46401308324_fcdc326670.jpg&hash=ab0d75a5e2863693e2909671807ce023) (https://flic.kr/p/2dGjQnj)
Here is what the image looked like adjusted for the out of bounds area, set to the correct crop/pan position on the full raw buffer size and ready to start working on the focus pixel map:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7887%2F33249573598_dee334bc9e.jpg&hash=21d680ea00b3d1bc8e9612d563699412) (https://flic.kr/p/SE9KQj)
I made separate fpm files using one of the scripts that is in the repository:
./img2fpm.sh -v 80000346_1808x1189_1.png
./img2fpm.sh -v 80000346_1808x1189_2.png
Then combined the fpm files like this:
cat 80000346_1808x1189_1.fpm 80000346_1808x1189_2.fpm > 80000346_1808x1189.fpm
Just thought I'd document the process in case anyone comes up with yet another resolution and I'm too busy--or just don't feel like working on it.
By the way, you can actually download the original sized files from flickr.com if you want to work through the tutorial I posted on Reply #564 (https://www.magiclantern.fm/forum/index.php?topic=16054.msg207474#msg207474). Here's a tip, once you get a pattern working on a section of the image you can just duplicate the layer, move it to another position using the previous layer to line it up, merge layers, rinse, repeat and you'll be done short order--just like the wheat and chessboard problem (https://en.wikipedia.org/wiki/Wheat_and_chessboard_problem).
I'm thinking about re-working the scripts to make it easier to work with multi-pass focus pixel map files. In the meantime I'm pressing on trying to get all the new resolutions working in MLVFS before moving on to the other apps.
We're getting to a point where we should make a decision. Do we make multi-pass map files for all possible settings or do we modify the algorithm to better match the map file to use? As it is MLVFS was the first app to deal with focus pixels and it uses pretty much the simplest method possible to match up the map file with the image. Camera ID and full raw buffer size. The newer builds show sampling methods so it is possible to make a better choice over which map file to use. However, how to deal with a problem like this particular example? Maybe throw every possible pattern at it? Some developers would probably cringe at that because all this processing must be affecting the image in some way. Maybe have a user selectable switch to decide how aggressively to work on the focus pixels like we have with chroma smoothing and bad pixel fix?
-
Looking good! ANd maybe not so strange after all:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg212214#msg212214
-
Keeps getting stranger and stranger.
100D_5008x708_14bit_5k
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7921%2F47088313722_e8f2159e69.jpg&hash=853d3ee3d443b78db2883997fcc3c36f) (https://flic.kr/p/2eK2V8Y)
Push it to clearly see yet another focus pixel pattern:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7826%2F46416843054_f79c6277fb.jpg&hash=f30d34e50b417b60e24cae58840b396a) (https://flic.kr/p/2dHGsiu)
Ok, so I went through the steps but when it came to finding the full raw buffer size:
raw_info:
...
height 738
width 5080
However, no matter how I processed the MLV file the height was 1180 so there is no way this fits in a full raw buffer that is only 738 pixels high!
Think I'll put this one aside for now. Anyone actually using this setting?
[EDIT] Ok--couldn't resist the challenge. Turns out that mlv_dump showed the actual size. ACR and dcraw were doing some aspect ratio resizing. Readjusting in Photoshop did the trick.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7920%2F32201150957_61a791c804.jpg&hash=c40fa026369f5df9d31671076d6a042f) (https://flic.kr/p/R4vjiT)
Here's a closeup -- no focus pixels!
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7833%2F40178164883_84b5160721.jpg&hash=6fc8a5abd2ec55a72ce1925b3aa689c8) (https://flic.kr/p/24dpACK)
@IDA_ML -- Your turn. See if these latest map files work with any setting and resolution you can come up with. This is still a work in progress so things might will change.
-
Downloaded your today's FP maps and shot a first set of test files in real-life conditions. I believe, this one is problematic:
https://we.tl/t-Q83GMJwuhS
It is a 4k (4056x2582) file that stll has a few FPs left - look around the chimney. Please let me know if you would like me to reshoot the sample with a white target.
I will keep testing and providing problematic files.
-
Let's take a look at this. At first it looked pretty good but my screen was dirty:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7897%2F46427714974_486033f186.jpg&hash=12a33280f0f3252a8496ae540a718802) (https://flic.kr/p/2dJEb9o)
Right, that's not on my screen, those are some leftover focus pixels. My experience is if we do "mlv_dump --dng --no-fixfp" followed by "dcraw -T" it usually brings out the focus pixels quite clearly:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7912%2F47151191071_ff0ba887e8.jpg&hash=2a05657980493f55f86b953df7bb6594) (https://flic.kr/p/2eQAboc)
Now what does the pattern look like on the map file?
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7811%2F32209182497_83720d166c.jpg&hash=7fb25c53e6dd557f5582db58b2a20b73) (https://flic.kr/p/R5dtNH)
That doesn't look right but that's the pattern I've been using on all of the 1:1 sampling and nobody seems to be complaining -- until now. What is going on? Not sure but since this image clearly shows the focus pixels I thought I'd build up the map file from scratch. The way I do it is start with a small area in Photoshop, copy the layer and move it until the pattern matches up. Looks like some time ago I must have mismatched the pattern.
That's better:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7851%2F47151191581_7d778bf0c7.jpg&hash=5c4f7c4314cb9f37448a5af7b8763aa7) (https://flic.kr/p/2eQAbwZ)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7873%2F47098814992_ca70db63a1.jpg&hash=a702b3dd27548638556369e3fbd3d0eb) (https://flic.kr/p/2eKXJNm)
A little bit of color grading and voilà!
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7880%2F32209197147_cce1ba9100.jpg&hash=90f5f3c4eef099df5c3c16a2892190b3) (https://flic.kr/p/R5dyai)
Now I need to go back and replace all those 1:1 map files with the right pattern.
-
Excellent work, Dfort! It is amazing to see how you manage to hunt down the FPs even without a white target. Obviously, you have falcon's eyes. ;)
-
Beautiful work done by @dfort!
Obviously, you have falcon's eyes. ;)
That’s exactly right! :)
-
Some cross reference from the EOSM M topic:
@Jonit: funny, that really works - overexposing by far, and then EV=-4, Highlight Reconstruction = on. You can see the focus pixels.
@Jonit: this does only work, when leaving Focus Pixel Fix enabled. When switching it Off, those pink dots are away.
@dfort: I also recorded some frames, one clip overexposed (@10bit), another with normal exposure (@14bit) using Danne's rewire preset @ 1736x1158. I uploaded the clips with an receipt, so you can see them good in MLVApp. Does this help? It seems there are different types of focus pixels - some are very good to see (white, center), others not so good to see (dark grey, everywhere). The dark ones are good visible in the overexposed clip, processed like Jonit did.
Things aren't what they first seems to be. Confusing?
I took another look at the samples Jonit posted to see if they work with the latest focus pixel map files I've been working on. The thing is, when something is so overexposed that there is no recognizable image and you max out saturation, contrast, vibrance and whatever it takes to see the focus pixels, the map file itself will start affecting the image. In other words, this overexposing trick isn't working out that well for me.
However, the normal exposure image masc uploaded is easy to work with. Here it is stretched to the max to get those focus pixels to pop. Looks like something that should be hanging in an art gallery:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7892%2F47201860651_b8ea2c501f.jpg&hash=e8cb9b138092a26228434865f8a86887) (https://flic.kr/p/2eV4SFD)
Looking at the entire frame you can see that familiar cross shape pattern that we've been seeing on the EOSM, 700D and 650D:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7887%2F46287424165_290a69f95f.jpg&hash=c3154abc2911d3e4a8c72ea63bdbf7b5) (https://flic.kr/p/2dwg9zk)
Note that I have this set up in Photoshop as per the tutorial I posted a few weeks ago (https://www.magiclantern.fm/forum/index.php?topic=16054.msg207474#msg207474).
This is a full raw buffer size I've never seen before so of course I don't have a map file for it. However, it looks very much like another pattern and here's where the plot thickens.
mlv_dump -v
Block: RAWI
...
height 1190
width 1872
...
Block: RAWC
..
sampling 1x1 (read every line, read every column)
Block: IDNT
...
Camera Name: 'Canon EOS M'
...
Camera Model: 0x80000331
The width of this full raw buffer is 1872 which is also the width when using Movie crop mode (mv1080crop) -- not to be confused with the crop_rec module. The height is 1190 which is the height of the mv1080 raw buffer. Now things are going to get crazy because the sampling says it is 1x1 (same as mv1080crop) but the focus pixel pattern looks exactly the same as the 3x3 sampling of an mv1080 file. In fact all I had to do was to pull in an image of the 80000331_1808x1951 map file, line it up, convert it to a fpm file and it works -- in MLVFS for now.
Here's what the normally exposed image looks like -- no focus pixels.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7834%2F40237121063_ef99426968.jpg&hash=08e6f763416532272532075dd68343f8) (https://flic.kr/p/24iBLfB)
It looks like my idea of looking at the sampling in order to help determine which map file to use won't work in this case. The inventory of map files keeps growing because MLVFS needs a file for every possible full raw buffer size for each of the cameras that show focus pixels. If you're using MLVFS and want to play around with the latest experimental crop_rec module settings make sure you're using the latest focus pixel map files (https://bitbucket.org/daniel_fort/ml-focus-pixels/src/default/focus_pixel_map_files/) from the ML Focus Pixels (https://bitbucket.org/daniel_fort/ml-focus-pixels) repository. I've still got some more work to do before coding the script that can be adapted for other MLV apps.
I've been reworking my scripts, updating map files and have switched from huge inefficient P1 type Portable Bit Map files to tiny Portable Network Graphic files. Well the scripts still work with pbm but if ImageMagic is installed on the system the scripts throw them out and save png files. This is helping speed things up when a new setting come up that requires a focus pixel map file, though admittingly my scripts are as slow as molasses.
-
Remarkable stuff @dfort and this is all great news so far!
-
As some of you know I've been adding map files for the latest bleeding edge crop_rec module experiments. Most of the new resolutions have been coming from the EOSM experiments but there has also been experiments done on the 100D and 700D. At this point it looks like we've got all the current settings covered so I thought it would be a good time to take a step back and reflect on where we are at with this focus pixel project.
Where are these so-called focus pixels on my camera's sensor -- Everywhere!
When I started this topic it seemed that the focus pixels were bunched up in the center of the sensor in a cross shaped pattern.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm1.staticflickr.com%2F578%2F22809705629_53bd142a86.jpg&hash=f3cd3fb2afddfd3330d4139d5d5fd756) (https://flic.kr/p/AKBGNH)
Next we found out that the patter stretched across the entire sensor.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm1.staticflickr.com%2F756%2F22108377514_3b888afd91.jpg&hash=e7254c189bdbbf186202a21d9fa58869) (https://flic.kr/p/zFDdoE)
This makes sense because it would be very difficult to manufacture a sensor with different types of receptors in a cross pattern. The main difference between the "older" 650D, 700D and EOSM sensor and the "newer" one used on the 100D and EOSM2 was the area covered by the focus pixels.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1511%2F24032131713_2cec87086e.jpg&hash=3fa616b6f0972065ed28bc72e06c7623) (https://flic.kr/p/CBCXuv)
At first there also seemed to be a difference in the color and pattern of the focus pixels of the newer sensor but after seeing many sample files it looked more and more like these sensors all shared the same pattern and showed up in images in pretty much the same way. Just like the discovery that the focus pixels weren't confined to a small cross of the older sensor or large box of the newer sensor, focus pixels started appearing in areas outside of these boundaries.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1649%2F24653928396_27b5e8aa48.jpg&hash=4ae9bc03b4f139af768fa4604c924c8c) (https://flic.kr/p/DyzPYY)
There is an area we call the out of bounds area of the full raw buffer. That can be clearly seen using the raw_diag module from the ISO research tools branch.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1572%2F24119537929_ca11676725.jpg&hash=e800cd1e93106be278bba4ada1d74ac7) (https://flic.kr/p/CKmWje)
Up until now I've been careful to keep from mapping the out of bounds areas but on this latest round of updates I started mapping the entire full raw buffer because some of the experimental crop_rec settings might have different OB zones and it made comparing the image files easier by lining up the upper right corners of the images and finally because it was easier. The savings in the number of mapped coordinates is rather insignificant. I mean we're already defining up to 154,440 coordinates on these map files, what's a few thousand extra dots between friends?
So to conclude this section--the map files are covering the entire area of the full raw buffer.
What is the focus pixel pattern? -- There's more than one.
When I first started I wanted to eventually map out all of the focus pixels on the full sensor then calculate all the pattern for the various settings. However, getting a good look at focus pixels on the full sensor wasn't possible until recently when @theBilalFakhouri came up with a setting that gave us access to the full 5280x3508 raw buffer--and it showed focus pixels. This is the pattern:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7910%2F47178121792_f6c4ba2fc7.jpg&hash=d0bf55abb970d638a439a0c13e45dd45) (https://flic.kr/p/2eSYcWY)
Notice that the pixels on the horizontal axis repeat every 24 pixels. File that because we'll come back to it.
The first 1:1 sampling options we had with these cameras was the zoom mode which is activated with the magnifying glass button and the Movie crop mode setting which is something that is unique to these cameras that show focus pixels. What I noticed when testing zoom mode was that the pattern would shift 8 pixels left or right. Why this happens is probably due to the way the "crop" value is determined. If you run mlv_dump -v you will see that each frame has a crop and pan setting. We're using the crop value to line up the map file. A while back there was a bug (https://www.magiclantern.fm/forum/index.php?topic=16054.msg160318#msg160318) with the way crop was calculated and fairly recently a similar issue came up with the lossless compression settings (https://bitbucket.org/hudson/magic-lantern/commits/5052fabbc8d21446d99cd743b904c90243082720). When I wrote the fpm.sh script I took this into account and calculated the crop setting from the pan values like this:
##
# Allow for legacy MLV files with cropX, cropY bug
#
cropX=$((panPosX + 7 & ~7))
cropY=$((panPosY & ~1))
This should no longer be necessary so I'll be taking it out.
Alright, so where am I headed with this and why did I ask you to remember 24? Because in the zoom setting you can move the capture area around the sensor and the pixels will jump left right in 8 pixel increments as it switches between the 21 full raw buffers used in that mode.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm2.staticflickr.com%2F1475%2F24510484991_4ab3b20762.jpg&hash=a1381a1c2369dcc55447f91533543f17) (https://flic.kr/p/DkUDfP)
To compensate for this I copied the map file so that the same map would still work even if the focus pixels moved left or right by 8 pixels.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7906%2F47178346842_52a73965f7.jpg&hash=9051d6e8620ba496f4d16f6148b97d01) (https://flic.kr/p/2eSZmR9)
Now I'm starting to question if this is necessary because on my new map files I didn't do this on the 100D yet it worked on all the samples that I received. Looks like more testing on the zoom settings are in order. My question is does this also affect the Movie crop mode setting? Yet more testing. Perhaps the code should be changed so that crop is forced into 24 pixel increments? (There's that number again.)
Maybe like this?
raw_info.width, (skip_x + 23) & ~23, skip_y & ~1,
Moving on to the 3x3, 5x3 and the new 1x3 sampling settings -- at one point @a1ex looked at the focus pixels (https://www.magiclantern.fm/forum/index.php?topic=16054.msg174482#msg174482) as part of his pixel binning research (https://www.magiclantern.fm/forum/index.php?topic=16516.0). I played around with his octave scripts (https://www.magiclantern.fm/forum/index.php?topic=16054.msg174582#msg174582) and using the 1x1 sampling pattern we were able to recreate the 3x3 and 5x3 patterns. Here they are:
3x3 sampling (mv1080) focus pixel pattern
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm6.staticflickr.com%2F5553%2F30227791343_e7b6e33cf0.jpg&hash=a0e4652fe17123201debedde57cbc395) (https://flic.kr/p/N48kfg)
5x3 sampling (mv720) focus pixel pattern
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm6.staticflickr.com%2F5519%2F30826838016_d34739808b.jpg&hash=801a323347e100d07fadf3bb20f8a780) (https://flic.kr/p/NY4AXb)
Here's the latest setting that is getting attention:
1x3 sampling focus pixel pattern
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7900%2F46507181914_afdaa0581d.jpg&hash=e38b2154fce4ec7573ded4db6412f9a6) (https://flic.kr/p/2dRFsU3)
Ok now, those of us who are better at remembering images than numbers might have just gotten an "ah ha" moment. That's the same pattern as the one we're using on the 1x1 sampling adjusted to take into account the 8 pixel left/right shifting. For those of you better with numbers, 24/3=8. Alright, I may not be a math wiz but I did pass elementary school.
This brings up an off topic point -- the horizontal binning pattern probably should be evenly divisible by 3 but we are forcing the crop setting to be evenly divisible by 8. Maybe that's another reason to change the code so that it forces the crop value to be evenly divisible by 24? I don't know, just a shot in the dark on this.
But wait, there's more! (Is this starting to sound like an infomercial?)
Recently @Danne came up with a setting that showed a different focus pixel pattern. Not that surprising if you have ever played around with the various PREFERRED_RAW_TYPE:
raw.c
#undef RAW_DEBUG_TYPE /* this lets you select the raw type (for PREFERRED_RAW_TYPE) from menu */
Here's the focus pixel pattern for his "reconfig_cat" setting or whatever he calls it:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7895%2F40265925043_e5889eb927.jpg&hash=a4fb240fa64b97de84046336fca7e823) (https://flic.kr/p/24maoEX)
What's with these "multi-pass" map files? -- When one pattern isn't enough.
When we were experimenting around trying to get mv1080 (3x3 sampling) working on the EOSM we started getting images where the map files weren't working (https://www.magiclantern.fm/forum/index.php?topic=16054.msg191208#msg191208)and focus pixels were starting to make a strange pattern, like two patterns were layered on top of each other. We also had a situation where the same full raw buffer size was used for both 3x3 and 5x3 sampling so what to do?
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm6.staticflickr.com%2F5665%2F30798763052_49c0ac022e.jpg&hash=cb872f87d417fd4820d025af5d6fbd61) (https://flic.kr/p/NVzHf5)
At first I made a map file that combined the focus pixel patterns but that didn't work because the coordinates were too close to each other and they started picking up the color from adjacent focus pixels and it turned into quite a mess. The solution was discovered by @bouncyball (https://www.magiclantern.fm/forum/index.php?topic=16054.msg191216#msg191216)which was to make a separate pass, one for each focus pixel pattern.
This trick also worked for situations where the same sized full raw buffer was used for different sampling, like what happened on that crop_rec setting. So even though other cameras didn't show both focus pixel patterns superimposed over each other like the EOSM, they did benefit from multi-pass map files. Now we've got to ask, isn't all this extra processing affecting the image? Yes, though it is much better than having focus pixels on your video. For the most part the image doesn't seem to suffer unless you clipped the highlights and bring the exposure way down in post so they turn grey. At that point, the focus pixel map starts showing up on the image but at that point the image is quite a mess anyway. A little bit of chroma smoothing helps. I've suggested that several times on this topic.
Now with these latest crop_rec settings I looked at MLV files that had the 1x1 sampling focus pixel pattern superimposed over the 3x3 pattern (https://www.magiclantern.fm/forum/index.php?topic=16054.msg212153#msg212153). Again, a multi-pass map file was able to deal with that.
Where do we go from here? A light touch or throw everything including the kitchen sink at it?
Here's the problem, we will continue to find new resolutions that require one or more of the focus pixel patterns. Right now which one(s) can't be predicted. So do we try to find a way to figure out which pattern is needed? With this latest round of testing it looks like that would be very difficult to do. How about making a pass with each of the known patterns? That might work as long as we can figure out how to line up the map files for every possible setting though it does seem like overkill.
Another problem is that the inventory of map files is growing exponentially because for every new resolution there are five cameras that might need that new map file. The good news is that it appears that every camera seems to be using the same patterns. Yes, even the "old" and "new" sensors are using the same patterns. That means that we shouldn't need to make a separate map file for each camera.
Well this post turned out much longer than I anticipated but if you got to this point perhaps you have some idea you might want to share?
(NOTE: Please, oh please don't quote this entire post on your reply!)
-
Danne's EOSM has some hot pixels. He found a way to deal with them in camera (https://www.magiclantern.fm/forum/index.php?topic=9741.msg212680#msg212680) but what about the footage you might have already shot? Like the old saying goes, "we'll fix it in post."
Let's see what we can do about this:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7920%2F33365265598_7b798e7c20.jpg&hash=a002c3a1605c079c7f9f1c406fac2e9b) (https://flic.kr/p/SQnH2Y)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7916%2F47189118102_92f0b65ca2.jpg&hash=efeb78a58f5d6b197a33adcaceaa2476) (https://flic.kr/p/2eTWyLw)
That one "hot" pixel is screaming for attention. I like using dcraw -T to create a tiff file that doesn't hide these defects like ACR does then use that tiff file in Photoshop or any other image editing application that has a ruler you can set to show pixels. Here's the ultimate in pixel peeping:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7883%2F46326497105_e3b9aa6476.jpg&hash=97b922993f353d93f3525b7b7823b04b) (https://flic.kr/p/2dzHpAH)
That is one big red hot pixel! If you look closely you'll notice we're looking at a group of 9 pixels. The debayering algorithm is blending the color of the neighboring pixels so the pixel we want to map out is in the center of that cluster - 1384x690. With mlv_dump and a little bit of basic arithmetic we can easily modify the focus pixel map file.
mlv_dum -v
Block: RAWI
...
height 1060
width 1872
...
Block: IDNT
...
Camera Model: 0x80000331
...
Block: VIDF
...
Crop: 72x84
The full raw buffer size is 1872x1060 so we add the Camera Model to that to find out that we will need to work with the 80000331_1872x1060.fpm file. To map out that pixel we add the location of the hot pixel to the crop values like this:
Width = 1384+72=1456
Height = 690+84=774
Now plug that into the .fpm file, it is an ascii tab delimited file that can be modified with any text editor:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7883%2F46326498045_ab09d5e38f.jpg&hash=f0b96ab3556599e24dbf93f4b430b1da) (https://flic.kr/p/2dzHpSV)
Done.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7918%2F46326498055_72db444b86.jpg&hash=1f193ac7f53718dac321fecbe1d64996) (https://flic.kr/p/2dzHpT6)
From now on the modified map file will map out that hot pixel along with the focus pixels.
-
Does anyone know if there are raw streams that don't show Focus Pixels ?
Can imagine that somebody already tried out, but in case not, I can imagine that there are raw streams that don't have Focus Pixels.
There's this debug option in magic-lantern/src/raw.c file.
#define RAW_DEBUG_TYPE /* this lets you select the raw type (for PREFERRED_RAW_TYPE) from menu */
This gives a raw stream value select option in debug menu.
If you use ML preview with half shutter press, you can quickly see how the raw stream looks, without the need to record.
So change raw type value, and check how it looks with half shutter press.
Not all values give valid raw stream, in that case it shows the 'raw detect' error.
-
Been down that path many times. Short answer. No better stream has emerged yet.
-
Managed to elminate red pixels by tweaking cmos 6:
Before:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.postimg.cc%2FvHbrmmgm%2FScreenshot-2019-03-01-at-13-51-50-png-500px.png&hash=e2eae655fb1c589ffb7376b9d7334b6c)
after:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.postimg.cc%2FfTCxShmf%2FScreenshot-2019-03-01-at-13-51-56-png-500px.png&hash=c7e527524dbf9f17c3469882de95b81a)
It´s not unproblematic. Image seems sometimes "weakened" on top. Black level and some vignetting but most of the times image is just as with untouched cmos 6. Part of the bottom is cut maybe 20 pixels(less of a problem).
Test files(MLV):
https://bitbucket.org/Dannephoto/magic-lantern/downloads/fp_eosm.zip
-
There's this debug option in magic-lantern/src/raw.c file.
Been there, done that. At one point I thought I found a setting where there were no focus pixels showing (https://www.magiclantern.fm/forum/index.php?topic=16054.msg176032#msg176032) but it turned out they were still there and were even harder to remove.
Danne's coffee cup looks great at first glance.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7816%2F33373587098_13710e7b07.jpg&hash=8d637189b8bffe5a76652eb21cfb8a0f) (https://flic.kr/p/SR7mJ7)
Stretching the images to make the focus pixels pop shows that they are still there. Maybe fewer and in a different location but still there.
"normal" setting
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7898%2F40284430163_e368814e8f.jpg&hash=d3debe2d25eded97eb45a91c30379831) (https://flic.kr/p/24nNeAK)
after tweaking cmos 6
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7809%2F33373587578_28529dcff4.jpg&hash=2c9dacc64fd1591bbd68d32410faca95) (https://flic.kr/p/SR7mSo)
The issue that I see is that the current focus map file works fine here:
"normal" setting
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7803%2F46335041705_6f4524833d.jpg&hash=55e46ee884ffd4c9f9645ba813f49680) (https://flic.kr/p/2dAtcBp)
but not on this one:
after tweaking cmos 6 [EDIT] Yes it looks exactly like the previous one but this one was run through with the current map file.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7837%2F47197543592_cd1e955dc3.jpg&hash=72b79dcad3e86953ce885de5fb594cb8) (https://flic.kr/p/2eUFKnA)
Running a second pass with that pattern will "fix" it.
I doubt there is any way we can predict if the settings were tweaked in such a way that it would require that second pass with a different map file so maybe being super aggressive and throwing all the possible patterns into the mix is the only way to guarantee focus pixel removal?
(Looking for some developers' feedback here -- hint hint)
-
One of these days we will hit the right reg :P
-
Talking focus pixels. Take a look at eosm when running regular mv720p raw in 6400 iso. A reg is set which causes a vast amount with pixels. Levas found the reg. Disabling it completely gets rid of the pixels. Added here to crop rec presets:
https://bitbucket.org/Dannephoto/magic-lantern/commits/2aa08f8e974c53dab8ad461cc2b680b95f3cf492
-
Probably no longer relevant, as dfort already mapped out these pixels, but I managed to get this picture by accident. I took a photo while crop_rec raw was enabled and this came out.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.ibb.co%2FS0PfGB4%2FIMG-0411.jpg&hash=6f98d64c207feaab0c43ecc0a3381485) (https://ibb.co/S0PfGB4)
EDIT: here is the .cr2 if anyone is interested https://mega.nz/#!PUF3nIaa!hjOU46_s7mXEA86B1j9c5KJP3YxybokBMdxYAbKfR-E
-
This is actually an interesting case. It is only the second CR2 file with focus pixels that I've seen. The other one was shared by a1ex on Reply #569 (https://www.magiclantern.fm/forum/index.php?topic=16054.msg207726#msg207726).
Let's take a closer look at these. First the one a1ex posted:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.staticflickr.com%2F7874%2F47280353381_4bfdfa9f23.jpg&hash=9fba65182d1730cef7c2e5e27f6045a5) (https://flic.kr/p/2f31aQz)
This is from a 700D and it clearly shows that familiar cross shape. The grey border is the full raw buffer and the image is properly lined up over that full buffer.
I wanted to take a look at the full raw buffer by running it through dcraw with the "-4 -E" options. We can't use mlv_dump on CR2 files so we'll turn to exiftool to see what we can learn from this.
Canon Image Width : 5184
Canon Image Height : 3456
Image Width : 5184
Image Height : 3456
There's quite a bit of redundancy in the metadata. The applications that support CR2 files confirms the size of the image as 5184x3456. Note that if we look at any MLV file (that wasn't shot with the silent module) in mlv_dump using the "-v" option it will this:
Block: RAWC
Offset: 0x000000e8
Number: 2
Size: 32
Time: 0.776000 ms
raw_capture_info:
sensor res 5184x3456
sensor crop 1.62 (APS-C)
sampling 1x1 (read every line, read every column)
As long as we're being redundant, let's check the user guide:
Recorded pixels: L (Large) : Approx. 17.90 megapixels (5184 x 3456)
M (Medium) : Approx. 8.00 megapixels (3456 x 2304)
S1 (Small 1) : Approx. 4.50 megapixels (2592 x 1728)
S2 (Small 2) : Approx. 2.50 megapixels (1920 x 1280)
S3 (Small 3) : Approx. 350,000 pixels (720 x 480)
RAW : Approx. 17.90 megapixels (5184 x 3456)
Ok--so the size of a CR2 file from the 700D and all of the cameras that show focus pixels for that matter is 5184 x 3456. (I kept the various JPEG sizes from the user guide for future reference--maybe there is something that we can learn from the way Canon came up with these dimensions.)
Now let's verify the full raw buffer. The file produced by running "dcraw -4 -E" is 5280x3528. Here is what exiftool shows:
Sensor Width : 5280
Sensor Height : 3528
Sensor Left Border : 84
Sensor Top Border : 64
Sensor Right Border : 5267
Sensor Bottom Border : 3519
If we treat the Sensor Left and Sensor Top Border like the Crop in mlv_dump the image lines up perfectly on the full raw buffer. The Sensor Right and Bottom Border is the location of the lower right corner of the image when it is lined up on the full raw buffer.
So at long last we can accurately map the focus pixels on the entire sensor. Note that the focus pixels extend over the entire sensor so we can't just map what is visible on this example.
Now let's take a look at the example @Jonit posted:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Ffarm8.static