Messages

crop_rec_4k_mlv_snd branch doesn't have this issue.
At least the latest build from Experiments downloads page (2018 July 22).
Sorry, can't build from latter sources for now.

"Still present" is related to your previous build for 650D (didn't observed this bug earlier in crop_rec_4k_mlv_snd branch).
May be it has nothing to do with crop_rec. Because this module is not even loaded on my camera.
It seems like mlv_lite itself produces the bug. When 1/25 happens, you may switch raw recording off, turn camera off and turn on again, than it shows real shutter again in 100% of cases, turn raw recording after that and that's ok. Till next reboot, when the luck decides )

25 fps bug still present (sometimes you turn the camera on and the shutter speed shows 1/25 instead of what it is set to. Recording performed at real shutter speed value, e.g. 1/48, but LV and even mlv info shows 1/25).
I don't know methods to reproduce or to avoid this. Behavior seems to be sporadic. It just happens. Or not. Whatever settings or sequence of actions.

When thinking about it... does this work on RAW data? I guess not...

This method works on RAW data of course:
The Paper

I have also considered the idea that values of focus pixel are just some kind of offset from original values.

It seems obvious at a glance, since FP pattern is brighter on highly exposed images and darker on underexposed. There are different types of their behavior though. Offset can be positive or negative, and its absolute value quite differs (i.e. central part is usually brighter) But the overall picture is clear. At this point we know that some kind of correlation exists for sure:


In order to verify this idea I shot the evenly lit grey card with the series of exposures from underexposed to fully overexposed, to see if this offset remains constant through the whole dynamic range of sensor.

The idea was simple: since I shot the uniform grey card, I can easily restore original value almost exactly, by averaging the neighbor "healthy" pixels of the same color. So I wrote a small utility that reads dng files, maps the focus pixels and plots the graphs for actual values in relation to what values should be as if they were usual pixel.
I got different types of graphs: most of them are almost linear, others have a noticeable roll-off in highlights, the curve slope differs from row to row:

(y-coord is the actual value of focus pixel, x-coord the original value that should be restored)

First obvious problem is that some focus pixels reaches the highest value very fast and then stay overexposed, therefore they do not carry any useful information after that exposure.
Ok, let's say we can use interpolation techniques for them, if they are overexposed. What about others? Hey, now we have all the data from graphs to restore original values, right?

Well, sort of. Or should i say "not at all".
I updated my script to do the opposite: read a dng file, take the previously saved graphs data for a given focus pixel, extrapolate it according to actual pixel value, and update the raw data in dng files.

When I processed the first test frame, I almost jumped from my chair with a joyful shout. But that was only beginning. The thing is the method kinda works only in the same lighting conditions, when shooting the same grey card, and with the same ISO.
Ok, I thought we can build the graphs for every ISO, that's not a problem. But what's the matter with the light?
I got a desk lamp with adjustable color temperature and went deeper. Changing the color temperature results in a noticeable value shift in FP pattern.

With further investigation by color overlay charts it turned out, that values of focus pixels depends not only on the intensity of the same channel, but the green as well.
For example, for a given red focus pixel its value would be:
R = R1 + G*x,
where R1 - the original value that should be restored, G - intensity of neighbor green pixels, x - some multiplier, different for each type of FP.
and for a pixel from blue bayer block: B = B1 + G*x

I updated my program to get these multipliers, store them alongside with the graphs data and to calculate and set the desired R1 and B1 values.

Now it seems to work better with any light color temperature. But only in blurred areas. As you may tell already, since the values are depending on green channel, to calculate them properly the green value should be averaged for this point somehow, but this turns into the same old problem: artifacts on contrast edges. Because the closest green pixel for a given point is, well, at least one pixel aside, its value may be on the other side of the edge. So this method requires some kind of interpolation to find proper green values.

Wait, interpolation? Ok, so we came back to where we started.
Unfortunately, this offset method still requires some sort of adaptive interpolation for all the focus pixels. For this reason it works well in smooth areas (where any regular interpolation works) and produces the artifacts of the same nature on contrast edges.

There's one thing though that still should yield better results: since green channel contains two times more information than red or blue, interpolating it should produce less artifacts.
But then i discovered that some pixels depends on all three values, e.g. R = R1 + G*x + B*y, so restoring them becomes a total mess. Enthusiasm diminished

By far, the modified adaptive interpolation (cross method I mentioned earlier) is the best approach:


The paper: Adaptive pixel defect correction

Can we update the focus pixel maps in a way that they affect only the areas where those pixels are (the central part of the image, not the whole frame).
May I ask the author to please read this post?

In order to avoid double posting, may I ask the developer to take a look at this post?

Basically, I have a couple of suggestions regarding the treatment of focus pixels. The idea is to make resulting video more usable by affecting only areas where those focus pixels are, and not the whole frame area, and to introduce an optional alternative interpolation algorithm, which works better in most situations.

[1. When shooting 14 bit raw in non-crop mode, focus pixels are located only in center part of the image.]

In order to evaluate how focus dots removal works in popular raw processors (MLVFS, MLV App, MLV Producer) I'm using this improvised test chart:


14 bit raw video (1736 x 976) shot on Canon 650D with mlv_lite module in non-crop 1080p mode.
PDR (Pink Dot Remover tool) with updated dot data and altered interpolation algorithm used as a reference guide.

A couple of observations and suggestons:

1. When shooting 14 bit raw in non-crop mode, focus pixels are located only in center part of the image.
No matter what AF points pattern used, what AF method selected in Canon menu or AF disabled, all the focus pixels are concentrated in central 290 pixels height area:


Therefore, assuming focus pixels all over the frame (like actual fpm tables does) we introduce color artifacts in areas which were clean originally. This behavior should be avoided:


Suggestion: let's update fpm tables the way they don't affect the top and bottm parts of the image.
Focus pixels should be treated only in 290 pix height central area in order not to ruin the originally clean image.
Above is applicable to 650D and may vary by camera models. If this is the case, may be we should detect camera model and limit the FP affected area accordingly, or even let user decide where to remove those pixels by dragging some selection area.

2. Interpolation algorithm works better if takes into account only horizontal and vertical neighbour pixels, avoiding diagonals.
Let's call it "cross" method for now. I've already mentioned this back in 2013, and this idea may sound strange at first, but here are some fresh examples. Judge for yourself:



While this is obviously exaggerated extreme test (although shooting let's say a book page in movies is not so rare), almost all real-world scenarios became a bit cleaner and seems more calm, when this "cross" method used.
So my second suggestion is: Let's introduce this interpolation method as an option in UI, so the user may decide which one is better in a given situation.
This applies to MLVFS and MLV App. MLV Producer uses its own method which is better and very close to what PDR does (is it the same?)

DNG's used for examples:
Original
Treated by MLV App / MLVFS
Treated by PDR

Modified interpolation algorithm explanation and code (java)

Seems like everything is working (recording with crop_rec 5k anamorphic, mv1080p 46 fps, mlv_play).
Should sd_uhc module work without corresponding lua script now?
Upd. Ok, quick benchmark shows recording speed around 60 MB/s, so it seems like sd_uhs working too.

@Danne, following your instructions I got these values for my 650D:
5. 0x4a601d4 (mv1080p @24fps)
6. 0x4540298 (mv1080p x5 zoom)
7. 0x42401e4 (x5 zoom, movie crop enabled)
8. 0x2d701d4 (mv720p @50fps)

Is it possible to make MLVP start in windowed mode instead of full screen (and remember window size and position) ?
rev. 3261

Is it right? Where is BD for second and third color? (R-G-B)

This is right. You forgot the Bayer pattern. There are only WxH pixels on the matrix. All three colors are among them.

You can use the usual formula to check if it fits in your camera controller's speed limit:
W * H * BD * FPS / 8 /1024 / 1024
(W &  H — frame width and height in pixels, BD — bitdepth, which now can be 10 or 12 instead of 14, FPS is frames per second, and converting to megabytes per second). Result is required speed in MB/s

For example, for 650D we have a speed limit in 40MB/s.
We wish to film 10 bit raw video 1728x736 at 25fps.
1728*736*10*25/8/1024/1024 = 37.9 MB/s which fits fine in controller's limit. So in practice we can shoot continuously with these parameters.

To activate RAW_DEBUG_TYPE, define it here:

Got it. Tested, and at least 0x10 PREFERRED_RAW_TYPE works like we used to (650D, raw_rec, both 10 and 12 bit). No tearing, no artifacting, no shifting (even PDR works). Absolutely usable for shooting already. Thank you dfort!

The pull request is quite an interesting read

Indeed. I'll try other raw types as well. By the way, when using mlv_rec instead of raw_rec, i got some tearing in 10 bit mode: every other frame's top third is slightly shifted (5 or 6 pix) to the top.

Are you planning on putting up a pull request for this

Shame on me, but i spent some years in different kind of activity, and i just forget everything about Git, Bitbucket, branching etc. Actually coding is way more understandable I'll try to refresh and join later. As for 650D's CONFIG_EDMAC_RAW_SLURP, it may be easier for you to just add the proper DEFAULT_RAW_BUFFER MEM(0x25B00 + 0x3C) in your commit. Other changes are basically the same as yours for 700D obviously.
Or I can share my test build for other 650D users if this make sense.

Thanks to dfort, i've managed to find the right address for 650D:

Code Select Expand

#ifdef CONFIG_650D
#define DEFAULT_RAW_BUFFER MEM(0x25B00 + 0x3C)
#endif

LiveView works during recording, no pink frames or glitches in extracted dngs (raw_rec, both 12- and 10-bit). Some shifting occurs though, focus pixel masks no longer works (MLVFS windows), need more testing.

Upd. Actually, there are no focusing pixels anymore on 650D (raw_rec, 14 bit as well as 12 and 10 bit), but there are these along the contrast edges (all over the frame). What am I missing? Were there some changes in raw framework?


What PREFERRED_RAW_TYPE should we use for 650D to get the old behavior (usual horizontal pattern of focusing pixel, removable by PDR etc.)?

this might also work for the 650D and maybe the 100D if we can get the DEFAULT_RAW_BUFFER address for those platforms.

Awesome!
Got 650D here, VM all set up and ready for testing. Have some experience with finding STUBS addresses. Can you give us some guidelines on how to find this god damn DEFAULT_RAW_BUFFER address?

selecting 12 bit is working fine on the 700D

So LV display during recording works on 700D too? No freezing?

So this is the way to go, if we want to fix LiveView for other cameras, am I right? If it works for 550D, it may work for others.
I have some time and some skills. Need some push to the right direction. Please give me some clues, where to dig in, if it make sense.

@teatotalTED - "recording works great, no freezing or odd behavior" - is that really true? You mean, no LV freezing during recording on 550D = realtime monitoring works?

Everything is green basically in your test
Try to shoot something colorful. PDR is way better solution if you interested in color fidelity. Because, as it says, cs algorithm, well, smoothes (just blurs) all the color information. But PDR is adaptively interpolates just the "pink dots" themselves.

http://www.magiclantern.fm/forum/index.php?topic=5780.msg80822#msg80822

what is the 2^20 representing?

Bits converted to megabytes (2^20 = 1024*1024)

10 bit RAW @ 1080p24 needs 190 MB/s throughput

That is not correct. 14 bit fullHD raw @24fps needs around 83 MB/s.

550D's SD-controller has write speed limit of 21 MB/s.
I think, this is the answer to all your questions.

You should update canon firmware first. Even if it's version is already marked as 1.0.9.

Which build are you using because there are some failed nightly builds.

That was unified from July, 8. I've edited raw_rec according to Dmilligan first changes and compiled. That changes were not in nightlies.