[IMPOSSIBLE] AF algorithm and parameters in LV?

[SIM62000]

Very basic questions:
1. Why is the contrast-detect AF on Canon (not only) DSLRs so slow?
2. Can ML control the algorithm and parameters of the CDAF in LV?
3. If it can, could their optimization translate into the higher AF speed in Shooting mode?

For example, if we made the AF box smaller, might it be processed faster? Actually, the ability to change the AF box size (e.g. with the control dial) without zooming in could be useful even if the AF speed remained unaffected.

[engardeknave]

It's slow because it has to wait for the lens to focus.

[dmilligan]

Don't you think that if there was anything that could be done in software to make it faster, Canon would have done so? It's clearly something they care about since the main new feature of the 70D is faster AF in LV. They had to change the hardware to make it happen (dual pixel sensor). So I doubt there is a software solution to this since a huge corporation with tons of engineers working for them couldn't come up with one. If that's the case it's extremely unlikely a handful of hacker/hobbyists could in their spare time.

[SIM62000]

Don't you think that if there was anything that could be done in software to make it faster, Canon would have done so?

I do not think that the manufacturers instantly implement their developments. Dual-pixel phase-detect? Well, since people buy it, it's OK for the time. It is non-hunting which is great for video but not important for shooting.

Why Panasonic and Canon itself could do very fast CDAF systems in their 4/3 and the like cameras? Is it only about the processor speed and/or a specific lens build? Otherwise, in what respect the LV mode in DSLRs is different from that in mirrorless systems? Does anybody know the answers? BTW, Canon now has STM lenses that might focus faster.

Even if Canon did their best, it could be good to allow the users to fine-tune the CDAF parameters according to their taste and aim (speed vs accuracy, AF box size). Still more probably for those using third-party lenses. No need for the ML developers to look for the best solutions for any camera/lens combination.

[dmilligan]

It really doesn't matter, it's probably not possible to change something like this anyway, the algo may very well be implemented in hardware. Even if it was possible to modify this algo, it would require so much reverse engineering effort, that no one would even try, as there is very little use for something like this.

[SIM62000]

It really doesn't matter, it's probably not possible to change something like this anyway, the algo may very well be implemented in hardware. Even if it was possible to modify this algo, it would require so much reverse engineering effort, that no one would even try, as there is very little use for something like this.

Doesn't matter, very little use for faster CDAF? You must be joking. Even if only for a certain camera/lens/subject/conditions combo. There are a lot of features in ML that I'd call less important yet they may be more useful for somebody else. As to the possibility assessment, are you speaking as a professional?

At the time, ML can contrast-detect the focus with different algorithms and vary the threshold value. It seems that it can also send commands to the lens focusing system. In principle, that is all what needed to control the CDAF system.

[dmilligan]

A professional what? Hacker? No. Electronics Engineer? Yes.

No I'm not joking. I don't think most photographers generally use LV for normal shooting, I sure don't. There are so many other disadvantages to LV: more heat -> more noise, lower batter life, etc. Professional videographers generally don't use AF at all.

But what I'm really saying is that I don't think that there are any developers out there who might be able to figure this out, who would care to. Best case scenario would be spending hundreds of hours working on something for a marginal improvement at best. If I had the skill, I certainly wouldn't waste my time trying.

At the time, ML can contrast-detect the focus with different algorithms and vary the threshold value. It seems that it can also send commands to the lens focusing system. In principle, that is all what needed to control the CDAF system.

This would all be through the CPU, since that's all we can control, which would be enormously slower.

[SIM62000]

I don't think most photographers generally use LV for normal shooting, I sure don't. There are so many other disadvantages to LV: more heat -> more noise, lower batter life, etc.

So, all owners of non-DSLR/SLT cameras do not count? For me, the CDAF allows for precise focus in any available light, which the PD on my 600D can not ensure. And of course I would like that the CDAF was faster. Slow CDAF is one of major drawbacks that DSLRs are blamed for. So, this ussue is no way unimportant.
Thank you for your explanation concerning the CPU, may be anyone else thinks otherwise?

[dmilligan]

So, all owners of non-DSLR/SLT cameras do not count?

Uhh, ML is only for DSLRs (with the exception of the M, which has decent AF in LV, so I assume you're not complaining about that camera), so no, they don't.

I realize that it is important for some. That is why Canon did something about it with the 70D, but their solution was a hardware one, do you think they would have gone to those lengths (a totally redesigned sensor) if simply improving the efficiency of the algorithm could do the same. That should tell you that there is not much room for improvement with the algorithm.

[SIM62000]

Canon did something about it with the 70D, but their solution was a hardware one, do you think they would have gone to those lengths (a totally redesigned sensor) if simply improving the efficiency of the algorithm could do the same. That should tell you that there is not much room for improvement with the algorithm.

The 70D's AF system is better for movies since it is not hunting. Probably its performance can be improved further, that's another story.

I understand your logic, however IMHO the manufacturers start making improved products not ASAP but rather when they consider it absolutely necessary. And I do not understand what prevents the DSLRs with conventional sensors from having a decent CDAF speed, while the mirrorless do it easily. This is why I would like to know a ML developer's opinion based on real knowledge.

[Audionut]

And I do not understand what prevents the DSLRs with conventional sensors from having a decent CDAF speed, while the mirrorless do it easily.

In that case, it might be best to increase your knowledge on the subject before proceeding further.

[SIM62000]

In that case, it might be best to increase your knowledge on the subject before proceeding further.

It would be a much longer and much less certain process than if a ML developer would say something, at least simply [IMPOSSIBLE].

[Audionut]

Well dmilligan is a developer.  He doesn't have the forum tag as of yet, but trust me, he knows what he is talking about.

Either way, arguing about your point when you clearly don't even understand that basics of how the system works is just poor form. 

[SIM62000]

Well dmilligan is a developer.

Thank you, I did not know that.

arguing about your point when you clearly don't even understand that basics of how the system works is just poor form. 

I know the basics but I do not know how exactly it translates into the performance. I wish that dmilligan had written a paragraph about that instead of his extensive logical reasoning. Either way, thanks:).

[dmilligan]

I just figured a logical explanation would be easier to understand than a technical one. But here's my best attempt at a technical one: As best I understand PDAF is fast because it only has to make one measurement to get at least close to the correct focus. Taking the phase measurement tells you which direction and gives you a good estimate of how to correct the focus. So we can get to the correct focus in just a few tries. CDAF, however, can only really answer the question is the focus better or worse than it was. It's not really possible to get an estimate of the error from a single contrast measurement. So with CDAF we have to take a bunch of measurements moving the focus a little each time. This is slow not really due to the speed of the analysis, but the time it takes to move the focus and take another LV frame. If the fps in LV is 30 and we need to check at least 15 focus points to get the correct focus simply due to the nature of how CDAF works, it doesn't matter how fast the analysis is it's going to take at least half a second.

As for why some cameras seem to be faster at this I think it is because they have some special pixels that are capable of phase detection. So these cameras actually use some kind of a hybrid mode (maybe? I could be wrong about this) These pixels create a problem in raw recording because they show up as pink dots (search the forum for pink dot problems). The revolution about the 70D is that every pixel is capable of phase detection.

The other reason this isn't possible or feasible has to do with the nature of ML. Unlike most software developers who write code against a well documented API or well document hardware specs, ML knows nothing about any of the details of the Canon firmware or hardware, other than what they can figure out via reverse engineering efforts which is a difficult, time consuming and tedious process. AFIAK ML knows very little about the specifics of the AF process, like what functions within the Canon firmware and also which pieces of hardware are responsible for making this happen. Also a lot of stuff having to do with focus and lens control is controlled by an external chip that ML can't control and knows very little about. ML does most of what it does by simply calling functions in the Canon firmware that we happen to figure out what they do. But changing the way Canon firmware works while not totally impossible is much more difficult and 'hackey' at best. For the most part ML tries to stay out of the way of Canon tasks as interfering with them can be dangerous.

We could theoretically make our own algorithm entirely but it would certainly be way slower since we'd have to do at least some analysis with the CPU and the very slow and limited lens control methods we have.

[Audionut]

Thanks for the extended explanation.

AFIAK ML knows very little about the specifics of the AF process

That was my understanding also.  I'll tentatively mark this as impossible.  If the time comes where ML has greater control over the AF process, this can be revisited.

[SIM62000]

dmilligan, thank you for your explanation. Actually I knew at this level why CDAF should be slower than PDAF but *fast* CDAF-driven cameras still exist. Like you, I am not sure whether they have any on-chip PD pixels. Logically, the manufacturers should have announced such a technological development. And in that case why don't such cameras offer "pure" PDAF like EOS-M does? Well, that's just logics:).

I understand the difficulties that the ML team encounters in their work but I would like also to point on the great importance of the issue under discussion. Hopefully some time ML will have faster lens control methods, which is, as far as I can see, the only remaining thing for building a competitive CDAF algorithm. Thank you again.

[SIM62000]

P.S. Concerning the separate AF chip, I guess you are speaking about the phase-detect one. The contrast-detection, I think, is going through the CPU since it works with the same data as the main image processor, and perhaps only the CPU is able to process this large amount of data. Either way, to calculate the contrast in a limited area does not took much time. The point is in the lens control.

My Sigma 17-50 f/2.8 OS HSM will move right with a 10 ms (minimum as of yet) time delay in Rack Focus, though won't without the delay. Possibly this or another lens could do it with even smaller delay. However, for CDAF to refocus from closeup to infinity in good light takes about 4s on my 600D! If your calculation 15 points x 0.3 s = 1.5 s was correct, that would be a great improvement. At least everybody could try to find the best settings for his needs.

[dmilligan]

This is an embeded/real-time system. If you are not familiar with the way these types of systems are designed and operate (which is VERY different from a traditional desktop computing platform) it is a bad idea for you to make speculations about what the CPU does and doesn't do.

As typical with embeded/real-time systems, esp. those responsible for handling/processing huge amounts of data, the CPU itself is responsible for almost none of the 'heavy lifiting'. Most image processing is done in hardware; there's just too much data to try and process things in real-time in software running on a dinkly little ARM CPU. Case in point: raw recording. Raw recording works because devs figured out 2 things: the location of the buffers containing the raw data, and how to use the 'edmac' or external memory controller. Even trying to just simply copy the raw data buffer using the CPU, was not fast enough. ML devs reverse engineered the memory controller which allows the transfer of large blocks of memory from one place to another without CPU overhead. Basically the CPU just tells a piece of hardware 'copy this block of memory over to this block of memory', and the hardware itself does the whole thing in very few clock cylces, b/c that's the way it was designed to work. Analysis/processing works the same way. There's dedicated hardware to do the processing, b/c it's much cheaper to design a piece of hardware to be very fast at one specific task, than it is to design a generic, general purpose CPU that is fast enough to accomplish the same thing 'in software'. Demosaicing, WB, Picture Style, color space conversions, noise reduction, jpeg/h264 encoding, are all examples of things done by specialized hardware not the CPU.

CDAF may or may not be done in that chip, but it is very likely being done in hardware, either way it doesn't matter, we can't change the way hardware works. You may be thinking that the 'CPU' is the entire DIGIC processor, but that is not the case. The DIGIC chip has a generic ARM CPU as part of it (that's the part that we can control), but it also has a lot of custom hardware that we don't know much about. Some of the hardware we have reverse engineered and know what it does and how to use it, but it's still hardware, so there's no way to change the way it works (all we can do is change 'how' we use it, not 'what' it does). ARM itself is a generic CPU design that was created for the very purpose of allowing hardware makers to embeded it easily into their custom chip designs or ASICs.

If your calculation 15 points x 0.3 s = 1.5 s was correct, that would be a great improvement. At least everybody could try to find the best settings for his needs.

That was not intended to be a realistic calculation about anything, but merely an example of the absolute best case scenario for a relatively small change in focus. I pulled the number 15 out of thin air, mostly because it is easy to divide from 30. Most lenses have hundreds of focus steps, and it may not even be possible to check focus every frame, that is simply the best case scenario. My point here is that there is a limit to the speed at which we can take measurements, and we'll probably have to take a lot of them simply due to the nature of CDAF. This means that it is much more likely that we are limited by the speed at which it's possible to move the lens, and the speed at which we can take measurements, than we are limited by the speed of analyizing the measurement (which is probably going to be really fast b/c it's done by custom hardware, NOT a CPU, by trying to use the CPU, there is only potentially for making things worse).

[dmilligan]

Logically, the manufacturers should have announced such a technological development. And in that case why don't such cameras offer "pure" PDAF like EOS-M does?

I think they did:

The 650D is the first Canon DSLR that has continuous autofocus in video mode and live view, which Canon calls "Hybrid AF". This feature, intended mainly for video recording, is implemented through a new sensor technology in which certain pixels are dedicated to phase-detection AF (traditionally used in DSLRs) and others to contrast-detection AF (used in camcorders).

See also:
http://www.dpreview.com/reviews/canon-eos-650d-rebel-t4i/15

but *fast* CDAF-driven cameras still exist

I imagine most of the cameras since the 650D have this "Hybrid AF". Are these the cameras you're refering to?

[SIM62000]

Thank you for the explanation, there is something I did not know before. I can not argue against that, just look at the following.

My system can do about 30 steps in one second (with StepSize=1 or 2 and TimeDelay=10 ms)  and get from closeup to infinity in just above 1 s (with StepSize=3, 18 steps.) Intuitively I feel that 50 points should be more than enough to get the precise focus provided the algorithm is right. (BTW, did you find out how many points the camera's own CDAF takes?). If I am right, then the speed of the command execution is already adequate. The question is how quickly the contrast can be calculated and how quickly this data can be converted into the focus commands. That is just FYI, in case you are interested.

Concerning the embedded PD pixels, you are right that the manufacturers announced the technology [at least in some cases] where it was implemented. But I did not hear nothing like that about some of the fastest as of yet CDAF systems. The Pana GH3, if I can remember, claimed about 0.1 s, ant it does not offer PDAF at all; the full-frame Sony A7R is apparently faster even than the 70D with its on-chip PDAF).