Magic Lantern Forum

Using Magic Lantern => Raw Video => Topic started by: csv on April 10, 2014, 06:29:36 PM

Title: misunderstandings...
Post by: csv on April 10, 2014, 06:29:36 PM
Hi
I read a lot, please believe me! And I wonder if I am stupid or something like that because these are a few points I absolutely don't understand.
But I need to understand: my aim is to buy a camcorder, and I am hesitating more and more each day. One of my options is the 5DM3 with ML.
First and foremost I have to thank the ML team: what a remarquable work! And please forgive me if my questions are stupids!
My questions are about the 5DM3, in raw, and especially about ratio and card compatibility. Let's ask:
1-the 5DM3 have a 4K sensor. Why is the video only usable at 1920*1080 max in raw?
2-or, I saw that it was possible to "crop" and use more of the sensor. But, at that point, I am lost: if one use a larger space on a sensor, he would expect that the field of view is larger (with the same lens, of course)! But it seems to be less large! I do not understand.
And I would like to use more of 2K to make movies, in RAW, with a larger field of view.
3-SD cards seem to be very short for making long footage. Can we expect that the 5DM3 will use brand new SD cards, which could accept filming at 2.5 or 3K in raw more than a few seconds?
Thanks very much for your helping (and sorry for my bad English...)
CS
Title: Re: misunderstandings...
Post by: Walter Schulz on April 10, 2014, 07:16:55 PM
ML team found a way to grab the data stream sent to the cameras display in video mode. And using a 5D3 native display resolution is just 1920x1280. (Other Canon DSLRs have lower resolutions. See here (https://docs.google.com/spreadsheet/ccc?key=0AgQ2MOkAZTFHdFFIcFp1d0R5TzVPTVJXOEVyUndteGc#gid=5).)

This mode uses the sensor's full area but not every pixel! That's your first misunderstanding.

In crop mode every pixel is used but not the whole sensor area. It's cropped.
Quite an oxymoron: Crop mode therefore uses a smaller area but is able to use more pixels.

And SD in 5D3 is limited to about 21 MByte/s transfer speed. Not able to keep up recordings in high resolution RAW video mode for more than a few seconds.

Ciao, Walter
Title: Re: misunderstandings...
Post by: rainless on April 10, 2014, 07:20:38 PM
Buddy you've got a loooooooong way to go :)

First I hope you're aware that the 5D3 records to SD and Compact Flash... and that CF can write around 160mbs if you have a fast one.

Also it's not about size... it's about speed. The higher the resolution... the faster a write speed you need.

Other than that Walter already covered everything. (You might also want to check out my 2014 Raw video Capabilities thread.)
Title: Re: misunderstandings...
Post by: chmee on April 10, 2014, 09:59:11 PM
Quote from: Walter Schulz on April 10, 2014, 07:16:55 PM
This mode uses the sensor's full area but not every pixel! That's your first misunderstanding.
I just thought about the 1x Filmmode and the technique behind it. Is it proven, it's not using every pixel? Is there any "Merger/Binning" between Sensor and Buffer, that is doin the job? Because the Aliasing/Moiré-Artifacts are not as aggressive, as it should state for a skipping-technique.. (talking 'bout 5DIII)

regards chmee
Title: Re: misunderstandings...
Post by: a1ex on April 10, 2014, 10:08:06 PM
If somebody takes some regular and silent pictures of a resolution chart from a really sturdy tripod (of course, with remote release, not by touching the camera), it should help clearing the mystery.

I said silent pictures and not video frames because silent pictures are not cropped, and I'm also looking for pixel-perfect alignment between the "noisy" picture and the LiveView frame. Mirror movement will interfere with this alignment; locking up the mirror with some adhesive tape should help minimizing the vibrations. A long exposure or Canon's LiveView silent shooting might help too, since the vibrations would get some time to settle.
Title: Re: misunderstandings...
Post by: ted ramasola on April 11, 2014, 02:02:35 AM
@chmee,

the mkIII has a more "agressive" OLPF than the previous mkII thats why in h264, this makes it appear better in handling the issues caused by line skipping(aliasing) but its there, it even appears softer than the mkII in h264 with all settings being equal. However in raw the OLPF seems to work to the mkIII's advantage as the mkII suffers more pronounced aliasing and moire in raw than in h264 mode due to less powerful OLPF, thats why it needs "another" in the form of the mosaic VAF. The mkIII does not need the VAF anymore.
Title: Re: misunderstandings...
Post by: a1ex on April 11, 2014, 07:53:55 AM
If it would be OLPF, it would be also soft for stills (just like VAF).

Hint: http://www.chipworks.com/en/technical-competitive-analysis/resources/blog/full-frame-dslr-cameras-canon-stays-the-course/

"Canon's X030334 uses an additional transistor (T5) that can be used to  sum the FD nodes of three rows of even (or odd) pixels of the same  color."
Title: Re: misunderstandings...
Post by: chmee on April 11, 2014, 09:47:51 AM
@ted ramasola
if the OLPF would be that aggressive, the sharpness-decrease would be visible in the nocrop-ml-videoframes as well, but it isnt. imho the details-difference between h.264 and nocrop-ml is caused by the "suboptimal" raw->jpg->h.264-transcoding-routines.

regards chmee
Title: Re: misunderstandings...
Post by: csv on April 11, 2014, 10:49:26 AM
Thank you very much!
Yes, maybe a long way to understand everything. I'm coming from video and cine, so DSLR are a very strange world for me.
I do understand now how cropped footage works!
I didn't notice that the 5DM3 uses CF at 160 Mb/s. But, since it uses it, how isn't it possible to shoot more than a few seconds in 2.5 or 3K? It's not at 160, but around 100/120 I think...
Thanks again!
CS
Title: Re: misunderstandings...
Post by: Walter Schulz on April 11, 2014, 10:58:38 AM
100 MByte/s? At 24 fps you will have to have RAW size about 4 MByte. Don't think the numbers match with 14 bit RAW at 3k ...
Title: Re: misunderstandings...
Post by: chmee on April 11, 2014, 11:08:26 AM
@csv simple Maths - one Frame[bytes] = width*height*14/8

example A: 1920*1080*14/8 = ~3.6MB -> *24fps =~87MB/sec (5DIII, unlimited rec possible with one fast CF-Card)
example B: 2500*1080*14/8 = ~4.7MB -> *24fps =~114MB/sec (5DIII, limited rec possible with one CF-Card due to CF-Slot-Limitations, unlimited with CF+SD)
example C: 3000*1276*14/8 = ~6.7MB -> *24fps =~160MB/sec (5DIII, limited some seconds?)
Title: Re: misunderstandings...
Post by: csv on April 11, 2014, 02:52:01 PM
Thank you Chmee, you answer the question as I wouldn't expect! So CF or SD, due to limitation, no more than 2K. Not so bad, of course, but I extacly would like to do something like 2500*1080. Too sad...
Why is CF limited???  :(
Title: Re: misunderstandings...
Post by: chmee on April 11, 2014, 03:14:28 PM
as a (ml)optimization theres is the combined mode, where you record on CF and SD-Card simultaneously. you'll get about 120-125MB/sec.

I cant tell you, why canon built such a CF-bus-controller inside, having in mind, 166MB/sec is just the theoretical max-speed - and being gentle, it is enough to do photos (the main purpose of this piece of electronics). with the komputerbay 1000x card i nearly can shot ~30 RAW-Pictures at 6fps, then it get hickups, but its quite fluid to shot without a pause.

https://www.youtube.com/watch?v=N6HWreyyo1g

Title: Re: misunderstandings...
Post by: csv on April 11, 2014, 05:51:50 PM
Whaou! Combined mode! Workflow must be terrific!
But you are right, of course: these are made for photos...
Title: Re: misunderstandings...
Post by: eoshq on April 12, 2014, 09:46:40 PM
@ted ramasola, You need to put your thinking cap on and stop posting BS about the 5D3. The 5D3 in stills mode is the highest resolving Canon DSLR to date. This obviously would not be possible if it had a super aggressive OLPF.

With every lens tested on both by DXO, it out resolves the 5D2, often by a larger percentage than the increase in megapixels would suggest. This actually lends evidence to the 5D3 having a weaker OLPF than the 5D2.

This is a good example:
http://www.dxomark.com/Lenses/Compare/Side-by-side/Zeiss-Carl-Zeiss-Apo-Sonnar-T-Star-F2-135-ZE-Canon-on-Canon-EOS-5D-Mark-III-versus-Zeiss-Carl-Zeiss-Apo-Sonnar-T-Star-F2-135-ZE-Canon-on-Canon-EOS-5D-Mark-II___1017_795_1017_483 (http://www.dxomark.com/Lenses/Compare/Side-by-side/Zeiss-Carl-Zeiss-Apo-Sonnar-T-Star-F2-135-ZE-Canon-on-Canon-EOS-5D-Mark-III-versus-Zeiss-Carl-Zeiss-Apo-Sonnar-T-Star-F2-135-ZE-Canon-on-Canon-EOS-5D-Mark-II___1017_795_1017_483)

Quote from: ted ramasola on April 11, 2014, 02:02:35 AM
@chmee,

the mkIII has a more "agressive" OLPF than the previous mkII thats why in h264, this makes it appear better in handling the issues caused by line skipping(aliasing) but its there, it even appears softer than the mkII in h264 with all settings being equal. However in raw the OLPF seems to work to the mkIII's advantage as the mkII suffers more pronounced aliasing and moire in raw than in h264 mode due to less powerful OLPF, thats why it needs "another" in the form of the mosaic VAF. The mkIII does not need the VAF anymore.
Title: Re: misunderstandings...
Post by: ted ramasola on April 13, 2014, 12:18:58 AM
Quote from: eoshq on April 12, 2014, 09:46:40 PM
@ted ramasola, You need to put your thinking cap on and stop posting BS about the 5D3. The 5D3 in stills mode is the highest resolving Canon DSLR to date. This obviously would not be possible if it had a super aggressive OLPF.

With every lens tested on both by DXO, it out resolves the 5D2, often by a larger percentage than the increase in megapixels would suggest. This actually lends evidence to the 5D3 having a weaker OLPF than the 5D2.

This is a good example:
http://www.dxomark.com/Lenses/Compare/Side-by-side/Zeiss-Carl-Zeiss-Apo-Sonnar-T-Star-F2-135-ZE-Canon-on-Canon-EOS-5D-Mark-III-versus-Zeiss-Carl-Zeiss-Apo-Sonnar-T-Star-F2-135-ZE-Canon-on-Canon-EOS-5D-Mark-II___1017_795_1017_483 (http://www.dxomark.com/Lenses/Compare/Side-by-side/Zeiss-Carl-Zeiss-Apo-Sonnar-T-Star-F2-135-ZE-Canon-on-Canon-EOS-5D-Mark-III-versus-Zeiss-Carl-Zeiss-Apo-Sonnar-T-Star-F2-135-ZE-Canon-on-Canon-EOS-5D-Mark-II___1017_795_1017_483)

I'm not talking about stills. I'm talking about video, especially h264 video native to both 5D2 and mk3. The OLPF even in older models are tuned for stills, that's why these cameras are so popular for stills. When the 5d2 became popular in video by "accident" it was not something canon expected, that's why 24p was added later due to the clamor.

The 5D3 produced cleaner video images compared to 5D2, lesser aliasing and color moire, but it "appears" softer as a tradeoff, this was actually not because the 5d@ was sharper but rather they appeared sharper due to the aliasing and enhanced edges.so many early adapters were surprised with this, a few daring souls even resorted to removing the OLPFs. However the OLPFs worked well with the ML raw dngs.
Title: Re: misunderstandings...
Post by: eoshq on April 13, 2014, 08:55:46 PM
ted, You still don't get it and you never will. Your logic is failed. You have been going around saying the "the mkIII has a more "agressive" OLPF than the previous mkII". When it is pointed out that this cannot be true, your EGO has you talking nonsense in circles, instead of just admitting you are wrong. There is no hope of understanding with you.
Title: Re: misunderstandings...
Post by: rainless on April 13, 2014, 10:53:58 PM
Quote from: eoshq on April 13, 2014, 08:55:46 PM
ted, You still don't get it and you never will. Your logic is failed. You have been going around saying the "the mkIII has a more "agressive" OLPF than the previous mkII". When it is pointed out that this cannot be true, your EGO has you talking nonsense in circles, instead of just admitting you are wrong. There is no hope of understanding with you.

Yeah but you seem to be more of a dick about it... You're talking about CAMERAS... why don't you chill the hell out?

Who cares if the 5DmkIII has more OLPF or not? Is anybody arguing that, given the choice between either camera for free, somebody's going to choose the 5DmkII?

No.

And in that case all the bickering is pointless.
Title: Re: misunderstandings...
Post by: chmee on April 13, 2014, 11:22:23 PM
@eoshq what about this: not the OLPF is the problem, but the microlenses-structure, its gapless in the 5DIII. think about it instead of complaining.
Title: Re: misunderstandings...
Post by: reddeercity on April 14, 2014, 12:39:21 AM
What Ted said is right for Video, it a known fact to anyone that shot video on the full frame Canon's :)
Title: Re: misunderstandings...
Post by: Audionut on April 14, 2014, 02:40:46 AM
@eoshq

Is your only evidence, a misunderstanding based on DxO results?
Title: Re: misunderstandings...
Post by: eoshq on April 14, 2014, 08:31:42 PM
@chmee

The gapless microlens structure is not acting like a VAF OLPF in the 5D3 for video. If it were, it would at the same time destroy the resolution of the optical system for 22 megapixel stills.

The key to the 5D3's lack of moire in video has everything to do with (5760 x 3840) and the Digic 5 processing. The fact that the 5D3 is 22.1 megapixels and not more or less, is no coincidence. That resolution was chosen to make it a better video camera. Think about these two numbers (5760 x 3840) again carefully and I think it will become quite obvious.



Quote from: chmee on April 13, 2014, 11:22:23 PM
@eoshq what about this: not the OLPF is the problem, but the microlenses-structure, its gapless in the 5DIII.
Title: Re: misunderstandings...
Post by: eoshq on April 14, 2014, 08:51:05 PM
@reddeercity,

We can't divide the 5D3's optical system into one for video and a different one for full resolution photos. Any idea that suggest such a theory is nonsensical. It is the same camera in both modes and anything that has been done in the optical system to minimize moire in video would also affect the optical system in photo mode.

As we can see with the VAF add-on OLPF's for the 5D2, 6D, 7D, D800, etc. An OLPF strong enough to stop moire and aliasing in these cameras can only be used in video mode and must be removed for photos, because it destroys the resolution of the camera. Obviously no such thing is going on with 5D3 as it has higher resolution in photo mode than the 5D2.

Quote from: reddeercity on April 14, 2014, 12:39:21 AM
What Ted said is right for Video, it a known fact to anyone that shot video on the full frame Canon's :)
Title: Re: misunderstandings...
Post by: eoshq on April 14, 2014, 09:05:28 PM
Quote from: Audionut on April 14, 2014, 02:40:46 AM
@eoshq

Is your only evidence, a misunderstanding based on DxO results?

@audionut,

This is a pretty classic attack method you have chosen against me. You have said that I don't understand DXO's lens testing without saying why. You want to discredit me somehow, but you are to lazy to explain yourself or you actually don't know what you are talking about. Either way, I challenge you. Explain yourself or be quiet about it.

I will even give you a hint. DXO's sharpness test are just another way of presenting the same data that has always been used to express the resolution of an optical system. Go ahead and make my day.
Title: Re: misunderstandings...
Post by: chmee on April 14, 2014, 09:37:59 PM
@eoshq
you're not capable doin' a smart, friendly dialog. using the wrong words. Dude, not with me, and me not with you. shame.
Title: Re: misunderstandings...
Post by: Audionut on April 15, 2014, 03:42:06 AM
Quote from: eoshq on April 14, 2014, 09:05:28 PM
You want to discredit me somehow, but you are to lazy to explain yourself or you actually don't know what you are talking about. Either way, I challenge you. Explain yourself or be quiet about it.

It was a simple question.

http://www.nikonusa.com/en/Learn-And-Explore/Article/gy43mjgu/moir%C3%A9-false-color.html
QuoteIn many D-SLRs an OLPF—also known as an anti-aliasing filter—prevents false color and moiré that can ruin a picture. The trade-off to using an OLPF is slight softening of the image. To counteract the softness, in-camera sharpening is used and can be further adjusted by the end user. In addition, photographers who shoot the NEF (Nikon Electronic File) RAW file format can add more sharpening in post-production.

Do you have hard evidence to to describe the amount of in-camera sharpening applied on the cameras in question?


Quote from: eoshq on April 14, 2014, 09:05:28 PM
I will even give you a hint. DXO's sharpness test are just another way of presenting the same data that has always been used to express the resolution of an optical system. Go ahead and make my day.


QuoteSharpness

Sharpness is a subjective (http://dictionary.reference.com/browse/subjective) quality attribute of an image or a lens. Sharpness indicates the visually perceived quality of details of an image or details reproduced by a lens. It is associated with both resolution and contrast of reproduced details (within an image or by a lens).

The DxOMark score for Sharpness is based on the Perceptual Megapixel (P-Mpix) concept that weights the Modulation Transfer Function (MTF) of the lens with the human visual acuity. Read more about Perceptual Megapixels.


I question DxO marks measure of sharpness, as a valid indicator, of the performance of an OLPF.  Subjective measurements are not necessarily accurate.


Now I challenge you.  Lose the smug arrogant trolling attitude, or go away.  Your choice!
Title: Re: misunderstandings...
Post by: eoshq on April 15, 2014, 07:12:40 AM
What I said from the very beginning was that the 5D3 does not have a stronger OLPF than the 5D2 and  I want people to stop saying that it does, because that simply is not true. And it certainly is not the reason that it does not suffer from moire and aliasing as much as many other DSLR's.

I used DXO's testing as one example of solid, scientific evidence about the 5D3's OLPF. What DXO is testing is the combined MTF performance of a lens and camera together as an optical system. They present it as Perceptual Megapixel (P-Mpix) and call it sharpness to be easier for most people to understand. They really should call that score resolution because that is what they are measuring. When they say that the 5D3 achieves 18 P-Mpix with the 40mm f/2.8 and the 5D2 achieves 15 P-Mpix with that same lens, they are just saying that the 5D3 reaches a higher MTF score.

"The Perceptual MegaPixel (P-MPix) is a proprietary ranking of lens-camera combinations, created by DxO Labs. It is intended to express the resolution a camera produces when paired to a particular lens."

DXO has tested 108 lenses on the 5D3 and 5D2. In every case, the 5D3 reaches a higher score. And this increased resolution can be also seen when comparing images in print or on screen. Full stop, that should be the end of the discussion about the relative OLPF strength of these two cameras. What more evidence is needed?

But wait, there is more:

We know that the 5D3 does not have any change in its optical system or sensor between still photo and video modes. There is no magical OLPF that moves in and out from the in front of the sensor.

We know from the folks at VAF who make addon OLPF's specifically to reduce alaising and moire of the 5D2 and other cameras in video mode, that these filters utterly destroy the full resolution image and must be removed for photography.

We know that early on a few people removed the OLPF filter from the 5D3 in hope that it would increase video resolution and this did not work. If they had thought to compare full resolution still images with the 5D2 beforehand they would have known this would not work.

We know that the 5D3 in raw video mode has increased resolution, again with no change in the optical system.

We know that the 5D3's full resolution is 5760x3840 or 5760x3240 in 16:9 ratio for a good reason. That reason can simply be expressed as 5760/3=1920, 3240/3=1080

We can speculate about the video processing inside the 5D3 that creates h.264 and why it is not as sharp as we would like, but I would guess that the reason is a lack of processing power and concerns with heat dissipation.
Title: Re: misunderstandings...
Post by: Audionut on April 15, 2014, 07:45:37 AM
I agree that your analysis seems solid.  However, you cannot conclusively say that the filter is stronger, weaker, or the same, based on a subjective measurement.  You can advise that the results suggest something, but you cannot be adamant.

You should consider this before accusing others of spreading BS, or otherwise expressing any opinion, that differs from yours.

Attack the post, not the poster.  ;)
Title: Re: misunderstandings...
Post by: eoshq on April 15, 2014, 11:03:13 AM
Thank you. I tried to make my analysis in an objective manner. I am basing my conclusions on actual MTF measurements made by DXO and my own analysis of images taken with both cameras in fully controlled tests.

Food for thought:
1. Both the A7R and D800 achieve maximum DXO P-Mpix scores of 29 megapixels even though they are 36 megapixel cameras. With each camera that score is only with one exceptional lens. The vast majority score much lower. Could it be that somewhere around 30 megapixels is the maximum usable resolution of a 24x36mm sensor and only with a nearly perfect lens? Why is it that lower megapixel cameras such as the D600 and 5D3 can achieve scores much closer to their full resolution?

2. Canon has made decent gains in resolution over the last few years even with zero or very small increases in megapixels. Witness the 1D-X, 5D3 and 6D's gains over the previous generation 1DMIV, 1DsMIII and 5D2 as well as the increases of the 70D/700D/100D over the 7D. What is Canon doing differently now with their sensors versus a few years ago?

Title: Re: misunderstandings...
Post by: eoshq on April 15, 2014, 11:16:06 AM
I am not good at being diplomatic but really all I wanted to do is give the information that I had found. Thank you for listening and for your contributions to the Magic Lantern and DSLR video communities.
Title: Re: misunderstandings...
Post by: ted ramasola on April 15, 2014, 11:49:53 AM
I concede that my previous observations regarding the behavior of the 5d mkIIIs image may have contained invalid conclusions.

eoshq's analysis on the image quality contains some important information that has provided me with better understanding.
Title: Re: misunderstandings...
Post by: Audionut on April 15, 2014, 12:04:23 PM
Quote from: eoshq on April 15, 2014, 11:03:13 AM
Could it be that somewhere around 30 megapixels is the maximum usable resolution of a 24x36mm sensor and only with a nearly perfect lens? Why is it that lower megapixel cameras such as the D600 and 5D3 can achieve scores much closer to their full resolution?

Pixel size is something that immediately springs to mind.  I would assume that smaller pixels can capture smaller detail.
There's obviously a defined limit, based on the sensor size.  No idea what that is.


Quote from: eoshq on April 15, 2014, 11:03:13 AM
2. Canon has made decent gains in resolution over the last few years even with zero or very small increases in megapixels. Witness the 1D-X, 5D3 and 6D's gains over the previous generation 1DMIV, 1DsMIII and 5D2 as well as the increases of the 70D/700D/100D over the 7D. What is Canon doing differently now with their sensors versus a few years ago?

Processing IMO.
Better processing could mean a reduction in sensor filtering, resulting in increased detail.

Everything is a game of pros vs cons.  IIRC, the new Sigma 50/1.4 could have had much greater sharpness, but the result was increased CA, and something else I cannot recall atm.  So they choose to reduce sharpness, for a better overall product.
I'll add a link when I find it.

Same with pixel size.  Smaller pixels may produce finer details, but they also capture less light.  When the manufacturers do away with antiquated RGB filtering, this won't be so much of a problem :)