5D2 RAW video Builds 14-Bit

[reddeercity]

Thanks A1ex for the explanation, now I understand what you tring to do
I will do some tests and post them, with my conclusion .

[romainmenke]

Camera-specific exceptions are not always desirable (they make the code harder to maintain and they have a negative effect on testing coverage). I've asked you to convince me that the extra effort to add the exception (and to maintain that exception from now on!) is worth it. I'm trying to simplify things, since this will reduce the burden of maintaining the code base (and this is one of these things that can be kept simple and portable).

There is a difference in quality (2048 looks amazing by the way) but for those who work with different cameras on one project I'd opt for 1856.
Camera-specific exceptions are also not desirable for post processing.
I'm not being lazy and I do care about quality, but keeping ML "simple" to use is important I think.

Also the 5D2 is no longer in production and keeping the code simple will make it more likely the camera will be supported by Magic Lantern for the coming years.

p.s. why is there so little moire in the 2048? Such a big difference between 2048 and the others.

[xvince1]

As far as I can come with my fresh judgement about 1856/1872/1880, I agree with Reddeercity : I always shoot with the maximum resolution I can get (that's why I bought this 2000€ camera for a hobbyist, instead of 1000€ like my brain should have). I obviously continue to use RAW beside MLV (and because I don't really use audio) just to get 1880 instead of 1872. But I really understand the coder's positions and portabilities arguments. To be honest, I think I will stay with last releases as I have found some compromises and because the present work (performances / reliability) seem's to me already convenient.

And as always : a massive thank for that...

[gabz]

p.s. why is there so little moire in the 2048? Such a big difference between 2048 and the others.

Probably because it's in 3x mode, so no line skipping

@ a1ex
When you ask if we want more pixels, the obvious response has to be yes, but ultimately i'm not a coder, and you know what you are doing.
So i'll go along with your choice either way !

Edit : and as pointed by xvince, we can still use previous builds for 1880

[dariSSight]

The horizontal resolution is restricted to multiples of 8 bytes and 16 pixels according to latest findings. This restriction is valid at least for 5D3 and 6D (didn't do much testing on other cameras), but I'd like to keep the code portable without camera-specific exceptions.

So, before including this change in nightly builds, I'd like to ask you which is better: 1888 with 8 pixels of black border that you will have to crop, or 1856 without any border pixels? Between these 2 values, there are no valid resolutions that respect the alignment restrictions.

This change was discussed here:

http://www.magiclantern.fm/forum/index.php?topic=3904.msg106087#msg106087
https://bitbucket.org/hudson/magic-lantern/pull-request/438/raw-recording-force-line-size-to-be/diff

In a world where Government and Self Victimization Citizen fight for paper and manufacturing jobs, you are a Breath of productivity. I will always vote for resolution but I'm grateful for the work you and the ML team do, so I'm good with your end decision.

[guentergunter]

Well, there are differences. But I guess, they barely exist in real circumstances.
Also, there's so much more degradation from Moiré or crappy lenses or ISO noise, etc.

[poromaa]

I would vote for letting the developers decide, but would encourage to keep it simple. Exceptions might be added later (if really needed). If kept simple, more time can be put into other work, and the releases might get even more stable.
Again, thanks all devs for great work!

[reddeercity]

I would vote for letting the developers decide, but would encourage to keep it simple. Exceptions might be added later (if really needed). If kept simple, more time can be put into other work, and the releases might get even more stable.
Again, thanks all devs for great work!

What do mean, it's all readly stable !
The problem is with all the other cameras e.g. 5d3,6d . Read the link that a1ex talk about.
There  asking the 5d2 user if there want 1888 or 1856 , Do Not leave This in the hands of the
Other people to decide for you ! Just because some more work , beside all that
The 5D Mark2 should be not limited , Its the Camera that Started
The DSLR  revolution and gave Birth to Magic Latern ! So let make this camera
The best it can be even it we have to put more work in the code. It's well worth the extra
effort .

P.S. My test for this are still on the way   

[xvince1]

Looking to ted's test, I thought about VAF on 5D2 and the difference between 2048 crop downscale and the 2 other non crop resolution. The aliasing seems to me really so huge and I was asking myself in which ways decreasing resolution for upscaling will play.

To me the 1880 looks sharper than 1856 and 2048 look clearly sharper than the 2 others. I made some tests in RAW mode last summer with an old 600x compact flash @ around 1580 then upscale to 1920. the diffence with 1880 upscaled was really huge. 

I agree that between 1856 and 1880, the difference is not important, but if I'm asked between 2 resolutions : I definitively vote for the maxima.

[terranaut]

if the only downside of the larger 1880 is the black column-edge pixels, any mlv converter could just have an auto-detect camera type, and when its a 5d2, crop the needed empty columns out to fit a desired ratio. or perhaps theres some type of plugin for premiere that intelligently doubles-up an images edge column pixels to fill in the adjacent black columns.

[guentergunter]

if the only downside of the larger 1880 is the black column-edge pixels, any mlv converter could just have an auto-detect camera type, and when its a 5d2, crop the needed empty columns out to fit a desired ratio. or perhaps theres some type of plugin for premiere that intelligently doubles-up an images edge column pixels to fill in the adjacent black columns.

Absoluteley! Cropping the image before encoding to dng is easy to program and computed instantly.
And if I got a1ex right, it's no special coding effort to go for 1888. So, yeah I vote for that!

I'm also looking forwand for the tests from reddeercity.

P.S.: I don't see much chances for a premiere plug-in. First: You'll have to adopt it to any new version of premiere. And then? What about Finalcut or vegas or after effects or...
The faster, easier, simply better way will be to crop the borders right in the beginning while converting to dng. Then there's one program and not dozens of plug-ins.

[ted ramasola]

So does an added resolution also help if in case we use a VAF filter? I tested it and also this time I also included how the max crop mode which is 2144x1076 compares downscaled.

This is a lower res image:


Click this to view higher res image.
https://fbcdn-sphotos-b-a.akamaihd.net/hphotos-ak-ash3/t31.0-8/10012698_453226301474826_470179409_o.jpg

[poromaa]

The problem is with all the other cameras e.g. 5d3,6d . Read the link that a1ex talk about.
There  asking the 5d2 user if there want 1888 or 1856 , Do Not leave This in the hands of the
Other people to decide for you ! Just because some more work , beside all that
The 5D Mark2 should be not limited , Its the Camera that Started
The DSLR  revolution and gave Birth to Magic Latern ! So let make this camera
The best it can be even it we have to put more work in the code. It's well worth the extra
effort .

Well, if I had to choose between some few extra pixels and perhaps higher dynamic range, faster recording (continous as say 1856 - I have a 1000x card but it still stops sometimes), shutter-free full-res timelapse etc I would go for the latter. That means, any time the developer needs to put into this few extra pixels is lost time in other development areas. Im just saying that the difference seems too small for me to care. Anyway its my oppinion - thats what forum are for
cheers

[reddeercity]

1856 will still be there so no worry about performance , there saying we will Loose
The upper resolutions  1880, 1872 no more ! You are missing the point here there will be no
Loss to Photo functions and have nothing to do with that Just Raw Video, That's what is
Is all about !

[1880]

Hi, and first, a massive THANKS to all the Devs who have made RAW on the 5D2., possible.

However, I am alarmred with the 'idea' of downgrading the 5D2 to only 1856 !
I have always believed that the core of Magic Lantern, was the pursuit of quality, not convenience.
Unified ?  Mass marketing - Consumer - " One size fits all".

I bought a 5D2 because of Magic Lantern, not because it was a Canon.
I am therfore sadened by the idea that it maybe downgraded, to 'fit' in with other cameras.

There is no argument, that can show that less pixels are, 'the same', or,  better than more.
In pure science, and in the 'real' world, this is NONsense.

I offer my total support to Reddeercity, who speaks sense.

Quality first,    Please !

[a1ex]

FYI, the very existence of raw video and all other features available on most cameras is based on portable code (unified, one size fits all).

I could have easily hardcoded the raw recording code to work only on 5D3, and fully optimize it for that camera only, since it has the most potential (especially regarding write speeds), but I chose to make the code portable. Yes, this means the code may not be fully optimized for any particular camera, but you have RAW on nearly all cameras that ML runs on. Would it have been better to have RAW only for 5D3?

If you are alarmed by the idea of us (who gave you all this stuff for free) trying to make our life easier by simplifying the code, feel free to get your hands dirty and implement a clean solution. Complaining without logical arguments will not do, and will only encourage us to stop the raw video development and focus on something else.

[romainmenke]

Hi, and first, a massive THANKS to all the Devs who have made RAW on the 5D2., possible.

However, I am alarmred with the 'idea' of downgrading the 5D2 to only 1856 !
I have always believed that the core of Magic Lantern, was the pursuit of quality, not convenience.
Unified ?  Mass marketing - Consumer - " One size fits all".

I bought a 5D2 because of Magic Lantern, not because it was a Canon.
I am therfore sadened by the idea that it maybe downgraded, to 'fit' in with other cameras.

There is no argument, that can show that less pixels are, 'the same', or,  better than more.
In pure science, and in the 'real' world, this is NONsense.

I offer my total support to Reddeercity, who speaks sense.

Quality first,    Please !

The one argument is for the devs who put their time into creating ML and maintaining out of production camera's.

I would ask you to look at the big picture. Camera's can be rented now with ML and big productions use many different camera's, giving the 5D2 footage with black borders is like killing it. People will just choose other camera's.

Not only will the 1888 exception make the camera less desirable for recording but if one day the devs decide they will no longer support the 5D2 because of exceptions like this, you will be forced to upgrade. That is what I think the essence of ML is. Upgrading existing gear through firmware updates, unlocking and creating features, so you don't have to buy new gear every year.

Future first!

[romainmenke]

If you are alarmed by the idea of us (who gave you all this stuff for free) trying to make our life easier by simplifying the code, feel free to get your hands dirty and implement a clean solution. Complaining without logical arguments will not do, and will only encourage us to stop the raw video development and focus on something else.

I was thinking the same thing!

[1%]

It might be worth testing the edmac flags that caused this resolution resolution restriction in general.

uint32_t dmaFlags = 0x40001000; // "enhanced"

vs
   
uint32_t dmaFlags = 0x20001000; //Original


I recorded some more on 7D/6D and not really sure about the improvement... On 6D ended up struggling with 720P and indicators took a long time to say continuous. Interestingly though at a higher res I got a few, maybe 20 more frames. 7D took longer to reach continuous too and write speed jumped all over the place. Slurp still seems like its set to the original flags and I think I may have changed them to the "enhanced" ones by mistake, leaving the originals for normal raw. Maybe the restriction + original flags are the best as it exposed some underlying problem and would explain why some resolutions performed "better".

[mWaltari]

What kind of room there is for optimization for 5DmkIII? Just interested and if someone has capacity to pick up and implement those

Is alex talking of code options or specific hardware related ones (like sensor etc)

[a1ex]

More accurate prediction of file write speeds, low-level file I/O, fully exploit the dual card slot, modify the source resolution to avoid the EDMAC copy step, hardcode all raw parameters, fine-tune task priorities, stop/slowdown Canon tasks not related to raw recording, refactor ML to use fewer tasks, optimize raw recording code to reduce idle times, stop YUV LiveView code, find more unused memory areas, find out how to change YUV resolution and crop window for a more efficient preview, exploit external storage mediums... I could fill a page with this stuff.

Some of these things are not camera-specific (e.g. task refactoring or code optimization). Others, like write speed prediction, may benefit from a mathematical model tailored to each camera, but I'll prefer a general solution (one size fits all, with automatic tuning). Others are highly camera-specific and I prefer to avoid them, unless the tweak brings significant improvement without major side effects.

[xvince1]

To be honest, there's something I don't get : what the trouble to just add 1888 (if it's not too much complex to add in code) : All people who do not want to post-prod crop still can use 1856, and others can use max resolution ? (perhaps some notice in the readme file to warn about black border on 5D2, and that's all)

To a1ex :

Just to be precise : Actually, I (we ?) don't know clearly the history of ML developpement team and I can't really figure what's the involvement or deserving of every dev' in digging Canon code. BUT I can clearly figure that for the original core of the team it can be very painfull to see other people forking, using the original code you'd share, without following your guidelines, and I clearly understand your concern about portability.

I REALLY don't want to be fresh or misrepectfull about your work, I've only tried to honestly answer your question about max resolution on 5D2.

[reddeercity]

It looks like all Hope is Lost for the 5D Mark 2 this sadden me Greatly ! 
What camera will be Next , All for the greater good of the 5D Mark 3 !
I guess in the end the dev. can write all the code there want but, if not one uses
It what good is it.
I thought Magic Lantern was user base group who all have say in the end result
Maybe I'm wrong.

[ted ramasola]

I can choose between 1856 and 1888 with no coding effort. Between these 2 numbers there are no values that respect the alignment restrictions from EDMAC (it happens to work on 5D2, but I don't want to rely on this for all other cameras).

Camera-specific exceptions are not always desirable (they make the code harder to maintain and they have a negative effect on testing coverage). I've asked you to convince me that the extra effort to add the exception (and to maintain that exception from now on!) is worth it. I'm trying to simplify things, since this will reduce the burden of maintaining the code base (and this is one of these things that can be kept simple and portable).

@a1ex,

Just to clarify, is that a typo you wrote 1888 or does the 5D2 actually can do 1888 aside from 1880 ?

Here's my take on the resolution issue since you wrote it can be done with "no coding effort".
I can understand the burden from the developer's side in maintaining camera specific code. However, we must also consider that the cameras themselves were never really intended to perform similarly. Each has its own strengths. So at some point in making code, I think one should consider allowing each camera model stand out and maximize what it can truly be capable of and not sacrifice it for the sake of efficiency. I think you can still squeeze out some "juice", whatever that may be, from this old lady the 5D2, and I personally think that it should be allowed to. Other cameras can't do it. The 7D for instance, may not have the resolution the 5D2 has in 1X mode, however it shines in other features like higher frame rates and higher resolution in 3X mode. And just as the 7D is allowed to shine in that area, I believe the 5D2 should also have its niche.

[Audionut]

The Magic Lantern project is for the community.  It is not developed solely for individual users.

If an individual wants a feature implemented, then an individual is free to make his/her own modifications to the source code.  This is an open source project after all.

You are granted no rights with Magic Lantern.  This includes the right to complain, in a manner that is unbecoming to the time and effort that developers place into maintaining and enhancing this project.


a1ex could have simply implemented any resolution he saw fit, instead, he choose to bring the decision to the forums, for open discussion by members.