crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView

[Greg]

Sorry it will not be possible.
It's just a hint for someone like reddeercity but with 500D.

[Danne]

I see. Thanks. Short on time over here.

[reddeercity]

It's just a hint for someone like reddeercity but with 500D.

If anyone is interested I think we have everything to do it for 500D (Full res LV others will be useless - too slow sensor).
3 years ago I increased the width, but I did not know how to increase the height.

I suppose these are the registers (my old notes) :

Code Select Expand

ADTG[2]1172:     0x3d5 (was 0x667)
ADTG[2]1173:     0x39b (was 0x6e6)
ADTG[2]1178:     0x3d5 (was 0x667)
ADTG[2]1179:     0x39b (was 0x6e6)
ADTG[2]1214:     0x3de (was 0x622)

So look at crop_rec.c - /* some modes require additional height adjustments */

Very useful , thanks  I was wonder on this , I was going to pm you if you have any notes that could help.
I still can't reproduce you test in the 500d , registers are different on 5d2 but close thou.
my problems is the timer A can't lower it lock to 610 , and ever time I try agjusting the Headtimer
I get a frozen camera/lockup (battery pull)  so if I can get timer A to lower I think I will have it
I have the width , but compressed to the left in a 2K crop window , anyways thanks it helps .

[Greg]

I'm afraid there may be a problem with raw buffer. Something like 10/12bit raw video with old cameras.

[Teamsleepkid]

@greg think it will be possible on eos m? I know it won't do 4K or 3k but any resolution boost would be good.

[mk11174]

This is where I am at with 700D so far, just trying to figure out how to change height without it getting all scrambled, was able to change width past 4096 but was getting borders so brought it down to an even number, am able to go to almost 1500 on height until it scrambles, so I am assuming other registers need to be adjusted to make it work.

Also, fps drops to 6.5fps with the register changes to make it work, was able to force it to 9fps with C0F0[6014], any higher caused some sort of overlap on image.

Here are some examples of resolutions I would get by adjusting C0F0[6804]. It seems as I adjust the 3 on the left, the height would change, the 3 on the right, the width would change, the middle 0 seems it has to stay 0. I can go much higher on the height, but as I said, it gets crazy looking, shifty and predator vision like.

Code Select Expand


4550204   1920 x 1080
4540298   1928 x 1144
4e40448   4248 x 1217   
4cc0422   4096 x 1193
45b0422   4096 x 1080  current
50b0564   5384 x 1256


[mk11174]

I managed to get height going by adjusting

Code Select Expand


6804  0xc3b0422     4096x3096
713c  0x4a7 to 0xca7
8172  0x6fd to 0x8fd
8178  0x6fd to 0x8fd


But I end up with this image, not sure how to fix this?

[dfort]

I was thinking about looking into the EOSM and 700D. Since there are others out there that are willing to fry their sensors to get 4k working I posted adtg_gui builds for these cameras.

https://bitbucket.org/daniel_fort/magic-lantern/downloads/

@mk11174 -- Did you already get 14-bit lossless compression working?

[a1ex]

is this any indicator for a bricked camera?

A bricked camera is easy to identify - it does not boot (in other words, it's as useful as a brick). So far, there were two cases, both successfully recovered.

However, the Apr04 null pointer bug might (or might not) have changed some of your settings somehow. Since you have not noticed any side effect yet, your camera is probably OK, or you may find out some strange behavior when using some functionality that you don't use very often. That doesn't count as "bricked", but still, those (persistent) changes that appeared because of that bug are not desirable.

By any chance could it be possible for some of us to have used this particular build and NOT be affected by the null pointer bug at all @a1ex? 

Of course - what happened was not deterministic.

I've just posted a new build that checks whether your camera was affected by this bug. All you have to do is to upgrade - if no message pops up, your camera was not affected by that bug at all. Otherwise, just follow the instructions.

For this particular bug, the persistent changes are easy to identify, because the value that could get written into ROM is always the same - 0xA5A5A5A5.

Note that any other code (from ML and even from Canon) can cause similar issues (and probably much harder to identify). We have patched one such issue many years ago - on most camera models, a bug in Canon's startup code overwrites one single (predictable) memory address with 0 (4 bytes). You can check it by looking for ARMLIB_OVERFLOWING_BUFFER and tracing the patched address in QEMU (e.g. set a value at startup and watch it getting overwritten). In some camera models, this address ended up right in the middle of ML code (thus causing unpredictable behavior that changed with... the amount of code before the affected spot - e.g. insert a printf and watch things working in a totally different way).

This is a limitation of the current CPU and operating system - any task is allowed to write at any memory address. On top of that, Canon code saves their settings to ROM (by reflashing), so all it takes to soft-brick a camera is a programming mistake that results in writing at the wrong memory address.

I can think of a run-time check that detects any writes to Canon's property data structures (some of these are the persistent settings that get flashed into ROM) from tasks other than (Canon's) PropMgr, but it will definitely have a noticeable overhead (impact on performance). I can also think about adapting g3gg0's mem_prot module (which, back then when g3gg0 wrote it, was way above my level of understanding), but this won't catch badly configured DMA transfers. I can also imagine a routine that would check the integrity of Canon's property data structure right before they get saved into ROM (and maybe lock up the camera if problems are detected).

[jankrueck]

nice work @a1ex. I tested new build and now message pops up.
I was/am curious about this error/bug saying cache is locked.

I also can't get 1920x1920 working, can't get any higher than 1728px.
Before the april6 I was able to record square footage.
Not sure if you changed something, or what happend.



Also If I do 3k ISO400 I get a crash when there is to much detail moving.

Code Select Expand

ML ASSERT:
0
at mlv_lite.c:2036 (compress_task), task compress_task
lv:1 mode:3


Magic Lantern version : crop_rec_4k.2017Apr10.5D3113
Mercurial changeset   : 2bbf0bbbaa74 (crop_rec_4k) tip
Built on 2017-04-10 08:54:19 UTC by jenkins@nightly.
Free Memory  : 164K + 3162K



hope my data is helping you in any way. if not, just tell me to stop it


cheers!

ps: Is there a chance to get some reference sound back in 3k mode, to have better syncing in post?

[a1ex]

See this post for the red dot and the first post for resolution:

* 1920x1920 @ 24p (1:1 square crop) - continuous*)
...

Code Select Expand


                                /*   24p   25p   30p   50p   60p */
    [CROP_PRESET_3X_TALL]       = { 1920, 1728, 1536,  960,  800 }, /* 1920 */


ps: Is there a chance to get some reference sound back in 3k mode, to have better syncing in post?

I'll probably stop adding features for a while and will look into ways for preventing this class of bugs.

[jankrueck]

uuuhm sorry for the timewaste.. I somwhow skipped page 8 completly.
It took me a while to understand the resolution"issue"
I probably switched to 25p whithout noticing this will affect something


maybe I have to get a filmclap, till you find time to add soundrecording



Cheers and thanks!!!!

[mk11174]

Posted by: dfort
@mk11174 -- Did you already get 14-bit lossless compression working?

I have only been working with resolution changes for fun just to see what happens and of course sharing what I find. 

[Markus]

Tried some of the new experimental modes on Fw 123 today. Looked really good but i could not get 1920x1080 45p continious on my Komputerbay 1066x I got around 10-20sek rec time. Has anyone achieved 45p continuous on FW 123? 

[DeafEyeJedi]

Of course - what happened was not deterministic.

I've just posted a new build that checks whether your camera was affected by this bug. All you have to do is to upgrade - if no message pops up, your camera was not affected by that bug at all. Otherwise, just follow the instructions.

Fortunately, no messages prompted up after upgrade w 2017-04-10 build though I am curious to see what the instructions would be for the affected ones.

Thanks for double checking @a1ex! 

Actually decided to try this (since it was reproducible last time) again w 1080p 45/48 3x3 under Crop mode while in 60p from Canon menu together with FPS override set to 45p and can now confirm that this bug has either probably gotten worse or I'm doing something incorrectly (which I doubt) but feel free to tell otherwise.



Hence the reason why I had to use an iPhone to snap this screenshot because no matter how quick I can try getting back into ML menu upon restart (after each battery pull) and getting to scroll around (both Canon or ML menus) within 5 seconds more or less was impractical. It just freezes up the LV.

[DeafEyeJedi]

Tried some of the new experimental modes on Fw 123 today. Looked really good but i could not get 1920x1080 45p continious on my Komputerbay 1066x I got around 10-20sek rec time. Has anyone achieved 45p continuous on FW 123?

How were you able to achieve 15-20 seconds without the LV display freezing up within 5 seconds more or less @Markus?

Meanwhile gonna try defaulting Canon settings to defaults and reload a fresh 2017-04-10 experimental build again just to double check.

[a1ex]

For now, FPS override only works in Low Light mode for reducing the frame rate (known bug).

For 1920x1080 at 45 fps, set 50 fps in Canon menu.

[goldenchild9to5]

@jankrueck I got the same thing Compression crash recording @ 3K running April 6th build.   

[goldenchild9to5]

@A1ex Gonna try April 10th build today   Thank you once again for all your hard work..

[Markus]

No lockups here. Only hard to get cont rec on alot of the sweetspot resolutions.
Some settings of mine:
Global draw on, canon in 720p all-I 50p mode, croprec 3x3 binning 1920x1080 47/48p @ 45p. Lossless 14bit.
Without global draw i could not get a bw preview in this mode.
Anything that sticks out?

The file size estimation got confused some times when switching between modes. It also got the wrong fps when switching. I went to photo mode and back as a workaround when this happend.

[DeafEyeJedi]

For now, FPS override only works in Low Light mode for reducing the frame rate (known bug).

For 1920x1080 at 45 fps, set 50 fps in Canon menu.

That did the trick. Haven't use PAL in years (I'm on NTSC land). Should have thought of that to begin with and Thanks for the tips.

So far 1920x1080 @ 45p MLV-lossless 14-bit seems to be continuous even w HDMI out to an monitor (clean native feed ) which is a PLUS for gimbal work because obviously the 5D3 will have the grey previews while mounted on the Ronin (which will go off from 'Turn off LCD' within 5 seconds during each recordings -- useful feature from 'Powersave in LiveView' under Prefs within ML menu) and can then focus on the external monitor instead with a peace of mind! 

[goldenchild9to5]

@a1ex Just tested April 10th Build so far so good.  One thing that I've noticed is that I can record at higher resolutions for longer now for example 3520x1498 - 2.35:1 I can get over 12 sec on a Lexar 128GB 1066x card before it would record for like 2 sec on the April 6th Build great work.  Also one thing that I've noticed is LiveView in 3K mode when recording @ 2.35:1 is all grayscale now, on the April 6th Build it showed both color and grayscale. 

[a1ex]

Any differences between Apr10 and Apr06, except the null pointer check performed at camera startup, are your own imagination (review the changes, please).

Or, do a controlled test. Same static scene, same settings (!), run 10 test clips on both builds (alternating the builds), and then draw the conclusion.

[goldenchild9to5]

@a1ex Cool no problem.. @ camera startup I don't see any test running am I missing something, or it just runs in the background and if anything is wrong it flashes a message. 

[DeafEyeJedi]

Upgrade camera again via Canon menu and wait for either a pop up screen notifying instructions or none will be prompt (meaning it isn't affected) @goldenchild9to5 if you haven't already.