crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView

[garry23]

@A1ex

Am I right in saying the key.press(KEY.HALFSHUTTER) wont (sic) work in the experimental 4K at the moment? It doesn't seem to.

I'm trying to explore triggering the experimental FRSP(4K version) in  Lua script.

Or am I jumping the gun 

[a1ex]

Am I right in saying the key.press(KEY.HALFSHUTTER) wont (sic) work in the experimental 4K at the moment? It doesn't seem to.

Right - will take care of this (as I want to merge lua_fix in regular nightlies anyway).

CTRL SRV Stack Overflow free=84 Used=163008176

Would be great if you can find a way to reproduce this.

[garry23]

@A1ex

Great 

[Kyle Kearns]

Does a 1:1 crop mean it's a 2x crop similar to a GH4?

[hyalinejim]

The references to 1:1 mean that every pixel on the sensor becomes a pixel in the image. The "crop factor" depends on the resolution you choose.

A 1920 x 1080 1:1 central section of the sensor (5760 pixels wide) gives a crop factor of 3.

2880 x 1620 is a 2x crop factor.

[goldenchild9to5]

@Kyle Kearns Naw.. 3K 1:1 I believe is 1.84 -  UHD is 1.5 -  True 4K is 1.4

[pc_bel]

Quote

    CTRL SRV Stack Overflow free=84 Used=163008176


Would be great if you can find a way to reproduce this.

After 2 hours I'm not able to reproduce CTRL SRV...

Suddently get this message in top of screen (in red letters):
[103] COMPRES_TASK: TASK WARNING: Free=164 Used=3932
Camera off.
When I boot camera again, I get this message full screen:
ML ASSERT: 0 at mlv_lite.c:2006 (compress_task), task compress_task vv:1 mode:3

srm_malloc_suite(1)...
srm buffer #1: 48600064
srm_malloc_suite(f1408)
Resolution changed: 2080x1587 -> 2080x961
Black level: 2047

Compressed size higher than uncompressed - corrupted frame?
Please reboot, then decrease vertical resolution in crop_rec menu.

ML ASSERT: 0 at mlv_lite.c:2006 (compress_task), task compress_task vv:1 mode:3

This ML ASSERT message appears when in crop_rec mode 1920 50p 3x3, I change (in Canon menu) from 1280x720 50 All-I to 1920x1080 25 All-I before change crop_rec to OFF. I need to reboot 2 times and then all come back to normal operation...

I will try again to reproduce the CTRL SRV message

[Doofii]

select 14bit lossless,and "frame and block size mismatch:0xcba800 0x20 0x88bc00" shows when play back.
ive tired many times,but the problem still exists.
what could i do?

[Quentin]

I shot something like 12 takes.
I tried 14lossless and got a warning at the top of the screen (couldnt read due to age)
and a series of many warnings.
I removed the battery, start from scratch, default ML settings, load modules.
I set 10bit
Set the Canon video to 1280x720, NTSC.
Everything as expected.

In order to reach optimum performance, I set the Canon video back to 1920x1080, PAL.
The resolutions 3K, UHD and 4K had less height that the promised in ML forums.
e.g 3072x1728
In these resolutions, I couldnt record more than 3 secs maximum.
Surprisingly I managed to record 3568x1320 for much longer duration. <--- pretty strange not to be able to record longer for lower resolution

[martinhering]

I was experimenting with the lossless lj92 compression in MLV files. I wanted to see if I can decode the image and reinterpret it as a raw cfa pattern. I came across a number of problems that could also be a problem in mlv_dump. I took the liblj92 implementation by Andreaw Baldwin found here:
https://bitbucket.org/hudson/magic-lantern/src/7a9b6805756c3b86f2174bac00433c544a976501/modules/mlv_rec/lj92.c?at=mlv_rec_lj92&fileviewer=file-view-default

But I could not make it work. I had decoding problems due to the fact that the library is missing some support for multiple components. I then came across the implementation in tiny_dng_loader by Syoyo Fujita here:
https://github.com/syoyo/tinydngloader/blob/master/tiny_dng_loader.h

I then extracted his changes back into a simple standalone .c and .h file and tried that and viola, the decoding result was much better:


I posted the extracted version of the library on GitHub:
https://github.com/martinhering/liblj92

Just saying, if you encounter any decoding problems in mlv_dump with the mlv_rec_lj92 branch, give my version a try.

[pc_bel]

I shot something like 12 takes.
I tried 14lossless and got a warning at the top of the screen (couldnt read due to age)
and a series of many warnings.
I removed the battery, start from scratch, defauly ML settings, load modules.
I set 10bit
Set the Canon video to 1280x720, NTSC.
Everything as expected.

Yes, same as me. Now message is:

[76] CEC: stack warning: free=128 used=3968

I'm totally lost...
@A1ex
I'don't know what to do, so if you can point me to what to do to help, just let me know.

After the messages I can reboot and I can record continuous without corrupted frames (allways talking about crop_rec April 4th build, 1920 50p 3x3 mode)... so this is not a big problem, but I have no idea of code so no idea what all this messages mean and what to do... sorry  I'm limited to explain what happens...

[a1ex]

Did some changes that might help with memory corruption (new build posted).

In particular, I think I've found a way to limit compressed data size when it ends up larger than the allocated buffer. Previously, passing an arbitrary limit for buffer size simply locked up the camera at certain resolutions - it appears this limit must be aligned at 4096-byte multiples (maybe a bit less, but in any case, at least 1024).

If you still get stack overflow, please report even if you don't have a way to reproduce it. This kind of bug is usually very serious.

[garry23]

@A1ex

Just checking: I assume this build doesn't include the Lua fix yet? I doesn't look if it does.

Cheers

Garry

[pc_bel]

Just tested April 6th 113.

Can't get continuous recording like before with april4th, only 10 sec. before message "didn't make it to stimated record time" and then recording stops. (1920 50p 3x3 mode).

[GutterPump]

This kind of bug is usually very serious.

Is this mean we could get a semi brick or definitive brick ? What is the long-term risk ?

[a1ex]

@pc_bel: nothing was changed on the recording speed side; I bet any difference is because of different scene complexity. Please test the stack overflow bug first.

@garry23: Lua changes are a bit too much when I'm troubleshooting a memory corruption bug. The changelog from the Experiments page is valid.

Is this mean we could get a semi brick or definitive brick ? What is the long-term risk ?

Yes, as with all ML builds. Explained a little here and also somewhere in the Tragic Lantern threads. Currently I'm diagnosing this report, and it appears to be, indeed, caused by some bad settings reflashed by Canon code into ROM - possibly after memory corruption caused by ML.

The warnings from the download page are not there for fun - this kind of errors cannot be prevented on the current processor design, as it doesn't have a MMU (also valid for many other simple CPUs that use non-volatile storage updated frequently). Canon started to use a processor with MMU with DIGIC 7, but they don't seem to use its memory protection features (they simply do a flat memory mapping, except a 0x1000-byte block unique to each core).chd

Just in case, make sure you have a copy of your ROM in a safe place (any recent installer tells you how to do so).

[pc_bel]

Just tested April 6th 113.

Can't get continuous recording like before with april4th, only 10 sec. before message "didn't make it to stimated record time" and then recording stops. (1920 50p 3x3 mode).

After reboot, I get the same message as with the april4th build:
ML ASSERT: 0 at mlv_lite.c:2006 (compress_task), task compress_task vv:1 mode:3

srm_malloc_suite(1)...
srm buffer #1: 48600064
srm_malloc_suite(f1408)
Resolution changed: 2080x1587 -> 2080x961
Black level: 2047

Compressed size higher than uncompressed - corrupted frame?
Please reboot, then decrease vertical resolution in crop_rec menu.

ML ASSERT: 0 at mlv_lite.c:2006 (compress_task), task compress_task vv:1 mode:3

[a1ex]

ML ASSERT: 0 at mlv_lite.c:2006 (compress_task), task compress_task vv:1 mode:3

Fortunately, that's no longer causing memory corruption (so it's a minor bug for now). It's simply invalid image data being compressed (and output size resulting higher than initial size). The extra data is no longer written to RAM on latest build (and before, it was written in the place reserved for future frames, not on DryOS data structures or persistent setting areas - to my knowledge).

The stack overflows are worrying me, though (as they do indicate memory corruption in the middle of DryOS).

[BBA]

Thanks for those groundbreaking efforts  !!!!!!

Don't know if it helps. If I can remember...

I got just while moving in the menu but without CF card (only SD):
white flashing "[93] audio-common-task stack warning free=180 used=3016"
then red flashing "[91] clock-task st(don't remember the end of the word) free=36 used=8156"

[jankrueck]

Code Select Expand

ASSERT: this->VramNv12Param.hJpegMemSuite
at ./Epp/Vram/VramStage.c:745, task Epp
lv:1 mode:3


Magic Lantern version : crop_rec_4k.2017Apr04.5D3113
Mercurial changeset   : e7e1fdb85f6b (crop_rec_4k) tip
Built on 2017-04-04 15:41:29 UTC by jenkins@nightly.
Free Memory  : 146K + 3094K


Code Select Expand

ASSERT: 0
at ./PathDrvWrap/EekoYuvToVramPath.c:606, task Epp
lv:0 mode:3


Magic Lantern version : crop_rec_4k.2017Apr04.5D3113
Mercurial changeset   : e7e1fdb85f6b (crop_rec_4k) tip
Built on 2017-04-04 15:41:29 UTC by jenkins@nightly.
Free Memory  : 158K + 3072K



this was on 4th. didn't see there is a new version.
testing up to date version atm.

looks good so far.



I have little knowledge on how you managed to achiev this update.
But I assume, there will be no way to use an external monitor on this, right?
HDMI connectors are limeted to 1080p huh?

[BBA]

Without ML (SD card with ML removed) and only CF card (data card):
In Canon menu, the sensor cleaning gets disabled when the camera is switched on.
(I am with the 4th april build)

[pc_bel]

For now, all I could get are these error messages:

ASSERT: hLvJob->hJpegMemSuite
at ./Epp/Vram/VramStage.c:868, task Epp
lv:1 mode:3
Magic Lantern version : crop_rec_4k.2017Apr06.5D3113
Mercurial changeset   : bc99f7502deb (crop_rec_4k) tip
Built on 2017-04-06 08:28:51 UTC by jenkins@nightly.
Free Memory  : 172K + 3779K

ASSERT: IsSuiteSignature( hSuite )
at ./PackMemory/PackMem.c:599, task RscMgr
lv:0 mode:3
Magic Lantern version : crop_rec_4k.2017Apr06.5D3113
Mercurial changeset   : bc99f7502deb (crop_rec_4k) tip
Built on 2017-04-06 08:28:51 UTC by jenkins@nightly.
Free Memory  : 172K + 3778K

ASSERT: fInitialize == TRUE
at ./System/PostPostman/PostPostman.c:243, task Epp
lv:0 mode:3
Magic Lantern version : crop_rec_4k.2017Apr06.5D3113
Mercurial changeset   : bc99f7502deb (crop_rec_4k) tip
Built on 2017-04-06 08:28:51 UTC by jenkins@nightly.
Free Memory  : 172K + 3781K

[a1ex]

Without ML (SD card with ML removed) and only CF card (data card):
In Canon menu, the sensor cleaning gets disabled when the camera is switched on.
(I am with the 4th april build)

Does clearing Canon settings help? Otherwise, do you have a screenshot, so I know what to look for?

[pc_bel]

Day of testing!!!

I returned to crop_rec 113 april4th build due to the short recorded times I get with april6th (I don't know why...) 

Really surprised for the extremely low rolling shutter in 1920 50p 3x3 mode... but panning, I get a cut line in the top part of the frame, only with panning! it dissapears in static frames.

This video explains better:
http://ovh.to/b2isZ5B

Edited: I can't see the cut line in live view, only in processed dng's.

[jankrueck]

Really surprised for the extremely low rolling shutter in 1920 50p 3x3 mode... but panning, I get a cut line in the top part of the frame, only with panning! it dissapears in static frames.
Edited: I can't see the cut line in live view, only in processed dng's.

same here. got this strange line, when panning.