crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView

[garry23]

@A1ex

I've continued 'playing around' with the latest experiments in still photography and am a bit confused. ML seems to be creating 'time dilation in the space-time continuum' 

I set up a long exposure test, 30s, and took a base .cr2 capture.

I then kept this exposure and used the full res LV in the experimental build.

As soon as I pressed the half shutter press, I got the saving message. I pull the card immediately.

I repeated and left the card in for a minute.

The two DNGs were the same, ie I seem to get a 30s silent in a couple of seconds.

The cr2 and the two DNGs were exposed OK, ie ETTR like and not overexposed.

I obviously don't understand how long exposures are working with the crop/silent stills.

When you have time, I would welcome any insight.

Cheers

Garry

[a1ex]

I returned to crop_rec 113 april4th build due to the short recorded times I get with april6th (I don't know why...) 

Proof that it has shorter recording times, please! A static test scene (without varying light levels and with the same focus), 10 test runs with each build, starting from a formatted card, at the same resolution and ISO, should do the trick.

I set up a long exposure test, 30s, and took a base .cr2 capture.

I then kept this exposure and used the full res LV in the experimental build.

As soon as I pressed the half shutter press, I got the saving message. I pull the card immediately.

The two DNGs were the same, ie I seem to get a 30s silent in a couple of seconds.

The cr2 and the two DNGs were exposed OK, ie ETTR like and not overexposed.

I obviously don't understand how long exposures are working with the crop/silent stills.

Best guess: you have tried in photo mode (which I didn't try yet), with exposure simulation. So, if you have metered a 30s ISO 100, the LiveView is probably at 1/FPS and a very high ISO. An example, if FPS override was set to 5, the LiveView frame might have been exposed at 1/5 seconds, aperture wide open - even if you have selected something different in menu - and ISO as high as required to get the same brightness as with a 30-second exposure (considering the other two variables).

I'm curious to see the DNGs.

[garry23]

@A1ex

No EXIF so can't tell other than looking at them.

Canon and ML ISO settings remain at 100.

Tried with expo sim off. Saving straight away, i.e. Doesn't do a 30 sec capture.

Cheers

Garry

[garry23]

@A1ex

Sitting here trying to think this through.

As this is coming from a video development, I guess we will be limited, at ISO100, to shutter speeds that are consistent with video framing rates.

I guess this means this still photography approach will solve fast shutter speeds, but not give ISO100 slow shutter speeds.

Or have I got it wrong?

[Frank7D]

pc_bel said:

but panning, I get a cut line in the top part of the frame, only with panning! it dissapears in static frames.

In other contexts, this was theorized to be a symptom of overloading the processor. I used to get it with greyscale ml preview on my 7D in 5x crop mode.

[BBA]

@A1ex
Thanks for your assistance ! Don't hesitate to ask more if necessary ! I am not stuck  .

a) more infos about my problem:

    a.1) without ML card (I use the SD card for ML)
    a.2) CF card in camera (formatted in camera)
    a.3) I go to Canon menu : yellow settings menu / SETUP3 / "sensor cleaning" and set "auto cleaning" to "enabled"
    a.4) I switch off the camera : the sensor cleaning screen appears as before; the red bottom led is not lit.
    a.5) I then switch on the camera : the sensor cleaning screen does Not appear; the bottom red led is lit.
    a.6) if I go again to the Canon menu : yellow settings menu / SETUP3 / "sensor cleaning", the "auto cleaning" has been set to "disabled"

b) reinit all settings : no change

    b.1) Canon menu / yellow settings menu / SETUP4 / "reinit all settings"
    b.2) the "auto cleaning" feature is actually set to "enabled" if I verify in the Canon menu
    b.3) then I get the same as above in a.4) , a.5)  and  a.6)

[EDITED 2017/04/07]
c) circumstances ? no CF card ?

Unknown

I think (not sure at all) it may be due to the fact that I tried to repeat the problem I got with no CF card in the camera (silly thing : should not be done without further investigation): the camera got stuck; no message ; I had to remove the battery.
Investigated by A1ex (thanks a lot) : no such problem when switched on with no CF card (see post #156). 

[a1ex]

@BBA: could not reproduce.

Side note: I've let the camera on for 10 hours, on a power adapter, with today's build, on the 3K video mode, 3072x1920 in mlv_lite menu, lossless compression, "framing" preview, in standby - so it would estimate the compression ratio every 2 seconds. Every 30 minutes, it closed LiveView for a second, and returned to LV (I could hear it when I came home).

Temperature after this experiment was 62 degrees C. The camera was enclosed in a drawer during the entire experiment.

I've now downloaded the previous build (Apr04) and will perform the same test overnight (or until it crashes).

edit: it crashed after 15 minutes or so of recording short test clips in 50p 3x3; back to Apr06, no crashes yet.

I'm doing these tests to check these reports:

I tried 14lossless and got a warning at the top of the screen (couldnt read due to age)
and a series of many warnings.

Trying to reproduce the live view disconnection, I let camera on... This time get this message:
CTRL SRV Stack Overflow free=84 Used=163008176
[...]
Suddently get this message in top of screen (in red letters):
[103] COMPRES_TASK: TASK WARNING: Free=164 Used=3932

[garry23]

@A1ex

Ok I get it now 

As you say on the first post

The last feature complements the well-known full-resolution silent pictures - the new implementation will be usable at fast shutter speeds, without the exposure gradient - but with rolling shutter (just like regular LiveView frames).

Photographers need to look to using the crop approach and the current FRSP, together.

I'm looking forward to trying to script these two techniques together in Lua 

Really exciting and fun times.

Cheers

Garry

[a1ex]

Please follow the same steps as ju64 and contact me on IRC.

[Walter Schulz]

Upper right corner:
IRC: #magiclantern

[a1ex]

Another case of bad setting flashed somehow into ROM. Reproduced the error in QEMU. Appears easy to fix, since these settings fortunately have backups (wear leveling?). I've already unbricked a similar case a while ago (also 5D3), and I've already booted the GUI in QEMU on the affected ROM.

Two failures on the same build are a bit too much to be a coincidence. For now, my advice would be to avoid the Apr04 build, even though some reports say it's faster (I'm willing to bet it's not, but I'll benchmark it, just in case).

Apr06 should be safe regarding this issue (see my tests).

[Kharak]

Thanks for the heads up!

 I'll update to Apr06.

[goldenchild9to5]

@a1ex Thank you for all your hard work.. I tried the April 4th build but did not encounter such problems on 1.1.3.  Will give April 6th a test drive tonight I'll post my findings. 

[Hans_Punk]

Amazing work @a1ex as always!

I'm using MLV_Dump for the newest crop_rec module with higher resolutions (4K, 1080p48 etc) build - And I'm getting vertical stripes on the .dng exports.
Is this normal for now?...or am I missing something simple? I have not used MLV_Dump before now (and I'm a code ignorant idiot)
Test shots were lower ISO 100-400, various resolutions tested from April 4th build - all exhibited some vertical stripes (some more noticeable at lower ISO)

Running this code on Win 7 works for me to export the .dngs's, I just don't know if vertical stripe correction is an option yet in this version of MLV_Dump...or I simply need to somehow amend the .cmd code to 'switch on' the correction in MLV_Dump?

@echo off
md "%~n1"
"Q:\NEW MLV\mlv_dump.exe" -o %~n1\%~n1_ --dng %~n1.MLV

Link to screengrab of most prominent vertical stripes:
https://www.dropbox.com/s/md2inmxhaj604gv/M06-1750%20%2800123%29.jpg?dl=0

.Dng Sample files:
https://www.dropbox.com/sh/fe5nve001l2vqx5/AADpbW7eVOqafc7XP4XnhgD0a?dl=0

[Danne]

Upload a sample file?
Shortened file:

Code: [Select]

mlv_dump -f 5 -o Output.MLV Input.MLVInput.MLV being your file with stripes. Do the command correct or you might loose the file.

[garry23]

@A1ex

I thought I would carry out a few experiments in photo mode looking at 'lowish' shutter speeds with the FRLV.

I decided to use an FPS twice the selected shutter speed.

It appears I can push the FPS down to 0.5 (ML says its 0.554) before it plateaus out.

I took the following combinations and all the DNGs looked OK in LR:

FPS = 2, ML:shutter = 1/10s
1, 1/4
1, 1/2
0.5(0.554), 1s

From my perspective this looks a healthy overlap with the 'normal' FRSP, which could use down to, say, 1/4s.

Cheers

Garry

[a1ex]

From my perspective this looks a healthy overlap with the 'normal' FRSP, which could use down to, say, 1/4s.

Sounds reasonable - as the full-res implementation can be updated decide whether to use LiveView or the "long exposure" version, depending on the shutter speed selected in photo mode.

[garry23]

@A1ex

Yes: I've worked out how to do it in my Lua bracketing scripts and I will confirm things as soon as I can. ie when I can trigger with the half shutter key press.

This should give photographers a 30s to 'ultra fast' seamless capture capability.

 

[pc_bel]

Two failures on the same build are a bit too much to be a coincidence. For now, my advice would be to avoid the Apr04 build, even though some reports say it's faster (I'm willing to bet it's not, but I'll benchmark it, just in case).

Ok, back to april6th...

192050p3x3, 5D3 113, ISO 100: 10 recordings 1min/each (camera didn't stop recording, I stoped recording assuming continuous).
192050p3x3, 5D3 113, ISO 400: 10 recordings 1min/each (camera didn't stop recording, I stoped recording assuming continuous).
192050p3x3, 5D3 113, ISO 800: 1 recording, camera stops recording after 10sec. (message:"didn't make it to stimated record time")
I turn off camera.
When I turn on camera, a full screen message appears like this one:

When I boot camera again, I get this message full screen:
ML ASSERT: 0 at mlv_lite.c:2006 (compress_task), task compress_task vv:1 mode:3

srm_malloc_suite(1)...
srm buffer #1: 48600064
srm_malloc_suite(f1408)
Resolution changed: 2080x1587 -> 2080x961
Black level: 2047

Compressed size higher than uncompressed - corrupted frame?
Please reboot, then decrease vertical resolution in crop_rec menu.

ML ASSERT: 0 at mlv_lite.c:2006 (compress_task), task compress_task vv:1 mode:3

So ISO was the culprit for not get continuous recording with april6th?... My fault.

No STACK OVERFLOW error for now...

Edit: in this tests I got the same corrupted image that before when panning (top part of frame with motion cutted).
Can I ask if is easy to adress this, or not? That would make the recordings usable...!!!! (at least fro me).

[squig]

Apr06 3104x1298 1600 ISO 93.4MB/s ~57%
5.26Gb
5.78Gb
Restarted
8.15Gb
25.06Gb (I got bored and stopped recording)

I was going do a comparison with Apr04 but the stack error appeared (for the 1st time) when I loaded the modules so I chickened out.

[a1ex]

Edit: in this tests I got the same corrupted image that before when panning (top part of frame with motion cutted).
Can I ask if is easy to adress this, or not? That would make the recordings usable...!!!! (at least fro me).

Here's a boring task: what's the number of pixels before the cut? (min/max/median would be great)

[pc_bel]

Here's a boring task: what's the number of pixels before the cut? (min/max/median would be great)

  Sorry for that!!!... Its only thing that makes image unusuable.

I've  measured px and the pattern is very strange... min distance I've found is 7px, max is 125px, but it doesn't follow a logic. At first it seems increassing with the panning velocity, but looking close, during panning, one frame have 98px (from top to cut) the next 112px, the next 18px, one frame (in between all of this) is correct, and for example, the next have 73px...

Min I found: 7px
Max I found: 125px
Average from random 17dng:83,8px
@A1ex
Is this what you are asking for?

[a1ex]

Yes. I thought I'd get away with it by increasing the optical bar size, but it looks like I should re-enable double-buffering (at least for certain video modes).

[pc_bel]

If you need to do some "mechanical-tedious" work, if you explain me how to do it, I can do it...

[BBA]

@A1ex

Could not reproduce

A good thing for other users : if you agree, I will edit my #155 post and remove the unnecessary warning about not switching on a camera without CF card

Sensor cleaning issue:

Maybe something interesting in the Debug menu: red dot in front of "memory patches"

https://www.dropbox.com/s/ap2zant3kn9vy2e/VRAM3.PPM?dl=0


memory patches, with a red dot:

https://www.dropbox.com/s/cafc5h767gbqgdv/VRAM2.PPM?dl=0


Could it be of any help for fixing the cleaning sensor issue of my camera ?