Danne's crop_rec_4k experiments for EOS M

[Disease]

How are you guys getting these high frame rates? Can you only do this in crop mode and in raw? With the cinematic aspect ratio? 2:35?
Is shooting in raw the only way to get rid of the horrible rgb noise and aliasing I get when in normal/codec/not cropped mode?

[masc]

How are you guys getting these high frame rates?

Select preset (1080p, 46/48fps), start recording.

Can you only do this in crop mode and in raw?

Use RAW.

With the cinematic aspect ratio? 2:35?

If you like you can setup this too.

Is shooting in raw the only way to get rid of the horrible rgb noise and aliasing I get when in normal/codec/not cropped mode?

Reduce noise by proper exposure and enough light. Reduce aliasing e.g. with 5K modes or cropped modes (1x1 recording) and fast apertures.

[ricardopt]

Hi everyone, i know i'm going to get "crucified" by asking this but here it goes... i got on the magic lantern bandwagon recently but i have yet to understand something that might be simple to understand to older people here in the community, in video mode i find the ratio option and when selecting the submenu raw video there's also aspect ratio option.

Why similar options in different menus? if i select 16:9 in movie tab what should i choose in the raw video submenu? i normally select 16:9 on both.

[Danne]

Because I want the user to be able to use both. Not at the same time obviously.
Preferred way is to stick with those outside raw video menu cause they are customized to my presets.
It's impossible for me to have been working almost two years refining my eos m build, at the same time create explanatory pages.
My point of view. Any user must be able to do the work testing through on their own and simply make up their own mind and path on how to use this build.
Or simply ask here.

[cmh]

I'm not entirely sure but to my understanding Movie Tab ratio is about liveview crop (it restrains the maximum resolution) but gives you the proper liveview and the raw video aspect ratio concerns the resolution, if not matched it might need a desqueeze in post. It is the way it is for technical and performance reason.
edit: matching both is perfectly okay.

[Danne]

Under raw video menu there's a good automated code for calculating ratios for all presets. Not always working optimal with my crop rec presets.
The way I started doing it was to set raw video ratios at max setting then limit raw buffer directly or with skip offsets or combining the two. This way I have control what part of the sensor to use and not. This is by no means perfect but It will let me refine and have better control on my experiments.
I was having some discussions with jiphop about taking out raw video resolutions but then again people tend to experiment on their own with resolutions so it stays.

[cmh]

Yeah I'd assume that it's too complicated to explain in details to non ML devs.
It's hard to find the right balance between usability, user experience and simultaneously satisfy power users.
That's one of the reason why a lot of open source softwares seems clunky for newcomers (but there's other factors, like budget and time constraints, the lack of bug/feature reports, stubborn dev team etc).
edit: I think you did a good job so far. I have my own little lua video presets that are just made off of the 5k anamorphic script (if I need 10 bits with a slight increase in resolution or 12 bits depending of the lighting conditions for exemple).

[canoneer]

I got some assert file messages yesterday and soon after that "card full" reported from canon sw (I know since message is in the M language selected). The assert file reported about mlv lite packaging (deleted it as I tried to release space (silly me)). Now I wonder if I should go back to an empty card, then reload Ml and start up again. But if the messages are due to original parameters - I guess it will still be there, because the card is absolute not full -  I must probably re-format and start from absolute scratch. Any advice is very welcome before I am back at the desk today.

nightly 13 jan eos m 202.

[canoneer]

Re-card full-

None of the visible files on the sd card hold vast volume - disk say full with 31.9gb data, ML folder hold 36.7mb, EOSM 66kb, ML setup 98kb, auto exec 459kb. Thats all. Mac´s disk utility confirm full. So I guess I go back to start.

[yokashin]

Maybe can help making the format SD card from the Canon menu with the option not to delete ml files?

[canoneer]

Thanks. Will do. I have another SD card set up and use that at the moment. Experienced a pink artefact in the raw files I produced for test today. If I am able to include an image I will. I seem not able to edit it away in Resolve after converting to DNG or in MLVapp prior to.

[canoneer]

Thanks. Will do. I have another SD card set up and use that at the moment. Experienced a pink artefact in the raw files I produced for test today. If I am able to include an image I will. I seem not able to edit it away in Resolve after converting to DNG or in MLVapp prior to.

https://www.dropbox.com/s/0d5mgub0se5ow57/Untitled_1.1.1.jpg?dl=0

[IDA_ML]

Earthquake shaking in the high-resolution modes of the EOS-M and 100D

Yesterday, I decided to film a few timelapses with the EOS-M at low frame rates (0,5 to 2 fps) using the Full Sensor Readout mode at 1736x2928 and the 4k (4080x2296) crop mode (1:1) at 12 bits lossless.  When I came back home, processed the footage with MLVApp and exported it at 24 fps, I was shocked.  It was entirely ruined by horizontal earthquake shaking, as we know it from the 7D before Dfort successfully managed to fix it in his latest experimental builds.  Initially, I thought, it was the wind causing the shake and my tripod was not stable enough but then I shot another test in my apartment and the shaking is still there.

What happens is that the image shifts horizontally by a few dozens of pixels back and forth from frame to frame.  This shift is also visible on the preview screen image while recording if there is a vertical contrasty edge in the scene.  This edge also moves back and forth every time the screen refreshes.  Fortunately, I was able to fix my footage by applying stabilization to it in Resolve.  If filming at higher fps (14 to 24 fps) the shaking does not seem to be so obvious and is hard to notice but stabilization makes it smoother meaning that it is probably there even at higher frame rates. 

I used the latest 29-th of April experimental build for the 2.0.2 FW version on the EOS-M, installed on the recommended 64 GB SanDisc Pro SD card but I found some unprocessed timelapse footage shot on the previous January 10-th, 2020 build and shaking is there too.

Here are my settings:

Shutter speed in the Canon Menu: 1/30 sec.

Video tab settings in the ML menu:
======================
presets: 5k anamorphic rewired
RAW video: ON 1736x2928 (16:9 ratio set in the submenu, Preview: Framing)
bit depth: 12 bit
shutter range: Full range
FPS override: 1 fps (actual shutter speed was 1 sec. at this setting)
everything else set to OFF

I used the same settings also with the 4k crop (1:1) mode, except for Presets.
===================================================

Here are a few samples before and after MLVApp conversion that illustrate the shaking phenomenon.  A stabilized sample is also included:

https://we.tl/t-F3KOCy4iGx

Has anybody else experienced this?  I have noticed that this phenomenon exists on the 100D too and is well reproducible too.  If necessary, I can post links to samples from that camera too.

Hopefully, it's an easy fix.


     

[canoneer]

ref to earlier post:

........ The assert file reported about mlv lite packaging (deleted it as I tried to release space (silly me)). ..........

(nightly 13 jan eos m 202.)

Clip - only audio played by mlvapp.

The assert files hold versions of this text when generated to the card for my issue:

ML ASSERT:
0
at mlv_lite.c:2751 (compress_task), task compress_task
lv:1 mode:3

compress_task stack: 1de948 [1de9d8-1dd9d8]
0x0009EC24 @ be6338:1de978
0x0009E564 @ 9ec80:1de948

Magic Lantern version : Nightly.2020Jan13.EOSM202
Mercurial changeset   : bdda3ceefa0c+ (crop_rec_4k_mlv_snd_raw_only) tip
Built on 2020-01-13 17:08:33 UTC by Windows@ALPHA.
Free Memory  : 232K + 2610K

I am clueless to if this is due to my handling - the lighting conditions, lens (18-55), card (64gb SanDisk extreme pro), or settings(raw, 1736x976, 5 sec clip, 25fps, focal 42mm, 1/50s, 180 deg, f9.9 iso100, 14bit lossless, audio 2ch,48kkhz)

Next clips normal take and play so I am not having a stable problem. But on my other card -same build - 32gb same type - I have had reported available disk space 36gb on 32gb card - and full card when 7gb available - after same assert reports.

[Danne]

@IDA_ML
I cannot reproduce. Works perfectly fine here.
It looks like your tripod wasn´t set alright. Is it happening still or only that time? Please test 5 or 10 more recordings and see if you can get steady issue reproducing.
If you want further examination please examine two dngs side by side that differs, extract metadata both with exiftool and from mlv metadata and see if anything changed. I have not time to do deeper analysis on my own.

I also used step 1 and two from here with fps override set to 1s shutter range ON and interval set to 3 seconds(should work with also interval of 1s). Also works perfectly:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg223302;topicseen#msg223302

[IDA_ML]

I checked the issue with the two latest experimental builds, starting from the default preset and shot dozens of timelapses with sometimes turning camera off and on inbetween to save battery power.  Shakiness was observed in all clips.  It is most obvious at 1 fps.  I also tested the 100D with your latest build - same situation.  I did not use the intervalometer on the EOS-M because the settings that I listed gave me perfect reciprocity (fps/shutter speed = 1) and I was happy with that.   I just press the record button to start and stop the video.  This worked perfectly on the 100D last summer, (no shaking), with the older builds back then.  I am surprised that you could not reproduce it with my settings.  Could you please just try without intervalometer, with settings from my previous post?  When you have time, of course!  It's definitely not the tripod!  Please take a look at the samples that I sent and compare with the stabilized one. 
I am too busy right now but will continue testing, as you suggest, when I find some time.  Will try the intervalometer option too. 

[Danne]

Already tested without intervalometer. Works perfect. I suggest you try and narrow down exact settings creating this effect. Could take a while so be patient. Good luck.

[IDA_ML]

Danne,

Just did another brief test with the camera placed on a very massive and heavy table inside.  It reveals that the shakiness is there also at 14,288 fps in the 5k anamorphic mode.  At this fps it is harder to see but it's there. This sample was shot at 14 fps, 10-bit lossless, then exported with MLVApp at 24 fps and finally, scaled down to 1920x1080 in Resolve to make the file smaller.  A stabilized version was created too.  The comparison of both samples you can download from here:

https://we.tl/t-xyV5n7dd57

Please watch very carefully!   The difference is hard to see if you don't know what to look for - the horizontal shifts!

Got to run now.

[Danne]

As I told you. Cannot reproduce. If you want more careful inspection I need screenrecordings, screenshots etc that fully in detail shows me what is going on. Sending me more files won´t solve the issue.

Edit: You also need to test other settings to see if you can narrow down the issue. Hopefully you follow exact steps to run basic set up leaving certain parameter alone but I suspect you do all kinds of personal changes not told here(or that I miss to see) so I really will need to see screen recordings of exactly what you do. Not only what you tell/describe.

[IDA_ML]

I will, don't worry!  Just let me get back from work and I will continue testing.  Timelapse videography is one of my favorite photographic genres and ML has always done a great job for me in that area.  Never had that problem before.  I will do my very best to figure it out and will report my findings here.  Thanks.

[Danne]

Yeah, strange. Tested both 4k and 5k anamorphic flv. What we need is one setting that will be 100% reproducible. Maybe it is af or fps override related, I have no idea really.

[Danne]

New version:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg208959#msg208959

- Fix for issue enabling correct map file in mlv app when in x3crop mode no ratio set. Thanks to @blackbar:
https://www.magiclantern.fm/forum/index.php?topic=16054.msg226786#msg226786
- Yet another upload. Refining centre for higher frame rates
- Testing consistent 48fps also for x3crop mode

[IDA_ML]

Danne,

After two more days of testing, I finally found out the reason for the shaking.  It was a totally unexpected hardware effect, that I never experienced before - it was the image stabilizer of the lens that I used - the EF 35/F2 IS.  I thought, it should automatically recognize when the camera is on a tripod and turn the IS off.  However, this was never the case.  It would stay on all the time while on the tripod providing a perfectly stable footage.  When I used this lens for filming timelapses in the past, I always left the IS on, even when using a tripod since it would nicely compensate for small tripod vibrations caused by the wind.  Not this time though - when on the tripod, the IS now causes shakiness itself.  I have no idea why.  It still works perfectly stabilizing hand held footage at 24 fps but when filming on a tripod at low fps the shakiness is there.  Turning IS off results in an idealy stable tripod footage.

It was a good exercise, for me at least, since I checked all timelapse functions thoroughly and they all work fine, with IS off, of course.  The settings that I described in my post #4077 can easily be reproduced and I can lower the fps down to 0,544 fps (shutter speed 1,8 sec.) - perfect for slowly moving clouds.  Anamorphic 5k Full Sensor Readout footage looks gorgeous on a 30-inch screen.  I use this mode a lot if I need high-resolution aliasing free footage at 14,288 fps.  If I need to film a timelapse in between, all I need to do is lower the frame rate to 0,5-2 fps.  Have not tested your today's build yet but the January 10-th one works great for timelapse work.

I apologize for the false alarm and inconvenience that I may have caused to you and thank you for your patience with me.

[Danne]

It was a totally unexpected hardware effect, that I never experienced before - it was the image stabilizer of the lens that I used - the EF 35/F2 IS.  I thought, it should automatically recognize when the camera is on a tripod and turn the IS off.

Well, good you cleared that one out.

[yokashin]

With the camera on a tripod, I turn off the IS. I've been doing this since the night I photographed the stars. They were so strange that I even advertised the lens Within 15 seconds of exposure, IS moved stars by 1-2 pixels ...