LiveView hacks (write speed improvement)

[koopg]

For dual ISO on 5d3, did you see flickering on the bright area?

no, had 2 hours of great dual iso@3.5k 14 bit lossless 2:20
beside first frame pink
had nothing but continues stunning raw video.


Sent from my SM-N975F using Tapatalk

[mlrocks]

no, had 2 hours of great dual iso@3.5k 14 bit lossless 2:20
beside first frame pink
had nothing but continues stunning raw video.


Sent from my SM-N975F using Tapatalk

Great news!
Now 5D3 in 3.5K super 35mm mode is no less than Arri Alex 35mm series.
Any one here knows if dual ISO on 5D3 in 5.7K 1x3 mode has flickering in bright area?

Why Most Movies Are Shot On Arri Cameras

Reading the follow up comments is very interesting. Many experienced DP there.

"I have an Alex Mini LF and a red V raptor, Helium, C70, Blackmagic,Panny EVA1. The Alexa  might have been way above over the competition 5-8 years ago, today? Not  so much. The Alexa Look , besides the out of the box colors is this. Alexa digs deeper into the shadows. Whatever you shoot with Alexa tends to look HDR'ish. because the log curve digs into the shadows. As soon as you shoot something with any other camera and lift the shadows in post.  Boom! Looks almost identical to the Alexa. Red V raptor is Cleaner also. Faster, cheaper, more compact, lighter. The only difference is the out of the box colors.  Once you color correct every camera side by side? Alexa is nothing special."

"Alexa mini LF owner here, fortunately have been able to use just every digital cine camera from bmpcc6k (which I still own and use), and at least 8 different types of red cameras(komodo is valid) up to the Mini LF. Arri is truely the best camera company in the world.  If you don't believe me, shoot a project on Arri with cooke 1.8 anamorphic or any high end glass and watch your mind be blown. Here's a music video I shot with that setup

. It  can make our jobs seem too easy at times but allows room for extreme creativity and style."

"Sony Venice has wider gamut, dynamic range and noise that is far more like film (variations in size over luminance). But the Arri is far harder to mess up the settings, and the Sony menu is a mess. I.e. the transition to digital from film was easier with the Arri. It’s easier to use, and harder to fruk up. I’ve posted over 100 features, and the Arri gets consistent results (I examine the metadata so I know what’s been done in the settings by the dop). imo the Sony creates better image both in it’s character and technical range, but is harder to learn how to use."

"The Sony Venice and especially the Venice 2 is absolutely beating out Arri even the Arri LF systems. Arri will always be considered an amazing camera system but they aren't adapting nearly as quickly as their competitors. We can't keep the Sony Venice on the shelf and have had DP's opt for the Venice 2 on major feature productions over Arri. Movies are no longer mostly shot on Arri cameras. Was this relevant 2 years ago, absolutely, but now the competition is getting too tight. Not to mention Panavision is still very popular and always was neck and neck with Arri."

Take home message: High end cameras are getting better and closer to each other now. Alexa still has an edge but not significant. 5D3 ML is getting there too. Once the realtime preview and the hdmi output is ported from 700D/650D by Bil the Great.

2h means 1TG footage? How long was the shooting session for this 2h footage? Did you experience the overheating due to sd card overclocking? What was the camera temperature? Did you buy all of the 2tg cf cards and sf cards? Will this take one to two weeks non-stopping pp computing time? Thanks.

[Wushuliu]

Only 650d/700d can do correct high resolution hdmi out. EOS M may do that also. Thanks to Bil's work. Hope he can figure out the 5d3's realtime preview and hdmi out.
650D/700D UHD 1X3 preset has AR 16-9, 4K 1x3 preset has AR 2. I tested that both HDMI out works fine with my Zacuto 3 inch EVF.
5D3's 5.7k 1x3 preset has incorrect framing unstretched hdmi out. I can use Zacuto EVF's anamorphic ratio of 2 to do certainly level of unstretching that is good enough for rough framing, but the hdmi out framing is not correct.

Hm. Ok. So 650D or possibly 5D3.
Hopefully some test footage will be posted soon.
Fascinating developments. Exciting.

[mlrocks]

Hm. Ok. So 650D or possibly 5D3.
Hopefully some test footage will be posted soon.
Fascinating developments. Exciting.

Better buy 700D over 650D, because Bil only has 700D. This means that 700D always has the latest dev. LOL.

[mlrocks]

Right now Sandisk Extreme Pro 170 MB/s 256GB CF card is about 250 dollars, SD card is about 60 dollars, micro SD card is about 30 dollars.
Anyone here used microsd card to sd, microsd to sd to cf on 5D3? SD overclocking still performs normally?

[Walter Schulz]

ML project does not support SD-to-CF adapters. If you have troubles (and there will be!) you are on your own.

If you go for cheaper CF: Komputerbay 1066x. 128 GB variety is somehow reasonable priced. 256 not. At least around here.

[mlrocks]

ML project does not support SD-to-CF adapters. If you have troubles (and there will be!) you are on your own.

If you go for cheaper CF: Komputerbay 1066x. 128 GB variety is somehow reasonable priced. 256 not. At least around here.

Thanks for the quick reply, Walter. Seems not able to save money here.

[theBilalFakhouri]

I have added stubs for other models in first post, in case if we need them later (70D/100D/60D/600D/1300D/1200D).

DIGIC 4/4+ won't benefited currently due to limited write speed. 50D/5D2/550D lacks these function.

[mlrocks]

Interestingly, compared with 10 bit footage, 14 bit 2k 1x1 AR 3-2 24 fps, when shooting with auto/manual ettr, and elevating exposure one stop in MLV App, the output is very nice, similar to alexa/amira's high key footage, the noise in the dark area is not apparent. So, 14 bit is actually better than 10 bit, when using ettr, may gain one to 2 stops dr. For 10 bit shooting, better avoid ettr, because much of the information pushed to the dark side is lost and not saved in raw files.
In this term, 14 bit 4.5k 1x3 with ettr and dual iso may have a dr of 16 stops, 12 stops from the 650d raw file, 1-2 stops from ettr, 2-3 stops from dual iso. This dr is on par with the legendary alexa and amira's dr. 4.5k 1x3 raw also should be on par with the alexa/amira's 2.8k prores. The sensor size in 4.5k 1x3 mode is about 230 mm2, almost the same as 4-3's 224 mm2, although a little bit smaller than alexa/amira's super 35mm, this is good enough for news shooting, which requires large in focus zones. Imagine a lowly 650D/700D is Alexa/Amira or Red One in disguise, this is really something to talk about.

https://www.provideocoalition.com/alexa-dynamic-range-its-all-in-how-you-use-it/
https://www.newsshooter.com/2020/05/27/the-arri-alexa-is-10-years-old/

Just checked some old test footage. ON 650D, UHD 1x3 10 bit lossless has significantly lower noise level than 14 bit lossless 1920x1080 1x1 24 p in the dark area. So it is a complicated issue. Seems larger sensor size is important to lower down the noise, in addition to the data bit rate. The former one may have much more effects than the latter one.

[ShittyWebsite]

yeah, i enabled both cards and spanning and i also now have continuous recording at 3616*1536 23.976 @14 bits (unless overexposed)

amazing...

bilal, you are putting the magic back into magic lantern, brother!

Hi, i might be late, how can i get this custom resolutions? i can only use the presets (3.3, 3.5k, uhd)


Oh i guess its UHD awith a small decrease in the horizontal resolution?

[mlrocks]

Hi, i might be late, how can i get this custom resolutions? i can only use the presets (3.3, 3.5k, uhd)

Crop mode new 2 module. Test thoroughly before real application. These are not optimized as well as Crop mode presets.

[ShittyWebsite]

Crop mode new 2 module. Test thoroughly before real application. These are not optimized as well as Crop mode presets.

Oh thank you, i was trying that on a 5D3 lol

[theBilalFakhouri]

Oh i guess its UHD awith a small decrease in the horizontal resolution?

Correct.
on 5D3 set crop mode preset to UHD, from RAW video submenu set resolution to 4096 and Aspect Ratio to 16:9 so you can get 3840x1536 then highlight "Resolution" and use "Main Dial" or "Multi-Controller" to decrease resolution to 3616x1536.

Crop mode V2 is not related here, and that's only for 650D/700D.

[ShittyWebsite]

Hi, sometimes when using "one more hack" the console shows

Is there any way i can remove the console? (not the "Show console: off" but deleting the console or maybe even just ignoring it so i can see what i'm recording without the console in front of it)

[Bender@arsch]

Hi, sometimes when using "one more hack" the console shows

Is there any way i can remove the console? (not the "Show console: off" but deleting the console or maybe even just ignoring it so i can see what i'm recording without the console in front of it)

If the console shows, it means there is a Failure somewhere. I have this sometimes too, deleting the last clip (because of corruption) and restart Camera is the best way.
Maybe you can read the failure message next time, maybe it is helpful for bug fixing.

[theBilalFakhouri]

Yeah, I simply can't turn off the console all time (or even only when RAW video recording). It's used mostly in every ML feature, disabling it is not a good idea at all, you may have corrupted frames while recording without realizing that which result in totally useless MLV clip.

I didn't have console showing when I used "One more hack". Could you share your settings? a steps to reproduce?
-Probably adding a little more delay after calling "One more hack" could solve this problem. I will do it soon.

-In general: avoiding using "One more hack" could be a better idea for now, since I really don't know what is it doing (this hack actually disables some EDMAC channels activity), figuring out how to silence EDMAC channels streams without using CartridgeCancel a.k.a "One more hack" in clean way is a solution for the side effects we are having when using this hack.

-Edit: also mind share printed message in the console? are you sure it's from "One more hack"?
It could be from something else.

[ShittyWebsite]

Hi, thank you, i couldnt reproduce consistently, i'm not sure its from "One more hack"


I managed to get the logs from yesterday, i'm not sure if this is helpful:


Preset: 3.5K 1:1
14bit lossless
Card Spanning on
Use SRM Memory: On
Small Hacks: Af Off
More Hacks: lvface+aewb
One more hack: On
SD Overclock: 160mhz
Dual Iso: Off

Code: [Select]

ML ASSERT:
0
at mlv_lite.c:2768 (compress_task), task compress_task
lv:1 mode:3

compress_task stack: 1ae530 [1ae5c0-1ad5c0]
0x0006A01C @ b451e0:1ae560
0x00069878 @ 6a084:1ae530

Magic Lantern version : Nightly.2022Apr07.5D3113
Mercurial changeset   : 712fa0bdd45c+ (crop_rec_4k_mlv_snd_isogain_1x3_presets) tip
Built on 2022-04-07 02:25:55 UTC by Bilal@DESKTOP-7RFHEE1.
Free Memory  : 213K + 2362K


When 1080p 40fps:

Presets: mv1080p 40fps
14bit lossless
Card Spanning: Off
Preferred Card: SD
Kill Global Draw: On
Use SRM Memory: On
Small Hacks: Af Off
More Hacks: lvface+aewb
One more hack: On
Recording Delay: 2s / recording time 5s
SD Overclock: 160mhz
Dual Iso: Off

Code: [Select]

ML ASSERT:
0
at mlv_lite.c:2768 (compress_task), task compress_task
lv:1 mode:3

compress_task stack: 1ae530 [1ae5c0-1ad5c0]
0x0006A01C @ b45440:1ae560
0x00069878 @ 6a084:1ae530

Magic Lantern version : Nightly.2022Apr07.5D3113
Mercurial changeset   : 712fa0bdd45c+ (crop_rec_4k_mlv_snd_isogain_1x3_presets) tip
Built on 2022-04-07 02:25:55 UTC by Bilal@DESKTOP-7RFHEE1.
Free Memory  : 213K + 3001K


[juno60]

Hi there,

I am awfully sorry to sound like a total bozo... but I cannot get ML to load the modules. I get only 2, namely adtg and bench.
Tried multiple times, no luck whatsoever. I am not new to ML and all crop rec releases have been working fine, including the experimental ones.
This time around I can't figure out what's wrong. 

Running firmware 1.2.3

Thanks a lot, guys!

[theBilalFakhouri]

Updated links in the first post for 5D3 and EOSM to Danne's downloads page.

It seems Danne added a delay after calling CartridgeCancel, please test latest Danne build for 5D3 and see if there is a first frame corruption when using "One more hack".

[Danne]

Tested briefly after adding but seemed to be working. Not sure what delay is best but added this to cartridge cancel:

Code: [Select]

    if (one_more_hack)
    {
        void (*CartridgeCancel)() =
        cam_eos_m ? 0xffa7e7d8 :
        cam_5d3_113 ? 0xff17fd68 :
        cam_5d3_123 ? 0xff181340 :
        0;
        CartridgeCancel();
        msleep(40);
    }

[mlrocks]

Scene: composed of several tree with dense green leaves, a parking lot with several cars, a gate with iron fences
Time: during the sunny noon at ISO 100
Lens: EFS lens 10-18 IS at 10 mm, F8, IS on, AF on
Camera: 5D3, ML Dann's version 20220517 1.13
Small hack: AF on
Three hacks on
Card spanning on
SD overclocking 160hz
Global draw always on
Auto ETTR on
Dual ISO off

1. 3K 1X1 preset, 1920x1920 AR 1:1 14 bit lossless, 24 fps, data stream 90 MB/s, the first several seconds orange, then always green, 1 minute recording, continuous.
2. 3.3K 1X1 preset, AR 16:9 10 bit lossless, 24 fps, data stream 110 MB/s, the first several seconds red, then always orange, 1 minute recording, continuous. 12 bit lossless is 140 MB/s and can only last for less than three seconds.
3. 3.5K 1X1 preset, AR 2.67 14 bit lossless, 24 fps, data stream 110 MB/s, the first several seconds orange, then always orange with large buffer like 15 seconds, 1 minute recording, continuous. AR 2.39 is 125 MB/s and can only last for less than 30 seconds.
4. Anamorphic 1x3, 1920x2340, 14 bit lossless, 24 fps, data stream 105 MB/s, the first several seconds orange, then always orange with large buffer like 30 seconds, 1 minute recording, continuous. Lens at 18 mm to cover the full frame.

Seems 120 MB/s is the limit for continuous recording.

[theBilalFakhouri]

Found another hack which appear to disable LVICV task which saves a little CPU and memory cycles, I completely don't know what this task does:

LVICV_ReleaseResource: during my tests this hack seems not very reliable (at least on my 700D, didn't make tests on 5D3), sometime I can record more frames with this hack enabled, sometime recording time becomes shorter and it's better to not use it, however card benchmarks appear to have little improvement and the improvement is constant (unlike when RAW video recording):

no hacks:                                                                                                     only LVICV_ReleaseResource:

Around ~1.9 MB/s more, but . . but . . I was about to post benchmarks with all previous hacks with LVICV_ReleaseResource, and . . write speed was identical (huh, I remember it was about ~ 1 MB/s increase during my initial tests some days ago), anyway read first general note down below.

General notes:
-It seems there is some kind of curve for write speed improvement, once we reach higher write speeds, the effect of a new hack become lower, e.g. if you applied only LVICV_ReleaseResource hack and run benchmarks you can get ~2 MB/s improvement, but when you apply LVICV_ReleaseResource hack beside all previous hacks the gain is only ~1 MB/s (same applies for other hacks).

-My Sandisk Extreme PRO UHS-I U3 170 MB/s SDXC card version (which I made these tests using it) is limited to ~90 MB/s maximum write speed, while other versions can go up to 98 MB/s (@Walter Hmm was that an Sandisk card? It seems it's not), but this card is for sure a Sandisk card and it performs 99 MB/s write speed, same as my old 32GB Sandisk Extreme PRO 95MB/s UHS-I U3 SD card (second photo), but that card wasn't stable for me at 240 MHz, anyway, what I am trying to say we actually can reach:



*~90MB/s write speed in LiveView!
so basically 3K 3072x1308 @ 23.976 FPS at 10-bit lossless is Continuous

*Only when using certain versions of Sandisk Extreme PRO UHS-I U3 170 MB/s (probably 95MB/s too) cards which reach 99 MB/s in benchmarks in Play mode. I think @Levas has 95 MB/s version? and it's stable at 240 MHz unlike my 95 MB/s version, I think it's time to me to get new card (is there a way to know it would perform 99 MB/s write speed before buying it? ).

Stubs:

Code: [Select]

0xff0d6444 700D.115
0xff0d62e0 650D.104

**Try to find them for other models**
I will include LVICV_ReleaseResource hack in my next build release for 650D/700D, to be tested.

[Walter Schulz]

while other versions can go up to 98 MB/s (@Walter Hmm was that an Sandisk card?)

Adata Premier ONE 64GB microSDXC. UHS-II. Rated 275MB/s, V90.

[Billsifr013]

I have added stubs for other models in first post, in case if we need them later (70D/100D/60D/600D/1300D/1200D).

DIGIC 4/4+ won't benefited currently due to limited write speed. 50D/5D2/550D lacks these function.

I think this time has come, could you make a build for 60d, I would try it myself, but I'm not a programmer)

[mlrocks]

SanDisk 170 mb/s sd card can only write 90 mb/s. maybe 300 mb/s card is needed for 99 mb/s write speed?