Author Topic: LiveView hacks (write speed improvement)  (Read 9106 times)

mlrocks

  • Member
  • ***
  • Posts: 151
Re: LiveView hacks (write speed improvement)
« Reply #100 on: April 23, 2022, 06:32:43 AM »
Interestingly, compared with 10 bit footage, 14 bit 2k 1x1 AR 3-2 24 fps, when shooting with auto/manual ettr, and elevating exposure one stop in MLV App, the output is very nice, similar to alexa/amira's high key footage, the noise in the dark area is not apparent. So, 14 bit is actually better than 10 bit, when using ettr, may gain one to 2 stops dr. For 10 bit shooting, better avoid ettr, because much of the information pushed to the dark side is lost and not saved in raw files.
In this term, 14 bit 4.5k 1x3 with ettr and dual iso may have a dr of 16 stops, 12 stops from the 650d raw file, 1-2 stops from ettr, 2-3 stops from dual iso. This dr is on par with the legendary alexa and amira's dr. 4.5k 1x3 raw also should be on par with the alexa/amira's 2.8k prores. The sensor size in 4.5k 1x3 mode is about 230 mm2, almost the same as 4-3's 224 mm2, although a little bit smaller than alexa/amira's super 35mm, this is good enough for news shooting, which requires large in focus zones. Imagine a lowly 650D/700D is Alexa/Amira or Red One in disguise, this is really something to talk about.

https://www.provideocoalition.com/alexa-dynamic-range-its-all-in-how-you-use-it/
https://www.newsshooter.com/2020/05/27/the-arri-alexa-is-10-years-old/

After so much consideration, I finally settle down on 14-bit lossless UHD 1X3 with auto/manual ETTR without dual ISO on my 650D for raw news shooting, due to its AR 16-9, relative good low light performance, good image quality, acceptable pp computing time, correct real time color high resolution preview and magic zoom 2, etc. 14-bit lossless UHD 1X3 for a complex scene has a peak writing speed of 85 MB/s, I will go to user interface 2 (almost equivalent to global draw off but with certain essential information) and underexpose the scene to maintain the continuous recording, or go 12-bit lossless. Most of the time, though, the writing speed is below 78 MB/s, allowing to use the global draw in user interface 1. 

koopg

  • New to the forum
  • *
  • Posts: 30
Re: LiveView hacks (write speed improvement)
« Reply #101 on: April 23, 2022, 09:01:46 PM »
For dual ISO on 5d3, did you see flickering on the bright area?
no, had 2 hours of great dual iso@3.5k 14 bit lossless 2:20
beside first frame pink
had nothing but continues stunning raw video.


Sent from my SM-N975F using Tapatalk


mlrocks

  • Member
  • ***
  • Posts: 151
Re: LiveView hacks (write speed improvement)
« Reply #102 on: April 24, 2022, 06:21:58 AM »
no, had 2 hours of great dual iso@3.5k 14 bit lossless 2:20
beside first frame pink
had nothing but continues stunning raw video.


Sent from my SM-N975F using Tapatalk

Great news!
Now 5D3 in 3.5K super 35mm mode is no less than Arri Alex 35mm series.
Any one here knows if dual ISO on 5D3 in 5.7K 1x3 mode has flickering in bright area?

Why Most Movies Are Shot On Arri Cameras
Reading the follow up comments is very interesting. Many experienced DP there.

"I have an Alex Mini LF and a red V raptor, Helium, C70, Blackmagic,Panny EVA1. The Alexa  might have been way above over the competition 5-8 years ago, today? Not  so much. The Alexa Look , besides the out of the box colors is this. Alexa digs deeper into the shadows. Whatever you shoot with Alexa tends to look HDR'ish. because the log curve digs into the shadows. As soon as you shoot something with any other camera and lift the shadows in post.  Boom! Looks almost identical to the Alexa. Red V raptor is Cleaner also. Faster, cheaper, more compact, lighter. The only difference is the out of the box colors.  Once you color correct every camera side by side? Alexa is nothing special."

"Alexa mini LF owner here, fortunately have been able to use just every digital cine camera from bmpcc6k (which I still own and use), and at least 8 different types of red cameras(komodo is valid) up to the Mini LF. Arri is truely the best camera company in the world.  If you don't believe me, shoot a project on Arri with cooke 1.8 anamorphic or any high end glass and watch your mind be blown. Here's a music video I shot with that setup . It  can make our jobs seem too easy at times but allows room for extreme creativity and style."

"Sony Venice has wider gamut, dynamic range and noise that is far more like film (variations in size over luminance). But the Arri is far harder to mess up the settings, and the Sony menu is a mess. I.e. the transition to digital from film was easier with the Arri. It’s easier to use, and harder to fruk up. I’ve posted over 100 features, and the Arri gets consistent results (I examine the metadata so I know what’s been done in the settings by the dop). imo the Sony creates better image both in it’s character and technical range, but is harder to learn how to use."

"The Sony Venice and especially the Venice 2 is absolutely beating out Arri even the Arri LF systems. Arri will always be considered an amazing camera system but they aren't adapting nearly as quickly as their competitors. We can't keep the Sony Venice on the shelf and have had DP's opt for the Venice 2 on major feature productions over Arri. Movies are no longer mostly shot on Arri cameras. Was this relevant 2 years ago, absolutely, but now the competition is getting too tight. Not to mention Panavision is still very popular and always was neck and neck with Arri."

Take home message: High end cameras are getting better and closer to each other now. Alexa still has an edge but not significant. 5D3 ML is getting there too. Once the realtime preview and the hdmi output is ported from 700D/650D by Bil the Great.

2h means 1TG footage? How long was the shooting session for this 2h footage? Did you experience the overheating due to sd card overclocking? What was the camera temperature? Did you buy all of the 2tg cf cards and sf cards? Will this take one to two weeks non-stopping pp computing time? Thanks.

Wushuliu

  • New to the forum
  • *
  • Posts: 21
Re: LiveView hacks (write speed improvement)
« Reply #103 on: April 24, 2022, 05:42:28 PM »
Only 650d/700d can do correct high resolution hdmi out. EOS M may do that also. Thanks to Bil's work. Hope he can figure out the 5d3's realtime preview and hdmi out.
650D/700D UHD 1X3 preset has AR 16-9, 4K 1x3 preset has AR 2. I tested that both HDMI out works fine with my Zacuto 3 inch EVF.
5D3's 5.7k 1x3 preset has incorrect framing unstretched hdmi out. I can use Zacuto EVF's anamorphic ratio of 2 to do certainly level of unstretching that is good enough for rough framing, but the hdmi out framing is not correct.

Hm. Ok. So 650D or possibly 5D3.
Hopefully some test footage will be posted soon.
Fascinating developments. Exciting.

mlrocks

  • Member
  • ***
  • Posts: 151
Re: LiveView hacks (write speed improvement)
« Reply #104 on: April 24, 2022, 05:46:57 PM »
Hm. Ok. So 650D or possibly 5D3.
Hopefully some test footage will be posted soon.
Fascinating developments. Exciting.

Better buy 700D over 650D, because Bil only has 700D. This means that 700D always has the latest dev. LOL.

mlrocks

  • Member
  • ***
  • Posts: 151
Re: LiveView hacks (write speed improvement)
« Reply #105 on: April 24, 2022, 05:52:42 PM »
Right now Sandisk Extreme Pro 170 MB/s 256GB CF card is about 250 dollars, SD card is about 60 dollars, micro SD card is about 30 dollars.
Anyone here used microsd card to sd, microsd to sd to cf on 5D3? SD overclocking still performs normally?

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 8254
Re: LiveView hacks (write speed improvement)
« Reply #106 on: April 24, 2022, 05:55:08 PM »
ML project does not support SD-to-CF adapters. If you have troubles (and there will be!) you are on your own.

If you go for cheaper CF: Komputerbay 1066x. 128 GB variety is somehow reasonable priced. 256 not. At least around here.

mlrocks

  • Member
  • ***
  • Posts: 151
Re: LiveView hacks (write speed improvement)
« Reply #107 on: April 24, 2022, 07:11:34 PM »
ML project does not support SD-to-CF adapters. If you have troubles (and there will be!) you are on your own.

If you go for cheaper CF: Komputerbay 1066x. 128 GB variety is somehow reasonable priced. 256 not. At least around here.

Thanks for the quick reply, Walter. Seems not able to save money here.

theBilalFakhouri

  • Developer
  • Hero Member
  • *****
  • Posts: 756
Re: LiveView hacks (write speed improvement)
« Reply #108 on: April 26, 2022, 08:15:03 PM »
I have added stubs for other models in first post, in case if we need them later (70D/100D/60D/600D/1300D/1200D).

DIGIC 4/4+ won't benefited currently due to limited write speed. 50D/5D2/550D lacks these function.
Announcement: 100D/SL1 users:
I am willing to port the latest RAW video achievements to 100D. Check this for more info: https://www.magiclantern.fm/forum/index.php?topic=26250.msg238692#msg238692
We are almost half-way from the goal! any help from any user is appreciated.

mlrocks

  • Member
  • ***
  • Posts: 151
Re: LiveView hacks (write speed improvement)
« Reply #109 on: April 30, 2022, 03:16:39 AM »
Interestingly, compared with 10 bit footage, 14 bit 2k 1x1 AR 3-2 24 fps, when shooting with auto/manual ettr, and elevating exposure one stop in MLV App, the output is very nice, similar to alexa/amira's high key footage, the noise in the dark area is not apparent. So, 14 bit is actually better than 10 bit, when using ettr, may gain one to 2 stops dr. For 10 bit shooting, better avoid ettr, because much of the information pushed to the dark side is lost and not saved in raw files.
In this term, 14 bit 4.5k 1x3 with ettr and dual iso may have a dr of 16 stops, 12 stops from the 650d raw file, 1-2 stops from ettr, 2-3 stops from dual iso. This dr is on par with the legendary alexa and amira's dr. 4.5k 1x3 raw also should be on par with the alexa/amira's 2.8k prores. The sensor size in 4.5k 1x3 mode is about 230 mm2, almost the same as 4-3's 224 mm2, although a little bit smaller than alexa/amira's super 35mm, this is good enough for news shooting, which requires large in focus zones. Imagine a lowly 650D/700D is Alexa/Amira or Red One in disguise, this is really something to talk about.

https://www.provideocoalition.com/alexa-dynamic-range-its-all-in-how-you-use-it/
https://www.newsshooter.com/2020/05/27/the-arri-alexa-is-10-years-old/

Just checked some old test footage. ON 650D, UHD 1x3 10 bit lossless has significantly lower noise level than 14 bit lossless 1920x1080 1x1 24 p in the dark area. So it is a complicated issue. Seems larger sensor size is important to lower down the noise, in addition to the data bit rate. The former one may have much more effects than the latter one.

ShittyWebsite

  • New to the forum
  • *
  • Posts: 26
Re: LiveView hacks (write speed improvement)
« Reply #110 on: May 01, 2022, 10:23:26 PM »
yeah, i enabled both cards and spanning and i also now have continuous recording at 3616*1536 23.976 @14 bits (unless overexposed)

amazing...

bilal, you are putting the magic back into magic lantern, brother!

Hi, i might be late, how can i get this custom resolutions? i can only use the presets (3.3, 3.5k, uhd)


Oh i guess its UHD awith a small decrease in the horizontal resolution?

mlrocks

  • Member
  • ***
  • Posts: 151
Re: LiveView hacks (write speed improvement)
« Reply #111 on: May 01, 2022, 10:38:20 PM »
Hi, i might be late, how can i get this custom resolutions? i can only use the presets (3.3, 3.5k, uhd)

Crop mode new 2 module. Test thoroughly before real application. These are not optimized as well as Crop mode presets.

ShittyWebsite

  • New to the forum
  • *
  • Posts: 26
Re: LiveView hacks (write speed improvement)
« Reply #112 on: May 01, 2022, 10:38:55 PM »
Crop mode new 2 module. Test thoroughly before real application. These are not optimized as well as Crop mode presets.

Oh thank you, i was trying that on a 5D3 lol


theBilalFakhouri

  • Developer
  • Hero Member
  • *****
  • Posts: 756
Re: LiveView hacks (write speed improvement)
« Reply #113 on: May 01, 2022, 10:55:54 PM »
Oh i guess its UHD awith a small decrease in the horizontal resolution?

Correct.
on 5D3 set crop mode preset to UHD, from RAW video submenu set resolution to 4096 and Aspect Ratio to 16:9 so you can get 3840x1536 then highlight "Resolution" and use "Main Dial" or "Multi-Controller" to decrease resolution to 3616x1536.

Crop mode V2 is not related here, and that's only for 650D/700D.
Announcement: 100D/SL1 users:
I am willing to port the latest RAW video achievements to 100D. Check this for more info: https://www.magiclantern.fm/forum/index.php?topic=26250.msg238692#msg238692
We are almost half-way from the goal! any help from any user is appreciated.

ShittyWebsite

  • New to the forum
  • *
  • Posts: 26
Re: LiveView hacks (write speed improvement)
« Reply #114 on: May 04, 2022, 01:44:11 AM »
Hi, sometimes when using "one more hack" the console shows

Is there any way i can remove the console? (not the "Show console: off" but deleting the console or maybe even just ignoring it so i can see what i'm recording without the console in front of it)

Bender@arsch

  • Member
  • ***
  • Posts: 107
Re: LiveView hacks (write speed improvement)
« Reply #115 on: May 08, 2022, 01:06:42 PM »
Hi, sometimes when using "one more hack" the console shows

Is there any way i can remove the console? (not the "Show console: off" but deleting the console or maybe even just ignoring it so i can see what i'm recording without the console in front of it)

If the console shows, it means there is a Failure somewhere. I have this sometimes too, deleting the last clip (because of corruption) and restart Camera is the best way.
Maybe you can read the failure message next time, maybe it is helpful for bug fixing.

theBilalFakhouri

  • Developer
  • Hero Member
  • *****
  • Posts: 756
Re: LiveView hacks (write speed improvement)
« Reply #116 on: May 09, 2022, 03:59:38 PM »
Yeah, I simply can't turn off the console all time (or even only when RAW video recording). It's used mostly in every ML feature, disabling it is not a good idea at all, you may have corrupted frames while recording without realizing that which result in totally useless MLV clip.

I didn't have console showing when I used "One more hack". Could you share your settings? a steps to reproduce?
-Probably adding a little more delay after calling "One more hack" could solve this problem. I will do it soon.

-In general: avoiding using "One more hack" could be a better idea for now, since I really don't know what is it doing (this hack actually disables some EDMAC channels activity), figuring out how to silence EDMAC channels streams without using CartridgeCancel a.k.a "One more hack" in clean way is a solution for the side effects we are having when using this hack.

-Edit: also mind share printed message in the console? are you sure it's from "One more hack"?
It could be from something else.
Announcement: 100D/SL1 users:
I am willing to port the latest RAW video achievements to 100D. Check this for more info: https://www.magiclantern.fm/forum/index.php?topic=26250.msg238692#msg238692
We are almost half-way from the goal! any help from any user is appreciated.

ShittyWebsite

  • New to the forum
  • *
  • Posts: 26
Re: LiveView hacks (write speed improvement)
« Reply #117 on: May 10, 2022, 01:25:15 PM »
Hi, thank you, i couldnt reproduce consistently, i'm not sure its from "One more hack"


I managed to get the logs from yesterday, i'm not sure if this is helpful:


Preset: 3.5K 1:1
14bit lossless
Card Spanning on
Use SRM Memory: On
Small Hacks: Af Off
More Hacks: lvface+aewb
One more hack: On
SD Overclock: 160mhz
Dual Iso: Off


Code: [Select]
ML ASSERT:
0
at mlv_lite.c:2768 (compress_task), task compress_task
lv:1 mode:3

compress_task stack: 1ae530 [1ae5c0-1ad5c0]
0x0006A01C @ b451e0:1ae560
0x00069878 @ 6a084:1ae530

Magic Lantern version : Nightly.2022Apr07.5D3113
Mercurial changeset   : 712fa0bdd45c+ (crop_rec_4k_mlv_snd_isogain_1x3_presets) tip
Built on 2022-04-07 02:25:55 UTC by Bilal@DESKTOP-7RFHEE1.
Free Memory  : 213K + 2362K


When 1080p 40fps:

Presets: mv1080p 40fps
14bit lossless
Card Spanning: Off
Preferred Card: SD
Kill Global Draw: On
Use SRM Memory: On
Small Hacks: Af Off
More Hacks: lvface+aewb
One more hack: On
Recording Delay: 2s / recording time 5s
SD Overclock: 160mhz
Dual Iso: Off


Code: [Select]
ML ASSERT:
0
at mlv_lite.c:2768 (compress_task), task compress_task
lv:1 mode:3

compress_task stack: 1ae530 [1ae5c0-1ad5c0]
0x0006A01C @ b45440:1ae560
0x00069878 @ 6a084:1ae530

Magic Lantern version : Nightly.2022Apr07.5D3113
Mercurial changeset   : 712fa0bdd45c+ (crop_rec_4k_mlv_snd_isogain_1x3_presets) tip
Built on 2022-04-07 02:25:55 UTC by Bilal@DESKTOP-7RFHEE1.
Free Memory  : 213K + 3001K



theBilalFakhouri

  • Developer
  • Hero Member
  • *****
  • Posts: 756
Re: LiveView hacks (write speed improvement)
« Reply #118 on: Today at 12:13:22 AM »
Updated links in the first post for 5D3 and EOSM to Danne's downloads page.

It seems Danne added a delay after calling CartridgeCancel, please test latest Danne build for 5D3 and see if there is a first frame corruption when using "One more hack".
Announcement: 100D/SL1 users:
I am willing to port the latest RAW video achievements to 100D. Check this for more info: https://www.magiclantern.fm/forum/index.php?topic=26250.msg238692#msg238692
We are almost half-way from the goal! any help from any user is appreciated.

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7236
Re: LiveView hacks (write speed improvement)
« Reply #119 on: Today at 09:20:50 AM »
Tested briefly after adding but seemed to be working. Not sure what delay is best but added this to cartridge cancel:
Code: [Select]
    if (one_more_hack)
    {
        void (*CartridgeCancel)() =
        cam_eos_m ? 0xffa7e7d8 :
        cam_5d3_113 ? 0xff17fd68 :
        cam_5d3_123 ? 0xff181340 :
        0;
        CartridgeCancel();
        msleep(40);
    }