Levas' crop_rec_4k experiments for 6D

Started by Levas, December 01, 2019, 03:52:51 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Levas

Quote from: alexboum on April 25, 2020, 08:14:04 PM
my SFCard is too slow for 4K14Bit LL so I test it with 3K 16:9 10FPS (~45MB/s) (No aliasing). Camera is hand helded. It's not

Not sure, but did you know, you can enter submenu of crop_rec with the 'Q' button when you highlight crop_mode in ML menu ?
You can change the bit depth in there, from 14 bit(Off setting) to 12, 10, 9 or 8 bit.
Should give you longer recording times.

Quetzalcoalt

Hey Levas, i've downloaded your build from the last page (6D 1.1.6), and i wanted to test a sandisk 95mb/s card. Everything works, i get around 60mb/s but i wanted to bench it.
I took the bench module from the latest nightly build but i get some errors when i try to load it.
I also tried the other way around, installing the nightly and getting the sd module from your build. Still doesn't work, but i get a different error.
The error appears only when the sd module is on, if it's off there are no problems.
I've already did a clean install on your module and i forgot to write down the error i got. But if you need it i can provide it.
I guess both modules are not compatible. Do you have the modules that will work together ?
Also is there a guide on building custom crop recording presets ? I wanted to see what is the max res i can do at 30fps.
50fps at 1824x818 is super!
Thanks!

alexboum

Quote from: Levas on April 26, 2020, 07:45:25 PM
Updated Crop_rec again.

And I added a 3072x1728 x 16fps mode (maybe 16 fps can become 24 fps with optical flow)
Unfortunately I couldn't get it to work with canon 5x liveview, so you need ML preview for this preset.


Great I will check it today, I already love the 16fps :) . No other request for the moment :P

Quote from: Levas on April 26, 2020, 08:00:31 PM
Not sure, but did you know, you can enter submenu of crop_rec with the 'Q' button when you highlight crop_mode in ML menu ?
You can change the bit depth in there, from 14 bit(Off setting) to 12, 10, 9 or 8 bit.
Should give you longer recording times.

8) Thanks, no I didn't think to go in this submenu, only try the mlv_raw bitrate setting but only 14 bit were available.
Thanks a lot for the new preset.

Levas

Found a problem while using dual-iso in combination with crop_rec presets, see this post for more information:
https://www.magiclantern.fm/forum/index.php?topic=7139.msg226389#msg226389

The problem is not very apparent in developed dual-iso files, but it is there, slightly color difference between frames, once you see it, you can't unsee it  :P
Found a way to fix this, so here's an updated crop_rec module file, to replace your old ones with, only works with the newer builds posted recently in this topic, doesn't work with the nightly builds.
New crop_rec module file:
https://drive.google.com/open?id=10o2StQq83_9H2bk1azkcbHEqcTAzefaD
Source crop_rec.c file, for those interested:
https://drive.google.com/open?id=1Xgv5NfesimMHPBprhEcHv8vFYQ23Nj90

Also updated the 1x3 modes, 1616x2048x25fps and 1640x2156x24fps.
They now show you stretched ML preview and if opened in MLVapp, they're automatically stretched to normal view.

Link to MLVapp:
https://www.magiclantern.fm/forum/index.php?topic=20025.0
So never been easier to record 4848x2048 resolution or 4920x2156 resolution on the 6d  :D


Danne

You know you can fix the 1x3 stretched preview in cam too? I mean in mlv_play while watching a recorded file. Fixed on eosm.

Did you go through all your presets to fix the dual iso issue?

Levas

Yup, did go through all my presets, it's not that much of work, you put something behind the B-timer in your presets that you can change from the advanced menu in crop_rec.

        case 0xC0F06014: return 0x88b + delta_debugging;

Enable dual-iso and ML-preview and see if lines/bars are static or not, if not go to advanced menu and make delta_debugging one unit higher (or lower) and check ML-preview.
Repeat until problem is solved.
The problem should be fixed within 4 units of the B-timer.

I've got ML preview working for 1x3 modes, playback in mlv_play is not unstretched, how did you fix that ?

Danne

Not by my computer. Could you drop mlv_play code from my branch into yours and you'll see the buffer fix in source tree.

Levas

Ah so the fix is done in MLV_play module.Thanks will check the source.

mineralof

Levas
Thanks for the development! The new 1440 crop mode works very well. I can record raw video for 10-15 seconds 12-14 bit losless. But the preview does not work (Half Press). The frame width is much larger than x5. it's great. sorry for my english

Levas

The 5x zoom liveview is indeed not correct framing, much smaller.
If you don't see ML-preview on half-shutter press, you probably need to push the 'info' button on your camera a few times.
With the 'info' button you normally get different overlays on liveview from Canon, like horizontal level meter or histogram and stuff, but it blocks ML-preview.
So try pressing the 'info' button until half-shutter press gives you ML-preview with correct framing.

mineralof

Quote from: Levas on May 03, 2020, 11:45:44 PM
The 5x zoom liveview is indeed not correct framing, much smaller.
If you don't see ML-preview on half-shutter press, you probably need to push the 'info' button on your camera a few times.
With the 'info' button you normally get different overlays on liveview from Canon, like horizontal level meter or histogram and stuff, but it blocks ML-preview.
So try pressing the 'info' button until half-shutter press gives you ML-preview with correct framing.
thank. I noticed today that the preview (half-press) in cropping mode does not work with the connected external monitor.  maybe fix it somehow? 

alexboum

Quote from: mineralof on May 03, 2020, 10:17:40 PM
But the preview does not work (Half Press). The frame width is much larger than x5. it's great. sorry for my english
Preview works great, did you set in RawVideo menu : Preview<=Framing? Or aren't you talking about Liveview (view while recording)

1X3 is really sharp compare to 3x3, but  I can olny record few seconds (because of my SD) with 10bit 2:1 ratio (bitrate ~ 59MB/s):
https://youtu.be/4nl6N2bHGwE
Dual ISO is smooth too:
https://youtu.be/jNgNw0Sn78s

It also open with the right ratio in MLVApp 8)
Thanks again for the new functionality.

mineralof

Quote from: alexboum on May 04, 2020, 09:14:24 PM
Preview works great, did you set in RawVideo menu : Preview<=Framing? Or aren't you talking about Liveview (view while recording)
I'm talking about liveview in crop x5.  in this mode, the screen displays a much smaller frame field than it actually is.  To see the actual size of the recorded video, you must press the shutter button halfway.  but this doesn't work with an external monitor connected :( By the way, here is my video where half the frames were shot in crop rec mode 2560x1440 24fps x5 crop. I really like the new features of my camera.
https://youtu.be/Sjxn-Ha7XaU

Quetzalcoalt

Quote from: Levas on December 01, 2019, 03:52:51 PM
To give a more simpler option to new people, I've uploaded the build I'm using myself.
It has all the modules needed and activated for raw recording.
...

Hey Levas, i've downloaded your build from the last page (6D 1.1.6), and i wanted to test a sandisk 95mb/s card. Everything works, i get around 60mb/s but i wanted to bench it.
I took the bench module from the latest nightly build but i get some errors when i try to load it.
I also tried the other way around, installing the nightly and getting the sd module from your build. Still doesn't work, but i get a different error.
The error appears only when the sd module is on, if it's off there are no problems.
I've already did a clean install on your module and i forgot to write down the error i got. But if you need it i can provide it.
I guess both modules are not compatible. Do you have the modules that will work together ?

Also is there a guide on building custom crop recording presets ? I really couldn't find on how to do it. I wanted to see what is the max res i can do at 30fps and at 50fps.
50fps at 1824x818 is super and i wanna see if i can try and record continues or for 1-2 minutes!
Thanks!

Levas

You probably need a benchmark module that's from the same Magic Lantern branch as the sd_uhs module.
Try this bench module:
https://drive.google.com/open?id=1UM5v5fnLt0fMMw07cQWFJlBLJ6s0WTqp

Never used it, only tested it once today, seems to work.
But in case you see something strange or your camera acts weird, pull out the battery and restart the camera  :P

Levas

Quote from: Quetzalcoalt on May 06, 2020, 03:27:11 PM
Also is there a guide on building custom crop recording presets ? I really couldn't find on how to do it. I wanted to see what is the max res i can do at 30fps and at 50fps.
50fps at 1824x818 is super and i wanna see if i can try and record continues or for 1-2 minutes!
Thanks!

As far as I know, there is no guide for that.
But you need to be able to compile your own magic lantern files for that.
So download source code to your computer, alter the code, compile for use in camera.

But I'm working on a preset where you can alter the settings yourself in camera (if I can figure out all parts, some are more or less trial and error for me :P ).
So keep an eye on this forum for a new crop_rec module.

Quetzalcoalt

Quote from: Levas on May 06, 2020, 10:22:36 PM
You probably need a benchmark module that's from the same Magic Lantern branch as the sd_uhs module.
Try this bench module:
https://drive.google.com/open?id=1UM5v5fnLt0fMMw07cQWFJlBLJ6s0WTqp

Never used it, only tested it once today, seems to work.
But in case you see something strange or your camera acts weird, pull out the battery and restart the camera  :P

:O it worked, I'm getting 38.4mb/s without the sd mod when camera is in movie mode at 720p @60. and 54.6 mb/s with the sd mod. I tried running the benchmark in photo mode and i got 68.6mb/s.
It's really interesting, why are we getting lower speeds in video mode ?
I guess there is no way to squeeze another 5mb/s in video mode, just so i can do the 50fps at 1824x818 continues.

will definitely be waiting for the custom presets crop module you are working on. !
Thanks!

EDIT: i saw some memory patches (not exactly sure what these are):
0xFF325A20: 0x92D47F0 -> 0xEA46A3A6 SD_UHS
0xFF325AA8: 0x59F7114 -> 0xEA46A394 SD_UHD
0xFF78F308: 0x92D40F8-> 0xEA34FD8C SDR104.

I guess these are that the sd hack module checks what is the max it can do and patches the values so that it uses slower OC on the controller or slower memory card mode.

Levas

 :D  :D  :D
Got something to play with.

New crop_rec build for 6d, with Custom Resolution preset, now you can dial in any resolution you like (The preset configures the corresponding max fps for that resolution).
The preset is made for use in 5x zoom and is 1:1 pixel readout.


Select the "Custom Resolution Experimental" crop preset from crop-rec menu.
After you selected the preset, you can enter crop_rec submenu with the [Q] button on your camera:


In here you can select Horizontal resolution (Goes from 1656 to 5472) and vertical resolution (goes from 640 to 3648)
Dial in any resolution you like and it probably works  ;D
Could be that there is a slightly difference between the resolution dialed in and the one selectable in RAW video menu (horizontal resolution can be 8 pixels difference  sometimes, and depending on the aspect ratio chosen, you may not see the full vertical video resolution, in that case choose another aspect ratio, like 1:2 or 1:1)

As you can see in the screenshot above, there is also a safetymargin setting, this is the overhead on the B-timer(coupled to vertical resolution).
A safety margin value of 64, seemed really robust to me. But in case you get some corrupted frames, you might want to try to increase this value.
The safety margin is directly coupled to B-timer, so it can be used to finetune fps. But if it is set to low, you might end up with corrupted frames.
Trial and error.
If your settings are too critical and you get corrupted frames (could just be only one corrupted frame at the beginning on your video file), you may start to see scrolling text on your display.
You can get rid of this text box on your display by going to debug menu and highlight 'show console' and press the 'set' button on your camera twice (turning camera off and on also helps, but the other action is slightly less annoying):


Now finally, here is the link to the module file:
https://drive.google.com/open?id=1mJkTJlPMmckeyM84f06jCLQ2IPM1yU6Y
Updated version, with Canon liveview (Canon liveview works between Horizontal Resolutions range (1840 - 2880) and vertical resolutions range (908 - 3648)

For anyone interested, the source file:
https://drive.google.com/open?id=1aBmQWSX6fDQRdtZwK-mApYjDPlEJyNBh
Updated version, with Canon liveview (Canon liveview works between Horizontal Resolutions range (1840 - 2880) and vertical resolutions range (908 - 3648)

Good to know, this preset is experimental, tested it myself, did get some camera lockups when accidentally recording in 1x zoom mode (error 70).
But haven't seen anything more worse then camera lockup (mirror suddenly goes down, and camera doesn't respond), which is fixed by pulling out the battery.
Re inserting the battery and turn on the camera again.

But With the safety margin of 64 (default setting) I think it is rather stable.
Now you might question and think about doing what all idiots would do with this tool...'How fast is the fps when the resolution is set to lowest possible settings'.
Well I can tell you that when 1656x640 is chosen as resolution, you get about 67fps.
Seems stable to me in 5xzoom (got lockup when accidentally recording in 1xzoom :P)


High resolution settings with slow fps, might give slow response in giving you ML preview. Could also be the case with low resolution settings with high fps.
In that case, just wait a few seconds, if that doesn't help, you might want to press the 'menu' button on your camera twice (this makes sure all registers are set to chosen value)

Real use for this preset could be finding a desired resolution for 30fps. In that case, you have to choose a resolution and see in ML preview how much fps that resolution is.
If you need more fps, you need to lower vertical or horizontal resolution.
When you have the desired fps, for instance 30 fps with a certain resolution, you can finetune your recording resolurtion in the Raw video menu. (by choosing proper aspect ratio and choosing the desired resolution(use top scroll wheel on your camera for small increment steps (8 pixel horizontal resolution).

EDIT: Canon liveview doesn't work in this extreme preset, you can make use of the ML-preview (see Raw video menu)
Furthermore, Whatever resolution is selected, horizontally, it always should be in the middle of the sensor, couldn't get vertical resolution in the middle, I believe it starts at the bottom of the sensor and goes up when resolution is increased.

EDIT 2: playing with it right now, think it works really well with framerates up to 30 fps. If you go higher then 30fps, it's sort of hit or miss. Trying to find decent 50fps, but small lower part of the frames are corrupten. Seen this before with 75 and 100 fps experiments. Will see if I can fix this.
1656x640 at 67 fps works like a breeze though  :P

alexboum

It's so cool, now we can choose resolution and framerate (by using safetymargin setting) :D.
Thanks a lot!

Levas

Some more about the new custom resolution experimental crop_rec preset:

I tested it some more now, and it seems really stable to me at resolutions which don't give more then 30fps.
Tried to find some nice 50fps setting yesterday, but couldn't get it to work, small lower part of the frames were corrupted...nothing new, high fps has always been a little tricky, will see if I can fix the same way I got 75 and 100fps to work.

If anyone experience any troubles with some settings, let me know and see if it is fixable...

At the moment, this preset is more like a proof of concept, I have some things in mind to implement.
-Try getting Canon liveview to work with this preset.
-Try getting the frame vertical centered on the sensor.
-Try getting high fps (>30fps) to work better, it's now a bit of a hit or miss, some settings work fine, others not.
-Try keeping the settings saved when camera is shutdown(should be possible, many ML settings are saved...no idea how I could do the same for this preset settings)

Levas

Quote from: Levas on May 08, 2020, 11:08:28 AM
-Try getting Canon liveview to work with this preset.

Quick fix for Canon liveview.
Has resolution restriction, Canon liveview works between Horizontal resolution range (1840-2880) and vertical resolution range (908-3648).

When preset is loaded, and ML-preview or Liveview doesn't look good, you might want to press the 'info' button 2 times.
Pressing the info button 2 times makes sure the preset is configured for all registers.

New build available in the post a few posts above this one, replaced the files and updated the links.

EDIT:
High fps seems more stable in this build.
1840x908 resolution gives about 46fps.
Works fine over here, first frame might be corrupt, but the rest is all good.
To prevent the first frame for being corrupt, you might want to increase the safety margin value to 80.
I didn't get a corrupted first frame with that setting.

Levas

Updated the Custom_resolution_experimental preset.

Last settings are saved in the preset. So if camera is turned off and on later again, the preset starts with the last resolution settings set in the preset.
And the preset now uses mostly the center part(horizontal and vertical) of the sensor for any set resolution.

I think most resolutions will work  ???
And as stated before, 5x live view is working for horizontal resolution range 1840 - 2880 and vertical resolution range 908-3648.

Not all settings with high fps work, some settings give corrupted bottom part of frames. Not sure how to fix it at the time.
So be sure to test your own custom settings. And check if ml preview works, if not try hitting menu button twice.

But here are some settings that work over here, so you might want to try dial in these settings yourself in the submenu of the "Custom_resolution_experimental" preset.

1840 x 908 at 48 fps (actual recording resolution can be altered in raw video menu, for example 1840x782 (for 2.35:1 aspect ratio)


2880 x 1200 at 24 fps


2240 x 1260 at 30 fps


1656 x 640 at 60 fps (actual recording resolution can be altered in raw video menu for more common aspect ratio)


Here's the new module file:
http://drive.google.com/open?id=1jZ6Z_PKshd-xOT8-YZDwuyzU-6aL7H8Z

And here's the source:
http://drive.google.com/open?id=1_flOViBGoF0qsCGXO0g2S8yxVU9ucS8A





jrose99x

Hey guys, I'm new to the forum so sorry if I'm not posting in the right place.

I have a canon 6d and I successfully installed the raw video 10 bit 12 bit build,

I have 3 questions

1. At the moment I try to set the recording resolution to 1920/24fps (on 14bit), but it says "1920 is not posible in current video mode(max 1824) and so it's currently on 1824x1026 obviously. (it says at 14bit it's at 74.8mb/s , which I know the 6d has max 40mb/s write speed to card natively)
The thing is that even if I lower the bit rate to 10 bits, the resolution still maxes out at 1824x1026, but says 53.4mb/s. (which should be ok for a few seconds as the buffer takes the other 13 mb's, right?)

So my question is why does lowering the bit rate (and therefore mb/s being outputted) NOT give more headroom for higher resolution? all it did was extend the amount of time I can record without skipping frames. what is the value I have to change to get higher resolution, what video "mode" is it talking about?

2. I'm thinking the best solution is the UHS/SD card overclock module I've read about in the forums, but can't find the actual module/download link/build ANYWHERE, on site or off. (I know there's risk involved but thats the whole thing with ML we're already risking it) so if anybody could link me to it, I would GREATLY appreciate it. I imagine having the 60mb's write speed could be what the camera needs to bump up the resolution? at least to 2240 and I'd be a happy camper.

3. So I'm currently recording in 1824x1026 10bit, and I switched the aspect ratio to 1: 2.39 (I did this because in the menu as I did this it said it estimates 39.8mb/s, which as I understand is under the 40 mb's cap, therefore allowing continuous shooting if I'm correct, and it also said "Expect 0 frames at 27 mb/s, which I have no idea what that's supposed to mean) So anyway it says 39.8 mb/s estimated (and the camera should also have the 250mb on the buffer too to compensate any overage over 40 if I'm not mistaken?) Ok so the thing is I also have the buffer graph/monitor on in live view and it says it's usually using about 27-30mb/s when I'm actually recording video and it stays like that through the recording, UNDER what it estimated... YET it still starts to skip frames at approximately 18 seconds, my question is again, WHY? 39.8 and 27-30 are both under the 40mb/s cap, it shouldn't even be going onto the buffer,  so what am I missing here exactly?

I tried the 5 different buffer methods (which I don't understand either) and still, skipped frames at approximately 18 seconds

Thanks so much for any help and excuse the long post I'm new to this, cheers! Thanks to the ML team for what you've accomplished thus far, just a noob trying to understand.

Levas

There are great things possible with Magic Lantern, especcially with raw video, but it's a big mess with different builds... :P

So to anwser the first question.
Resolution and write speed have no connection to each other besides how long you can record (unlimited recording time or less).
So higher writing speeds doesn't unlock higher resolutions, it does mean however that you can record longer in higher resolutions.
For higher resolutions in the build you're using, you need to press the magnifying button on your camera to go into 5x zoom mode.
Once you're in 5x zoom mode, press the trashcan button to go into ML menu, and in raw video menu, you'll see that you can now record in bigger resolution (probably ~2688x958).
Now you can record 2688x958 while in 5x zoom mode.
Once you press the magnifying button and go back to normal mode, the resolution changes back to 1824x1026.
You will probably also see that in 5x zoom mode, you're not using the whole sensor width, it's only using about half of the sensor width (2688 pixels of the 5472 total horizontal pixels). This is called cropping, you're cropping the recording area of the sensor, that's why this is also sometimes referred to as crop mode.

Now standard 5xzoom has max resolution of 2688x958 pixels, now with a some special ML build, which contains a crop_rec module, you can select bigger resolutions then 2688x958.
I'm working on a crop_rec module, only the menu system is a bit messy :P
I'll see if I can upload it this week in this 6d forum. Complete build, which also contains the module to speed up write speed.

Which brings us to question 2, will upload a complete build this week, with the SD/UHS module.
But there is a caveat, it doesn't work with all SD cards out there, sandisk extreme pro with the 170mb/s label works for me and also the older sandisk extreme pro with the 95mb/s label.

Question 3
Not sure, but I think the 27mb/s you mention is the data rate your SD card can handle. Makes the most sense to me, otherwise I can't explain why you're not getting ~40mb/s.
Also I see you're using the 10/12 bit build for 6d.
But there is also something as lossless raw recording. Same raw quality with lossless compression, so lower data rates needed for recording.
For that you need the "4K raw video recording; lossless compression" for 6d build from the experiments downloadpage, where you probably also downloaded your 10/12 bit build.
With this "lossless" build you can select lossless recording option in raw video menu, and you can even record in 12 bit lossless. Lossless bitrates use ~30% less data.
So this will give you longer recording times.


Oleg_K

Sorry, maybe off topic, I have Canon 6d, the latest build from Levas - MagicLantern - 6D, when shooting Silent Pictures, time lapse in simple mode is fine, but in full mode the picture is very bright, everything is bad. Maybe you need some settings? Sorry for the english.