Magic Lantern Forum
Developing Magic Lantern => Modules Development => Topic started by: Danne on November 09, 2018, 05:11:37 PM
-
Due to misassumptions let us move on!
https://www.magiclantern.fm/forum/index.php?topic=23041.msg208432#msg208432
From crop_rec thread:
https://www.magiclantern.fm/forum/index.php?topic=19300.msg208429#msg208429
In case anyone missed this:
https://www.magiclantern.fm/forum/index.php?topic=10111.msg208136#msg208136
A special ML build here for 5D3113:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_isogain_1x3_5D3113_presets.zip
build for 5D3123(Not tested! bleeding edge, be careful)
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_isogain_1x3_5D3123_presets.zip
commit:
https://bitbucket.org/Dannephoto/magic-lantern/commits/75b6e5aedbbb23c22b5055ddea6712f662681955
I added a few items to the crop_rec menu and also rearranged a few things which makes it possible to get cleaner output. Thanks to a1ex crop_rec.c code it´s possible for a player like myself to get some decent workflow out of this.
How does it work?
Pretty straight forward. If you want to shoot regular 14bit or use the regular 12/10 bit menu in RAW video you can use Crop mode menu as before. However, if you want to get cleaner iso and lowering analog gain to get 12/10bit output a different way do following:
1 - Enable crop rec and select any setting in Crop mode menu. Be sure to start off in 14bit-lossless
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.postimg.cc%2FJhkZ9m0H%2FVRAM0-PPM-500px.png&hash=efdbb8ef3bb4f886b0228508fec39ea7)
2 - Press q on Crop mode menu and select any iso in there. This will result in a 13bit iso which will get you cleaner files than the regular iso. You won´t see the change by eye in liveview but comparing files recorded you´ll see the difference in shadows. By enabling iso in this submenu you will override any analog iso already set before. Set chosen iso to off if you want to get back to old iso settings
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.postimg.cc%2FFsLcBN6t%2FVRAM2-PPM-500px.png&hash=b9b1398cb5cc653e6fd75e5c6c3faea8)
3 - You can now choose to add 12bit or 10bit on top of your chosen iso to further lower analog gain registers
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.postimg.cc%2FXJ1FTJ3W%2FVRAM1-PPM-500px.png&hash=db3d80f39e08d1d86b89e548fb6b038d)
Original iso 100
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.postimg.cc%2FR09ctyGk%2Fiso100-png-800px.png&hash=8b24d9f1b011c35adf9408123c19e29c)
10bit iso 100 More or less same as iso 1600. Let´s work the other bits if you not after lowering bitrate.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.postimg.cc%2F50yQqVzR%2Fiso100-10bit-png-800px.png&hash=6aa3d7d2a0df5e8d918796a7561e90f5)
12bit iso 100
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.postimg.cc%2FtgXxJP4y%2Fiso100-12bit-png-800px.png&hash=9b7b22c05f03ff346a749ed5a65dcdc3)
13bit iso 100
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.postimg.cc%2F0Q4MVBd2%2Fiso100-13bit-png-800px.png&hash=757aefa399db7f30bf918978e5778eea)
Have fun! And be careful. It´s bleeding edge...
Noticed two bugs on the way when testing:
1 - When 1080p45/1080p48 3x3 is set you will get stuck in 10x zoom mode if you go there.
2 - adtg_gui.mo will interfere with slowish "framing" previewing. You need to disable adtg_gui.mo altogether to have framing mode working again.
Thanks A1ex, Levas, bilal, 70MM13 and others for contributions.
-
for 5D3123 , nothing ? :-\
-
10bit iso 100
This is actually about ISO 1500. Try comparing it with Canon's ISO 1600, in both highlights and shadows.
Yes, it's slightly better than Canon's 1600, but labeling it as ISO 100 is plain misleading, sorry to say that.
Edit: further test reveals the actual ISO is actually *above* 1600. Test files (white level tweaked manually, right before the last histogram peak):
iso1600.DNG (https://a1ex.magiclantern.fm/bleeding-edge/iso50/danne/iso1600.DNG) (unmodified Canon ISO, assumed exact)
iso100-cmos0-443.DNG (https://a1ex.magiclantern.fm/bleeding-edge/iso50/danne/iso100-cmos0-443.DNG) (estimated ISO 1527)
iso100-cmos0-443-adtg-888x-low.DNG (https://a1ex.magiclantern.fm/bleeding-edge/iso50/danne/iso100-cmos0-443-adtg-888x-low.DNG) (estimated ISO 1491)
danne-iso100-10bit.DNG (https://a1ex.magiclantern.fm/bleeding-edge/iso50/danne/danne-iso100-10bit.DNG) (estimated ISO 2043)
ISO estimation method in octave, from median value in the raw file, normalized with black and white levels (not 100% right; estimation should be done from clipping point):
[a,ea] = read_raw('iso1600.DNG'); a = a - ea.black_level; a = a / (ea.white_level - ea.black_level); median(a(:))
ans = 0.33647
[a,ea] = read_raw('iso100-cmos0-443.DNG'); a = a - ea.black_level; a = a / (ea.white_level - ea.black_level); median(a(:))
ans = 0.32112
# 1600 * 0.32112 / 0.33647 = 1527
[a,ea] = read_raw('iso100-cmos0-443-adtg-888x-low.DNG'); a = a - ea.black_level; a = a / (ea.white_level - ea.black_level); median(a(:))
ans = 0.31355
# 1600 * 0.31355 / 0.33647 = 1491
[a,ea] = read_raw('danne-iso100-10bit.DNG'); a = a - ea.black_level; a = a / (ea.white_level - ea.black_level); median(a(:))
ans = 0.42959
# 1600 * 0.42959 / 0.33647 = 2043
Edit2: further testing to figure out why Danne's picture is brighter reveals a different shutter speed when crop_rec with the "iso 100 10-bit" preset active: rather than 1/100 as dialed in Canon menu, after enabling crop_rec, shutter speed becomes 1/73.31 (as read from the Expo menu or from the DNG metadata). This gives a resulting ISO of 2043 * 73.31 / 100 = 1498, i.e. as expected.
QED, initial hypothesis confirmed.
-
Oh, quite so. Same quality more or less. Tested. 10bit a no go. Thanks for pointing out. 10bit could still be useful for lower bitrates with modes not working with anything else than 14bit lossless.
@jimiz
Updated build for 5D3123 in my above post. Not tested so please report how it works.
EDIT:
Edit: further test reveals the actual ISO is actually *above* 1600. Test files (white level tweaked manually, right before the last histogram peak):
iso1600.DNG
iso100-cmos0-443.DNG
iso100-cmos0-443-adtg-888x-low-iso1500.DNG
danne-iso100-10bit.DNG
Confirmed, slightly above 1600iso comparing your files.
That octave image analysing is really something else.
EDIT 2:
So conclusion is I have been simply rearranging the signal pushing down for instance iso 400 to iso 100(12bit setting)
200 to 100(13bit). Fools gold 8).
Well, now I pragmatically now what´s going on and had a good c code practice...
-
Ok guys, here's the proof everyone needs...
I used a manual aperture lens and made absolutely sure that both exposures are identical. This is a serious stress test.
See for yourself.
There's a bit of loss of highlights, but for practical purposes it is negligible.
I've been doing this by hand for all of my videos for the last year, and I am overjoyed to be able to do it so easily now.
Overwhelming thanks to Danne!
And of course to the wizard a1ex!
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fthumb.ibb.co%2Fgj4bQA%2F100-gain-dropped.jpg&hash=11340e26ba01951761fef3a175e577af) (https://ibb.co/gj4bQA)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fthumb.ibb.co%2Fii8aJV%2F100-stock-iso.jpg&hash=3a6a12d15bf421a14c83de6fb39a6968) (https://ibb.co/ii8aJV)
-
Thanks for sharing. Well, it goes like this. If you shoot at iso 100 then set it to iso 100 with gain drop you could actually then switch again to regular canon iso 200 and it should give you the same result as the one with the gain drop. Just keep the same settings, aperture, shutter(iso 200 will become brighter and classic ettr will take place). So what´s really happening here is that I have set iso hundred but in reality we are shooting iso 200. Conclusion. Fools gold 8).
One sidequestion. I know the question was asked that dual iso could be disabled for previewing purposes when entering 10xzoom mode. How would it work if dualiso previewing could act like HDR showing alternating iso image on display, high low with intervals? When hitting record dualiso should be enabled like before so it would only be a previewing feature. Even showing only base iso could be really useful, maybe even better than alternating isos.
Now that 1x3 recording is possible it´s even more useful with dualiso but previewing is a bit problematic.
-
200 stock ISO, same exposure...
The proof is in practice.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fthumb.ibb.co%2FmawCyV%2F200-stock-iso.jpg&hash=a9354655ec9098639ff13c63e472b164) (https://ibb.co/mawCyV)
-
DNG, please.
-
All three tests, zipped DNG.
https://drive.google.com/file/d/17hlqCmZtawshYnzvHpaMCi2zvuDxc6w_/view?usp=drivesdk
-
@jimiz
Updated build for 5D3123 in my above post. Not tested so please report how it works.
THANKS I will try!!! ;)
-
All three tests, zipped DNG.
Clip areas:
- 0446: 0.0863% (WL=16381)
- 0447: 0.1373% (WL=9138)
- 0618: 0.1101% (WL=16381)
Pretty sure 0447 was based on ISO 400 or greater, possibly 1600; please compare with these.
Additionally, your clipped area is extremely small. If you use a higher stock ISO, the highlight difference in the final output will be negligible. Please try.
Please consider using a lamp shining into some large flat area (such as a white wall); look at my test scene a few posts above. There, even a tiny change in ISO (e.g. from 1600 to 1500) is noticeable in clip areas, which are pretty large, btw:
iso1600.DNG: 15.34% (WL=16381)
iso100-cmos0-443.DNG: 13.98% (WL=16381)
iso100-cmos0-443-adtg-888x-low.DNG: 13.53% (WL=6040)
danne-iso100-10bit.DNG: 23.39% (WL=2885) [note: different exposure time on this one]
TLDR: please compare with stock ISO 400 and 1600.
-
At these exposures, stock 400 and especially 1600 are just a pile of noise when the shadows are lifted to my liking.
Maybe this isn't for everyone, but it sure works for me!
I'm a happy camper getting excellent results, and the last time I checked, that's what really matters...
-
Please post the DNGs.
At these exposures, stock 400 and especially 1600 have much cleaner shadows than stock 100 or 200. Please try them.
-
Stock 400 did surprisingly well, but not as good as Danne 100.
Stock 1600 was unusable when pushed. Not worth the data...
For me, 100 with clean shadows is heaven :)
Here's the DNG:
https://drive.google.com/file/d/1htZlLLEZTsXqRujzj7SV90sP1GcxzrxC/view?usp=drivesdk
-
Please make sure you use the same shutter speed, aperture, scene light and so on. Otherwise, the test is not relevant.
This last file shows 1/1000 exposure time; previous ones were at 1/62.5.
If scene light is no longer the same as in the original test, please capture the entire sequence again, at stock ISO 100, 200, 400, 800 and 1600, and at whatever settings you have used for Danne's ISO 100 (not known to me at the time of writing). Also, please minimize the camera movement (not with subpixel accuracy, but the test images should have the same framing).
Anyway, crop area in the last file is 0.1349%; not sure whether to consider it or not, given the different exposure time.
-
I did a bunch of tests and found that the benefits are primarily for 100, falling off to equal by 400.
I was exposing to the right in the tests, using the bulb as the reference point by eye at 10X zoom, adjusting first by increasing the f stop and then by shutter speed when I hit the aperture limit.
I also don't know what Danne has done "under the hood", and so it may be that the test is invalid for actual ISO comparison.
Until today, I've been doing this by hand with the iso registers using your modules, knowing for example that I'm dropping 200 to 109, my most common setting.
I know exactly how well that works for me in extremely challenging circumstances, and I know that the iso 109 I am using really acts like 109 but with cleaner shadows.
I have to get back to the editor. Hopefully something good will come from this, I know that I will be using it!
-
Not by my computer atm. If running yesterday build I think stock iso 400 equals to setting iso to 100 in submenu. Leave out 12/10 bit.
If running todays build I am now focusing on getting continuous shooting rather than trying move isos around.
A1ex is right. Lowering analog gains compensating with pushing higher isos will only move base iso etc to another place. Getting cleaner files will need tweaking of other registers. The analog gains however is still a major player when getting continuous to 1x3 modes and modes that doesn't work with regular 10-14bits workflows.
-
I also don't know what Danne has done "under the hood", and so it may be that the test is invalid for actual ISO comparison.
Analog gain lowered and CMOS[0] overridden for higher ISOs; see his commit.
For example, ISO 100 presets are:
- with his "iso100" option enabled: CMOS[0] as for ISO 200, analog gain 512
- with "iso100" + "12-bit": CMOS[0] as for ISO 400, analog gain 250
- with "iso100" + "10-bit": CMOS[0] as for ISO 1600, analog gain 60
I believe you have use one of the last two presets, just not sure which one.
In my tests, reducing analog gain only improves things by about 0.1 EV.
In other words, I'm pretty sure the results you were getting were 0.1 EV actual improvements, and 1.5 ... 3 stops were misunderstandings (possibly coming from the mislabeled ISO values in crop_rec menu). I wanted to prove it on your own images, but that requires identical test conditions for each setting (which you did not provide).
You made a very strong statement:
Ok guys, here's the proof everyone needs...
yet, you did not provide a valid proof.
I'm a happy camper getting excellent results, and the last time I checked, that's what really matters...
Yet, this particular topic (ISO tweaks) was about getting results *better* than with stock Canon ISOs.
My argument is that, in Danne's build, the improvement is about 0.1 EV over stock Canon ISOs (according to my measurements). In your example, the shadow difference is much stronger than that, but I'm pretty sure that difference is *not* an improvement over stock Canon ISOs (except for that 0.1 EV, which is not going to be noticeable without careful examination).
There's a bit of loss of highlights, but for practical purposes it is negligible.
Pretty sure the loss is about 1.9 EV if you have used the 12-bit "iso100", or 3.9 EV if you have used 10-bit "iso100". Clearly negligible ;)
-
My statements were not in the right context. My fault.
I was referring to my experiences with reduced gain by hand, not with Danne's experiment. I have only used it to take those test shots. It seemed like it was behaving the same way, which it may indeed, but I was going by my impression of how it behaves the same way under the familiar test conditions.
It may be that on a statistical level, the reduced gain is only marginally better, but in practice, I find it vastly superior. There's still noise in the shadows, but it's more like film grain, and it is usable when pushed hard. I can't do the same with stock ISO.
It works for me.
-
I was referring to my experiences with reduced gain by hand, not with Danne's experiment. I have only used it to take those test shots. It seemed like it was behaving the same way [...]
No, it does not.
Besides, your test scene is not ideal for comparing different ISOs, simply because the clipped area is extremely small. On your test image, whether you use ISO 100 or 400, it doesn't make a big difference in highlights (the clipped area was close to 0.1% in both cases). This is exactly why you said:
There's a bit of loss of highlights, but for practical purposes it is negligible.
Yet, the difference in highlights between these two ISOs is 2 full stops. In your test scene, these two stops are on the tiny light bulb; that's why you found the highlight difference to be negligible. In other scenes, the last 2 stops may cover a very large area of the image.
I'm still asking you to capture a full set of DNG images, with the same exposure time, on your test scene, at the following settings:
- stock ISO 100
- stock ISO 200
- stock ISO 400
- stock ISO 1600
- Danne's ISO 100 (all 3 flavors)
- the preset you use with manual tweaking from iso_regs
I bet you'll have a little surprise at ISO 1600. You'll be tempted to try 3200 and 6400. Not joking.
Caveats:
- enabling both crop_rec and iso_regs at the same time does not work; if you enable both and then disable one of them, it won't work either.
- enabling Danne's crop_rec will alter the shutter speed; to cross-check with iso_regs you may have to use the shutter fine-tuning feature.
Just make sure shutter speed is set to Original (not Full Range) in crop_rec menu.
-
- enabling Danne's crop_rec will alter the shutter speed; to cross-check with iso_regs you may have to use the shutter fine-tuning feature.
I wonder why it messes with the shutter speed and was wondering if it could be the case that if 'full range' option is selected in crop_rec, it messes with shutter time. And could it be that if this option is set to 'original' it won't mess with shutter speed ?
-
Since there is still some interest in doing tests around my lowering gain builds I reupload them as they were published a few posts back:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_isogain_1x3_5D3113_presets.zip
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_isogain_1x3_5D3123_presets.zip
Feel free to do the tests suggested by a1ex two posts above but be sure to use the builds above. No time personally atm for testing.
I also uploaded two other versions which works pretty similar but with a few personal rearrangements which are to be considered a work in progress:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_isogain_1x3_5D3113_presets-ver2.zip
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_isogain_1x3_5D3123_presets-ver2.zip
On a sidenote it would be cool if we now could continue testing pregain etc and see if we could get it rightly done. I have to little experience in this field but got a few hints and pointers here:
https://www.magiclantern.fm/forum/index.php?topic=10111.msg208037#msg208037
-
I wonder why it messes with the shutter speed and was wondering if it could be the case that if 'full range' option is selected in crop_rec, it messes with shutter time. And could it be that if this option is set to 'original' it won't mess with shutter speed ?
That's correct. My bad, forgot this setting enabled.
On a sidenote it would be cool if we now could continue testing pregain etc and see if we could get it rightly done.
Keep an eye on the iso-research (https://bitbucket.org/hudson/magic-lantern/commits/branch/iso-research) branch.
My experimental modules (https://builds.magiclantern.fm/modules.html#iso-research) (iso_regs, raw_diag and adtg_gui) are compatible with crop_rec_4k and derived builds (including Danne's build), btw.
-
Have been trying to get 1x3 binning mode to work with increased height on the 100D the but have been running out of ideas. No matter how I tweak registers I cannot move above 1736x1188 before I get either corrupted recordings, every other white, the next contains image content or if going higher I get completely white live view image.
Commit here:
https://bitbucket.org/Dannephoto/magic-lantern/commits/a6170f148121d281b9b17a807f264f7afce038d3?at=crop_rec_4k_mlv_snd_isogain_1x3_presets
Now I tried changing around a lot in raw.c. Testing excluding height-- among other stuff but this don´t seem to help.
Is there anything in the timing registers that could be tweaked or did I hit roof at height 1188?
-
Unsure if it could help but you might try to define timer A+B (https://bitbucket.org/Dannephoto/magic-lantern/src/a6170f148121d281b9b17a807f264f7afce038d3/src/fps-engio.c?at=crop_rec_4k_mlv_snd_isogain_1x3_presets&fileviewer=file-view-default#fps-engio.c-253) to it's orig value. Maybe my customisations interfere somehow.
-
Might very well be. SOrry but what are the timer A+B original values?
Tried this but not really helping:
#elif defined(CONFIG_100D)
#define TG_FREQ_BASE 32000000
#define FPS_TIMER_A_MIN (fps_timer_a_orig)
#undef FPS_TIMER_B_MIN
// no need to cause confusions as recording speed cannot handle such high fps in crop mode
// (ZOOM || MV1080CROP ? 1288 : 1970)) <-- these are ok while not recording.
// Hybrid CMOS AF II uses 60fps by default in LV/MV for the camera display
// to achieve a "snappy" autofocus by doubling the fps
// MV720 is not LV so we need to extend the definition for the LCD.
#define FPS_TIMER_B_MIN (fps_timer_b_orig)
-
fps_timer_a_orig
fps_timer_b_orig
-
Hello all,
I was finally able to borrow a 5D3 for testing the latest very exciting 1x3 recording developments but only for a very short time - I have to return it tomorrow. Nonetheless, I thought, it might help if I provide some feedback on what I was able to test in that short time.
I tested briefly the two 17fps options in Danne's build:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_isogain_1x3_5D3113_presets-ver2.zip
and all I can say is that this is another break through for Magic Lantern. I would have never dreamed of shooting at 400/3200 Dual ISO at 1,0 crop factor and get usable results. Not only are they usable and quite clean, but also look very natural compared to previous techniques for improving dynamic range which in most cases provided very HDR-ish looking images. In my opinion, this 1x3 technique, in combination with Dual ISO, provides a radical solution to low-light shooting high-quality FHD video in RAW without or with very little noise in the shadows. I don't think, too many of the professional cinema cameras can do that. Congratulations to all of you making this possible!
Also, I greatly appreciate Danne's efforts to provide continuous recording options, even at the expense of reduced frame rate. There is nothing more annoying than early camera stops in the middle of a shooting. And I don't think that 17 fps is that much of a problem. Human eye accepts everything above 12 fps as motion and earlier silent movies were shot at 16 fps. I shot a wedding film recently that I filmed at 18 fps and nobody noticed that. Nowadays we also have quite good optical flow techniques. Of course, if you do a lot of camera motion, (slider, gimbal, flycam work, etc., requiring fast camera motion) you may end up with some choppy results but there is a solution to that too. Motion should be slower for 17 fps, compared to 24 fps, and as smooth as possible, so training is required. If you shoot from a tripod, monopod or even handheld and do not shake the camera too much, results are indistinguishable from 24 fps and the motion blur of moving objects in the scene really looks nice. So, 17 fps is not necessarily a bad thing!
I didn't have time to test all the options, especially the 10/12-bit ones at different ISOs in the crop menu - this requires a lot of time and very systematic efforts to pixel peep all combinations and select the ones that provide the cleanest results with the highest dynamic range. I hope that other 5D3 owners will jump in with more test results very soon which will be of great help to the developers. However, I did test some of the high-resolution crop modes (I use them all the time, especially with the 100D) and am sorry to say that they are all broken in the above build. The 3k crop mode that I have been successfully using to film the wedding at 3072x1728 resolution and 18 fps does not seem to be working at all. It does not allow me to set any resolution but stays all the time at 0x0. The 4k and Full-resolution Live View modes seem to be kind of working but the results are horrible - a lot of aliasing along horizontal and vertical edges, noise and loss of resolution - both, in the normal and Dual ISO modes. Camera operation in those modes is very unstable either - lots of camera freezes requiring battery pulls. These high-res crop modes used to work very well even in the very early builds from April 2017.
At this time I would make the following suggestion to A1ex and the other developers: Please try to implement working 17 fps presets from Danne's build above into one of the experimental 5D3 builds. Please use the July 4-th build #78 that you can find here:
https://builds.magiclantern.fm/jenkins/view/Experiments/job/crop_rec_4k/
The later ones, including the July 22-nd build, do not seem to be working properly - lots of instabilities. Once such an experimental build works properly and is stable enough, I am sure many professional 5D3 video shooters will start working with it and will greatly appreciate it. This is because they will have a very powerful tool in their hands that they will use for a long time in their work. This is the dream of every video shooter, including me. I would like to see a build that can provide the following modes:
1) Normal 1,0 crop factor FHD (1920x1080) RAW video with maximum quality and minimum noise that can be used in low-light conditions with Dual ISO. The 17 fps presets already do that;
2) 3x-crop FHD (1920x1080) RAW video (1:1 mode) that will allow me to quickly change the focal length of my lens from 1 to 3x on a push of a button without changing lenses or using heavy zooms. A 35/F2 IS lens is perfect for that and I use it all the time switching from 35 to 105mm focal length.
3) A 3k mode that I can use at 3072x1728 resolution and 18 fps for continuous recording (also with Dual ISO). This one is perfect for capturing scenes with a lot of fine detail (group shots with 50 or more people on them, landscape videography, etc.). Here is a film of the ancient city of Plovdiv that I made recently and that was filmed in that mode with the #78 build:
https://cloud.mail.ru/public/Grpx/iLK4KHaKi
(To watch it in the highest quality, please download it with the "Скачать" button from the top left of the above link).
4) A Full-resolution Live View mode with up to 5 fps continuous recording for smooth timelapses.
5) The usual 50/60 fps modes
For the beginning, 14-bit lossless compression would be perfect but wherever 13, 12 or 10 bit lossless could be implemented, would be very welcome to reduce file size.
And one final thought. Now that we have all the above functions more or less working and ready to use and we are close to the ultimate quality ML RAW video, in my opinion, future research should be directed towards increasing the interface bandwidth for higher frame rates and/or higher continuous recording resolutions. This can be done in two ways:
A. Reimplementing card spanning on the 5D3. I don't know why this extremely useful function was removed;
B. CF-card overclocking in a way similar to SD-card overclocking on the 100D - a God sent function.
It also would be very useful to create similar builds for other camera models such as the 100D, 7D, 650D and others. The 5D3 is still quite expensive and the majority of ML users, including me, use the other camera models. We all would love to see a long time awaited update to our cameras too in a way similar to what Bilal did with the 700D already.
Please let me know what you think about my suggestions.
-
Thanks for feedback IDA_ML. I checked and surely a lot of presets were broken. I readded all my presets to the crop_rec.c from crop_rec_4k branch and added it to my branch instead. Seems to work better:
https://bitbucket.org/Dannephoto/magic-lantern/commits/8846065dfd242f82de3942cc3295f5f33225f7e4
Downloads:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_isogain_1x3_5D3113_presets.zip
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_isogain_1x3_5D3123_presets.zip
In addition to 1x3 presets and isostuff in the submenu following is added:
- Added changes for 1920x1080 48 fps
- Changed following registers:
adtg_new[8] = (struct adtg_new) {6, 0x8179, nrzi_encode(fps_timer_b - 5) }; /* PowerSaveTiming OFF (5D3/6D/700D) adtg_new[9] = (struct adtg_new) {6, 0x8197, nrzi_encode(fps_timer_b - 5) }; /* PowerSaveTiming OFF (5D3) */
One very annoying bug is when camera mode is set to mv720p. Zooming will require battery bull when trying to get out of x10 zoom mode after previewing. No idea why it gets stuck. Happens with crop_rec presets.
-
In addition to the previous bug I also noticed that "framing" preview isn't working when hdr is set. Anybody can confirm? Known issue a1ex? Really useful working when using 1920x1080p 48fps.
Also uploaded builds which should be more stable shooting with above mentioned setting.
-
Danne,
I now remember one more bug that I noticed when I had the 5D3 in my hands but just forgot to report it in my post #1954 above. It concerns "Full-resolution LiveView: 5796x3870 at 7.4 fps" which is very useful for shooting smooth timelapse sequences at low frame rates using the full sensor resolution. When activated, this mode records just one single frame and then camera stops. This behavior is observed at any resolution set. In older builds (those from April, 2017) this Full-resolution LiveView mode was working just fine, providing continuous recording at 5 fps as also stated on the Experimental page. A fix should be greatly appreciated.
-
Now you mention it, I've made a full resolution live view crop preset for 6d.
And it behaves the same as the 5d3, only one frame is recorded.
The 6d has about 250Mb of free buffer, so you would at least expect to fit about 7 uncompressed frames in that buffer ??? (Not sure when lossless kicks in, is that before or after the buffer ?)
-
Hi Danne,
Can I use, your latest crop_rec.c to eos m if I build your "crop_rec_4k_mlv_snd_isogain_1x3_presets" branch build to eos m?
thank you for Your hard work! :)
-
For eosm it's the same presets as with crop_rec_4k.
-
Danne,
I have been inspecting the files that I shot on the 5D3 with the latest 1x3 crop modes recently and found that now the 5D3 is one BIG step closer to perfection. The Crop ISO (CISO) settings work so well in providing clean shadows that the results are really amazing. Here are a few more minor issues that need to be fixed to get the dream 5D3 video shooting build:
1) The 6400 CISO setting still has vertical stripes in the shadows. If these could be fixed and just the noise is left, this high CISO setting may be a life savior in some really dark situations, especially keeping in mind that Resolve does a pretty good job with noise cleaning. Please note that the CISOs 3200 and lower settings are perfect already - no vertical stripes at all.
2) MLVApp does not recognize and report the correct CISO at which the clip was shot. It reports Dual ISO 100/800 all the tme, regardless of what CISO was used.
I also have a few questions:
I. We now have the possibility to use the normal FHD MLV RAW video mode with Canon's ISOs and the mv1080p_ mv720p mode with CISO and Canon's ISOs. What exactly is the difference between these two modes and does one of them provide advantages vs. the other in terms of video quality (fine detail, dynamic range, etc.)?
II. Do the CISO settings apply to all crop modes in the Crop menu? Will they clean the Shadows of the high-resolution modes in the same way and strength as they do with the mv1080p_ mv720p mode?
III. I have noticed that it is possible to activate several CISO settings at the same time - e.g. the 200 and the 1600 setting. Which one will be applied in this case? Could these clean CISOs be used to also make Dual ISO with two CISO settings simultaneously active?
IV. This question is for A1ex and is actually a tool request. Would it be possible to add an indicator showing how much space is left on the CF card while recording? In the 1x3 crop modes the 5D3 eats card space for breakfast and you never know when you will run out of card space.
V. I understand that there are problems with overclocking the CF-card interface. Would it be possible to overclock the SD-card interface instead? If it could be overclocked to say 30 MB/s and card spanning could be applied, then the 5D3 would write at a speed of about 150 MB/s and this would significantly increase continuous recording times at the high-resolutions and continuous recording at 24 fps at the 1920x3240 1x3 mode would be possible.
VI. Danne, could you please provide some more information on how to use the "Shutter range" and the "Advanced options" in the Crop menu? In what situations should those be used?
Thanks.
-
Since my iso experiments was built on faulty assumptions about how analog gain registers and analog iso registers were related I decided to simply erase those code parts in following builds:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_isogain_1x3_5D3113_presets.zip
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_isogain_1x3_5D3123_presets.zip
What still is valid testing and developing is 1x3 presets together with playing around with analog gain registers creating 10/12 bit. Those two settings can still be found in the sub menu of crop mode.
In a longer perspective iso tweaking is still valid and could be developed further I´m sure. I might dig into this and revive some code parts in crop_rec later on but without mislabeling menu items.
Commit:
https://bitbucket.org/Dannephoto/magic-lantern/commits/4e18649356067e49e15134e432025acdb2b6f0a7
-
Since my iso experiments was built on faulty assumptions about how analog gain registers and analog iso registers were related I decided to simply erase those code parts in following builds:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_isogain_1x3_5D3113_presets.zip
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_isogain_1x3_5D3123_presets.zip
What still is valid testing and developing is 1x3 presets together with playing around with analog gain registers creating 10/12 bit. Those two settings can still be found in the sub menu of crop mode.
In a longer perspective iso tweaking is still valid and could be developed further I´m sure. I might dig into this and revive some code parts in crop_rec later on but without mislabeling menu items.
Commit:
https://bitbucket.org/Dannephoto/magic-lantern/commits/4e18649356067e49e15134e432025acdb2b6f0a7
@Danne
Can you compile crop_rec_4k_mlv_snd_isogain_1x3_presets_2019Feb14 also for 5d3 123 ?
thanks lot !!!
-
THANKS!!! :)
-
Continueing from here:
https://www.magiclantern.fm/forum/index.php?topic=19300.msg216898#msg216898
Worked on my 5D3 crop_rec.c code in this branch:
https://bitbucket.org/Dannephoto/magic-lantern/branch/crop_rec_4k_mlv_snd_isogain_1x3_presets
Commits:
https://bitbucket.org/Dannephoto/magic-lantern/commits/c89688d60f2c0da642c9460deb1cf1cf79553148
https://bitbucket.org/Dannephoto/magic-lantern/commits/0d200f7cb6d6158d83d38c9092e4bcdef208d76f
- Refined anamorphic presets so it now gives continuous:
1808x2300(2:35.1) = 5425 x 2300
1360x2300(16:9) = 4080 x 2300
- Also adressed the battery pull issue happening when in 48fps or mv720p 3x3 mode. Workaround allows for looping through x5,x10 and back into x1 mode.
- cine.lua included
- set_25fps in crop mode sub menu working with anamorphic mode
- bitdepth modes in crop mode sub menu working for all presets
- not included sd_uhs(don´t see why sd patching would be needed with mlv_lite anyway)
Downloads:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_isogain_1x3_presets_2020May27.5D3113.zip
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_isogain_1x3_presets_2020May27.5D3123.zip
In terms of image quality and performance the anamorphic mode on this camera outruns anything coming from eosm, 100d etc...
-
@Danne
modules loaded: crop_rec, mlv_snd, mlv_lite, mlv_play
res: 1920x1080p /14bit lossless (not crop mode)
CF (KB1066x) card seems to be working/saving fine when I revert to a previous 4K_crop-rec Danne build 2/4/19 :)
Still odd…
Try fomatting card in camera (non exfat) reinstall clean build.
-
now it works im not sure what i had done in my first test. now i just clean install and turn on the anarmophic preset in crop rec and no more greenish black preview. i have a very slow 4gb cf card atm. wil test more as soon a i recieved another card
Yes, bleeding edge stuff. Faster card will probably help. I set regs all over the place. A safety press on menu button and thrashcan once in a while to settle all regs won´t hurt too.
-
Try fomatting card in camera (non exfat) reinstall clean build.
Tried this. Get a Canon menu error:
Cannot format
Change card1
Yes, am saving to the CF card (1)
Still getting CARD FULL in top left of ML menu when I press record in any res. My CF card has been faultless til today.
Files into MLV App produce error: 'File header is missing: invalid MLV" or "file length too short Invalid MLV'
-
Sorry to hear but I suspect this is the reason but I can´t say for sure why it would break your card:
https://www.magiclantern.fm/forum/index.php?topic=19300.msg216884#msg216884
Tried formatting your card to fat32 on computer and then format in camera?
-
That I copied only the crop_rec.mo file?
I've done that before without issue…
Have I tanked something?
-
Tried formatting your card to fat32 (or fat, can´t remember exactly what it´s called) on computer and then format in camera?
Well, I can´t say what would work or not here. It´s all guesswork atm.
-
Tried formatting your card to fat32 on computer and then format in camera?
Well, I can´t say what would work or not here. It´s all guesswork atm.
Yes! See four posts above. :)
-
Your answer indicates formatiing in camera but not on computer to fat. Do you have another card you can test my build on? If that´s working clearly your card is having an issue.
-
Oh right, sorry. Did you mean FAT32 instead of exFAT? Thought it needed to be exFAT?
-
Oh man, we need @Water Schultz on this one. Find one of his rescue procedures and see if that will work.
-
I'm going to revert to previous build and see if I can get back the status-quo!
-
Clean format of KB1066x CF card in mac to exFAT
Clean re-install of previous 4K_crop-rec Danne build 2/4/19 on SD card
All seems restored and working – except for one shot which didn't respond to stop button. Restarted camera, shot a few more tests at various res. All import working into MLVApp.
Tried a different Sandisk Extreme CF card with current state – all is well.
Totally weird! :o But very happy my camera is still alive! :)
-
Clean format of KB1066x CF card in mac to exFAT
Clean re-install of previous 4K_crop-rec Danne build 2/4/19 on SD card
All seems restored and working – except for one shot which didn't respond to stop button. Restarted camera, shot a few more tests at various res. All import working into MLVApp.
Tried a different Sandisk Extreme CF card with current state – all is well.
Totally weird! :o But very happy my camera is still alive! :)
Still very unclear but I would say nothing is wrong with the ml builds. One thing is for sure. I would never trust that KB card.
-
I would never trust that KB card.
It's been rock solid til now – but today I'm starting to wonder… :( I had a Lexar before that which was a pile of poop!
-
Try a windows formatting:
https://www.magiclantern.fm/forum/index.php?topic=5473.msg171046#msg171046
Seems different card readers can affect too. Doesn´t really matter that it worked perfect until today. Suddenly issues starts to happen.
-
Try a windows formatting:
Didn't know that – interesting! Will look into it, but I need a PC!
Does anyone know when formatting on mac whether the eXFAT scheme is crucial? (e.g. GUID, Master Boot Record, APM)
-
Sorry, haven't much time (and it doesn't look like I can help).
Formatting with ExFAT will enable Exfat on low capacity cards where Canon's firmware denies to apply Exfat within its own format routine. A1ex has a patch for it.
But I'm afraid it will not rectify the situation with given card. I had CFs going south without rhyme or reason. No expertise in flash CPR, sorry.
-
i can increase fps override til 39fps in full HD without any corrupt/pink frames
-
Hey @andy kh with below mode you can do 45 or 48 fps mv1080p depending on if in 50fps(pal) or 60fps(ntsc) mode.
1080p45/1080p48 3x3
Worked before with 48fps 1920x1080 but reverted back to 1920x1040 because om pink frames.
-
Posted two new builds:
https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899
Mainly I separated 45/48fps modes into two and they run in either 50 or 60p mode. Upscaled 48fps to 1060p:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.postimg.cc%2FzB9rF1w7%2FVRAM0-PPM-scaled.png&hash=181458eb29e8f69513c42c9cb85b5d7b)
-
@danne problem wit 45/48fps is the live view. If i set to real time framing is not correct and if i set to framing live view is not in real time so very difficult to shoot a moving object in practical.
If i increase the fps in 1080p mode i get real time preview wit correct framing. There may be a way to increase the fps beyond 40
-
Wil try your new builds tomorrow as im away from home atm
And today i get greenish black live view while recording wit your yesterday's build. I was shooting 3.5k.
-
@andy kh
Thanks for report. Found the issue and now fixed. New builds posted just now:
https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899
Commit:
https://bitbucket.org/Dannephoto/magic-lantern/commits/178845d7d86e8b078a0d1345e05f4e35ee209fb2
Do note that preview will look a bit greenish because of the workaround to get bitdepth reduced with analog gain regs. Shouldn´t be almost black anymore though and image files will look good.
-
Do note that preview will look a bit greenish because of the workaround to get bitdepth reduced with analog gain regs. Shouldn´t be almost black anymore though and image files will look good.
the live view gets much darker as soon as i hit the record button making me difficult to shoot outdoor but not completely black anymore
-
Well from now on I expect some deeper feedback and testing. I will check into it myself som more though. I do see some extra greenish stuff happening. WIll look.
MAny of the presets can be run with the ordinary implementation of bitdepth changes so if that´s the case it will solve workaround issues.
-
@danne problem wit 45/48fps is the live view. If i set to real time framing is not correct and if i set to framing live view is not in real time so very difficult to shoot a moving object in practical.
very strange as framing in real time works only in ntsc . i have been testing in pal. today i tried both and found that framing in real timein pal is not correct
-
Are you referring to 45/48 fps presets? They are not gonna show the correct full mv1080p real time live view. They will behave the same both in pal and ntsc so don´t know what you are getting. Please film your screen or do more tests and explain further in detail.
Real time limitations known already since building the preset being in mv720p mode will yield faster fps.
-
now i understand
i tried in 60p and i get 1920 by 800 whereass in pal i get 1920 by 818 so in 60p the bottom is crop by 18
and its very near to what i see in real time
now i can get exact framing in any mode by goin to reg_skip_botton and increase the number by 350
-
Well increasing reg_skip_bottom also removes resolution.
Uploaded two new builds. Fixed some testing stuff I forgot to erase which affected mv720p modes.
Trying to fight the greenish tint but the regs I use for this on eosm and 100D is not very effective on 5D3. Will take a deeper look into this when I can.
-
New uploads for the 5DIII:
https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899
- Fixed a bug when recording 50/60fps
- Better handling mv720p mode 45/48 fps when going in and out of x10 zom mode
-
New uploads for the 5DIII:
https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899
- Fixed a bug when recording 50/60fps
- Better handling mv720p mode 45/48 fps when going in and out of x10 zom mode
simply THANKS Danne!!! ;)
-
You´re welcome.
-
Is it already possible to use magic zoom in full screen mode on 5D2 ?
-
Ask in a thread specific to 5D2.
-
just install this new build and tried shooting some slomo video. working fantastic so far.
thank q so much for this new build danne
-
Hey man. Thanks for reporting bugs and such.
-
i stil cant shoot in 50fps with your latest build. 60fps is fine. i have been shooting 60fps for the last couple of hours without issue. just tried 50fps this evening and i get some error which is very strange. if i restart the camera canon setting jumps to ntsc from pal. i tried setting to pal again and shoot at 50fps and whenever i get this error i try restarting and jumps to ntsc
http://ibb.co/ygDJBJt
-
New uploads for the 5DIII:
https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899
- Minor change. Maybe it affects 50fps mode positively. Can´t test, no cam here.
Changing between PAL and NTSC is like you describe sometimes not really changing and goes back on restart. Because of this I want to separate 45/48, 50/60 fps to independent presets. This means they can be run in either ntsc or pal as long as camera is set to mv720p. 45/48 is already like this but havn´t had the time for the other two.
Anyway. When I tested myself before it seemed to work. Are you using 5D3.1.1.3? Maybe reinstall?
If people are testing please take alook at 50/60 fps mode and report back if working or not.
About your screenshot it´s related to compression handling in mlv_lite.c. Something is affecting it so it correctly reports files are getting corruption issues.
And as always. Changing presets also likes pushing canon menu button to settle regs...
-
New uploads for the 5DIII:
https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899
stil the same problem. sometimes it works fine. the problems occurs after recording 3/4 times
About your screenshot it´s related to compression handling in mlv_lite.c. Something is affecting it so it correctly reports files are getting corruption issues.
i get som corrupted pink frames
i dont understand why there is no such problem with your 2nd Apr build!!i have been using it for so many days and i havnt found any problem so far
-
2nd april from my branch/builds?
What firmware used 1.1.3 or 1.2.3?
Could you share exact settings?
-
yeah 2nd Apr if i remember from your branch/build. it works perfect
im on 1.1.3
my settings are
pal in canon menu
720p@50fps
activate crop mode 1920 50/60 3X3
edit : i set to 1920X818 spect ratio 2.35:1
14bit lossless
real time
-
Could you test this version of mlv_lite together with the latest build?
https://bitbucket.org/Dannephoto/magic-lantern/downloads/mlv_lite.mo
mlv_lite from 2nd april. Want to know if it´s related to something in there or in crop_rec.c.
-
no it doesnt work
-
Doesn´t work at all or work and gives compression errors? I need my camera I believe...
-
Doesn´t work at all or work and gives compression errors? I need my camera I believe...
gives compression errors.
-
Posted two new builds:
https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899
Mainly I separated 45/48fps modes into two and they run in either 50 or 60p mode. Upscaled 48fps to 1060p:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.postimg.cc%2FzB9rF1w7%2FVRAM0-PPM-scaled.png&hash=181458eb29e8f69513c42c9cb85b5d7b)
New build anamorphic test in5Diii 123. Thank you
-
New uploads for the 5DIII:
https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899
- Separated 50p and 60p into two separate presets same as 45p and 48p. Means they will work both in PAL or NTSC mv720p. Also modified numbers some. Seems more resistent from compression errors in 50p mode now
Commit:
https://bitbucket.org/Dannephoto/magic-lantern/commits/e6f87460df08f511b2f3790bdaa49a62d8f2623e
Thanks to andy kh for reports.
-
Awesome!
Is it possible to have these presets (45/48/50/60 FPS) in 1920 1:1 ?
-
Might be. I could have a look in implementing the x3crop in Crop mode sub menu similar to how it's done for eosm.
-
Awesome!
Is it possible to have these presets (45/48/50/60 FPS) in 1920 1:1 ?
mv720p mode and x3zoom seems restricted to reduced height like what you get when in mentioned 1920 1:1 mode. I have a hard time figuring out a1ex pack modes on cmos regs but maybe once those numbers are "decrypted" we can revert to some more obvious numbers and start constructing more freely.
EDIT: Nudged cmos1 and some other regs and seems I´m on to something. Stay tuned. I post when I gt something working.
-
New uploads for the 5DIII:
https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899
Commit:
https://bitbucket.org/Dannephoto/magic-lantern/commits/ddd1c56678d2601be04091bb6fe8b08a63dfb838
Key are these two regs combined or resolution looks like sheit:
adtg_new[3] = (struct adtg_new) {6, 0x8000, 5 + reg_8000};
adtg_new[17] = (struct adtg_new) {2, 0x8806, 0x6088};
- x3crop now works with higher frame rates 45/48/50/60 fps. Simply be in one of the mentioned presets in mv720p and then select x3crop from within Crop mode sub menu:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.postimg.cc%2FTwR53Yq8%2FVRAM0-PPM-500px.png&hash=81c2382e5110f091659a1c0fe107f22b)
-
:P
-
done a quick test so far so good
great job danne
-
New uploads for the 5DIII:
https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899
- x3crop toggle working for 45/48/50/60 fps modes. Function same as for eosm here:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg218141#msg218141
-
Hi Danne, just to say thank you for the work you do (and all FM devs of course), I've got a 5dmkIII and last week I needed 720p footage at 48FPS, so I've just overwritten my previous ML 113(experimental, the 4K raw video recording 22-7-18) and the 2 hours shooting (not continuous) went all great!!
I'm going to update in the next days as long as you go on coding...
-
- Refined anamorphic presets so it now gives continuous:
1808x2300(2:35.1) = 5425 x 2300
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_isogain_1x3_presets_2019Jul02.5D3113.zip
Wow, this one is awesome! Works without problems.
Hope, sometimes we'll have a realtime liveview to have a possibility to shoot moving objects, not only static frames from tripod.
Thanks for your job!!
-
Yes, 5D3 can produce some heavy resolutions.
There's real time preview by the way. If in framing mode press halfshutter for cropped real time preview.
-
There's real time preview by the way. If in framing mode press halfshutter for cropped real time preview.
Yes, I understand what you mean, but what is a real purpose for such a preview? The proportions aren't correct, we can only see a massively stretched top part of an image.
-
Part of the workaround technique. Framing is done from real buffer very slow and focusing can be done while holding halfshutter while recording. Then back into framing again. In and out. I can work with that and capture pretty much anything I want.
-
Yes, I understand what you mean, but what is a real purpose for such a preview? The proportions aren't correct, we can only see a massively stretched top part of an image.
Danne, for me continuous recording in anamorphic 1600X2332< 16:9! :) :) :) another way round to preview framing before recording is press hold the AF Lock button (the lens should be in AF Mode)Cheers...
-
Yes. That should work just fine but not really 16:9 when upsampled in post :).
-
I am not certain if it is possible, to drop resolution for a higher FPS.
For instance 1280x720 @ 96FPS
-
Is it possible to enable proxy recording from 1920 3x3 preset?
-
What exact preset are you referring to?
When using proxy recording function also make sure you are selecting 2nd card(SD) in canon menu. This way proxy goes to sd and raw to CF.
-
I've tried your latest build with 2:35.1 10bit lossless 25fps (i've got 1720x2200 px).
When I enable proxy - it's stretched and cropped. I understand, that it's somehow made from 1x3 proportions, that's why it's not usable. So, I assume, aren't these two processes independent (MLV and proxy)? Can't we somehow use nice proxies from Alex's build 1920 (3x3) with your 1x3 raw 1720x2200 px?
-
Na, that would mean a corrected liveview output parallell to shooting raw. The holy graal if you so like ;)
-
Hi, Danne!
I'm sorry, maybe it was mentioned before, but I cannot find an information about what your lua script exactly do? I mean step-by-step
-
Why not test it? It sets optimised settings for anamorpic 2.39:1 recordings.
-
Why not test it? It sets optimised settings for anamorpic 2.39:1 recordings.
I did try. But I shoot mostly with 2.35:1, so I want to be sure, if I change 2.39 to 2.35, I won't mess anything else.
-
No. That's fine. Once you run the script you are free to change anything after.
-
Once you run the script you are free to change anything after.
So, what the script is actually doing? Can I do the same manually from ML menu?
I'm just curious, it would be nice to read some description of it. But if you're busy, don't bother ;)
-
Yes, manually. It's only a helper script to get going. Originally created eosm on mind.
-
Anyone knows how to record continuous at 14-bit lossless at 1440p 24fps? I get pinky/black preview on my LCD screen.
These are my selected settings:
Global draw: liveview
fps override: 24 fps
crop mode: 4K 1:1 and x3crop set
RAW video:
2656x1494
16:9
14-bit lossless
preview: realtime
croprec preview: auto mode
kill global draw on
sound on at 48khz
Any tips on what I am doing wrong?
Using build 2019-07-20 on 5d mark III firm.123
-
Anyone knows how to record continuous at 14-bit lossless at 1440p 24fps? I get pinky/black preview on my LCD screen.
These are my selected settings:
Global draw: liveview
fps override set at 24 fps
crop mode: 4K 1:1 and x3crop set
RAW video:
2656x1494
16:9
14-bit lossless
preview: realtime
croprec preview: auto mode
kill global draw on
sound on at 48khz
Any tips on what I am doing wrong?
Using build 2019-07-20 on 5d mark III firm.123
-
With a slight modification of the aspect ratio (1:2 in raw menu and 2.35 in crop rec menu) and resolution to 1808x2300 in 14-bit lossless of the cinema lua script I managed to get automatically the following in MLVapp:
5424x2300
Just to make sure I have understood this script correctly. Does this signify that the streching by 3-fold 1808x2300 to 5424x2300 does not result in any loss of pixels? Meaning that pixels had been squeezed and then streched back making this true genuine 4k footage?
-
I've used the cinema lua script using an external monitor. The issue I am having is to see the lower end of my footage. I know that there is a stretching going on vertically, which is why I cannot see the lower end of the filmed footage. Is there a way to say shrink the entire frame so that the lower end can be seen? Any other solution?
-
1. iso climb: in menu it is mentioned INFO(5D3), but it does nothing
2. x3crop toggle (halfshutter press (5d3)) in Press down or Set mode: x10crop not compatible with x3toggle - but i'm not in x10crop
-
There is a misprint for iso climb: in menu it is mentioned INFO(5D3), but it does nothing
Is this related to the 2019July20 or 2019Oct13 build from @Danne's bitbucket download page?
-
2019Oct13.5D3113.zip
-
New builds 5D3:
https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899
- Removed a few features due to lack of time bughunting
Removed: x3 crop toggle, iso climb.
All good stuff still in there so shouldn´t make any difference.
-
Thanks a lot for all the good job you're doing.
I'm testing a lot of your builds, and this one will be tested tonight. I'll be putting a review.
Danne, if you need me to test something special feel free to ask. As I'm a developer too I'll try to clone and build your version myself.
-
Ok, sounds good. General tests of the presets please go ahead. Not all sub menu items will work here and I am not up to date with this branch atm since I put all focus lately into the eos m. If you and others can find bugs and start working on fixes just go ahead. I will help out with pointers etc.
-
I've just tried to compile on debian testing, the classic unified branch for 5DM123.
I had to comment out the include module_strings, because the file is not present and it gives a compilation error.
Make zip made me a zip, I do not yet have a working qemu to test it, it'll come soon !
-
Edit:
Everything is working so far, with your branch build on my debian.
I don't get how I can correctly fork your branch and see it on bitbucket. We'll see what to do on that.
-
Part of the workaround technique. Framing is done from real buffer very slow and focusing can be done while holding halfshutter while recording. Then back into framing again. In and out. I can work with that and capture pretty much anything I want.
I tried 2019Oct15.5D3113 and 2019Oct15.5D3113. This feature is very very useful but sometimes(mostly) does not work for me. When it works and after pressing half shutter, it backs streched view again.
When I run your Lua script, everythings are fine and recording is ok. Low fps and laggy preview but at least not streched, framing is good. (I can get 5424x2270 24fps 10bit/12bit continuous)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.ibb.co%2FY8bpTcH%2Fcropped.png&hash=b42e572bbcb072991daf6aa7031d1052)
When pressing half shutter for focusing:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.ibb.co%2FvHmvBLT%2Fstreched.png&hash=f7ba5b4cf065a615aa5e8d8f9366d770)
And it stucks there. I can't see correct framed version again. Sometimes works 1-2 times, then streched view again. Also freezing camera when change aspect ratio on crop_rec menu.
Am I doing something wrong?
Thanks.
-
.
-
I don´t know if you have x10 focus aid enabled. Try disabling that if it´s the case.
-
I don´t know if you have x10 focus aid enabled. Try disabling that if it´s the case.
Tried both on and off. Nothing changed. But for now i use zoom button 2 times then focusing. Not best solution and very slow, but works.[emoji3]
May I ask, do you use 123 or 113?
-
Screen recordings preferred that shows your issues. Too much guesswork.
-
Screen recordings preferred that shows your issues. Too much guesswork.
There is a video that i record. I hope it helps.
https://drive.google.com/open?id=1t4SrL51Cvcn_pFS0AusOpHd_CL3QuE15
Also my settings:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.ibb.co%2FXtzkcJh%2Fdanne-crop-rec-settings.jpg&hash=66e620060118e2962ea1840925a7b1f2)
I guess I'm missing something to do. :)
Thank you very much for your interest.
-
All is working over here. Just tested. Did you set Preview to framing? You find this under RAW video menu settings.
-
All is working over here. Just tested. Did you set Preview to framing? You find this under RAW video menu settings.
Yes its Framing.
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.ibb.co%2FRDqbHM3%2FDF514-BF2-ED51-4-DE0-99-CC-0173-F6-BB26-A8.jpg&hash=d30a628a5fc78e845ad2e2c5b468b371)
-
Hard to say what is going on but keep testing different settings. Maybe you find the answer eventually. I'm on 1.1.3 by the way.
-
Hard to say what is going on but keep testing different settings. Maybe you find the answer eventually. I'm on 1.1.3 by the way.
I guess i found the problem. It’s about adtg_ui module. I disabled the module and everything works expected.
Does this module use any code with your crop_rec? I tested and looks fine. Just want to know.
Thanks again Danne!
-
Ah, yes. It will interfere with framing. Good catch. I should remove this module as it´s only needed for when looking for registry nad development.
-
Ah, yes. It will interfere with framing. Good catch. I should remove this module as it´s only needed for when looking for registry nad development.
Or just leave it disable when installing. Enable when needed for troubleshoot and tests.
-
nothing warms my heart more than seeing progression here
-
@Danne
Thanks for new build 'crop_rec_4k_mlv_snd_isogain_1x3_presets_2019Oct15.5D3113.zip'
If I use with iso_regs.mo ON I get strange behaviour of liveview: e.g. in Anamorphic 1360x2300, the liveview image changes its centre point and becomes squished up. Turning iso_regs OFF and restart resets the liveview to normal behaviour.
Is iso_regs not designed to work with crop_rec.mo now?
-
Both adtg_gui and, also seemingly, iso_reg module, will affect outcome in some way. Have no idea why so feel free to take a look if you have the time.
-
@Danne Righto. Thanks. Maybe something's changed, as previous versions seemed to play nicely with iso_regs.
-
If so, try to narrow down the build where it starts to break.
-
@Danne
Reverted to an older build to try and trace errors with iso_regs. Have only got a backup of crop_rec_4k_mlv_snd_isogain_1x3_presets_2019Apr02.5D3113.zip which on my camera is buggy with compression errors. Errors go when I update to most recent – but then whenever iso_regs is enabled the crop_rec previews are all messed up.
Trying to find more recent builds from July19 (which were working well on my camera) on your downloads but can't find them. Have you taken them down?
-
I can't keep every build. I'd prefer users to take time to compile on their own so I guess if this issue is important enough to fix you simply learn to compile. No priority over here. Need to focus on other things. Good luck.
-
Ok. I hear you. Thanks!
-
@Danne I'm going to try to have a go and jump into ML dev. :o I've installed via your compiler.app as per here https://www.magiclantern.fm/forum/index.php?topic=21882.0
The example given at 7) is for 17 EOSM.202
Could you give me some pointers how I'd change this to recompile your earlier Jul22 4K_crop_rec build for 5D3113 ?
Thanks!
PS I have managed to compile today's build, but am stuck on how to find an earlier build!
-
Good. I didn't include jumping around in source tree in any easy way with compiler.app so it needs to be done in terminal. You can also work from atlassian source tree gui. In there you can see commits and doubleclick back in time.
Sorry, not a lot of time lately...
-
You can also work from atlassian source tree gui. In there you can see commits and doubleclick back in time…
Thanks!
Lots of stuff I don't understand there! I know you're time-poor at moment so could anyone else point out how I work in the source tree gui to compile a specific build?
Thanks
***UPDATE***
I've searched back to find the commit for Jul22 and have managed to download a zip of this by doing https://bitbucket.org/Dannephoto/magic-lantern/get/f0e391b.zip
Now I have this folder, I need to compile it into the final zip… How do I do this?
-
I've got Danne's "crop_rec_4k_mlv_snd_isogain_1x3_presets_2019Jul20.5D3113" build. If you need it I can upload it somewhere.
-
@DeadEyeJedi I have got my install working to the same level as seen in this video now. What I'd like to know is how to compile a build from an earlier commit date in Danne's repo. Do you or anyone else know how? :o
Cheers all!
-
If you downloaded and got into the correct branch you can still compile from compiler.app after you select a new historical commit. I advice to use Source tree:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.postimg.cc%2FTPgQQ0hY%2FScreenshot-2019-10-28-at-18-05-11-png-scaled.png&hash=09f4a82e8efd441d6b371c09efe30132)
-
@Danne
Thanks. I have sourcetree installed now.
I'm doing: Open > Remote > Clone a repository > https://bitbucket.org/Dannephoto/magic-lantern > Destination: /Users/me/magic-lantern
From here I don't know how to select the historical commit or compile. Any pointers?
Thanks.
-
hg update [name of the branch you want to switch on, without the braces]
make clean
make zip
It can complain about module_strings.h missing. On debian testing as I don't know very much yet about it, I commented the include and got it to compile.
I take pointers if any.
-
Hmm. Thanks, but that just gets me the current files.
In sourcetree:
1) I have cloned the branch to my local repo
2) Found the commit point (rev 19200) from which I want to compile
3) Image shows my options:
https://drive.google.com/file/d/1vLiEctKach9jIPcELzfzMGcjtdMwTJtx/view?usp=sharing
I think what I want is 'checkout' but I don't see that option – and I don't want to break the internet by pressing the wrong button! :o
I'm also not sure when I switch from Sourcetree app to Danne's Compiler.app to do the compiling.
Any help gladly received. Thanks.
Tim
-
1 - In compiler.app source tree needs to be here:
crop_rec_4k_mlv_snd_isogain_1x3_presets
2 - On top of Source tree gui in your picture where it says "All branches" also select crop_rec_4k_mlv_snd_isogain_1x3_presets
3 - Double click backwards on any commit back in time and source will get updated into the past revisions
4 - Go back into compiler.app and compile
-
Thanks @Danne :)
When I double clicked on the commit point the message said: 'Switching working copy' (Hope this is correct)
Then I compiled with compiler.app
The resultant compiled zip is labelled with today's date rather than the date of the commit. e.g. crop_rec_4k_mlv_snd_isogain_1x3_presets_2019Oct29.5D3113.zip
Is this correct?
-
New uploads for the 5DIII:
https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899
- Fixed a bug when recording 50/60fps
- Better handling mv720p mode 45/48 fps when going in and out of x10 zom mode
Is X3 back, compatible with 50/60 ?
-
Yes, I included x3zoom for these modes. Don´t remember how to reach them. Look in the Crop rec submenu if not present as a static preset menu item.
-
Hi Danne and everyone involved, been following a lot of your work , especially with the eos M. And i thank you all so much for it. Workinh wonders.
I just bought myself a used mark iii. And tried the anamorphic mode, its awesome and the noise is so much cleaner in low light. And image quality superb. Love it. So happy, thanks guys.
I have one question about the anamorphic build, using the latest, on a 1.2.3 firmware.
I tried to understand the problem by reading this thread all over twice but i might not fully understand. When i use the anamorphic mode , the liveview halfshutter x10 works great, and zooms into the center of the image. But the ordinary liveview is focused on a different area, the upper part of the frame, so i cant really know exactly where the middle point of my shot is. Is this a thing that can be changed or is it ”common” knowledge to use the method of combininh halfshutter back and forth to know where center point rly is?
Hope it makes sense
Kind regards
S
-
There's certainly room for improvements. Check out 2blackbar managing to find center spot for anamorphic mode on the eosm. Same should be doable for the 5diii.
-
New builds:
https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899
Added my tlapse scripts originally built for the eosm here:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg223238;topicseen#msg223238
-
Who needs Christmas or Santa when we all have @Danne's never-ending magic compiling upon on us every now and then.
Let's all make it snow for once and y'all get your hands dirty on these tests builds and please do report your findings.
Including myself. Especially on the 100D since this so-called DSLR definitely needs some love again. EOSM is just scary good!
Thank YOU for annihilating Santa as always w your charming talents annually, @Danne! :)
-
New builds:
https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899
- Added centered regs for UHD 1:1 preset(Thanks to Ilia3101 who tweaked these)
-
Wow! Is there a chance your anamorphic to be centered or it has nothing common with this?
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fbitbucket.org%2FDmytro_Kn%2Fml%2Fdownloads%2Fcine.jpg&hash=cf9f3216f308154ed09ae25e5906f028)
-
Yes, it should be fixed too. No time atm, maybe Ilia3101 could jump in here :P
-
Wow! Is there a chance your anamorphic to be centered or it has nothing common with this?
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fbitbucket.org%2FDmytro_Kn%2Fml%2Fdownloads%2Fcine.jpg&hash=cf9f3216f308154ed09ae25e5906f028)
New version:
https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899
- Rearranged cmos1 and seems more centered now for anamorphic modes. Needs testing.
-
Testing:
Anamorphic mode has a single horiz 1px(?) white line at base.
If you come out of crop mode, LiveView shows only half the screen.
Or it could just be me… ???
-
Thanks for testing. I uploaded two new builds. Hopefully fixed now. Please test anamorphic again.
-
Hopefully fixed now. Please test anamorphic again.
I have a green picture and a wrong information on top.
Here is a regular picture:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fbitbucket.org%2FDmytro_Kn%2Fml%2Fdownloads%2FIMG_6330.jpg&hash=5cbf0710aa75bbf32b67b7df1fcf833a)
Here is a 1920 preset:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fbitbucket.org%2FDmytro_Kn%2Fml%2Fdownloads%2FIMG_6330-1920.jpg&hash=acf7aa79336265466ea3c77b603fa590)
Here is anamorphic preset:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fbitbucket.org%2FDmytro_Kn%2Fml%2Fdownloads%2FIMG_6330-anam.jpg&hash=89979d59dc468778aee91a0c171da60d)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fbitbucket.org%2FDmytro_Kn%2Fml%2Fdownloads%2FIMG_6330off.jpg&hash=7cebc908c3b15e94d62bacf03d78df91)
-
1920 preset? I only changed for anamorphic mode. Did you push menu button and back? Fps set to 24 in canon menu?
-
1920 preset? I only changed for anamorphic mode. Did you push menu button and back? Fps set to 24 in canon menu?
1920 preset - is just for comparison.
I've just found - green picture in anamorphic with problem on top appears when Set 25 fps is ON.
It's ok when Set 25 fps is OFF
-
Ok, I'll take a look tomorrow.
-
Wow! It got fixed by Danne :-* I shall try it out tomorrow aswell! Superb work man..
Cheers
Chris
-
New version:
https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899
- Slightly different centering for set_25fps but at least it works now. (Anamorphic mode)
-
@Danne Both UHD and Anam are now good here! Preview fixed. :) Thanks plenty!
-
Ported iso average,frame burst and frame stop functions to 5D3. New uploads, usual place.
On eosm iso average goes like this:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg222737#msg222737
Uploads:
https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899
-
Hey Danne Great work, Cheers, I should try the iso average etc asap. :)
About the centered liveview, It doesn't seem to be centered to me yet. (I don't think my account has the permission to upload images so I can't really show you.)
But to explain in words - When I zoom in 10x to center, it shows another center then what is showing in Live View. The center is further up than the actual center.
Have anyone else got other results?
//Chris
-
Centre while recording better. Also if holding down halfshutter. X10 zoom follows its own protocol.
-
New version:
https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899
- Ported frame burst refinements same as on eosm here:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg223648#msg223648
-
@Danne
Started playing around with your latest 5D3 build, with the frame stop feature, but this build is clearly different to the experimental download version that runs.
First, my script doesn't run, but that is not the issue.
I set an exposure of 0.8s.
I have expo sim and override on.
I switch on crop FLV.
I switch on RAW video.
I switch FPS on and to 1 fps.
But, unlike with the experimental build, I don't get a LV image and more critically the shutter switches to 1/33.
Not looking for help/answer, just giving update at the moment.
-
I get full range shutter when set so cannot reproduce what you are doing.
Anyway. Would help if you narrow down what works first then focus the bad new at the bottom of the list. Meaning. Is frame burst acting like it should when not moving fps override and such? Then give specifics of what happens when fps override is on. Do you turn off audio by the way? FLV do not like audio on etc.
Are we testing frame burst?:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg223648#msg223648
Also check here:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg223238#msg223238
Also, if you are not getting a liveview image I doubt you are in framing mode. So I would say there´s a lot more to exercise around movie modes here.
-
@Danne
It's a mystery ;)
I've switched back and forth between the Experimental build and your frame burst one.
The experimental one works as expected, ie when I switch from LV to Movie mode the shutter stays at the correct shutter, eg 0.8s if I'm using a 1 FPS.
When I load yours it switches the shutter to 1/33 when I switch from LV to Movie mode.
As I say, I use the same settings for each build.
As I say, a mystery: but I'll keep playing around.
-
Hm, I'l have a look later today. I might have messed some with a1ex fps values in crop_rec at some point. Didn't work this branch for a while.
Can you give feedback on the frame burst function itself. Tlapse script as well?
-
Frame stopping working great: looking forward to trying that in my script, once I get the shutter fixed.
-
I try it here and all works. I need to push canon menu buttons once in a a while and at one time I restarted camera since framing wasn't working but it works. I choose Full-res LiveView then Shutter range to Full range, FPS override set to Desired fps 1. Then I push canon menu twice and when coming back shutter is fullres 0.8. Disable sound and start recording. Works perfectly...
-
BTW where do you disable sound?
-
Got it!!!!!!!!! :)
then Shutter range to Full range
This was the problem!
Shutter OK, but still can't see mage in LV, even after Menu pushes, ie black/purple diagonal lines fill the image.
I'll carry on playing around.
Thanks for your support.
BTW rather strangely the File Manager viewing of MLVs doesn't seem to work with your build, I get an "Allocation Failed" message. Not a priority for me.
-
Did you select framing under RAW video? If still not working simply restart.
What's working or not can't be determined by one preset selected tests. Please try other presets with file manager as well. Still trouble? Maybe you were lucky on the other build?
Edit: also note that adtg_gui should never be enabled if not in use. It kills framing for instance.
-
UPDATE 29-Dec-2019
See https://www.magiclantern.fm/forum/index.php?topic=24663.msg223841#msg223841
@Danne
Have spent most of the day switching between your build and the experimental page version.
I have assured myself that the settings are the same each time, eg shutter at 0.8s and an FPS of 1.
With the experimental build I can always see a stable LV image in movie mode.
In your version I mostly see a solid black & purple image. No amount of switching the menu on and off fixes this. Occasionally the image from the non Movie LV mode transfers to move mode, but freezes.
The perfect world for me would be the experimental build with your framing feature ;)
But then again, we don’t live in a perfect world :)
-
I see. I would need your SETTINGS folder or a printscreen showing your settings in RAW video menu and in Movie tab to be able to see what's going on. Too many variables will go wrong otherwise. It works here, what can I say.
Edit: also what modules do you have enabled?
-
Thanks Danne for the hard work.
I updated my repos and it just built fine.
More testings tomorrow, have good end of year :-)
-
Yes. Lots of changes. Rearranged menus and modules enabled from start and a few other things...
-
just download an tested crop_rec_4k_mlv_snd_isogain_1x3_presets_2019Dec30.5D3123.zip
5d mark 3
version 1.2.3
I am really impressed.
my 'C' days are over long ago, (I do python as a qa automation developer)
I never was a videographer, only stills.
my English should get better
and when reading here I can understand one every two sentences
so please forgive me in advanced
the only issue
experimentals and your build from April (I belive) as well gave me an option to set live view and when pressing '*' I could frame my shot
it worked on all profiles. however on this build when pressed I get a x10 zoom which will not go away after release, and if I pressed many times on '*'
and the 'zoom' button It went away but image is broken, ML menue will not open an nothing else is working until restart
thank you so much for this it's all really amazing that I don't really care if this would be fixed
5k raw video from 5d3, and continueas, and live view ! what else it would shoot rockets to space :)
amazing! !!!!!!!
few questions. . .
what is 'crop rec preview' should it be on?
why in video menu it say '12 bit' and raw video menue say '14 bit'?
what is 'shutter range' I never sew this. logic tells me variable fps but didn't we had this all ready ?
what iso is optimal?
and last I do not own anamorphic lenses, I shot my doter using the anamorphic 5k and let mlvapp do its own streaching magic
I don't see distortion and the frames look AMAZING, do I need an eye doctor ?
is it disorted in a way I don't see it ?
what am I missing ?
-
- Set focus aid to OFF for * button and halfshutter working normal way
- Keep 14bit lossless in RAW vide menu. As a workaround you can add 10 or 12 bit in the Movie tab menu for longer recording times. It actually lower analog gain acting 10 or 12bit
- Crop rec preview set to auto mode will select either framing or real time preview but automated depending on what preset chosen. Turn it off if you wish to work manually between presets.
- Yes, anamorphic is a pixelated stretch(I call it anamorphic) but without the need for any anamorphic lenses. You already noticed that Mlv App fixes the upscaling in post just fine. And yes, it´s good.
- Iso climb. Enable i and move joystick up or down for iso control, even while filming
- Shutter range. As it says ORIGINAL will set canon shutter. Full range will allow for much lower shutter speeds. Works with FPS override down to around 0.5s shutter
-
@Danne
- Set focus aid to OFF for * button and halfshutter working normal way
When I try this on the 5D3, the halfshutter works normally, but * does nothing, ie no zoom.
Is it me...again ::)
-
zoom? You mean focus? Check your manual. Think it should look like this(except the pinks :P:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.postimg.cc%2Fx8V4PpBF%2FVRAM0-PPM-300px.png&hash=7e335e530b5f091343c45987e0cff94d)
-
No I meant zoom.
With that feature enabled the image zoom in to x10, I guess, when I do a half shutter press. If I disable that feature, the half shutter returns to normal, but the * doesn't 'zoom in', like the half shutter did.
Are you saying it shouldn't zoom in, just do an AF?
-
Either x10 zoom enabled or the rewired button disabled and then it´s back to canon settings for both halfshutter and ae button.
-
I meant something else. not focus and not zoom.
in the experimental buildm my preview was realtime and when press+Hold '*' I get framing preview .
the change you suggested does help with the x10 but not bringing back old method. maybe it was a settings I did some how and don't remember how.
but I can totally live without it.
I made a few more test, and everything work like charm.
this build is the best new year gift :)
Happy new year
-
It should enter framing when pushing halfshutter and set to real time. Will take a look tomorrow. What preset are you using by the way?
-
Everything is working fine so far here.
The crop rec integration is more intuitive this way.
There are a few modes that says preview broken, is it fixable ? What's the cause of it ?
Happy new year BTW
-
@Danne
At the risk of annoying you, a quick thought on the focus aid.
Currently, when on pressing * will do a x10 zoom, which stays on until you do a half shutter press.
Doing a half shutter press brings up the x10 zoom, while the half shutter is pressed.
The ‘problem’ with this approach is that, if one uses the half shutter in a script, as I do, you can’t also use the focus aid.
One way round this is if your focus aid only used the * button and that it also toggled the x10 on and off. Thus freeing up the half shutter.
Just a thought and Happy New Year to you.
Cheers
Garry
-
* button comes with haflshutter button for good or worse. The other way around is tricky. I assume yoù know a way to assign the ae(*) button cause I don't.
Cheers
-
Ok
-
Hi!
Just cloned your crop_rec_4k_mlv_snd_isogain_1x3_presets_2020Jan01 repo and tried to add my iso-regs code into it.
Think maybe I've done something wrong as the new 'Presets' tab is there – but when I try and select it, it doesn't activate. :(
Can you see something I've done wrong? Maybe I need to add things in different places now… :o
Ignore pls!
For some reason my crop_rec.mo wasn't compiling at all… Compiled separately and it works now. Sorry… :-[
Cheers!
-
Cool menu edit + new toys… Thanks! :)
Notice that with Sound recording = ON, the camera start/stop beeps don't sound. Turning Sound recording = OFF brings them back. Is this expected behaviour?
-
There's a reason for most things in here ;).
-
Got these errors every other couple of shots today with latest build:
Live display; text in red:
'fatal no buffers'
then a few seconds later
'audio failed to stop. read state 3'
only a 3kb file is saved with no content.
Pressing menu then allows a proper recording, but the whole cycle repeats again.
I was recording in 3K as usual, with NO audio, so that's odd!
Anyone replicate this error?
Thanks.
-
Tell me exact settings, preset, etc. Preferrably do screenshots or a screenrecording to be sure not to miss anything.
-
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.ibb.co%2FnP8VwwJ%2FIMG-3322.png&hash=158038f594192ef353802de7fa17c9db) (https://ibb.co/MB5jgg0)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.ibb.co%2Fm8b8cwC%2FIMG-3323.png&hash=d275f791932bb2166b062b4f76c460ea) (https://ibb.co/DpzpRyD)
These are the error screens, but I noticed that Sound recording was enabled – my bad. However, I was running raw_diag.mo. I disabled that module and it seems to be ok again. Never seen that error before! :o Good to know!
-
How are you running raw_diag module?
Next time reporting issues tell build, set up, enabled modules etc. Tiring with most, not all, 'test reports' just thrown out there without even basic info.
-
How are you running raw_diag module?
Just by loading and enabling the module. I only really use it for the raw info readout window.
-
Yes, but that indicates you are running older builds. In later build you would see that enabling modules is not there. So, build info is vital.
-
I cloned your repo at commit #5a6f55b 2019‑12‑31 and compiled locally.
I can see that you have now enabled modules by default and that the modules tab doesn't display. But if you add other 'extra' modules in – as I did with a1ex's raw_diag.mo – the module tab then appears and can be accessed to enable these extra modules – but the rest of the modules still remain 'invisible'.
So I don't think it is an 'older build'. ?
-
Oh, that´s nice. Didn´t know it worked. Jip-Hop worked out the module automation routines. Nice find.
-
Added mlv capabilities as for eosm here:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg224216#msg224216
EDIT: Buggy. Took out the mlv_play fix for now. Needs more testing...
EDIT: Reuploaded. Hopefully bug fixed. Needs testing
-
The anamorphic mode is superb, so much less noise in low light situations.
Right bow i have the slow framing preview and a live preview on atomos shinobi working.
Tho the framing is correct on the canon camera its a bit off on the external monitor. I wonder if its possible to arrange it so that i can get the center position on my external monitor aswell?
//Chris
-
@Danne do you know why the 5D3 can only record 14 bit with compression in UHD preset?
Could 12/10 bit be added? Or is this something not easy? (I can go in to the code again if needed)
-
It could probably be added but I don't know how to do it the "right" way. There are comments about reducing bits in mlv_lite.c.
But, meanwhile it should work with my bitdepth workaround by keeping 14bit lossless in RAW video menu and selecting 10 or 12 bit from Movie tab menu. Should do more or less the same.
-
Ah ok, this is available with the new "eos m" layout? I will try it out.
-
Yes, works, just tested.
-
Nice! I will try it very soon. (once I empty my card)
Is this only possible with the recent layout update?
-
Can´t remember. I know I added my bitdepth workaround not so long ago to the uhd so that might very well be true. But it´s better to use the latest as I did a lot of cleanup as well so nu reason to stay on the older version.
-
I think it's true then, I remember I just couldn't find any way to change bitdepth in the previous build.
I will use the latest of course, I have always done so with magic lantern :)
-
I think it's true then, I remember I just couldn't find any way to change bitdepth in the previous build.
I will use the latest of course, I have always done so with magic lantern :)
Strange. Should have been in the sub menu. Anyway. It´s in Movie tab menu now so can´t be missed in the latest version. Just make sure that it´s set to 14bitlossless in RAW video menu:
https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899
-
Ah I didn't even think it could have been in crop_rec! Spent an entire holiday choosing between 4 second recording times and 20fps :D
Also why are the modules gone in the new build? How does it all work? How do you load a module? Do you think this is the future of magic lantern layout? It feels a whole lot less modular to me :(
-
Future, modular? I rearranged and modified to be able to just start the camera and get on using it. It's just another open source project.
Code is there so you can build your own tweaks if there's something you're missing but since you didn't even find the crop mode sub menu in the last build at least enjoy the oppurtunity to lower the bitdepths again ;).
It's not like the modules are gone. They are simply autoenabled. Is there a module you're missing?
-
Future, modular? I rearranged and modified to be able to just start the camera and get on using it. It's just another open source project.
Well your branch is clearly the most advanced and bug free right now, so it is the future. When referring to 'future' I meant: would a1ex want to merge all of these changes back in to the main source tree? I imagine the user interface changes (module aspect) are a bit too drastic (for exmaple some users don't want any video features), but the internal code changes are of course great.
I rearranged and modified to be able to just start the camera and get on using it. It's just another open source project.
It is great and very usable, I really like the new video tab. I just want to know which modules are enabvled by default now... (and to get module screen back if I can change your mind ;))
I remember on my 5D2, loading the smallest amount of modules would increase recording times. I just think it's best to let users disable whatever modules they want, but I am not against having some modules load by default like you have done.
Code is there so you can build your own tweaks if there's something you're missing but since you didn't even find the crop mode sub menu in the last build at least enjoy the oppurtunity to lower the bitdepths again ;).
Yes thanks :D
It's not like the modules are gone. They are simply autoenabled. Is there a module you're missing?
Nothing I'm missing, but I want to see what's enabled and disable what I don't need. I just think it's better for module screen to stay, what if someone wants to write a new module, it's useful for them to toggle it on and off easily.
I only want to suggest improvements, sorry if anything I say sounded rude.
-
Nothing rude at all. I just answered :). I welcome feedback.
These modules are default:
mlv_lite \
mlv_play \
mlv_snd \
dual_iso \
silent \
lua \
crop_rec \
file_man \
ettr \You can add any additional module by simply putting it in the module folder on your card and it should appear the normal way.
You can also look for this line in module.c in src folder:
char *core_modules[] = {"mlv_lite", "crop_rec", "mlv_play", "mlv_snd", "sd_uhs", "lua", "file_man", "dual_iso", "silent"};And erase:
char *core_modules[] = {};Or erase all code related to core_modules and it should be back to normal. Didn´t dig in to hard on this. It´s mostly Jip-Hops code parts in this case and works really well.
About bugs I created a couple of pull requests. All the rest is small quirk fixes here and there, nothing big really. A1ex also developed a refined crop_rec code for the 700D which might be the new standard in the future. Who can tell really?
-
Ah good to know where that array is.
Also do you knokw why mlv_rec.mo stops working with crop rec enabled? I would be interested in trying to fix that (unless it's a really huge task), as I want to try card spanning.
-
No idea really.
By the way. You should port card spanning to mlv_lite ;)
-
By the way. You should port card spanning to mlv_lite ;)
+1 :)
I'm looking into this as well! I see the code in sd_uhs.c for enabling it in 5D3 (untested), but I assume additional code has to be ported into mlv_lite.
In which module does the code that needs to be ported live now?
-
No idea really.
😢
By the way. You should port card spanning to mlv_lite ;)
I like the sound of this too. I just looked at mlv_rec.c, and it seems not too complicated (an extra thread and loads of if statements).
But I need to gain some understanding of magic lantern code first. It's not as simple as c library and pthreads :'(
What branch would be the best starting point for this modification? Unified, crop_rec_4k or something else?
-
I would work in this branch directly. I see no reason no too. I ported global draw off a while ago. Not as hard as card spanning though but as you say. Not impossible to understand but It's a bit over my head.
-
I would work in this branch directly. I see no reason no too.
Yeah ok. I cloned crop_rec_4k_mlv_snd_isogain_1x3_presets, this is the main one right now?
I ported global draw off a while ago. Not as hard as card spanning though but as you say. Not impossible to understand but It's a bit over my head.
I am not sure where to start really. I see in mlv_lite theres loads of "NO_THREAD_SAFETY_ANALYSIS" which is a tiny bit concerning. Currently trying to compare how both modules do writing, it seems to be making a little bit of sense, but there's no clear comments like "this line writes the frame", so it's a bit difficult.
Are the FIO_* functions used for writing actual big frame data? Or are they just for headers? I can't tell what they are doing... just 'jobs' according to variable names.
-
has anyone else experienced blacked out playback as if something went wrong with recording, but when loaded into MLVApp the image is there? I've been noticing this the past few builds, I've tested and changed various settings/resolutions to see if I had accidently turned something off, but every shot comes back blank. The liveview before shooting is there, just no playback.
sorry if this has been discussed already, i'm short on time so I just jumped to the end of the thread to ask. :)
-
More information needed.
-
@Danne how often do you update mlv_lite.c? I hope we do not make changes at the same time that will be difficult to merge.
-
I only do minor changes. Did a small change today. Just keep on with your stuff. WHen ready I can merge it into mine branch.
By the way, where´s the branch you are working on?
-
I hg cloned crop_rec_4k_mlv_snd_isogain_1x3_presets yesterday - and am editing mlv_lite.c in there. Same branch as you?
-
Yes, same. Could you link your repo so I can follow?
-
Hmm I don't have it on bitbucket, I just typed "hg clone https://bitbucket.org/Dannephoto/magic-lantern -b crop_rec_4k_mlv_snd_isogain_1x3_presets" and started making changes. Should I make a branch of my own to my bitbucket account? How do I do that?
So far, I have been saving versions by copying the file, not through commits because I don't know how to use mercurial, however I can put those versions in to commits once I get the mercurial stuff set up properly.
-
I'm guessing maximum number of chunks of an MLV is 101? as you can have only one .MLV, and 100x of .M00-.M99
-
It's actually really difficult to adapt mlv_lite for spanning. I will try again with a different approach, it's a bit of a mess this first attempt, I just tried to copy most of the way mlv_rec does it.
In mlv_rec writing loop is so much simpler, here it is tangled in to everything, so it's difficult to have two running at the same time. I wish we could have mlv_rec working with crop_rec.
-
Hm, keeping mlv rec is hard to justify. No lossless recording if I recall correctly? Only reason to hold on to it would be if both lossless and card spanning were working but ultimately if you manage to get it working on mlv_lite personally I would think it's the better way.
-
Hm, keeping mlv rec is hard to justify. No lossless recording if I recall correctly? Only reason to hold on to it would be if both lossless and card spanning were working but ultimately if you manage to get it working on mlv_lite personally I would think it's the better way.
I am trying again on mlv_lite, this time without copying how mlv_rec does it, as it's just too different.
Also I didn't realise mlv_rec didn't have compression, I always thought it was the more "advanced one" :D That's what I felt from using the 5D2 anyway :D But I guess I was wrong :-\
I think one advantage to mlv_rec is that it writes EXPO blocks for each exposure adjustment during recording, while mlv_lite does not, is that correct? Or is this also outdated information?
-
Yeah, mlv_lite is stripped for maximizing speed. Originally dmilligan started out building this module.
-
Hi, I love the anamorphic mode, and i do not mind that it is squeezed in live view, and its perfect to be able to frame The desqueezed view before recording.
But i have one question that i rly havent found clearity to yet.
Even tho the live view is squeezed, is it possible to move the center position so that it matches the center better?
I made a video including external monitor to show what i mean.
It would be very very useful, (atleast for me;) ) if it was possible to match the squeezed views center point to the live view center point. In laymen terms, move the image up;)
4aed6ecab1
Is this complicated / goes much deeper than what I may think?
Thanks!
Kind regards
//Chris
-
I am almost done with card spanning, I have a very simple solution. just need to sort out one final thing: sharing the "slots" thing between two threads. Need to figure out what it is (I think it's a queue) and how it works, then how to share it. Going to leave this for tommorow.
Yeah, mlv_lite is stripped for maximizing speed. Originally dmilligan started out building this module.
So it would be nice to get mlv_rec working again and add compression support ;)
-
Hehe, well. Who knows what tomorrow brings. Nice to hear about your progress on card spanning!
-
I thought some of you videographers :) ;) may be interested to know that this 4mm fisheye lens us now available in an m mount, and it's cheap as chips.
https://www.venuslens.net/product/laowa-4mm-f-2-8-mft/
-
Should I add another struct semaphore * for protecting the slots array / writing queue when accessing it from two threads? Or should I extend the use of settings_sem to protect that?
BTW I am not adding any of this "guarded by" stuff to anything I add, as I don't know how to use it correctly. But as I understand that only affects static analysis, not running the code.
Also why does the code sometimes use '0' instead of NULL for pointers?
-
Ok nevermind about all that for now. First I need to understand how write_frames works...
Could anyone who knows explain: why it is only called once for multiple slots? it only gets the pointer from the first slot as the ptr argument, yet somehow writes all of the slots to the card?
And as far as I can tell a "slot" is an MLV block waiting to be written, which can be a frame or any other type - correct?
-
Also why does the code sometimes use '0' instead of NULL for pointers?
NULL and 0 have the same value. But, NULL implies pointer. So many people think you should use NULL if it's a pointer, because it lets other people know they're dealing with a pointer. I agree with this but it's basically a style choice.
-
Could anyone who knows explain: why it is only called once for multiple slots? it only gets the pointer from the first slot as the ptr argument, yet somehow writes all of the slots to the card?
Ok firgured this one out, all the blocks are actually in a single continuous block of memory behind the scenes, somehow. Therefore the pointer to the first block is actually a pointer to all the blocks needing to be written.
I guess this must roll over back to the beginning of the buffer when it reaches the end, but where is the code for detecting the roll over?
I think can make spanning work now I kinda understand what's going on.
These are just notes for myself, but I'm writing here so anyone who knows better can correct me.
-
Indeed, your understanding is correct, so far. Most of these things are not related to Danne's changes; mlv_lite has the same quirks in the main repository (so, the discussion here is probably off-topic).
Anyway - mlv_lite is pretty much the codebase from the original raw_rec (i.e. most of that is my code, updated by dmilligan to support the MLV format at the same recording speeds as the original raw_rec implementation, if not a bit faster (https://www.magiclantern.fm/forum/index.php?topic=17091.0)). Long before that, mlv_rec was written by g3gg0 in an attempt to completely replace raw_rec; unfortunately, it was a little slower, and its menus and internals were a bit too complex for my taste, so... we ended up maintaining both. Not ideal, I know. It would make sense to merge both modules somehow, keeping both the raw recording speed and the features, if possible.
Regarding 'loads of "NO_THREAD_SAFETY_ANALYSIS"' - there are only two, that I could find, and that's because the static analysis (https://www.magiclantern.fm/forum/index.php?topic=18907) does not handle cli/sei correctly. I don't see this as a reason to be concerned - on the contrary. But I cannot guarantee 100% correctness either - maybe rewriting some parts of ML in Rust could help with that. No, I don't know Rust, besides trying to run Hello World some years ago, but it seems to be something worth learning.
About what things are executing - there are our own DryOS tasks, more or less straighforward to understand, and there are hooks placed at various locations in Canon firmware (usually in their DryOS tasks). The most interesting one here is the VSYNC hook, aka "EvfReadOutDone" on recent models (i.e. triggered once after capturing every single LiveView frame, in Canon's Evf/LiveView task, which runs with very high priority). That's where each frame is processed, and the next slot is chosen (ideally continuing the previous memory chunk, as it's a lot faster to write several megabytes at once).
However, what exactly happens within each LiveView frame was only recently understood (https://www.magiclantern.fm/forum/index.php?topic=19300.msg207723#msg207723), and only to a very limited extent.
-
Thanks for the response a1ex
True it is a bit off topic, but the goal is adding card spanning which is on topic I think.
Is the reason for mlv_rec being slower that it writes one frame at a time? It would be nice to merge them. Would unified be the best starting point for this kind of change? Also about mlv_rec not loading in crop_rec_4k, is that a difficult fix?
Regarding 'loads of "NO_THREAD_SAFETY_ANALYSIS"' - there are only two, that I could find, and that's because the static analysis (https://www.magiclantern.fm/forum/index.php?topic=18907) does not handle cli/sei correctly. I don't see this as a reason to be concerned - on the contrary. But I cannot guarantee 100% correctness either - maybe rewriting some parts of ML in Rust could help with that. No, I don't know Rust, besides trying to run Hello World some years ago, but it seems to be something worth learning.
Hm rust, can it even be compiled to run on these cameras? Might it use any dynamic memory allocation behind the scenes?
Therefore, if a function is called from more than one task, it must be thread-safe.
This is a good way to think about it, will keep in mind.
About what things are executing - there are our own DryOS tasks, more or less straighforward to understand, and there are hooks placed at various locations in Canon firmware (usually in their DryOS tasks). The most interesting one here is the VSYNC hook, aka "EvfReadOutDone" on recent models (i.e. triggered once after capturing every single LiveView frame, in Canon's Evf/LiveView task, which runs with very high priority). That's where each frame is processed, and the next slot is chosen (ideally continuing the previous memory chunk, as it's a lot faster to write several megabytes at once).
However, what exactly happens within each LiveView frame was only recently understood (https://www.magiclantern.fm/forum/index.php?topic=19300.msg207723#msg207723), and only to a very limited extent.
Interesting to know. So the VSYNC hook is how ML redirects the image data to it's own memory?
And back to implementing spanning in mlv_lite:
My simplest (most threadsafe) idea was: the main thread will split each write in to an SD and a CF card part, then tell the SD card thread what to write (memory pointer and number of bytes). The splitting should be a 1:5 ratio, or learnt as the recording goes on. But this is not very good as it will make smaller chunks be written each time, and if the SD card has a hiccup, the main thread will be waiting for it to finish, and the other way round, the SD card would not be fully utilised. And if there is not enough frames in the queue, it can't be split very well.
I want to figure out a better way to share the queue between two threads. Do you think that is possible?
Haven't really made sense of the whole writing loop yet, so I can't tell.
-
Mlv_rec works here. Only tested with eosm and with that particular branch. I thnk mlv_rec works with 5d3 too. At least with a1ex crop_rec_4k branch.
With that said only thing personally missed in mlv_lite is card spanning :)
-
oh it does? not with crop_rec.mo enabled though?
-
Crop_rec works with eosm.
A bit off topic but it needs certain other modules enabled to work I guess. I'm sure it can be solved with a little trial and error. But let's fix card spanning on mlv_lite first ;)
-
But let's fix card spanning on mlv_lite first ;)
Sure :)
-
I think this line:
writing_queue_head = after_last_grouped;
should be moved to before the frame(s) start being written, so that when the other thread starts looking for frames, it will get unwritten ones.
Or will moving this line affect whatever process is actually putting the frames in to the buffer? In that case, I can add an extra version of this variable for the write threads, updated at the start, and leave the original one for the process actually inserting frames, updated at the end as before (will need some care when both threads are updating it)
I also think the SD card thread should only write one frame at a time, otherwise it will be wasting time and memory, especially when the buffer starts getting full.
This is my current best idea for spanning.
@Danne Do you know how much extra write speed mlv_rec spanning adds? Does it perfectly sum CF and SD speed or is there overhead?
-
I used it briefly when it was implemented. Wild guess it gives around 20mb extra. But. 5d3 can do sd_uhs hack so we might get like maybe 40mb extra rough estimation?
-
Woahh! Didn't know sd_uhs worked on 5D3 as well. So we may get 140MB/s which could give continuous UHD 😱
-
Well, a1ex would know this for sure but 125-130 might be more realistic maybe.
-
Wrong thread, sorry
-
I HAVE MADE WORKING SPANNING!!!!!! This is what the commit looks like: https://github.com/ilia3101/mlv_lite_spanning/commit/0684bab9e5937a563af8409f5c226af73b8e7c74
I have used github as I don't understand how to use bitbucket or mercurial. On that github repo you can see the before and after.
unfortunately: there is one case where the threads collide and recording stops with "Slot check error" - it happens rarely, and I know exactly why, I just haven't found a way to do mutual exclusion yet.
I am pretty sure the semaphore is what I should use as a "mutex", but I could not get it to work, could you help me a little bit please @a1ex? I just want to know how to use the semaphore for mutex purposes.
-
-
Wow. Impressive. No cam today but will test soon! Hope a1ex can take a look.
Bravo Ilia3101!!
-
I will upload test builds, I just don't recommend it for anything other than testing!
Recording WILL STOP due to an error sometimes!
Edit: I won't upload builds actually, until I have got it in to a more usable state. Testing right now won't mean anything.
-
When I add a semaphore eveything just freezes forever, and says something like "25 frames" or "31 frames" until I turn the whole camera off.
All I did was add a pair of take_semaphore and give_semaphore, timeout 0 which I guess means no timeout. I might just try something dumb like a spinlock.
-
Nice progress!
To debug these semaphores, you may use the wrappers at the beginning of menu.c. These will tell you who acquired the semaphore (what line of code), and you can also add the task name (get_current_task_name). What you print on the display, should stay there even if the camera locks up.
However, a hard lock up (when everything stops responding) generally means some kind of invalid hardware (MMIO) access, or DryOS memory overwritten, i.e. messed up badly enough that vanilla firmware can no longer continue. If some task ends up "simply" waiting at semaphore, that shouldn't bring down the entire machine.
For 32-bit integer assignments, on a single-core ARM CPU, semaphores should not be needed - afaik, this operation is atomic. That's probably not true on multi-core systems.
For very short critical sections, you may use cli/sei, but no other tasks will be able to run between those two. You should not call any DryOS API either (such as msleep), as these might enable the interrupts on their own.
Also take a look at the selftest module - that also serves as "documentation" of expected behavior and return value of various DryOS functions. The VxWorks documentation should be helpful, too, as the high-level behavior is pretty much identical (but the internals are not).
-
However, a hard lock up (when everything stops responding) generally means some kind of invalid hardware (MMIO) access, or DryOS memory overwritten, i.e. messed up badly enough that vanilla firmware can no longer continue. If some task ends up "simply" waiting at semaphore, that shouldn't bring down the entire machine.
None of my lockups were hard, just raw recording thread(s) locking up - I could still enter ML menu and it would just say "stopping"
For 32-bit integer assignments, on a single-core ARM CPU, semaphores should not be needed - afaik, this operation is atomic. That's probably not true on multi-core systems.
Doesn't apply to the current situation (I think), but good to know.
For very short critical sections, you may use cli/sei, but no other tasks will be able to run between those two. You should not call any DryOS API either (such as msleep), as these might enable the interrupts on their own.
This may be the perfect solution! I will try it out. Didn't realise what cli/sei were until I just googled it. Wow!
Is "ASSERT" and beep/bmp_printf allowed between cli/sei?
Will have a look at menu.c and selftest module as well.
-
I added sei/cli and it seems to fix the problem, but I think the code runs between them for too long, so I will still try and sort out the mutex.
Just one last question for today: There is loads of "Frame order error" messages at the end of recording - it only seems to happen when the video's bitrate exceeds or almost exceeds total write speed. Is this a major issue?
With UHD total write speed tops out around 111MB/s right now, not as amazing as I expected. However I haven't tried sd_uhs yet. Update: I compiled sd_uhs in your branch @Danne and it says not supported on the 5D3 - where do I get the latest sd_uhs?
-
Check the code in sd_uhs.c(my branch):
/* Below cams not tested/supported atm. Try it by enabling sd_overclock_task(); */
if (is_camera("5D3", "1.1.3"))
{
/* sd_setup_mode:
* sdSendCommand: CMD%d Retry=... ->
* sd_configure_device(1) (called after a function without args) ->
* sd_setup_mode(dev) if dev is 1 or 2 ->
* logging hook is placed in the middle of sd_setup_mode_in (not at start)
*/
sd_setup_mode = 0xFF47B4C0; /* start of the function; not strictly needed on 5D3 */
sd_setup_mode_in = 0xFF47B4EC; /* after loading sd_mode in R0, before the switch */
sd_setup_mode_reg = 0; /* switch variable is in R0 */
sd_set_function = 0xFF6ADE34; /* sdSetFunction */
sd_enable_18V = 0xFF47B4B8; /* 5D3 only (Set 1.8V Signaling) */
SD_ReConfiguration = (void *) 0xFF6AFF1C;
/* sd_overclock_task(); */ }
if (is_camera("5D3", "1.2.3"))
{
sd_setup_mode = 0xFF484474;
sd_setup_mode_in = 0xFF4844A0;
sd_setup_mode_reg = 0;
sd_set_function = 0xFF6B8FD0;
sd_enable_18V = 0xFF48446C; /* 5D3 only (Set 1.8V Signaling) */
SD_ReConfiguration = (void *) 0xFF6BB0B8;
/* sd_overclock_task(); */ }
return 0;
}Enable the function with this:
/* sd_overclock_task(); */to this:
sd_overclock_task();
Don´t forget, the module will autoenable when starting camera.
in modules.c
char *core_modules[] = {"mlv_lite", "crop_rec", "mlv_play", "mlv_snd", "lua", "file_man", "dual_iso", "silent", "ettr"};
Just tested over here and seems to work but havn´t checked agains any benchmark tests.
-
Didn't work, SD card just benchmarks at 0.0MB/s and there is some error when raw recording. (I'm firmware 1.1.3)
-
Well, I can´t really help you on that as I have no issues here.
You could test sd_uhs.mo from the sd_uhs branch. Here is a compiled version:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/sd_uhs.mo
Try and replace it with what´s in your modules folder now. This version needs to be enabled from with debug tab. Not as convenient but check if it works better.
-
Thanks! I will try that version.
Also this card might be wacky, it's an 600x pink transcend. I'll try a different card later.
-
Yes, could be it. My version don´t really like cheap cards :P.
-
Anyone have any idea what this error could be about:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.postimg.cc%2F5tc3X3bF%2FIMG-20200123-202510.jpg&hash=7b4d6b2451fd05a2c66a908b031f69f2)
It only happens rarely, but does happen.
Since that error happened, the camera has been recording for 10 more minutes without any more errors.
I think spanning is getting close to usable now!
-
Wonderful Ilia3101!
Compression error. Usually includes corrupt frame/frames. I had them when recording was disturbed of sone engdrvout routine changing regs in crop rec. But root could as well be overhead or recording edge of speed I think.
Sd_uhs working?
-
About to try sd_uhs with a proper card!
Edit: the sd_uhs didn't work with any of my two cards I have tried. I will try with some other cards once they are free.
I have uploaded some testing builds with spanning:
https://github.com/ilia3101/mlv_lite_spanning/releases/download/test-builds/magiclantern-Nightly.2020Jan23.5D3113.zip
https://github.com/ilia3101/mlv_lite_spanning/releases/download/test-builds/magiclantern-Nightly.2020Jan23.5D3123.zip
I decided to post them as it does not seem to be causing any dangerous crashes or lockups. I have only tested 5D3.113, don't know if it will be different for 123. Curious to see people's results.
My modified mlv_lite.c and mlv_lite.h code can be seen here: https://github.com/ilia3101/mlv_lite_spanning
I did have issues at one moment, but it was just a faulty lens adapter chip messing with the camera. Not from my spanning code. Pretty certain it's safe.
-
Gave it a go with version 1.1.3 and could get continuous 24fps 10bit with full expansion of UHD 1:1(3840x1536). I needed sd_uhs enabled but had to get the old routine patching with the very long wait from sd_uhs branch. With my sd_uhs version it would create some File create error and then stop recording. I guess this module needs some testing to get automation working. On a side note I wish for continuation of the sd_uhs patching module. I never really liked the way I stripped and got it working reliably mostly for the eosm but not with other cams.
Compression errors were present when trying out higher frame rates like 48/50/60 fps modes. I guess these modes are too much for the sd card to handle.
Crop marks dir is missing and i also couldn´t get the mlv_lite.c and mlv_lite.h to work with my ongoing branch but I guess it could be done later on.
Great work Ilia3101.
-
Glad to see you testing it out :)
I just sorted out the semaphores (https://github.com/ilia3101/mlv_lite_spanning/commit/3f110176b170949d2668217a0e0e236664a07d4d), and it has added megabytes of performance, at least with the weird transcend card. Before I was using cli/sei which would block the processor for too long and cause the compression errors (I think).
I am recording a long test clip right now to see if any errors come up.
When do you think we can start figuring out how to put it back in to your bitbucket branch? How many changes have you made to mlv_lite since?
-
Only one slight change jan 18th:
raw_set_preview_rect(skip_x, skip_y, res_x, res_y, 1);
raw_force_aspect_ratio(0, 0); */
- if (!get_halfshutter_pressed())
+ if (!get_halfshutter_pressed() || rec_trigger == 3)
{
autofocusing = 0;
long_halfshutter_press = 0;
So it won´t be an issue. You could test compiling and working right into my latest branch. I won´t be messing with it for a while anyway ;).
You changed mlv_lite.c and mlv_lite.h but no other places? Modules needs to be turned on I see at least but I also had some warning about dryos. Didn´t have time to check it too hard.
-
Nice. Just added that change to my mlv_lite.c, now it can easily be put back in to your branch whenever it is time.
-
Great. I´ll check. I think I was in the eosm branch before so My last message was testing the wrong branch :P
-
Ah ok. Undid the change for now.
Also updated the builds.
-
ah, no, the change is ok, I meant another error so you can add it back.
Tested again and skipped messing with the sd_uhs module. 48/50/60fps working just fine and what was nice there were no compression errors at all. Could very well be related to sd_uhs.
Nice part with this is that practically all presets can be run in 14bit. Only missing the sd_uhs module for the UHD 1:1 before patching I think sd card will run around 20mb and after patching at least double so getting some usable sd_uhs patch working is something to look into.
Let´s keep your testing branch open for a while and keep on testing from it and when ready I can put it into my own branch.
Remember folks. BLEEDING EDGE! Be brave, and maybe rich too ;)
EDIT: Will test full 48fps 1920x1080p and see if spanning stuff makes this work...
-
Redid the change :D
Yeah, let's leave testing open for a little while. @everyone please test SD/CF spanning! Download in this post (https://www.magiclantern.fm/forum/index.php?topic=23041.msg224571#msg224571)
-
This is super exciting, i have some good cards that will try it out on this weekend.
UHD in 14bit 24fps with liveview working?? Sounds to good to be true ???
-
The only thing that has changed is write speed (if you enable card spanning), now instead of 100 it's more like 112 for me (I really want to hear other people's results, I feel like it should be 120). With sd_uhs we might even get a lot more ;)
Note: just remembered: in my build I removed core module auto loading array, so you'll have to load them yourself sorry :) @Danne feel free to make a slightly more official build, the mlv_lite.c/h on my github is the latest
-
I took your changes and also added sd_uhs patching into 5D3 into this branch:
https://bitbucket.org/Dannephoto/magic-lantern/branch/crop_rec_4k_mlv_snd_isogain_1x3_presets_card_spanning
In time I´ll merge changes into the regular branch again. Feel free to grab code changes and compile/test new builds. Note that sd card I test with are sandisk extreme pro 95 or 170mb/s. Slower cards might behave fishy when patched.
When patched the first time it takes a few seconds for the changes to show. Once OFF turns to ON it´s patched.
Patch function enabled from with in Movie tab menu:
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.postimg.cc%2FV6tg3Jbs%2FVRAM0-PPM-scaled.png&hash=42ea362ae2acea7ebaf491c77201efb6)
Do remember:
BLEEDING EDGE! Briefly tested. I did not test to patch while only using CF-card so don´t know how it will behave if no sd-card is in the memory card slot...
-
Holy fuck. My sandisk 45MB/s card just wrote at 48MB/s!
However the CF did slow down to ~75MB/s... whyy!!!! Is the camera's ram becoming a bottleneck?
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fi.postimg.cc%2F441qtb8F%2FUHD.jpg&hash=007772d43734c3819fee3faad935005a)
^ That is UHD in 12 bit
Edit: yep, getting data corruption, need a really good card for this to work. At least 60MB/s
-
Would love to try this build to help the community !
I have 5DM3 with:
SDHS I Sandisk 95 MB/s
CF lexar 160 MB/s 1066x
But which build to use ?
And what is the settings to try ?
(I have read last 5 pages but I'm not enought in it to fully understand...)
-
@ilia3101 Thank you for your hard work bringing this to 5D3! :)
I have tested both a build compiled from Danne's branch :
https://bitbucket.org/Dannephoto/magic-lantern/branch/crop_rec_4k_mlv_snd_isogain_1x3_presets_card_spanning
and yours:
https://github.com/ilia3101/mlv_lite_spanning/releases/download/test-builds/magiclantern-Nightly.2020Jan23.5D3113.zip
Fastest card I have is Sandisk Extreme 90Mb/s U3.
On Danne's branch: I get almost instant stops in UHD 3850px with message: "Frame order error…Slot 10(?)…e13…expecting 19…" The message disappears very quickly so I can't read it all! A file has been saved to SD I notice: M25-1206.M00 – file is 50mbs but I can't open it in MLVApp.
**EDIT** I have just tried turning Spanning = On, but Overclocking = Off. It now records one file to CF and one to the SD, both with same timestamp but the SD one with the .00 suffix. Can't open .00 file in MLVApp or MLVFS.
On ilia's I get an error printed about: Missing crop-marks and also sd_uhs will not run on this camera. (I'm on 5D3 1.1.3)
-
Would love to try this build to help the community !
I have 5DM3 with:
SDHS I Sandisk 95 MB/s
CF lexar 160 MB/s 1066x
But which build to use ?
And what is the settings to try ?
(I have read last 5 pages but I'm not enought in it to fully understand...)
Wait for Ilia3101 to compile and share or learn how to compile. Seems you have the right card to use so should be ok.
-
Updated:
https://github.com/ilia3101/mlv_lite_spanning/releases/download/test-builds/magiclantern-Nightly.2020Jan25.5D3113.zip
https://github.com/ilia3101/mlv_lite_spanning/releases/download/test-builds/magiclantern-Nightly.2020Jan25.5D3123.zip
@timbytheriver What should happen is that the main file (.MLV) will be recorded on the CF card, and .M00 will be on the SD, just move both of them to the same folder/card, then you will be able to open it with MLV App.
And there may also be .M01 .M02 etc from 4GB splitting.
-
@ilia3101 Aha! Thank you.
just move both of them to the same folder/card, then you will be able to open it with MLV App.
I see this to be true now. :)
I will try your updated builds and report back.
Thanks!
-
@ilia3101
Works good over here. Is frame order "issue" due to spanning just recording frames differently? Seems indexing fine though so maybe we should exclude the check when spanning is on?
Next I want to increase anamorphic resolutions when spanning is enabled vs defaults now without spanning on.
Also noticed a previewing issue when recorded longer takes, 3 or more mlv chunks. Will print exct message when back home.
-
I am almost certain the frame order issue is just due to the nature of spanning. Makes each file look like it has loads of skipped frames.
So yeah I think we should disable this check when spanning is used.
-
Makes each file look like it has loads of skipped frames.
Ah, makes sense.
-
@ilia3101 With your newly posted build I get:
Card spanning + SD overclocking = on, I get msg: File Create Error. Stops recording immediately.
SD overclocking = off, Card spanning = on, I get Frame order… issue (Think you have discussed this?) and stops after 8 secs or so.
Upon restart had msg twice: Camera not shutdown properly. Skipping module loading.
Cards: CF Sandisk Extreme Ultra Pro 160mb/s (Been using for 3K fine for a year) + SD Sandisk Extreme 90mb/s. (Also been fine for ML core files).
-
speaking of cards, i want to join in the party but i don't have any decent sd cards. the local retailer here only has this in stock:
SanDisk 64G Extreme SDXC UHS-1 Memory Card - 150MB/s, C10, U3, V30, 4K UHD, SD Card - SDSDXV6-064G-CWCIN
opinions? will it overclock?
my other option is amazon, but then the wait for shipping begins ;)
-
Only tested realible sd_uhs patching with sandisk extreme pro 95mb and 170mb card. Most other cards either failed or even slowed down speed performance.
-
adding the "pro" to my stock availability search drops the results to zero :(
-
@timbytheriver
Hmm, strange. I didn't get to try out sd_uhs with a proper card so I can't tell what's going on. I will upload a build where the sd_uhs module does not auto load, maybe that will improve things for you.
And those "Camera not shutdown properly" happened only with sd_uhs for me.
However I did get much improved record times even without sd_uhs.
@70MM13 still valuable if you try it out without sd overclocking.
-
i'll try it with my ancient sandisk 30mb/s card while i wait for amazon shipping :)
any improvement is a good thing, right? :)
-
I will upload a build where the sd_uhs module does not auto load, maybe"
However I did get much improved record times even without sd_uhs.
Patch itself is not autoenabled. Problem is due to card itself. Don't patch and problem goes away. Reluctant to answer some users who neglect to share info in other posts but I don't want Ilia3101 to waste his time.
I get increased recording times when patching. I'd say around 125mb/s.
-
Oh yeah it doesn't automatically do the patch. And I forgot to post the build, not home now sorry. If you still want to try disabling sd_uhs tim you can just remove it from the card, though it shouldn't be the source of the problem.
Also if you get file creation error it could be because the card was inserted badly (I had that once), but I can't help any further with sd_uhs for now.
@Danne I also hit 124MB/s with sd_uhs and a 45MB card, still need to check if the file can be read, the camera couldn't play it (it was a 3 minute UHD)
-
Yeah, good for you the patch works on a 45mb card. I don't think you can reach higher speed with a faster card.
I also had preview issues but file itself works in mlv app.
The file creation error is explained in mlv_lite code. I can recreate it with older sd_uhs patch code. The issue isn't triggered with latest build.
@70mm13
Try patch on your 30mb card. Maybe it works.
-
i'm trying it on a few sd cards i have here, (all slow but a couple are brand new) and no matter what i do i don't get any files to show up on the sd card. i tried toggling through all of the canon menu card recording options, but it makes no difference. any suggestions? obviously i must be missing some crucial and obvious (but not to me) step!
while i await any suggestions i will try another test without a CF card, especially just to try the patching!
-
Don´t know what builds you are testing but you need to enable card spanning in RAW video menu.
Patching will only work with my latest code.
-
i'm using the latest one ilia posted, and yes, i did miss the obvious "card spanning" in the raw video menu! :P
meanwhile here's my sd-only overclock test results:
both new kingston cards (crappy cheap cards only intended to load ML and capture stills) said "card full" even though they were empty, and my ancient sandisk "ultra" 30mb/s worked, although i didn't see any real improvement, only a few extra frames recorded but that's within margin of error...
EDIT: PS: what formatting block size settings are you guys using on the sd cards?
-
Yeah, sd_uhs will not work with those builds. Have to wait for new uploads.
Preview issue comes from mlv_play. Could reproduce with longer takes, three chunks:
From mlv_play.c
/* no, its another chunk */
if(main_header.fileGuid != file_hdr.fileGuid)
{
bmp_printf(FONT_MED, 30, 190, "Error: GUID within the file chunks mismatch!");
beep();
msleep(2000);
return;
}
-
no worries, i'll wait patiently :)
any suggestions for optimal sd card format block size?
-
Ah I figured out why the fileguid is different, I will fix it this evening.
Update: fixed (https://github.com/ilia3101/mlv_lite_spanning/commit/3c88cb06c238c2f129a4d700705bbdac2e5aaf3b)
-
Cool. Works nicely with mlv_play and previewing longer takes is not longer an issue. I included the fix and also excluded frame order text when card spanning is set to on also in this branch:
https://bitbucket.org/Dannephoto/magic-lantern/branch/crop_rec_4k_mlv_snd_isogain_1x3_presets_card_spanning
-
Getting some errors around mlv_play. Locks camera:
[104]compress_task: stack overflow: free=0 used=4096
I record 14bit lossless anamorphic mode. Deleting and then previewing the next file got me the above text. card_spanning on but sd overclock is set to OFF.
Seems also happening with 10bit selected. Maybe my sd card is corrupted.
Formatted sd card in camera and all seems working again. I guess sd_uhs related. Hard to say. My sd card have been doing all kinds of tests lately :P
-
compress_task does not touch mlv play, it is only for compressing frames while recording. No idea why it would have a problem, I did not touch it's code or do anything that should affect it.
-
Probably sd_uhs patch related. Card chokes causing issues. After formatting all works again.
-
Have new card now: Sandisk Extreme Pro SDXC UHS-I 64gb (170MB/s read, 90MB/s write)
Compiled latest updates. Card-spanning and SD overclocking working well now:
I can get 1min+ 3600 x 1532 / 24fps / 2.35:1, continuous at 3072p. 1080p 14bit on SD card alone – continuous! :)
Many thanks @ilia3101, @Danne.
-
Have new card now: Sandisk Extreme Pro SDXC UHS-I 64gb (170MB/s read, 90MB/s write)
Compiled latest updates. Card-spanning and SD overclocking working well now:
I can get 1min+ 3600 x 1532 / 24fps / 2.35:1, continuous at 3072p. 1080p 14bit on SD card alone – continuous! :)
Many thanks @ilia3101, @Danne.
Wow, great work, need to try this when i get back home. Is the liveview working as normal in this cases?? :o
-
@Stousen Yes!
In 1080p = Real Time.
In Crop modes = Framing.
-
my new sandisk extreme pro 170 card arrived the other day, and i have put it through its paces...
formatted with 4mb blocks for my first test.
like tim, the sd card alone does 1920x818 continuous!
3072x1308 also continuous!
3620<> x1536 5 minutes (x resolution at maximum for true 2.35:1 - is there a way to push y above 1536?)
all 14 bits lossless, with lots of brightness and motion.
i've been putting off my next music video until i could shoot 3k+ continuous (it's a performance) and now i can do it! thanks!
-
A postscript to @70MM13's post: For Mac users!!
I tried to format a custom 4Mb block size in exFAT using the terminal like this:
sudo newfs_exfat -b 4194304 -v NameThePartition /dev/disk3s3
It appeared to have done so, but I couldn't get it to mount under any circumstances. The card will work in the camera – but you will be unable to mount it to retrieve your files.
The closest I could get to a larger custom block size was 1Mb (1048576) (These numbers have to be powers of 2 apparently.) This will mount, but it caused problems in the camera and I started to get only about 15sec @ 3072p with or without card spanning/sd overclock on or off.
Not sure if it was related but I then kept getting card errors: 'Card1 cannot be accessed' after formatting custom block sizes from the Terminal. Reflashed the Firmware, finally erased formatted to exFAT from disk utility with the default [tiny] block size of 32768 (0.03Mb).
All appears to be ok again now. But bottom line may be that Mac really doesn't play well with exFAT/exFAT custom block sizes. I wish I'd left well alone! :(
PS If you format the cards in-camera I noticed (both my SD and CF cards being 64Gb) that the SD card gets exFAT, while the CF gets FAT32. Just noting…
-
@Stousen Yes!
In 1080p = Real Time.
In Crop modes = Framing.
I managed to get it to work, i get avout 110-115 in w/speed!
Tho my live view doesnt work while recording in crop 3k? :(
-
I managed to get it to work, i get avout 110-115 in w/speed!
Tho my live view doesnt work while recording in crop 3k? :(
It's hard to see what you expect or even if there is a problem involved. To minimize communication mishaps you need to be specific with settings used, preferrably with screenshots or a screenrecording. Use contextual terminology instead of making up new words etc.
-
It's hard to see what you expect or even if there is a problem involved. To minimize communication mishaps you need to be specific with settings used, preferrably with screenshots or a screenrecording. Use contextual terminology instead of making up new words etc.
LoL, i hear ya. Will try not to do a Hegel and be swift but clear. :-*
Card spanning: on
1/1 3k crop
14 bit lossless
2.35:1
Required 110 write speed
Live view: framing
Kill global draw: no, also tried yes
Sd uhs hack : on
Live view before recording: framing
Live view after pushing rec: framing
Live view some seconds after pushing rec: pink screen but still recording
70 ish write speed CF - 40-50 ish write speed on SD.
Tho thought the liveview would function better if the cardspanning actually increased the bandwith? Or am i out biking on a road with no light?
Cheers
-
You need a better bike 8)
Previewing capabilities will not change with different cards. Framing...
-
You need a better bike 8)
Previewing capabilities will not change with different cards. Framing...
Aye, thanks for the enlightenment :D :D ;)
-
Fantastic build crop_rec_4k_mlv_snd_isogain_1x3_presets_card_spanning_2020Jan25.5D3113!!! I've got (1920X3)X2332- 2.39:1 continuous 24fps, 14bit lossless, Dual ISO at anamorphic Presets. Is it possible to add settings for 1920 -2,35:1?
I've tried Dmytro_ua's Fast video presets, updated for Danne's build 16th of Jan - https://drive.google.com/open?id=18-Wo0yNVBCVwfjyTlB7O7UK5bMPUtsCd, but there is no this lua in Scripts menu. ?
-
I've tried Dmytro_ua's Fast video presets, updated for Danne's build 16th of Jan - https://drive.google.com/open?id=18-Wo0yNVBCVwfjyTlB7O7UK5bMPUtsCd, but there is no this lua in Scripts menu. ?
Those presets are actually easy to update to your needs with a simple text editor on your computer. I've shared them like an example to everyone who needs smth special for themselves. I'm sure there is no universal set of presets suited for everyone. So you can edit any of them to your needs. And delete others which you don't need. Actually it's not hard at all.
Sorry, didn't understand at first what you are asking for. As I mentioned before, I'll try to update them as soon as I have time to test everything in a proper way.
-
It seems to me that menus structures in builds of 16th Jan and 25th Jan are the same. My lua script works fine in 16th Jan build, but there is no this lua in Scripts menu in 25th Jan build. All builds are Danne's build.
-
It seems to me that menus structures in builds of 16th Jan and 25th Jan are the same. My lua script works fine in 16th Jan build, but there is no this lua in Scripts menu in 25th Jan build. All builds are Danne's build.
Post screenshots or try describe more in detail what you arw missing. I use lua all the time and it´s still included as a module but who knows what you´re asking here. Maybe try and do a screenrecording?
-
New versions:
https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899
- Added card spanning implemented by ilia3101 and also sd_uhs overclocking
Faster sd card(sandisk seems to work better overclocking). Patch only works on sd card.
Only briefly tested so tests and reports are welcome especially now at early stages. Bleeding edge folks. I trust card spanning more than I trust the sd_uhs hack. When patching works it works really well, but otherwise it could give give surprises like corrupted files, no header etc, or maybe even affect the sd card permanently. Well, other than that. Happy testing :)
And thanks ilia3101 and a1ex for the card spanning feature(formerly implemented in mlv_rec by g3gg0).
-
Post screenshots or try describe more in detail what you arw missing.
When I open "Scripts" menu in ML build 16 Jan I see my lua and can run it. But when I open "Scripts" menu in ML build 25 Jan, I can not see the same lua in menu.
-
Post the scripts. More info needed if you compare. Try shorten the name of the script that is showing in menu.
-
Wonderfull work Danne & ilia3101 and a1ex !
I'll try it asap !
-
Try shorten the name of the script that is showing in menu.
Yes! Thank you. If SD card FAT32 formated - no problem with long file name. If exFAT only 8 length works.
-
No Problem with shooting so far. I'll try recording this evening.
-
I can't make screenshot on build crop_rec_4k_mlv_snd_isogain_1x3_presets_2020Feb01.5D3113. Debug\Screenshot-10s, SET, Shutter butt. - half-press. There is a display with ML top and bottom bars. Global Draw - LiveView. There is no new VRAMx.PPM file. Is it OK?
By the way in crop mode there is no "Focus distance info" in low bar, because there is no room for it. How to reduce crop made info in low bar or even delete it? Also WB info is useless for me.
-
Worked last time I tried. Could someone else test the screenshot function?
Code is out there. If you want to modify screen text you can check in lens.c for instance.
-
Worked last time I tried.
It works for me only for ML menu.
-
Turn off RAW video while doing screenshots. I did some automation to temporarily disable RAW video when using screenshot function but maybe it fails under certain conditions. If you could present clear cases reproducing the issue I could take a look. Screenrecording would do.
-
Thank you. Will try this evening.
-
I am trying pull request your branch with command :hg clone https://bitbucket.org/hudson/magic-lantern && cd magic-lantern && hg update crop_rec_4k_mlv_snd_isogain_1x3_presets && cd , but got an error:"abort: unknown revision 'crop_rec_4k_mlv_snd_isogain_1x3_presets'" I am on cygwin (Win10) What is the problem?
-
Ask google.
-
Turn off RAW video while doing screenshots. I did some automation to temporarily disable RAW video when using screenshot function but maybe it fails under certain conditions.
Yes, it works. Thank you.
-
How does the bit depth option work in crop_rec? I am really concerned about the black level staying the same as it is at 14 bit. I switch to 12 bit, black level is still at 2047. Is it just cutting the top 3/4 of values off?
-
Is it just cutting the top 3/4 of values off?
Values of what?
The workaround is very simple. There are analog gain registry that when lowered gives reduced bitdepth. Only white level needs adjusting. Maybe a bad thing but it works so I just kept working with that.
You could test and compare 12bit in crop rec vs raw video.
-
Ah I see, so it is gain. You are putting a 12 bit range in a 14 bit container (still getting more efficient compession). Not cutting anything off then (GOOD!).
Analog - this explains why black level stays the same. I wonder what the effects on noise are, I seem to get stronger than usual stripes in the shadows, that don't go away with stripe fix.
ProTip: You could do this gain by only 0.1 stops and call it increased dynamic range ;) ;) ;)
-
I really want to add real 12 bit to the UHD preset.
Wish I knew how😭 @Danne didn't you mention there's comments somewhere about reducing bitdepth??
-
I don't think it's less real. Quality wise probably the same. What's your issue by the way?
-
I have no issue, but:
The black-white range with your gain adjusted 12 bit seems to be approximately 2047-5600 (max 16383)
With real 12 bit it will probably be 512-3800 (max 4095)
So it is effectively 12 bit.
But I suspect the compression algorithm can be slightly more efficient in true 12 bit mode.
I just got an excessive amount of stripes recently shooting in crop_rec 12 bit, which got me wondering about this.
-
A1ex explained from time to time about bitdepth routines. I can't find the posts right now. I don't think those stripes are related to bitdepth in crop rec but who knows.
-
Ok thank you. I wasn't saying I was certain that the stripes are caused by it. Probably not. The 5D3 is really quite stripy.
-
Can i go a bit off topic here ?
I hvae bought a KB 128Gb from amazon
I have used it twice and yesterday its simply died.
I want to buy a new card, but KB feesls like out of the question with this type of quilty.
So, is there anything else to use on a 5d3 with more or less the same speeds but more reliability ?
Used cases are danna builds 3.5k raw o a 5d3
Thanks
Sent from my SM-N975F using Tapatalk
-
How did it die?
I use transcend. Never failed.
-
I use Lexar. Maybe it's not a top speed but a solid proof for many years
-
Thanks
How did it die?
I use transcend. Never failed.
Pressing record and its gone, will not recognize by any system
Can you tell me which type and does it do 3.5k with danne build ?
I use Lexar. Maybe it's not a top speed but a solid proof for many years
Same .. you are saying that its not top speed, but does it work with danne build 3.5k raw on 5d3 ?
Sent from my SM-N975F using Tapatalk
-
If going Lexar 1066x: Take care, there are 2 versions: https://www.magiclantern.fm/forum/index.php?topic=24461.msg220607#msg220607
My KB card went dead some years ago and got replaced. No problems since. Same with my 2 Transcends.
-
Dear Danne,
Is it possible to record 12/14 bit 2656x1484 (16/9)?
Or is 1308 height a hardware limitation for continuous recording 12/14 bit?
The 3072x1308 (2.35:1) mode is not useful to me as I rarely use 2.35:1 and make 99% of stuff in 16/9, so I will have to crop 3072 down to 2326 and lose 33% of the recorded pixels.
I don't have a 5D3, but I might buy one if 16/9 is working with >=4 megapixels Raw.
3072*1308=4,018,176
2688*1512=4,064,256
2656*1484=3,941,504 so it maybe possible
What is the highest resolution that 16/9 can be recorded continuously for 12/14 bit?
-
Impatient to try this vey exciting card spanning feature, I borrowed the 5D3 and tested the latest February 01. build using the following two cards:
1) Transcend SDHC 32 GB 300x, (write speed: 40 MB/s, USB 3.0 card reader test)
2) Sandisk Extreme Pro 64 GB 160 MB/s, (write speed: 114 MB/s, USB 3.0 card reader test)
I. With Card spanning ON, SD overclock OFF spanning works fine, in-camera preview too. The 00-files get nicely recorded on the SD card and merging with the main files on the CF card goes flawlessly. Just dump the contents of both cards in one directory and you are ready to go. MLVApp processing works fine too. I get continuous recording at 3072x1728 10-bit lossless resolution and 24 fps with about 1,5 stops underexposure at these settings. With the same settings but without spanning, I get continuous recording only at 17 fps. So, card spanning makes perfect sense and is extremely useful.
Important:
=======
To get maximum recording times, warming up cards is necessary. With the above settings and both cards cold, I get a maximum of 200 frames recorded. Immediately after that, I get 1000-2300 and even more frames recorded. I guess, it's the SD card that needs warming up because, without spanning, CF card warming is not necessary.
II. With Card spanning ON and SD overclock ON, recording does not work. Upon pressing the record button, I get three beeps along with the message "Recording stopped automatically", then recording stops and nothing gets recorded.
------------------------------------------
Unfortunately, I had to pay a price for my impatience. My beloved SD card died in these experiments. Here is how this happened:
III. I decided to check the SD-overclocking feature alone. I removed the CF card and left only the SD card with ML on it in the camera. Then I activated SD-overclocking, waited until it is patched, (status ON) and pushed recording. Recording did not work, I got the error message. Then I decided to format the SD card in the camera without turning SD-overclocking off. Formatting failed and I was prompted to replace the card. That was it. All further attempts to fix the card through formatting in the camera and PC via card reader failed. The card is dead. Not a big loss but still ... After that, I did not try other SD cards.
----------------------------------
Excellent work Ilia3103, Danne and everyone else involved. 12-bit UHD RAW-video recording at 24 fps is a real break through for the good old 5D3 !
-
What is the highest resolution that 16/9 can be recorded continuously for 12/14 bit?
There is a 3K preset that does 3072x1920, so you can record 16:9 in 3072x1728 (it's continuous I think)
@IDA_ML thanks for the great confirmation that it's working! Getting quite certain it's reliable now.
Unfortunately, I had to pay a price for my impatience. My beloved SD card died in these experiments. Here is how this happened:
III. I decided to check the SD-overclocking feature alone. I removed the CF card and left only the SD card with ML on it in the camera. Then I activated SD-overclocking, waited until it is patched, (status ON) and pushed recording. Recording did not work, I got the error message. Then I decided to format the SD card in the camera without turning SD-overclocking off. Formatting failed and I was prompted to replace the card. That was it. All further attempts to fix the card through formatting in the camera and PC via card reader failed. The card is dead. Not a big loss but still ... After that, I did not try other SD cards.
Sad about your card. Sometimes it's hard to tell if a card is dead or the file system is really messed up. Sometimes mkfs.vfat can fix things like that.
-
New builds
Usual links
- Removed sd_uhs module. Too fishy. Not worth it. If anyone wants to test it check commit to compile it again:
https://bitbucket.org/Dannephoto/magic-lantern/commits/a61ec50df0059d3ecd7e2356d4e0b8ad9157d37d
-
There is a 3K preset that does 3072x1920, so you can record 16:9 in 3072x1728 (it's continuous I think)
Can anyone please test this and confirm that it works continuously with 12/14 bit?
3072*1920 is considerably higher resolution at 5.9 megapixels, I'm surprised to hear the camera can support it with Raw.
If 3072x1920 is truly possible for 12/14 bit with 5.9 MP -- could 3264x1836 be supported?
That would be even better, pushing the max for 16/9.
3072*1920=5,898,240
3264*1836=5,992,704
3232*1818=5,875,776
3200*1800=5,760,000
Could it be possible to add 3264x1836 or 3232x1818 and/or 3200x1800 modes?
If 5.9 megapixels is the true limit, 3.2K would be the best possible 16/9 Raw output for this camera.
3200x1800 also has a nice upscale ratio as it's a perfect 6/5ths upscale to 4K.
I think it's important to add a preset supporting the maximum stable 16/9 resolution for Raw, as that's one of the most basic functionalities to get right before other unique aspect ratios.
A 3.2K 16/9 preset supporting 12/14 bit would be fabulous.
-
@Boobie Just tested 12 bit 3072x1728 - it's continuous with card spanning, but just on the edge. If your shot suddenly becomes overexposed, recording indicator goes orange and it will be at risk of stopping.
If 3072x1920 is truly possible for 12/14 bit with 5.9 MP -- could 3264x1836 be supported?
That would be even better, pushing the max for 16/9.
3072*1920=5,898,240
3264*1836=5,992,704
3232*1818=5,875,776
3200*1800=5,760,000
I've thought about that before, but have no idea how to implement it. It may be possible, but it's just as likely not - control of the sensor isn't very exact and there's always unpredicatble black areas coming out, they may be bigger or smaller depending on how it's configured. (at least from what I've heard from reddeercity with the 5D2, maybe its not like this on the 5D3)
-
@Boobie Just tested 12 bit 3072x1728 - it's continuous with card spanning, but just on the edge. If your shot suddenly becomes overexposed, recording indicator goes orange and it will be at risk of stopping.
Thank you very much for testing. Did you record at 24p or 30p?
I'm uncertain if MLRAW is RGB 36 bits or RGGB 48 bits pre-compression, would love to learn more.
-
I'm uncertain if MLRAW is RGB 36 bits or RGGB 48 bits pre-compression, would love to learn more.
MLRAW is 10/12/14 bit. RAW is B/W (one channel). Color comes with debayering.
-
MLRAW is 10/12/14 bit. RAW is B/W (one channel). Color comes with debayering.
I understand, I'm saying 12+12+12 versus 12+12+12+12.
-
I shot 24p (23.976fps)
The raw without being compressed is not RGB, it's black and white (Bayer pattern), each pixel has only one channel, red/green/blue (nothing bad all raw cams are like that)
And it's tightly packed 12/14/10 bit, no 16 bits anywhere. Uncompressed bitrate in MB/s is (width*height*bitdepth*fps)/(8*1024*1024)
For example (3200*1736*12bit*23.976fps)/(8*1024*1024) = 190.5 MB/s
+ compression makes that smaller by ~40%
-
hey everyone,
i'm a fan of the sd overclock! don't drop it, it's worth keeping! i used it to shoot my new candlelight music video continuously at 3072*1308 (4 minutes continuous!) i've also used it on the sd card alone for 1920 continuous with no cf card in the camera - all 14 bit :)
here's a few observations from my experiences with it...
it only works for me on the new sandisk extreme pro card i purchased for using it. none of my other cards work with it.
it only works reliably for me if the camera is started with overclock DISABLED. enable it after startup, and then use it. disable it again before shutdown.
i suggest for the time being to make it disabled at startup by default. the loss of a few seconds to enable it is far better than potentially unreliable operation...
it's a real drag that formatting the card with it enabled could fry the card. that's valuable information!
one other thing to mention: the camera definitely heats up when pushing it this hard. we should really be paying close attention to this. i shut down and let it cool down after a few takes when the temperature gets scary! these temps could have consequences, so everyone should be aware and careful!
thanks for the great work, everyone!
check out my candlelight video to see it in action :) i'll eventually post a 4k version when av1 finally gets adopted.
-
Sorry. It´s disabled. As mentioned. Follow my commit to compile the module locally. Might look into fixing it in the future.
-
Removed sd_uhs module. Too fishy. Not worth it.
Not only is it worth it! SD-overclocking is the key to the success of this entire card spanning project! The enormous extended functionality of the EOS-M and the 100D would have been impossible without SD-overclocking. These 10-13 MB/s on top of the 40 MB/s factory write speed were the key to the success of these two cameras. Same for the 5D3 too. Very little more write speed is needed to achieve continuous recording at key modes such as the anamorphic 3840x2160/12bit lossless/24 fps or the 3072x1728/12bit lossless/24 fps or 1920x1080/48 fps without the need to reduce frame rate or lower exposure. I wish, I hadn't mentioned my damaged SD card at all. I just wanted to warn people not to repeat my mistake. A damaged card should not be a reason to stop the development on this issue of crucial importance!
-
So... good opportunity to bring back the module screen 😉
-
Opportunity for others to get to learn code/compiling basics...
-
I woukd like to add
If you remember in the begging of the week
I told you my card die
It was a cf card
And after a few tests o wanted to format the card to record my actuall takes
So i format it and %×@@#@ a big explotion happend
No just kidding, no explotion just a died card
Could it be a good tip to not format the card?
Sent from my SM-N975F using Tapatalk
-
Getting quite certain it's reliable now.
Ilia3101,
Indeed it is very reliable. I shot tons of footage with card spanning ON, (SD-overclocking OFF), over the weekend and never had a glitch. Moreover, it is very user friendly too. You activate card spanning just once and forget about it. In the end of the shooting day, you just dump the content of both cards in the same directory and forget about merging too. MLVApp just does the rest and never complains. Safe and reliable SD-card overclocking, as in the 100D and EOS-M, is the only thing needed to make card spanning perfect. If that is not worth it, I don't know what is.
-
I would like to try and correct my bad english.
I lost (died) a card (CF KB128GB) this weekend due to a format (usnig canon manu) when the card was full.
After format the card it became un accessable from any system.
Was testing a scene, recorded few sampales
And for unknown resone I got a massage saying card full
I didnt gave it much thinking and went strait to format it from canon manu.
Maybe formating the card right after recording is a bad move ?
And a shutting off the camer and let is rest before would lead diffrent results
Sent from my SM-N975F using Tapatalk
-
Bug report: when recording uncompressed 12/10 bit, the white level metadata is always set to 16200 (that value is only suitable for 14 bit)
-
Good news guys! This morning I decided to test some spanned card footage with MLVFS to use it with DaVinci Resolve directly. And it works absolutely fine. I tested anamorphic 1920x2332 and 3k 1:1 3072x1728 at 24 fps and 10, 12 and 14 bits lossless and Resolve crunches it like cereal for breakfast. Average export times of the processed video on an i7 laptop with a 4 GB of RAM GPU are 7,5 to 9 fps with Davinci Resolve Lite 16 using the MPEG4 export format at the original resolution. This card spanning functionality is a real break through for the 5D3! Thanks so much, everyone involved!
-
Bug report: when recording uncompressed 12/10 bit, the white level metadata is always set to 16200 (that value is only suitable for 14 bit)
That is a problem also with official experimental builds?
-
I don't know. But it's caused by line 3081 in mlv_lite.c - just comment it out and everything is fixed - it makes the wrong assumption that if crop_rec bitdepth is not set, that the image must be 14 bit:
if (!bitdepth) raw_hdr.raw_info.white_level = 16200
https://github.com/ilia3101/mlv_lite_spanning/commit/033c0d4801043b0d2e31ae101854fabbd40616f9
@IDA_ML Great!
-
So you record uncompressed 12 or 10 bit plus adding reduced bitdepth in crop rec?
-
No, just uncompressed 12/10 bit set from mlv_lite.c - without setting bitdepth in crop_rec (set to OFF).
-
I see. Will check it. Never used uncompressed since lossless came into the picture.
-
I never use it either. I was just generating some files to test LibMLV.
-
Is there any SD vs CF size proportion recommended? For example, if I have 128Gb Lexar CF what size SD card should I buy? I assume there is no any reason to buy 128 SD card?
-
you should be fine with a 64gb sd card, but there's nothing wrong with having extra capacity!
the 170mb/s sandisk extreme pro is almost cheap on amazon... it works beautifully for me!
-
I have tried this version : crop_rec_4k_mlv_snd_isogain_1x3_presets_2020Feb01.5D3113
The mlv files can't be open in any software (Photoshop, Ae, MLVFS, MLV to CDNG on the fly). Only MLVproducer were able to do but it gave me a glitch pink image almost like snow black and white tv...
To record I have tried almost every option in the presets...
Do somebody know this problem ? Do I do something wrong ?
Thanks (:
-
Do somebody know this problem ? Do I do something wrong ?
No problem at all! Just try MLVApp. It opens MLV files from your camera directly.
-
I have tried this version : crop_rec_4k_mlv_snd_isogain_1x3_presets_2020Feb01.5D3113
The mlv files can't be open in any software (Photoshop, Ae, MLVFS, MLV to CDNG on the fly). Only MLVproducer
Did you try MLVApp?
-
Wonderful Thanks a lot MLVapp work so well !
-
Seems that there is no true 12/10 bit compressed recording? It's all gain based?
I recorded in "12 bit lossless" data format, and white level was at 6000 or something. Not below 4096 like I expected. And black was 2047 just like 14 bit (would be 512 in true 12 bit)
So there is no difference between using crop_rec bitdepth and mlv_lite data fromat lossless options?
-
Seems that there is no true 12/10 bit compressed recording? It's all gain based?
I recorded in "12 bit lossless" data format, and white level was at 6000 or something. Not below 4096 like I expected. And black was 2047 just like 14 bit (would be 512 in true 12 bit)
So there is no difference between using crop_rec bitdepth and mlv_lite data fromat lossless options?
Told you already ;)
But what is true 10/12-bit according to you?
-
came over from the EOS M and Dannes builds... love those - but now completely blown away by what the 5D3 is capable with this build! Just awesome.
One thing that I stumbled over today was the following behavior: when switching on the camera or going from still mode in movie mode, while having one of the presets "1920 1:1" or "mv1080p_mv720p" active then at first i just get a black screen. If i half push the shutter button to go in zoom mode, liveview appears and stays... behavior is from then on perfectly normal. Same if hit menu button (ore.g. also trash button) and come back. It`s not happening if the cam goes in standby, and mirror flips back when waking it up again. I can`t remember having this behavior the days before... Is this a known thing? Only difference I can think of, of doing today is changing frame-rates in canon menu for 60 fps recording. But of course I went back afterwards to 1920/25/All-I.
Already set back everything and did a completely new installation of ML. I am using "crop_rec_4k_mlv_snd_isogain_1x3_presets_2020Feb19.5D3113"
Thanks for any reply if the fault is on my side - if not, maybe a hint for an open ML issue.
Best, Larouso
-
I didn't have time to look into quirks too much changing from movie to photo mode.
As you say. 60 fps could cause some issues as well as waking cam from sleep mode. Hopefully it will be useful with some workaround as I pretty much have no time to dig into the code atm.
-
Told you already ;)
But what is true 10/12-bit according to you?
Sorry, was going to answer but forgot.
I meant "true 10/12 bit" as in: read from sensor the same way as usual (14 bit), but converted down to 12/10 bit in digital space (bits cut off)
Also "true" 12/10 bit would be within 0-4095 or 0-1023, currently it's 2048-~6000 for 12 bit ;)
It works fine though, compression is improved.
Only problem is how the MLV App stripe algorithm sees it as 14 bit, so it can't find any "highlights" to measure stripes.
-
Hopefully it will be useful with some workaround as I pretty much have no time to dig into the code atm.
yes of course - like mentioned, the workaround is easy and fast - just wanted to mention it... since the build looks besides that pretty damn perfect :)
Happy easter days for all of you!
-
Hello, I would like to know if you have to install the ML firmware on the 2 sd and cf cards at the same time, to record with both simultaneously, how the expansion sd works and what limitations this type of recording has on 5d3 in short.
Thank you very much.
-
Hello, I would like to know if you have to install the ML firmware on the 2 sd and cf cards at the same time, to record with both simultaneously
No, you only need to install it on one of the cards.
how the expansion sd works and what limitations this type of recording has on 5d3 in short.
It works by writing files to both cards at the same time (.MLV on CF card, .M00 on SD card), you must put files from both cards in the same folder before you can process them as videos. No limitations anywhere, only increased speed.
-
Sorry guys! I'v just read whole "crop_rec on steroids" topic from first message to last and now see that there is another topic about same theme. My mind already blowing so please could you tell me in short what advantages in this another crop module (with compare of last experimental one from july 22) at this moment? I use 5dm3 1.2.3
-
Great work,Danne!
Solved so many problems!
Keep going!
-
Hello, thanks for the answer the other day but I can't find the menu to record the 1920 x 2332 anamorphic adding the two sd + cf cards and at the moment I can only get to 1808, how and where are the settings in the magic latern menu Simultaneous recording and sd speed unlocking.
Thanks again.
-
I kindly ask if is possible to have an updated (last) zipped compiled file to load into 700D 1.1.5
Now I have crop_rec_4k_mlv_snd_isogain_1x3_presets_2019Apr02.700D115 from Danne repository
I'm able to compile from the unified main branch but I'm not able (I'm sorry) to compile from this branch :'(
Thanks for your patience :)
-
Nothing new for the 700D so no need for updating.
-
Nothing new for the 700D so no need for updating.
Thanks a lot @Danne
-
Hello, please I can't find in the last update for 5d3 firmware 123 from February 19th where is the module sd_uhs card spanning nor the overclocking and the menu to record sd+ cf, someone can guide me.
Thank you.
-
Read answer #363.
-
How to turn off 1.67 stretch in the raw video mode?
Is there a option？ Or How to have the option?
Thanks a lot.
-
New version:
https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899
- Added Custom modes same as for eosm here:
https://www.magiclantern.fm/forum/index.php?topic=24993.msg226973#msg226973
-
Hello, I would like to see how the m00 files are reproduced and edited in the mlv app, the time recorded on the sd card with the spanning function is lost when trying to import the mlv file from the cf card, there is some operation to be done with the m00 file to be recognized by the mlv app ?, I have not found a solution for the 5d3 with the experimental version May 17.
Thank you.
-
Just make sure the .M00 files are in the same folder as the .MLV files. Then just open the MLV file and MLV App will use the .M00 as needed.