As you probably have guessed from the latest developments (QEMU (http://www.magiclantern.fm/forum/index.php?topic=2864.125), EDMAC graphs (http://www.magiclantern.fm/forum/index.php?topic=18315.0), JPCORE (https://www.magiclantern.fm/forum/index.php?topic=18443.0), EEKO (http://www.magiclantern.fm/forum/index.php?topic=13408.msg175656#msg175656)), our understanding on how LiveView works has improved considerably. Finally, all my fiddling with QEMU, at first sight with little or no purpose for the everyday users (http://www.magiclantern.fm/forum/index.php?topic=2864.msg169970#msg169970), starts paying off.
Today, Magic Lantern proudly announces new ground-breaking features that were previously thought impossible or very hard to achieve.We proudly present....
4K RAW Video Recording!
(https://a1ex.magiclantern.fm/2017Apr01/fishy_4k_april_fools.png)
DOWNLOAD (http://builds.magiclantern.fm/experiments.html)
Twitter (https://twitter.com/autoexec_bin/status/848102541305020416) announcement (https://twitter.com/autoexec_bin/status/848313311141789696)
(http://a1ex.magiclantern.fm/2017Apr01/4k.png) (https://a1ex.magiclantern.fm/2017Apr01/50p.png) (http://a1ex.magiclantern.fm/2017Apr01/allmodes.png)
On the 5D Mark III, you now have the following new resolutions:
* 1920x960 @ 50p (both 1:1 crop and full-frame - 3x3 pixel binning) - continuous
*)* 1920x800 @ 60p (same as above) - continuous
*)* 1920x1080 @ 45p and 48p (3x3 binning) - continuous at 45p
*)* 1920x1920 @ 24p (1:1 square crop) - continuous
*)* 3072x1920 @ 24p (1:1 crop)
* 3840x1536 @ 24p (1:1 crop) (corrupted frames at 1600)
* 4096x2560 @ 12.5p (1:1 crop) - continuous
*) at 8 FPS
* 4096x1440 @ 25p (1:1 crop)
* Full-resolution LiveView: 5796x3870 at 7.4 fps (128ms rolling shutter) - continuous
*) at 5 FPS!
* Full-width LiveView - decrease vertical resolution in the crop_rec submenu, all the way to 5796x400 @ 48 fps :)
The last feature complements the well-known full-resolution silent pictures (http://www.magiclantern.fm/forum/index.php?topic=12523.0) - the new implementation will be usable at fast shutter speeds, without the exposure gradient (http://www.magiclantern.fm/forum/index.php?topic=12523.msg120750;topicseen#msg120750) - but with rolling shutter (just like regular LiveView frames).
*) Continuous recording for the above resolutions can be achieved as long as you can get a LJ92 (http://www.magiclantern.fm/forum/index.php?topic=18443.msg182074#msg182074) compression ratio (compressed / 14-bit uncompressed) of about 50-55%, with preview set to Frozen LV (previously known as Hacked Preview) for an additional speed boost. Otherwise, you'll have to reduce the resolution or the frame rate.
The following table shows how compression rate changes with ISO and bit depth; please check the figures for your particular scene in the raw video submenu, as they can vary a lot, depending on the scene content.
Quote from: a1ex on April 15, 2017, 01:12:36 PM
Bits per pixel 14 12 11 10 9 8
ISO 100 1/100 61% 53% 50% 48% 46% 43%
ISO 200 1/200 62% 54% 51% 49% 47% 44%
ISO 400 1/400 63% 54% 51% 49% 47% 45%
ISO 800 1/800 65% 55% 52% 50% 48% 46%
ISO 1600 1/1600 67% 56% 53% 50% 48% 46%
ISO 3200 1/3200 70% 57% 53% 50% 49% 47%
ISO 6400 1/6250 76% 60% 55% 52% 50% 48%
ISO 12800 1/12500 79% 63% 57% 53% 50% 49%
Credits: Greg (full-width LiveView (http://www.magiclantern.fm/forum/index.php?topic=10111.msg123909#msg123909)), g3gg0 (video timer (http://magiclantern.wikia.com/wiki/VideoTimer), DIGIC registers (http://magiclantern.wikia.com/wiki/Register_Map) documentation and lots of other low-level insights).
Complete list of new video modes:
/* 24p 25p 30p 50p 60p */
[CROP_PRESET_3X_TALL] = { 1920, 1728, 1536, 960, 800 }, /* 1920 */
[CROP_PRESET_3x3_1X] = { 1290, 1290, 1290, 960, 800 }, /* 1920 */
[CROP_PRESET_3x3_1X_48p] = { 1290, 1290, 1290, 1080, 1080 }, /* 1920; 1080p45/48 <- 50/60p in menu */
[CROP_PRESET_3K] = { 1920, 1728, 1504, 760, 680 }, /* 3072 */
[CROP_PRESET_UHD] = { 1536, 1472, 1120, 640, 540 }, /* 3840 */
[CROP_PRESET_4K_HFPS] = { 2560, 2560, 2500, 1440, 1200 }, /* 4096 half-FPS */
[CROP_PRESET_FULLRES_LV] = { 3870, 3870, 3870, 3870, 3870 }, /* 5796 */
What else could you wish for?
FAQWhere's the catch?This is only a very rough proof of concept. It has not been battle-tested and has many quirks. Some of them may be easy to fix, others not so. In particular:
* It feels quite buggy. I'm still hunting the issues one by one, but it's hard, as Canon's LiveView implementation is very complex, and our understanding on how it works is still very limited.
* Write speeds are high. For example, 10-bit 4096x2500 at 15 fps requires 180 MB/s. 1080p45 should be a little more manageable at 111 MB/s.
* Canon preview is broken in most modes; you need to use the grayscale preview in the raw recording module.
* High-resolution modes (in particular, full-res LiveView) may cause trouble with memory management. This is very tricky to solve, as we only get 3 full-resolution buffers in LiveView, with restrictions on the order in which they must be freed, and lots of other quirks.
* Since these settings were pushed to limit, the risk of corrupted frames is high. If it happens, decrease the vertical resolution a bit (
from the crop_rec submenu).
* When refreshing LiveView settings, the camera might lock-up (no idea why). Pressing MENU twice appears to fix it.
May I fine-tune the new modes?Yes! I've included some of the knobs on the user interface. Normally you shouldn't need to touch these buttons, but if you do, you might be able to squeeze a few more pixels.
Does it work with FPS override?Sort of. It's not reliable at this point, so it's best not to try yet.
Overheating?During my tests, I didn't manage to get a sensor temperature higher than 60 degrees. Your mileage may vary.
Risks?This mod changes some low-level sensor parameters that are not well understood. They were all figured by trial and error, and there are no guarantees about the safety of these changes.
As usual, if it breaks, it's your fault, sorry.
Will it work on other camera models?I hope so; however, this is an area where I hope to get contributions from others (yes, from you). If these new features don't motivate you to look into it, I wonder what else will.
I'll explain how all this works in the coming days or weeks.
Is it difficult to port to other camera models?So far, the 3x3 720p mode from crop_rec was ported to EOS M (rbrune), 700D (dfort) and 100D (nikfreak). So it shouldn't be that hard...
Will you port this to my camera model, please?No, sorry. I have better things to do - such as, preparing the April 1st prank for next year :)
Wait a minute, didn't you say you are primarily a still photo user? Why are you even doing this?If you look close, the usefulness for video is fairly limited, as the write speeds (and therefore the recording times) are not practical.
But the full-resolution LiveView is - in my opinion - very useful for still photo users. Although the current implementation is not very polished (it's just a proof of concept), I hope you'll like the idea of a 7.4 FPS burst mode, 100% silent, without shutter actuations.
Right now, you can take the mlv_lite module with pre-recording and half-shutter trigger: at 10 bits per pixel, you get 5 frames pre-recorded, and saved to card as soon as you touch the half-shutter button. Or, you can capture one frame for each half-shutter press, with negative shutter lag! (since the captured frame will always be pre-recorded).
And if a burst at 7.4 fps is not enough, you may also look at the 4K modes (12-15 fps).
(I know, I know, GH4 already does this, at much higher frame rates...)
The help menu for full-res LiveView says 5796x3870, but MLV Lite only records 5784x3856. What's going on?The raw recording modules have a couple of alignment constraints (e.g. can only start cropping from a multiple of 8 pixels, and the size of the cropped area (that goes into the MLV file) must be multiple of 16 bytes (that is, W*bpp/8 + H mod 16 must be 0).
To capture the full resolution, you may use the silent picture module. However, this module is not the best when it comes to memory management and buffering. Currently, you'll get an impressive buffer of 2 frames in burst mode :)
But hey - it outputs lossless DNG!
What about that lossless compression routine?It's included, although I didn't manage to test it much. There is a lot of room for improvement, but for a proof of concept, it seems to work.
update: also got lossless compression at reduced bit depths (8...12-bit).
P.S. The initial announcement was disguised as an April Fools joke, just like the original crop_rec (http://www.magiclantern.fm/forum/index.php?topic=17021.0).
Twitter announcement (https://twitter.com/autoexec_bin/status/848102541305020416)
From original April Fools post:
Quote
With our latest achievements in wizardry with ARM programming and DIGIC reverse engineering, we can speak of a new era of raw video recording.
On models like the 5D Mark III, the next upcoming releases will feature an improved version of our crop_rec module (http://www.magiclantern.fm/forum/index.php?topic=17021.0) that delivers the following new resolutions:
* 1920x960 @ 50p (both 1:1 crop and full-frame - 3x3 pixel binning)
* 1920x800 @ 60p (same as above)
* 1920x1080 @ 45p and 48p (3x3 binning)
* 1920x1920 @ 24p (1:1 square crop)
* 3072x1920 @ 24p (1:1 crop)
* 3840x1600 @ 24p (1:1 crop)
* 4096x2560 @ 12.5p (1:1 crop)
* Full-resolution LiveView: 5796x3870 at 7.4 fps (128ms rolling shutter).
The last feature complements the well-known full-resolution silent pictures (http://www.magiclantern.fm/forum/index.php?topic=12523.0) - the new implementation will be usable at fast shutter speeds, without the exposure gradient (http://www.magiclantern.fm/forum/index.php?topic=12523.msg120750;topicseen#msg120750) - but with rolling shutter (just like regular LiveView frames).
Please understand that providing the source code for those highly DIGIC optimized routines is a bit troublesome and will need some extra legal care. After this step is taken and as soon we are finished with ensuring the product quality you are used from Magic Lantern, we will upload the code to our repository.
Consider this being a huge leap towards our next mind boggling goal:
8K RAW Video Recording!
(https://a1ex.magiclantern.fm/2017Apr01/fishy_8k_april_fools.png)
Sample DNG from 5D Mark III, to show that our proof of concept is working:
8k.dng (https://a1ex.magiclantern.fm/2017Apr01/8k.dng)
Stay tuned for more information!
Who is ML's officiail bookie? I want to risk some Euro ...
Shut up and take my money!
(https://www.awesomegifs.com/wp-content/uploads/psy-elevator-gangnam-style.gif)
this is wonderful! i cant wait to test it on my camera at home!
INCREDIBLE!!!!! Wonderful job alex, i really appreciate all the hard work you and and the other developers have put in over the years to get magic lantern to where it's at right now.
ULTRA HD RAW 24p on 5d mark iii wooah :) ,so the most obvious question is when will we be able to test out this new update.
Feels like xmas. :)
Alex you are a genius.
Incredible. It,s art.
april 1st
Nope, I would rather say it's the whole truth as long as no one talks about expects continuous recording:
http://www.magiclantern.fm/forum/index.php?topic=18300.0
8) :P 8) :P :D :D ;D 8) 8)
KooooooooL 8) any chance of anything for the mark 2 or other cameras?
Please guys, I can't take the heart ache. Is this an April fools? Specifically the 960px 3x3?
I love how you get a 8K DNG sample from a 5d mark III, there's some real magic going on there ;D
Quote from: beauchampy on April 01, 2017, 01:29:47 PMIs this an April fools?
You don't expect an answer without tricks, traps, false mirrors and double meaning, do you? No today, not today ...
WOW! :o
8K is a bit noisy.
Thank you so much for everything you've done! Now, not to be demanding, but 16K is obviously doable so when can we expect it?
..it works perfect
Committed some of the sources; still playing with it, but those who can compile from source can already give it a try.
a1ex, which of the crop modes offer a real-time, colour preview?
Only the simplest crop mode (1920x1080) has good real-time preview. The modes with higher vertical resolutions have color preview too, but it's not centered (only the top of the frame is shown).
In the regular crop_rec branch, there's also centered 5x zoom. Here I've removed it temporarily, as it increased the code complexity quite a bit, but I'll probably add it later when things will settle.
I read all this and thought Great Joke ! :P but then I read the Source Code Holy Cow :o :o :o :o
There goes next few days , I'll be living in front of my computer exploring this and hopefully I can compile for 5d2 :D
reddeercity do you think that's possible yet? Wouldn't it require tons more reverse engineering?
Damn it I just happen to be away from proper computers that can compile ML for a week right now so I won't be able to try it if it is possible :'(
Build for 5d3 available? I just spent a week shooting 50p 3x: so would love to put it through its paces
Top of page -> Downloads
Download page -> Downloads (drop-down menu) -> Experiments
Quote from: Ilia3101 on April 02, 2017, 12:11:53 AM
reddeercity do you think that's possible yet? Wouldn't it require tons more reverse engineering?
Will not sure yet but I'm looking in to it .
If you read this post Full-resolution silent pictures (http://www.magiclantern.fm/forum/index.php?topic=12523.msg120497#msg120497) there's info about 5D2 and lead me to believe it maybe possible 8)
I'm currently try to compile the code now , but I get some error's .
some tweak.o stuff , I working of older code , so I think that could be the problem so I'll update to the latest source code
and try again . I don't think 4k is a reality on 5d2 but maybe 3K as the bandwidth for 4K is around 150MB from what I read.
Since we are limited to max. 80MB/s and by your Yet Another RAW Video Calculator (http://rawcalculator.bitballoon.com/calculator_desktop) at 10bit 3x crop 2496x1134 @ 2:2.1 A.R. is 80MB/s
unless Lossless compression is used and that's another thing by it's self .
Greg's original proof of concept (full-width LiveView (http://www.magiclantern.fm/forum/index.php?topic=10111.msg123909#msg123909)) was done on 500D (same generation as 5D2 and 50D).
Sure, it probably needs a bunch of reverse engineering, but it probably boils down to changing some ADTG registers and video timers.
I've actually got these presets working in adtg_gui first. A proof of concept for 3K was committed back in 2016 (https://bitbucket.org/hudson/magic-lantern/commits/50d8f06018) (also mentioned (http://www.magiclantern.fm/forum/index.php?topic=17021.msg171809#msg171809) in the "classic" crop_rec thread).
The registers interesting for changing resolution are documented either on the Register Map wiki page, or in adtg_gui source. Interesting commits: 8bbce9c (https://bitbucket.org/hudson/magic-lantern/commits/8bbce9c), 7333869 (https://bitbucket.org/hudson/magic-lantern/commits/7333869), 4a66a48 (https://bitbucket.org/hudson/magic-lantern/commits/4a66a48), 6c62a6b (https://bitbucket.org/hudson/magic-lantern/commits/6c62a6b), 38e78de (https://bitbucket.org/hudson/magic-lantern/commits/38e78de).
Thank you :)
Getting some compile error with mlv_rec and mlv_snd in crop_rec_4k branch.
Building module mlv_rec...
Updated HGVERSION
[ README ] module_strings.h
[ CC ] mlv_rec.o
mlv_rec.c: In function 'raw_rec_update_preview':
mlv_rec.c:4160:5: error: too few arguments to function 'raw_set_preview_rect'
raw_set_preview_rect(skip_x, skip_y, res_x, res_y);
^
In file included from ../lv_rec/lv_rec.h:24:0,
from mlv_rec.c:76:
../../src/raw.h:148:6: note: declared here
void raw_set_preview_rect(int x, int y, int w, int h, int obey_info_bars);
^
make[4]: *** [mlv_rec.o] Error 1
Don´t have my camera to test but I follow the error and change the line in mlv_rec to below which seems to get rid of the error code for both mlv_snd and mlv_rec.
In mlv_rec
raw_set_preview_rect(skip_x, skip_y, res_x, res_y);
to this:
void raw_set_preview_rect(int x, int y, int w, int h, int obey_info_bars);
Or maybe change to:
void raw_set_preview_rect(skip_x, skip_y, res_x, res_y);
?
this is wonderful. need to start looking for a used 5D mark III
Well I guess the original 4K trolls are no longer trolls.
But if 4K is possible that must mean 1000fps is as well.
Respect to a1ex and the whole magiclantern team.
No more excuses for not producing world class imagery.
Quote from: a1ex on April 01, 2017, 11:15:41 PMIs it difficult to port to other camera models?
So far, the 3x3 720p mode from crop_rec was ported to EOS M (rbrune) and 700D (dfort). So it shouldn't be that hard...
I'll take that as a compliment! :D :D :D
That reminds me, I should do a pull request for the 700D but it looks like there's more work to do now. How about getting some other cameras up to speed on the crop_rec module? By the way, don't expect 4k on anything other than the 5D3--then again maybe there will be more surprises.
Yep - my point was that people outside the core team are definitely able to play with this code (as in, it's not pure black magic).
Took this build out for a test (nothing fancy, just taking pictures of the kids with full-res LiveView and pre-recording). Noticed a major limitation (could not set shutter speeds fast enough for daylight) and pushed a fix for that.
After the fix, modes with reduced FPS (4K and full-res LiveView) now have the usual range of 1/4000...1/30 (or 1/60) mapped to 1/15000 to 1/fps. That's right - 1/15000 exposure time with 128ms rolling shutter :D
Next annoyance was the grayscale preview, which is slow and not working at all in 10-bit. Also, lossless MLVs are not playable in mlv_play. Will look into those.
a1ex, if you can improve the greyscale preview you will be the king of kings
These are just approximate calculations. You can initially estimate the speed of the sensor.
5D3 22MPx x 6fps = 132MPx/s
4096px x 2560px x 12.5fps = 131MPx/s
7D 18MPx x 8fps = 144MPx/s
70D 20MPx x 7fps = 140MPx/s
6D 20MPx x 4.5fps = 90MPx/s
500D 15MPx x 3.4fps = 51MPx/s :-[
Btw, my 4 year old smartphone 8MPx x 30fps = 240MPx/s :P
More accurate calculations - https://www.magiclantern.fm/forum/index.php?topic=12656.0
Of course we must remember about the speed of writing.
Quote from: a1ex on April 02, 2017, 08:58:03 PM
Yep - my point was that people outside the core team are definitely able to play with this code (as in, it's not pure black magic).
My point too. I'd like to get some more people to start playing with the code. There's lots of information in the comments and on the forum that takes the mystery out of what's going on--though some of what you're doing sure seems like pure black magic.
@Greg
and the rather slow 5D2: 21.2MPx x 3.9fps = 82.68MPx/s. Does this basically tell us the limit of the sensor? So this calculation works: 82.68/23.976 = 3.45MPx limit at 24p on this camera? So I guess it could do 2560x1320(3.38MPx).
:D how does one implement the "Greg's resolution hack" that I've been hearing about?
Fantastic! Thank you very much!
One Q about * 1920x960 @ 50p (both 1:1 crop and full-frame - 3x3 pixel binning).
1920x960 @ 50p (3x3 pixel binning) works good, but i can not set 1920x960 @ 50p for 1:1 crop, only 1920X632.
I set Crop mode - 1920 1:1 and 50p in Canon menu.
Quote from: Ilia3101 on April 02, 2017, 10:29:41 PM
:D how does one implement the "Greg's resolution hack" that I've been hearing about?
http://www.magiclantern.fm/forum/index.php?topic=19300.msg182149#msg182149
@Ilia3101 I'm working on it , just had some compiling issue with adtg_gui yesterday should have that resolve
today , then I can work off a1ex's links ;D
Quote from: vstrglv on April 02, 2017, 10:31:53 PM
1920x960 @ 50p (3x3 pixel binning) works good, but i can not set 1920x960 @ 50p for 1:1 crop, only 1920X632.
I set Crop mode - 1920 1:1 and 50p in Canon menu.
1920 1:1 tall (meaning "with increased vertical resolution", but couldn't fit that into menu)
Thank you! It works good!
Updated the preview with something a little more usable. It's color (though still low-res and non-realtime) and only drops to grayscale when recording speed becomes critical.
Unfortunately, g3gg0's raw_twk (https://www.magiclantern.fm/forum/index.php?topic=16682.0) routine doesn't appear to work in LiveView, so we are stuck with (slow) CPU-based previews for now :(
I can probably get faster previews in 14-bit lossless mode, by configuring the source raw stream as 16-bit (unpacked), since the compression routine accepts 10/12/14/16-bit input (http://www.magiclantern.fm/forum/index.php?topic=18443.msg181620#msg181620). For uncompressed modes, I'm afraid we currently don't have a way to get both unpacked data (for fast previews) and bit-packed (for recording) while in LiveView (so all my previous ideas (http://www.magiclantern.fm/forum/index.php?topic=5601.msg177531;topicseen#msg177531) are currently fiction).
New build posted on the Experiments page.
Trying to see what the next step is to see if there are any modes 700d can get, first i messed with atdg tool to try to reproduce the 3x3bin mode while in 720p by changing 0x4 to 0x2 on 800c just to understand how the tool works since those values are posted. I then set the tool to show values that have been changed more then twice and screen captured every value for 1080p, 1080p x5 and x10, 720p, 720p x5 and x10, crop mode in 1080p and also photo mode just to track all the values that are changing in case it was needed.
Thats where i am at now though, not sure what values to change though, gets scary when you see the crazy images on live view that make u think u burned sensor out. Only thing i saw that did anything as far as the size was 800c changing to 0x0 makes a zoomed version but its stretched tall. Other then that another one moved the image up and down.
Dont know how to cross reference with the 5d3 cause i see Pack12(?,?) for example and the values are not like what you see in the tool. So pretty much clueless on where to go from here??
Wow! That's a hell of a lot more useable and the preview is centered which is absolutely kickass.
I get 30 frames in UHD, but 3K is still continuous for me in crop_rec. The preview does drop back to greyscale about 50% of the time in 3K. The colour preview is more detailed and has a higher refresh rate... it's pretty much usable, I think.
I'm trying it out but unfortunately when I try to watch it on MlRawViewer or export it to DNGs using RAWMagic I'm left with this: https://drive.google.com/open?id=0B1lko58z3g6lRTlKUWc1YzlzOFk. I tried re-downloading the build and formatting my card but I can't seem to get anything out of any of these crop modes...
I've found a couple of issues in both builds. The first is nothing major: when you change a resolution in the crop mode menu, say 3K to UHD, you have to exit the ML menu before you can adjust the resolution size in the raw video menu. The other issue is making me pull out what little hair I have left: the lossless compression ratio appears to have a mind of its own, it's like a recalcitrant AI. At one point it was compressing at 51% on the 1st build:
(http://i.imgur.com/6IX4XKI.jpg)
When I loaded the 2nd build it started at 85%:
(http://i.imgur.com/HB2uNHV.jpg)
So I had to scale down to:
(http://i.imgur.com/bXY36uK.jpg)
After a few seconds of recording the compression went to 70%:
(http://i.imgur.com/Ojd4ykL.jpg)
Then when I try to increase the resolution I get this [Expect xxxx frames]:
(http://i.imgur.com/NckArF4.jpg)
A battery pull gets rid of the [Expect xxxx frames], but on the 2nd build it reappears when I try to scale up the resolution. So I went back to the 1st build to see if I could record at 3.3K 51% lossless again. Initially I was able to get this setting:
(http://i.imgur.com/Mj734Xe.jpg)
That was a bit much for the card, it stopped recording after a few seconds resulting in the dreaded [Expect xxxx frames]:
(http://i.imgur.com/kR4G8cb.jpg)
Taking me back to square one where I had to scale the resolution back down:
(http://i.imgur.com/gNnZrKY.jpg)
Ideally the compression ratio would either be fixed, or manually adjustable.
Quote from: squig on April 03, 2017, 04:11:54 AM
Ideally the compression ratio would either be fixed, or manually adjustable.
That's mathematically impossible, and not how lossless compression (https://en.wikipedia.org/wiki/Lossless_compression) works. The compression ratio you get is totally dependent on the scene, and it's even theoretically possible that the compression results in larger file sizes than the original uncompressed data. This typically happens when you give a lossless compression algorithm data that has very little redundancy (already compressed), or that is not of a type the algorithm is "tuned" for, for example feeding the compressor 10-bit raw data results in the compressed version actually being larger (http://www.magiclantern.fm/forum/index.php?topic=18443.msg176699#msg176699).
For the most part, lossless compression algorithms work by simply eliminating redundant data via more efficient encoding. Take some text for example, encoded using ASCII, each character takes 1 byte to store and therefore all characters have equal weight even ones that are rarely used. However we know that in any particular language there are characters (or even sequences of characters) that typically appear more often than others, so if we gave the characters that appear more frequently, shorter encodings (which subsequently requires less common characters to have longer encodings), then overall we can save space. However if we were to feed random gibberish to such an algorithm, it might actually take more total space the store than the original simple encoding, since the characters with longer encodings are just as likely to appear as ones with shorter encodings.
Understood, but from a cinematography perspective; I need to figure out what resolution will give the card enough headroom to write at that resolution continuously at least 90-95% of the time; so far it looks like that's somewhere in the 2.5-2.6K range for 24p 2.39:1, which is still great.
2.7-2.8K 2.39:1 looks like the sweet spot for repeatable continuous recording on the Toshiba 1066x card. By "repeatable" I mean repeatable near 100% of the time. Same goes for 3.3K anamorphic scope: 2272x1364 with a 1.5x anamorphic lens.
Quote from: a1ex on April 02, 2017, 11:59:08 PM
Updated the preview with something a little more usable. It's color (though still low-res and non-realtime) and only drops to grayscale when recording speed becomes critical.
DAMN ! This update is very promising. I did some tests and yes it's more comfy for framing.
Thanks you for all these exciting features A1ex and all the team.
OK, I'm pretty sure that the 14bit lossless compression varies with ISO.
In crop_rec 3K (3072 x 1286 ) I can get continuous recording at ISO 100
But if I whack the ISO up to 6400 I only get eleven seconds.
that makes sense because the more noise of 6400 iso would result in less compression
Happy Aprils fools day :)
How do I go about processing these mlv files with the mlv_dump included? I can't figure out how to spit dng's out
I use this method on Windows:
http://www.magiclantern.fm/forum/index.php?topic=10526.msg178538#msg178538
I salute the dedication but I wish squeezing every last pixel of out of the camera would be secondary to allowing audio with simple RAW. I would trade every last improvement for 1080p RAW with audio. For some reason, this worked for me once, but it just randomly stopped. It used to create a WAV file along with the RAW file.
100 ISO - black screen ~49%
6400 ISO - black screen ~60%
6400 ISO - busy scene, fast movement ~69%
Ah that explains a lot.
High iso less compression, low iso high compression.
So for low iso shooting, compressed 14 bit is more efficiënt then shooting in 10 bit
Quote from: hyalinejim on April 03, 2017, 03:17:46 PM
I use this method on Windows:
http://www.magiclantern.fm/forum/index.php?topic=10526.msg178538#msg178538
Thank you, exactly what I was looking for.
Quote from: squig on April 03, 2017, 05:15:03 AM
I need to figure out what resolution will give the card enough headroom to write at that resolution continuously at least 90-95% of the time
Latest update estimates compression ratio on the fly, from the current scene and exposure settings. Also a few minor bug-fixes.
To get the best possible compression, all you need is to set ISO to maximum, shutter to 1/FPS, aperture wide-open and use a bright scene. You'll get close to 35% (http://www.magiclantern.fm/forum/index.php?topic=18443.msg181620#msg181620) (apparently the limit for this codec, or maybe just for Canon's implementation).
Generally, if pixel values are predictable (from their neighbors), the image will compress well. That usually means low noise levels (low ISO, shadows, fully clipped highlights) and few details (large out-of-focus and/or low-contrast areas). Note that absolute noise values are higher in highlights, and the SNR is higher too, so exposing to the right will give higher file sizes (but that shouldn't stop you from using it).
Fast motion probably gives better compression, because of motion blur (didn't test, but that's my understanding with this codec).
Invalid image data (that includes e.g. 14-bit raw data interpreted as 10-bit, or pure noise, or machine code if you want) will give compressed output larger than initial size. In current implementation, this gives memory overflow. For some unknown reason (alignment issues? no idea), setting a maximum size for compressed output causes camera lock-up at certain resolutions, with no apparent logic (see this commit (https://bitbucket.org/hudson/magic-lantern/commits/903c4828d514d3a8f2e3bced77318acdd8fffc86); minimal example here (http://a1ex.magiclantern.fm/2017Apr01/crash.c)).
For 1.23 only? I don't have a crop mode option in 1.13.
Did you get the latest experimental build, and load the crop_rec module?
I wonder if there is a potential for ordinary resolution recording but with higher frame rates.
Something like 100 FPS ?
One question to the developpers do you know if the 3 things below would be possible one day:
1/ 1080x1080 (from 3x3 bining using the 3240x3240 sensor size). for a big anamorphic s65 lens.
2 / in the same way of the 14 bit losless mjpeg compression is an 8 bit mjpeg compression possible? so we can do low file size and get the real sharpness the 5d3 can offer (clean HDMI recorder on prores does not get any better sharpness than the in camera h264 file - whereas the raw files is always resolving much more sharpness)
3/ finally how far would you say you are from a working preview for the mode 3K, UHD, or zoom x5 and is it even possible?
Also you probably have thought this but the basic full sensor preview with blacktape on the sceen could make it and if the 1:1 sensor readout cannot show original preview then maybe a x3 preview instead of the x5. Would not be perfect but would definitly make it usable.
Thanx for the work it's a dream. Where is the TIP JAR already?
Bests.
I started testing 14 bit lossless with 1080P 24 because thats what I use the most. File sizes are much smaller, thats nice. Something funky is happening though. Using the histogram overlay with 80% or more of my exposure on the right half I'm getting horribly, horribly underexposed results in davinci. The live view looks great, histogram looks great, but then it hits davinici and the footage looks awful. Not sure what to make of it, but thought you guys might want to know.
So after a few hours playing around with A1ex's second release of the 1.1.3 firmware, here's what I'm getting with 14bit lossless option:
3520 x 1320 = max. 1314 frames - ISO within 200-400 range - scales up to UHD with no perceived loss of quality
4096 x 1862 = max. 507 frames @ 12.5 fps - Not sure how useful this would be for landscape work or slow shots. Again ISO within 200-400 range.
Previews: At the moment, not fantastic but definitely better than a black screen. I was able to rack focus with a moderate degree of success.
Camera crashes persistently when using 50/60p modes.
I'm using Komputerbay cards 64GB 1066x
IQ, even shooting at home with crap light, is just beyond words.
This is mighty fun to play with,
I was able to shoot up to 19 seconds at 3504x1312 at regular crop with Framed preview and Compressed Raw 14 bi, lots of movement, I was panning slowly downwards. But I only got so much recorded this one time, I never achieved the same length of recording again.
Is there a way of centering the crop? It looks weird not being centered.
2 things I learned:
1. If you half press shutter button while recording the preview stays in colour rather than alternating between colour and greyscale. Frame rate still lags but it makes things a hell of a lot easier to know what's going on in front of the camera.
2. Filming the moire pattern of a computer screen's pixel array and refresh rate will absolutely kill the compression and stop the recording, even at lower resolutions!
Quote from: Kharak on April 04, 2017, 01:48:08 AM
Is there a way of centering the crop? It looks weird not being centered.
It should already be centered. Are you loading the crop_rec module of the latest experimental build and then selecting 3K or UHD? Framing is off in 5x but should be centred in these modes. Sometimes 3x is shifted up for me. I just restart the camera and it's fine.
Why doesnt the latest package have the mlvrec 2.0 module .. its only has light. for 5d3123.
Hi,
Ive been testing the latest build (April 03)for 5dIII123.
So far what i noticed there is no mlv snd module
or mlv_rec. Also there is no gray scale preview.
Yeh I figured it out. I guess I was blind while looking at the modules or I wrote the files to the wrong sd card from my lexar workflow hub.
Getting pink frames and crashes in the 50fps modes.
Getting unreadable files in some other modes like the 3k and UHD. It was just an initial test, I will test more tomorrow. I am using 1066x 256gig lexar cards. However, they aren't formatted and have a lot of shots on them. I will format one and see what difference that makes.
Shooting with the Canon 5d3, 1.2.3 firmware and using the latest experimental build(April 3rd 2017) with Lexar 128gb 1066x cf card.
1080p 48FPS seems to be working (It seemed to have recorded fine, but I must have deleted the file.)
3520x1320 24FPS(ISO 100-200 iso seems to be continuous and at a 5x crop mode) ISO 400 recording stops after a few seconds. In no crop mode preview was colorful noise. in 5x crop preview worked great. I recorded 1510 frames and I ended it.
3584x1320 24FPS I recorded 1443 frames and I ended it.
1080p 60FPS(buggy, stops recording quickly) gray preview was glitchy and lagged.
3k (3072x1920@24fps) I got 66 frames.
UHD (3840x1536@24fps, couldnt get 3840x1600) I got 66 frames as well. It states its 120mb/sec, this might require the lossless compression with 10bit?
Both 3k and UHD frames are good, used MLV_DUMP from the page I downloaded the firmware to extract these.
I am also very tired, I will try and play around with them some more.
@A1ex, you are my king.
Ah, a little late to yet another party once again and after playing around in the past 48 hours or so I've finally come to a conclusion to be able to confirm that 3584x1320 in crop_rec (3K 1:1) can be recorded continuously
if shot in between ISO 100-200 and
more often than so in 400 depending on scene.
This was all tested with both 128gb 1066x & 256gb 1066x CF cards from KomputerBay on two separate 5D3's running 2017April03.5D3123 builds.
(https://c1.staticflickr.com/3/2828/33014736263_11f475c323.jpg) (https://flic.kr/p/Sip9SK) (https://c1.staticflickr.com/4/3929/33698823611_26f5ec5f2d.jpg) (https://flic.kr/p/TkRhdc)
Seems that ISO 800 and higher doesn't get along well with 14-bit lossless compression which makes sense as previously stated by others in here.
Here's an example of what it looks like upon running this beautiful hack via LiveView from 5D3 below and watch what happens when I push it up to ISO 800 (and higher) towards the end:
I was able to convert all 14-bit lossless files with ease from @Danne's latest cr2hdr.app_4k (includes the required certain mlv_dump which was modified directly from @a1ex) so I am more than flabbergasted by the fact that all of this is storming down on us like there's no tomorrow.
Love it and will continue to play with and hopefully run into some more bugs!
Also I can confirm that if I point the camera at a computer screen while recording and it'll crash once I focus upon the pixealated monitor (indeed this is quite strange but good find @hyalinejim) and I am actually
loving the half shutter press for FULL COLOR LIVE DISPLAY or whatever that's called (plus it zooms in a bit more to be sure everything looks spot on) before recording which is very handy.
Thank you @a1ex for yet another petrifying update (in a good way!) and don't ever change who you are! Ha. 8)
Quote from: DeafEyeJedi on April 04, 2017, 09:12:19 AM
Here's an example of what it looks like upon running this beautiful hack via LiveView from 5D3 below and watch what happens when I push it up to ISO 800 (and higher) towards the end:
https://vimeo.com/211439899
I cant seem to get that lovely liveview on 113. Mine is low fps, desaturated and choppy.
Running crop_rec_4k.2017.Apr03.5D3113 and I copied your settings..
I was just wondering...
Who has twixtor or something alike and creates some stunning 6K - 24p video out of that 7 fps full resolution silent video mode ;D 8)
Quote from: beauchampy on April 04, 2017, 09:36:31 AM
I cant seem to get that lovely liveview on 113. Mine is low fps, desaturated and choppy.
Running crop_rec_4k.2017.Apr03.5D3113 and I copied your settings..
Hmmm then perhaps I should downgrade to 113 and try to reproduce this?
Did you press 5x zoom after using ML menu or before? Sometimes it goes into a weird phase which may seem like having its own quirks but a simple restart (on/off) of the camera would help at least it does oh my end.
But it could very well be a bug for 113 and I'll let you know once I've downgraded.
Thought not too long ago @a1ex made a pull request for both firmwares (113 & 123) to be merged together after each update or does this only apply to the Nightlies and not experimental?
Well I'm actually tempted to check out the good old speedy 113... Just been using 123 as of late for HDMI output on gig's.
Actually this one trick may be worth to try (Thanks @dfort for the tips) perhaps reset your settings to default (from Canon menu) not the ML version and let me know if that helps either way.
Hi,
I don't know if it will be useful or not...
Testing latest build (April 3th crop_rec) on 5D3 113, for now only this mode because is which I find more useful for me: 1920x872 50p 3x3 (aspect ratio 2.20:1) 14bit lossless.
Yesterday I was able to record without problems in continuous with normal preview, in color.
Today, with the same settings as yesterday, it has been impossible to boot the camera. In the SD card there are text files called ASERT, one for each boot attempt. In them the following appears:
ML ASSERT:0at mlv_lite.c:1935 (compress_task), task compress_tasklv:1 mode:3
Magic Lantern version : crop_rec_4k.2017Apr03.5D3113
Mercurial changeset : 7b0cb9532629 (crop_rec_4k) tip
Built on 2017-04-03 14:54:32 UTC by jenkins@nightly.
Free Memory : 173K + 3807K
If I change SD card for another with 2017Feb11th 5D3113 (crop_rec proxy) all works again.
Hope it helps!!
Are you guys running ML off an SD card?
The best I can do @ 100 ISO is 3.3K 2.39:1. ML is on the CF card.
QuoteHi,
I don't know if it will be useful or not...
Testing latest build (April 3th crop_rec) on 5D3 113, for now only this mode because is which I find more useful for me: 1920x872 50p 3x3 (aspect ratio 2.20:1) 14bit lossless.
Yesterday I was able to record without problems in continuous with normal preview, in color.
Today, with the same settings as yesterday, it has been impossible to boot the camera. In the SD card there are text files called ASERT, one for each boot attempt. In them the following appears:
ML ASSERT:0at mlv_lite.c:1935 (compress_task), task compress_tasklv:1 mode:3
Magic Lantern version : crop_rec_4k.2017Apr03.5D3113
Mercurial changeset : 7b0cb9532629 (crop_rec_4k) tip
Built on 2017-04-03 14:54:32 UTC by jenkins@nightly.
Free Memory : 173K + 3807K
If I change SD card for another with 2017Feb11th 5D3113 (crop_rec proxy) all works again.
Hope it helps!!
I've erased all in SD card and copied clean April3th Crop_rec_4K build.
All works ok again like yesterday.
Same again:
ML ASSERT:0at mlv_lite.c:1935 (compress_task), task compress_task
lv:1 mode:3
Magic Lantern version : crop_rec_4k.2017Apr03.5D3113
Mercurial changeset : 7b0cb9532629 (crop_rec_4k) tip
Built on 2017-04-03 14:54:32 UTC by jenkins@nightly.
Free Memory : 177K + 3923K
Need to pull battery, and then camera doesn't boot anymore :(
Quote from: squig on April 04, 2017, 10:14:03 AM
Are you guys running ML off an SD card?
The best I can do @ 100 ISO is 3.3K 2.39:1. ML is on the CF card.
Yes. I was under the impression this frees up more bandwidth for recording to CF. Also, when you change CF cards your settings don't change.
OK cool, I'll give it a shot.
Seeing as we're all madly trying to squeeze every last MB out of the oldfella to get closer to 4K, would it be possible to add card spanning to these experimental builds?
I just downloaded the latest experimental build, but didn't get far.
I would welcome a little help, as I'm keen to test, but, I don't do video. That is I'm not up to speed on MLV etc.
If there is someone out there who has the time, maybe they would create a 1 page 'here is how to test', i.e. The steps.
In particular, how do I set up, run and process the full Res 'photo mode'. Once I know how to get things running, I can test.
Cheers
Garry
So what is the bottle neck for transfer speeds in the 5D3? Is it still the card reader in the camera? I thought cameras card reader was limited even with really fast cards?
Is spanning the trick for UHD?
Quote from: pc_bel on April 04, 2017, 10:13:46 AM
ML ASSERT:0at mlv_lite.c:1935 (compress_task)
This one indicates frame corruption, and should also print this (unless obscured by other messages):
printf("\nCompressed size higher than uncompressed - corrupted frame?\n");
printf("Please reboot, then decrease vertical resolution in crop_rec menu.\n\n");
Quote from: hindra on April 04, 2017, 06:28:06 AM
1080p 60FPS(buggy, stops recording quickly) gray preview was glitchy and lagged.
Took a quick look - after the latest fixes (see below), the following resolutions are continuous (1.2.3, lossless ~55%, Preview: Frozen LV, with grayscale preview):
- 1920x800 60fps
- 1920x960 50fps
- 1920x1080 45fps
1920x1080 48 FPS is a bit unreliable (corrupted frames with Global Draw on, but appears OK with it off). This mode is really pushed to the limit (and that's the reason I've also added a 45-fps option). Feel free to play with the knobs from crop_rec submenu, if you think you can find the sweet spot without corrupted frames.
I would expect the 1.1.3 firmware to have a little more headroom (maybe even allow a few more pixels), but didn't test it. You can fine-tune vertical resolution from the crop_rec submenu (and if you have corrupted frames, decrease it from there).
New build posted. Short summary:
- fixes for high-FPS modes with Frozen LV (first frame was corrupted, which didn't play nice with lossless compression)
- continuous full-res recording with lossless compression (about 4-5 FPS, depending on ISO and scene complexity)
- print current FPS and rolling shutter in the FPS override menu (visible from main Movie menu tab)
- two experiments with a small risk:
- single-buffering (risk of tearing, alongside with graphical checks)
- use-after-free for SRM memory (to fix half-shutter quirks)
- minor fixes here and there.
Thanks for the update A1ex, today it's my birthday, it's a nice gift from you :)
I believe this has already been said, but the best way to use the lossless 14bits is to forget to shoot in ETTR.
Do you think there is way to take the control on the compression codec ?
Quote from: garry23 on April 04, 2017, 01:47:28 PM
In particular, how do I set up, run and process the full Res 'photo mode'. Once I know how to get things running, I can test.
Simplest way:
- crop_rec.mo, silent.mo
- crop mode: full-res LV; silent mode: simple, any file format
- press half-shutter
- for preview, also load
mlv_lite.mo mlv_rec.mo, enable raw recording and global draw.
- you can review 14-bit regular/lossless DNG (pic_view) or MLV (mlv_play); for 10-bit full-res MLV, raw_twk runs out of RAM
Continuous full-res recording at 4-5 FPS:
- crop_rec.mo, mlv_lite.mo
- crop mode: full-res LV; raw recording: max resolution, lossless compression
- FPS override: 4 or 5 FPS, low light (important)
- press record
- mlv_dump foobar.mlv --dng
Fast half-shutter trigger (with negative shutter lag):
- crop_rec.mo, mlv_lite.mo, configured as above
- raw recording: enable REC trigger (half-shutter: pre only; without pre-recording enabled, that means only 1 frame)
- no need to use FPS override, as we are only saving the frames triggered by half-shutter
- start recording
- press half-shutter every now and then to capture a frame
- stop recording
Capture unpredictable action (animals, kids etc):
- crop_rec.mo, mlv_lite.mo
- crop mode: full-res LV or 4K half-FPS
- raw recording: max res, 10-bit (here, pre-recording is a bit confused by variable frame sizes, so let's not be adventurous)
- pre-recording: 1 or 2 seconds (actually "up to")
- half-shutter trigger: "hold" (which will pre-buffer up to about half of available RAM), or "pre only" (which will pre-buffer all the available RAM)
- start recording, press half-shutter etc
- caveat: preview is not working at all with 10-bit (try 14-bit)
- to review in camera: 4K 10-bit can be played back with mlv_play + raw_twk, full-res 14-bit with mlv_play only.
Quote from: GutterPump on April 04, 2017, 06:11:34 PM
I believe this has already been said, but the best way to use the lossless 14bits is to forget to shoot in ETTR.
Do you think there is way to take the control on the compression codec ?
Very little. (http://www.magiclantern.fm/forum/index.php?topic=18443.msg181620#msg181620)
Quote from: Quentin on April 03, 2017, 07:39:02 PM
I wonder if there is a potential for ordinary resolution recording but with higher frame rates.
First post:
Quote
* 1920x960 @ 50p (both 1:1 crop and full-frame - 3x3 pixel binning)
* 1920x800 @ 60p (same as above)
* 1920x1080 @ 45p and 48p (3x3 binning)
In particular, 1080p48 is quite pushed to the limit, which is why I've also included a 45p preset (hopefully a bit more reliable).
Quote from: etienne on April 03, 2017, 10:13:35 PM
1/ 1080x1080 (from 3x3 bining using the 3240x3240 sensor size). for a big anamorphic s65 lens.
Already possible on regular nightly.
Quote
2 / in the same way of the 14 bit losless mjpeg compression is an 8 bit mjpeg compression possible?
Yes... (http://www.magiclantern.fm/forum/index.php?topic=2803.msg13469#msg13469)
Quote
so we can do low file size and get the real sharpness the 5d3 can offer (clean HDMI recorder on prores does not get any better sharpness than the in camera h264 file - whereas the raw files is always resolving much more sharpness)
... but it would simply compress Canon's YUV422 buffer (with their color science, picture styles, sharpness etc).
Quote
3/ finally how far would you say you are from a working preview for the mode 3K, UHD, or zoom x5 and is it even possible?
Far, far away. The trick I've used to center the 5x zoom preview in the regular (non-4K) crop_rec branch didn't work for the new modes :(
Also, I have no idea how to configure image scaling. Adtg_gui doesn't reveal any difference between x5 and x10 zoom modes (both use identical raw buffer configurations - just the preview differs).
I can try to include the centered x5 zoom mode (which you can already use in the "classic" crop_rec builds), if that helps.
Anyone trying to turn the eos m into the blackmagic cinema camera? 2.5k possible? Maybe... hopefully...
This is amazing for movie still photographers!
@a1ex
Many thanks for the simple insight, for a simple still photographer ;) That is:
QuoteSimplest way:
- crop_rec.mo, silent.mo
- crop mode: full-res LV; silent mode: simple, any file format
- press half-shutter
- for preview, also load mlv_lite.mo, enable raw recording and global draw.
- caveats:
- preview doesn't work
Because of the preview limitation, I simply composed with crop_rec and silent switched off, used ETTR, at 0%, and switched them back on after the ETTR.
Grabbed a DNG which I inspected in LR.
Fantastic!!!
OK, I know you know the limitations at the moment, eg no EXIF data, but what I step forward for silent full res photography, ie in those churches I go to etc.
One thing I have noticed is that the ETTR exposure at 0% highlights (SNRs off/0) does show over exposure. Not sure if this is real or important at this stage of the developments.
As I said, I'm keen to help test the still photography side of the experiment: I'll stay clear of the MLV world...for now.
Cheers and, once again, thanks for this new step forward for ML.
Garry
I'm having trouble using mlv_dump to decode the compression. Any help appreciated. (sorry, I know this is a development forum. what I am I doing here if I can't use Terminal properly?) :)
Assuming you're on OS X -- Have you even tried @Danne's latest cr2hdr.app which includes the modified mlv_dump yet @lostfeliz?
@a1ex
Hello thanx for taking the time to answering the question this is highly appreciated. i'm on the curve to go to URSA 4.6K but this new fonctions let me think my olds 5d3's have still some life to live.
regarding the fact that mlv mjpeg 8 bit compression
Quotewould simply compress Canon's YUV422 buffer (with their color science, picture styles, sharpness etc)
Are you absolutely sure regarding the sharpness. I'm asking because this is a fact that the clean hdmi as absolutely zero additionnal sharpness from the h264 files i have tested this thousands times. But the raw definitly resolves way more sharpness similar to what produces an A7sii, gh4 or gh5 in HD so i was sure that the 5d breaks something in between what you catch with rec_mod and what you get as h264 or hdmi out.
Now what i'm undestanding from your saying is that the compress Canon's YUV422 buffer is what is sent to the hdmi out right? If it's the case it's useless for sure but can it be something else. Can you think of any other way to compress to something like 15~25% even if it means we loose all the color depth from the raw but just keep that sharpness from it. I'm insisting on this because Personnally I would be so much be happy with an MLV without any color depth additional value but which would just have the real sharpness we can get from the current raw and that could allow like 60~90 minutes on a 128GB card.
Also I want to congrats on the slowmotion mods all options are awesome i just did a 1080 60P in 2:35 flawlessly. As mentionned the 1080 48P as a few corrupted frames and its preview is also a little off towards the down of the image compared to the canon preview but appart from that we could say this is 99% usable.
Finnally. The 3K Mode and UHD are absolutely awesome. I personnally don't see how we could use over 3360x1528 with spanning and sound in the future so any preview that would be x2 would just be the ultimate thing to make this usable.
I undestand that for now it's complete unknow hope someone will find a solution one day otherwise in regards of you're sayinf:
QuoteI can try to include the centered x5 zoom mode (which you can already use in the "classic" crop_rec builds), if that helps.
Sure It's definitly better than nothing to have access to canon x5 center preview again instead of pink frame eventhough it will not help with framing but it will help to pull focus correctly durin shooting.
Bravo again keep the good work. Looking very forward to integration to MLV2 with spanning and sound enabled...
Good Luck.
Hi a1ex, thanks for the new build.
I noticed that the preview framing in UHD seems to be too far up. I tried fiddling with the crop options, to no avail.
Also, for anybody concerned about low light shooting, it looks like I can get continuous 16:9 with around 2.4k horizontal resolution at ISO 3200.
Can you attach a screenshot + DNG?
(I did the same, but couldn't notice any mismatch...)
Hi! What great stuff is happening here!
One question: I try to avoid raw because of inconvenience.
The 1:1 1920 and 1:1T work awesomely fine for me with crop_rec enabled and raw disabled. It would be so good to have 1920 60p (1920x800) in non raw mode but if I choose either 720 60p or any other Canon frame rate/resolution combo, I just get the standard rates.... Is there a trick to it or is the crop_rec just for ml_raw ??
So this is full frame mode, just to show the framing:
(https://s13.postimg.org/8h05h652f/20170404_224558.jpg)
And this is the DNG from UHD mode:
https://www.dropbox.com/s/xoxmrxn0hcj19qo/M04-2243_000000.dng?dl=0
You mean, the recorded area is not centered? That's probably true, as it can only be adjusted in rough increments. But the preview should be OK.
You can fine-tune the centering (in both directions) from the crop_rec submenu. Start values for vertical centering:
case CROP_PRESET_UHD:
cmos_new[1] =
(video_mode_fps == 24) ? PACK12(4,9) :
(video_mode_fps == 25) ? PACK12(4,9) :
(video_mode_fps == 30) ? PACK12(5,8) :
(video_mode_fps == 50) ? PACK12(12,9) :
(video_mode_fps == 60) ? PACK12(13,9) :
Then, you can check with raw_diag (Downloads -> Modules) using the OB zones tool, to see whether the optical black areas are clean and useful data is present in all edges.
If you (or anyone else) finds better values, I'll include them.
WOW!!!!! :o just saw this new development are you serious 4K on the old 5D Mark III. Like I've said before Magic Lantern Team never sleeps, thank you @a1ex and the whole Magic Lantern Team for those awesome new features, and previously impossible features :D
Actually, the preview is off as well as the file. In 3K mode framing is correct in preview:
(https://s11.postimg.org/xpb3jor03/20170404_233407.jpg)
But in 4K it's totally off (in the last build it was fine)
(https://s14.postimg.org/n9x110mb5/20170404_233433.jpg)
I tried altering CMOS[1] lo hi values. At lo 10 the image was centered vertically but with a weird overlay on half of the screen:
(https://s10.postimg.org/e8armmym1/20170404_235712.jpg)
It's a transparent overlay of what's actually beneath the frame. I tried playing with different values for CMOS[1] lo but with no luck. In the previous build this problem cropped up (excuse the pun!) from time to time, but I could restart the camera and it was back to normal. With this build UHD is vertically shifted whereas 3K is correct.
TANGENTIAL SUGGESTION: If there's some way to reliably control the positioning of the crop area, this is a great way to turn a wide angle lens into a digital shift lens, for those interested in architectural stuff and precise composition.
Another quick suggestion: It would be nice to have more control of the vertical resolution, if that's possible. Let's say I wanted to shoot in 4:3 using a 2x anamorphic. The max res I can get is 2048 x 1536, even though my card could handle a higher resolution.
I got that overlay when I messed with the 1920 tall positioning on the 1st or 2nd build.
I shot standard 1080p raw vs 2.8K raw at the same ISO and aperture, taking a few steps to get close to the same framing; the 1:1 2.8K crop appears to be about half a stop brighter than the 3x3 binned 1080p. Somebody else might wanna test that to confirm.
Getting back into the ML menu is quite tricky on the Apr04 build.
@DeafEyeJedi Thanks! I didn't know it'd been updated to handle the compressed MLVs.
So, any hints on how i can port any of the high resolution options to 700d, or are the high res options just not going to work for the lower end models like the rebels?
I looked around and read about needing to look at fps timers, but also see there's mem_engio_write address that has to be found, no idea how to find that one?
Quote from: squig on April 05, 2017, 02:46:38 AM
Getting back into the ML menu is quite tricky on the Apr04 build.
?! (http://www.chiark.greenend.org.uk/~sgtatham/bugs.html)
@a1ex Can you please point me to a guide on how to use mlv_dump for mac can't seem to get it to work with the latest development.
mlv_dump --dng INPUT.MLV
Quote from: a1ex on April 05, 2017, 07:35:23 AM
?! (http://www.chiark.greenend.org.uk/~sgtatham/bugs.html)
After recording, I have to restart to get back into the ML menu.
Are you getting the Canon spotmeter appearing during recording? Busy's still showing too.
Can't reproduce.
Canon's focus box appears for 1-2 seconds when starting recording, then again when stopping. No BUSY message (I was getting it with previous builds). Menu works fine, both before and after. Tried 3K 24p and 1080p45, starting from ML defaults. Also tried frozen LV.
If the link wasn't obvious: http://www.chiark.greenend.org.uk/~sgtatham/bugs.html
@Danne Can you breakdown the process for me.
http://www.magiclantern.fm/forum/index.php?topic=19300.msg182376#msg182376
Breakdown?
If on mac
1 - Drag mlv_dump to terminal
2 - Print --dng drag your mlv file after the word --dng in terminal and press enter.
If your version of mlv_dump says permission denied do following:
Open up terminal and print chmod u=rwx and drag mlv_dump terminal and hit enter then redo steps 1 and 2.
cr2hdr for mac is very easy. even I can use it. read that it has new mlv dump. id try that.
frame and block size mismatch:0xcba800 0x20 0x88bc00
Sent from my SM-N920I using Tapatalk
@Danne Thanks a lot..
In case anyone finds what I've done helpful, I offer the following as we experiment with this exciting new ML development.
This is only for the still photographers ;)
As at the moment LV preview is not working, so composition and exposure setting are not possible, I've written the following simple script to toggle the crop and silent features on and off (only works in LV).
This way you can compose without the feature on, toggle to use the feature (I use RATE as the toggler), do a half shutter press to get your DNG (I use DNG for stills), then toggle back.
The only 'strangeness' is that after toggling you will need to go in and out of the ML menu (TRASH) to reset the shooting state. The bottom left of the ML menu will give you feedback, eg FLV.
Here is the simple script:
--[[
Simple script to toggle Experimental FRSP on and off
Garry George April 2017
http://photography.grayheron.net/
--]]
-- Declare some globals
mod_button_1 = KEY.RATE -- key to modify
toggle_state = true
function test4mod(arg)
if lv.running then
if XFRSP_menu.submenu["Turn On/Off"].value == "On" then -- be prepared to toogle X-FRSP state
if arg == mod_button_1 then -- toggle
if toggle_state == true then toggle_state = false else toggle_state = true end
if toggle_state then
menu.set("Movie", "Crop mode",8)
menu.set("Shoot", "Silent Picture",1)
else
menu.set("Movie", "Crop mode",0)
menu.set("Shoot", "Silent Picture",0)
end
return false -- steal key press
else
return true -- do nothing
end
else
return true -- modified key works normally
end
end
end
event.keypress = test4mod
XFRSP_menu = menu.new
{
parent = "Shoot",
name = "X-FRSP",
help = "Simply toggles the X-FRSP state",
depends_on = DEPENDS_ON.LIVEVIEW,
submenu =
{
{
name = "Turn On/Off",
help = "Switches the script on/off",
choices = {"On","Off"},
}
}
}
As I say, until the previews (hopefully) get working, this little 'hack' will allow you to experiment with still experiment full res silent.
Quote from: a1ex on April 05, 2017, 08:46:05 AM
Can't reproduce.
Canon's focus box appears for 1-2 seconds when starting recording, then again when stopping. No BUSY message (I was getting it with previous builds). Menu works fine, both before and after. Tried 3K 24p and 1080p45, starting from ML defaults. Also tried frozen LV.
Things started to get a bit screwy yesterday after I put ML on an SD card. I'm thinking doing the install with an experimental build may not have been a great idea. I'll do it over with a nightly.
EDIT: It was sticky half shutter. All good now.
Hi,
Testing the last build (magiclantern-crop_rec_4k.2017Apr04.5D3123) on 5D 123 and after few tests camera freezed.
I had to remove the battery and now the camera doesnt' boot as usual.
The LCD stay black, back buttons don't respond, video mode no longer works, camera can only take pictures.
On the SD card I had crash log :
ASSERT: hLvJob->hJpegMemSuite
at ./Epp/Vram/VramStage.c:891, task Epp
lv:1 mode:3
Magic Lantern version : crop_rec_4k.2017Apr04.5D3123
Mercurial changeset : e7e1fdb85f6b (crop_rec_4k) tip
Built on 2017-04-04 15:42:08 UTC by jenkins@nightly.
Free Memory : 168K + 3151K
And assert log :
ML ASSERT:
PROP_LEN(80040002) correct:0 called:4
at ../../src/property.c:342 (prop_request_change), task ml_init
lv:0 mode:0
Magic Lantern version : crop_rec_4k.2017Apr04.5D3123
Mercurial changeset : e7e1fdb85f6b (crop_rec_4k) tip
Built on 2017-04-04 15:42:08 UTC by jenkins@nightly.
Free Memory : 176K + 3964K
I went back to previous working version (compressed_raw.2017Apr01.5D3123), same symptoms and I had this crash log :
ASSERT: pStruct->Data
at GuiProperty.c:1020, task PropMgr
lv:0 mode:0
Magic Lantern version : compressed_raw.2017Apr01.5D3123
Mercurial changeset : 706183bf6459 (compressed_raw) tip
Built on 2017-04-01 11:58:46 UTC by jenkins@nightly.
Free Memory : 170K + 3753K
And this assert log :
ML ASSERT:
PROP_LEN(80040002) correct:0 called:4
at ../../src/property.c:342 (prop_request_change), task ml_init
lv:0 mode:0
Magic Lantern version : compressed_raw.2017Apr01.5D3123
Mercurial changeset : 706183bf6459 (compressed_raw) tip
Built on 2017-04-01 11:58:46 UTC by jenkins@nightly.
Free Memory : 192K + 3983K
Sometime I get Err 80 too.
Any idea of what happen to my camera ? :-[
Idk if anyone ran into this issue like I did. But if you're having trouble with dumping the files or importing once converted here's what i did.
1. Open Notepad and type in
@echo off
for %%a in ( %* ) do ( md "%%~na" 2>nul )&( "INSERT FILE LOCATION HERE" -o %%~na\%%~na_ --dng %%~na.MLV )
ALSO don't type in INSERT FILE LOCATION. Thats where you place the location of the folder you're working in. :p
2. Save that file and what ever you name it (I NAMED MINE DUMP) be sure to add .cmd at the end of it. Which will make it a CMD file.
3. Place the MLV_DUMP file into the same folder as the the previous file we just made.
4. Drag you're MLV file to that location and then drag it over the CMD you made.
5. You should now have a folder with a bunch of DNG files.
6. If you can't open the dng files and get errors in let's say After Effects. You need to deleted the The first DNG file in that folder. It's the one with just zeros at the end of it.
I'm use to magic lantern MLV files working with my normal programs. So if this is already old news i'm sorry for posting. I did however see a few questions regarding this issue. Some are getting pink dots or weird rainbow-ish frames.
Quote from: ju64 on April 05, 2017, 10:10:55 AM
PROP_LEN(80040002) correct:0 called:4
at ../../src/property.c:342 (prop_request_change), task ml_init
This one sounds bad. My advice:
1) make a backup copy of your ROM if you haven't already (it's on the card, under ML/LOGS)
2) start the camera without cards and try to clear Canon settings
If that doesn't help, we'll have to investigate what happened.
Thanks a1ex
1) Backup of ROM is already done
2) How can I clear Canon settings ? LCD doesn't works...
Got it, let's chat on IRC for further troubleshooting.
QuoteQuote from: pc_bel on Yesterday at 10:13:46 AM
ML ASSERT:0at mlv_lite.c:1935 (compress_task)
This one indicates frame corruption, and should also print this (unless obscured by other messages):
Code: [Select]
printf("\nCompressed size higher than uncompressed - corrupted frame?\n");
printf("Please reboot, then decrease vertical resolution in crop_rec menu.\n\n");
Now with new April4th build, yesterday problems are gone. Testing 1920 50/60 3x3. Camera boots every time I turn on as expected... WAIT A MOMENT !!!
While I was writting this post with camera on, I've heard the camera disconnect live view and connected again. On screen find for a moment a error message. I didn't have time to read it, just "created a LOG file" or something similar.
This is the LOG:
Wed Apr 5 11:31:33 2017
0: 15.149 [STARTUP]
K285 ICU Firmware Version 1.1.3 ( 6.9.4 )
1: 15.177 [STARTUP]
ICU Release DateTime 2012.05.16 14:14:46
2: 16.597 [PROPAD] PROPAD_CreateFROMPropertyHandle DRAMAddr 0x41744000
3: 16.886 [SEQ] seqEventDispatch (Startup, 0)
4: 16.930 [STARTUP] startupEntry
5: 129.073 [SEQ] seqEventDispatch (Startup, 1)
6: 129.148 [STARTUP] startupPrepareProperty
7: 141.147 [SND] Seq LPC fin
8: 142.415 [RSC] hMemoryQueue (0x6A0010) hStorageQueue (0x6C0012)
9: 144.775 [RSC] MemoryStatusMasterResultCBR
10: 188.740 [MRK] MRK_RegisterSpaceNotifyCallback
11: 188.883 [MRK] MRK_RegisterSpaceNotifyCallback
12: 188.935 [FM] FM_RegisterSpaceNotifyCallback
13: 189.503 [JOB] InitializeJobClass (ID = 858, Num = 13)
14: 196.161 [EM] emLockControl (TYPE_JOBSTATE = 0x0)
15: 199.904 [JOB] InitializeInnerDevelopJobClass (ID = 858, Num = 16)
16: 199.946 [JOB] InitializeMultipleExposureShootJobClass ( Num = 27,11 )
17: 199.981 [JOB] InitializeMultipleExposureSaveAndEndJobClass ( Num = 13 )
18: 200.020 [JOB] InitializeHDRShootJobClass ( Num = 16 )
19: 200.053 [JOB] InitializeHDRSaveAndEndJobClass ( Num = 33 )
20: 200.321 [TERMINATE] SHUTDOWN init comp
21: 200.429 [TERMINATE] Abort init comp
22: 202.132 [STARTUP] startupPropAdminMain : End
23: 209.359 [FM] fmPrepare : Completed
24: 209.511 [SEQ] seqEventDispatch (Startup, 2)
25: 209.884 [SHTRD] [MNAV] MEMNAVI_Initialize
26: 210.049 [SHTRD] [MNAV] MEMNAVI_SetMemoryMap: 1
27: 211.874 [ENG] [ENGIO](Addr:0x41700000, Data:0x 44000)
28: 247.456 [SEQ] seqEventDispatch (Startup, 3)
29: 249.164 [MC] PROP_GUI_STATE 0
30: 250.138 [MC] [MC] PROP_HEADPHONE_VOLUME_VALUE : 0
31: 250.286 [MC] [MC] PROP_MOVIE_PLAY_VOLUME : 5
32: 251.295 [MC] regist master CardCover
33: 253.856 [PRP] M:98 F:97 L:0 P:32 T:7
34: 254.010 [STARTUP] update coded version.
35: 254.110 [STARTUP] internal version 06:09:04:98:97:00:32:07:3c:05:d0:d0:d0
36: 259.367 [STARTUP] startupPrepareDevelop
37: 262.974 [PROPAD] Done Rebuild Tune
38: 264.107 [MAC] MAC_Initialize
39: 264.479 [MAC] K_BOARDID4
40: 264.521 [MAC] K_BOARDID3
41: 264.555 [MAC] K_BOARDID2
42: 264.586 [MAC] K_BOARDID1
43: 264.757 [MAC] Key=0x0 Board=0x0 Body=0x734bbf81
44: 277.644 [SHTP] Regist RPC Handler Master ALO
45: 281.440 [PROPAD] PROPAD_CreateFROMPropertyHandle DRAMAddr 0x41b2c000
46: 393.855 [MR] mvrChangeAckCBR : Video - Mode=0, Type=1, Rate=50, GOP=1
47: 393.981 [MR] mvrChangeAckCBR : Sound Manual(C=2,R=48000,B=16)
48: 394.008 [MR] mvrChangeAckCBR : AE_MODE_MOVIE(3)
49: 394.024 [MR] mvrChangeAckCBR : ISO(104)
50: 394.178 [MR] mvrChangeAckCBR : MOVIE_REC_VOLUME(L:20 R:20)
51: 394.228 [MR] mvrChangeAckCBR : WindCut Off
52: 394.333 [MR] TimeCode Base:11526,11,530583872,0
53: 394.378 [MR] TimeCode Mode:1
54: 394.424 [MR] TimeCode DropFrame Mode:0
55: 394.468 [MR] TimeCode Disp Mode:0
56: 394.503 [MR] PROP_FILE_PREFIX:8W1A
57: 394.534 [MR] PROP_USER_FILE_PREFIX:PAU_,
58: 394.561 [MR] PROP_USER_FILE_PREFIX:PAU,
59: 394.588 [MR] PROP_SELECTED_FILE_PREFIX:PAU_,1
60: 402.263 [LVCOM] LiveView Gain Table Ver0.4
61: 403.395 [HPC] ReserveHPCopyChannel (1, 116)
62: 411.083 [MR] encdrwrapHPCopyAsyncCBR
63: 412.885 [LVGMT] PROP_TEMP_STATUS : STATUS_NORMAL
64: 418.280 [CAPD] LiveView Gain Table Ver0.4
65: 421.293 [LVCOM] InitializeLiveViewDefectDetection
66: 421.798 [LVCOM] ExecuteDefectMarge Start
67: 421.882 [LVCOM] ExecuteDefectMarge1[0x41dd9b00:61440]
68: 423.164 [LVCOM] A:[59] B:[9040]
69: 427.618 [LVCOM] ExecuteDefectMarge2[0x41de8b00:61440]
70: 428.697 [LVCOM] A:[35] B:[7713]
71: 432.633 [LVCOM] ExecuteDefectMarge3[0x41df7b00:32768]
72: 435.697 [LVCOM] ExecuteDefectMarge End
73: 436.838 [LVEPP] [REMO] Jpeg 16:9 PosW: 0 PosH: 300
74: 440.914 [PROP_AF_ASSIST_BEAM_FIRING] 0
75: 441.074 CurrentMedia = 2
76: 443.927 [STARTUP] startupPrepareRemote
77: 445.087 [AUDIO] RegisterCBRSDIOCableConnect
78: 452.462 [WFT] PROP_ADAPTER_DEVICE_ACTIVE: Can Not Change(0) Connect(0)
79: 463.393 [RMT] QuickTransmissionEnable (0, 1, 2, 2)
80: 463.438 [RMT] PROP_CONNECT_TARGET (0x0 <= 0x0)
81: 463.475 [RMT] PROP_CONNECT_TARGET_WFT (0x0 <= 0x0)
82: 463.531 [RMT] PROP_WFT_SYSTEM : 0
83: 463.961 [PTP] PSI UnlockTransImg : Trans List is NULL
84: 463.995 [PTP] PSI TarminateTransImgList : SessionTarget [1]
85: 464.020 [PTP] PSI TarminateTransImgList : Trans List is NULL
86: 464.133 [PTP] PSI UnlockTransImg : Trans List is NULL
87: 464.158 [PTP] PSI TarminateTransImgList : SessionTarget [10]
88: 464.180 [PTP] PSI TarminateTransImgList : Trans List is NULL
89: 464.259 [PTP] PSI DisconnectViewFinder
90: 464.763 [PTP] PROP_SCREEN_SAVER 0
91: 464.926 [PTPCOM] SetPtpTransportResources:0,323a
92: 465.189 [PTP] PSI UnlockTransImg : Trans List is NULL
93: 465.221 [PTP] PSI TarminateTransImgList : SessionTarget [10]
94: 465.242 [PTP] PSI TarminateTransImgList : Trans List is NULL
95: 475.488 [WFT] Regist PropCBR PROP_WFT_BLUETOOTH
96: 786.917 [SD] sdSendOCR: 1.8V Signaling Card
97: 789.340 [SD] sdSendOCR: Hi-Capacity Card
98: 795.978 [SD] CARD CAPACITY is 15193Mbyte( 31116288Sec )
99: 812.798 [SD] SD_GetAccessMode=3
100: 812.816 [SD] Set Hi-Speed Mode( 48MHz )
101: 821.024 [FSU] fsuGetPart: Block(8192, 31108096, 31116288)
102: 862.189 [FM] FileNo (DriveNo = 2, 0 <-> 157)
103: 862.304 [FC] _FC_OpenCatalog : wNumOfDir 1
104: 867.175 [MRK] MRK_Initialize
105: 867.760 [MRK] MRK_PrepareMrkFileObject
106: 870.803 [SEQ] seqEventDispatch (Startup, 4)
107: 870.977 [DP] DP_Initialize()
108: 871.656 [DP] Std T_PrintPage[0] T_PrintNo[0] R_PrintNo[0]
109: 871.694 [DP] Std C_PrintNo[0] C_PrintPage[0], C_Copies[0]
110: 871.788 [DP] Dp_SetPD_RegistApi Prev:0 New:0
111: 872.148 [DP] PROP_CONNECT_TARGET [0x00000000]
112: 872.268 [DP] RD_Cancel No Issue
113: 872.501 [DP] [GYO]DpFunctionList[0x10117fff]
114: 872.612 [DP] X FreeMem Address[0x00000000]
115: 876.710 [DISP] HPD before InitializeDisplay !!!
116: 876.961 [DISP] [ME] PropMgr (0) [SlaveResultCBR] (S_OFF)
117: 877.080 [DISP] [ME] PropMgr (0) [ChangeCBR] (S_OFF)
118: 877.103 [DISP] Display_flag = 1
119: 927.579 [IMPP] H264E InitializeH264EncodeFor1080pDZoom
120: 927.638 [IMPP] H264E InitializeH264EncodeFor1080p25fpsDZoom
121: 954.964 [DP] DP_RegisterCBR() Id=1 Count:12
122: 955.003 [DP] DP_RegisterCBR() Id=2 Count:12
123: 955.030 [DP] DP_RegisterCBR() Id=4 Count:12
124: 964.382 [GUI] MainEventHandler PROP_QR_DIDNOT_EXECUTE(0)(0)
125: 966.664 [GUI] MainEventHandler PROP_ERROR_FOR_DISPLAY(0)
126: 993.693 [LVGMT] Is PROP_OUTPUT_TYPE Real??[H:1][V:0][O:0]
127: 1001.225 [MR] mvrChangeAckCBR : Video - Mode=0, Type=1, Rate=50, GOP=1
128: 1002.062 [GUI] MainEventHandler PROP_LV_OUTPUT_DEVICE(0)
129: 1003.208 [LVGMT] Is PROP_OUTPUT_TYPE Real??[H:1][V:0][O:0]
130: 1004.069 [MAGIC] (Empty Func) MVW_RegisterXmpDataCallback
131: 1005.200 [HDMI] HDMI CEC Initialize
132: 1006.225 [MR] mvrChangeAckCBR : Video - Mode=0, Type=1, Rate=50, GOP=1
133: 1007.428 [GUI] ERROR ***** ConvertHdmiVideoCodeToGuiDisplayType unknown Type(0)
134: 1007.973 [HDMI] HPD ON
135: 1007.997 [HDMI] ConnectHDMI
136: 1008.023 [HDMI] InitializePortForHDMI
137: 1008.049 [HDMI] ActiveHDMIDevice
138: 1008.072 [HDMI] I2C_Init
139: 1008.162 [HDMI] Pana_Init
140: 1010.391 [SEQ] seqEventDispatch (Startup, 5)
141: 1010.413 [STARTUP] startupInitializeComplete
142: 1010.499 [MC] cam event guimode comp. 0
143: 1011.314 [MC] cam event guimode comp. 0
144: 1012.443 [DISP] BrightnessControl from GUI 1
145: 1013.765 [DL] PROP_ACTIVE_SWEEP_STATUS (0)
146: 1013.799 [DL] PROP_DL_ACTION (1)
147: 1018.304 [MC] PROP_GUI_STATE 2
148: 1021.169 [MC] PROP_GUI_STATE 2
149: 1022.224 [GUI] MainEventHandler PROP_LV_OUTPUT_DEVICE(0)
150: 1023.042 [MC] PROP_GUI_STATE 2
151: 1023.944 [GUI] IDLEHandler START_WARNING_LENSLESS_MOVIE_MODE
152: 1024.027 [MC] cam event guimode comp. 2
153: 1034.709 [WINSYS] Zlib maxsize update 884
154: 1034.763 [WINSYS] Zlib maxsize update 1396
155: 1060.186 [GUI] IDLEHandler START_WARNING_LENSLESS_MOVIE_MODE
156: 1060.259 [MC] cam event guimode comp. 2
157: 1096.103 [GUI] IDLEHandler START_WARNING_LENSLESS_MOVIE_MODE
158: 1096.175 [MC] cam event guimode comp. 2
159: 1118.924 [FA] ChangeCBR(ID=0x8003000a)
160: 1119.598 [FA] RequestFromCheckData
161: 1119.866 [FA] MpuMonSpecificFromPartner : COM_FA_CHECK_FROM 0
162: 1128.724 [SEQ] seqEventDispatch (Startup) : End
163: 1133.365 [HDMI] RESET_UNCOMPLETE
164: 1136.719 [HDMI] RESET_COMPLETE
165: 1148.561 [HDMI] KEY LOAD OK
166: 1148.595 [HDMI] HOT_PULG_DETECT OK
167: 1198.979 [HDMI] EDID_PREREAD OK
168: 1199.001 [HDMI] Rx Sense is OK
169: 1199.621 [HDMI] pfHdmiPbCBR Done
170: 1253.694 [HDMI] TMDS Check
I was pretty sure that everything was going pretty well until this happened.
Hope it helps for developement...
Were you using a HDMI monitor? I didn't test this configuration yet.
Spreading the word ;) http://photography.grayheron.net/2017/04/magic-lantern-does-it-again.html (http://photography.grayheron.net/2017/04/magic-lantern-does-it-again.html)
Quote
Were you using a HDMI monitor? I didn't test this configuration yet.
Yes, a Zacuto EVF... It can be the problem?
Trying to reproduce the live view disconnection, I let camera on... This time get this message:
CTRL SRV Stack Overflow free=84 Used=163008176
Camera off.
Camera on.
I can record continuous 1920x960 50p 3x3
Camera remains on.
Suddently get this message in top of screen (in red letters):
[103] COMPRES_TASK: TASK WARNING: Free=164 Used=3932
Camera off.
When I boot camera again, I get this message full screen:
ML ASSERT: 0 at mlv_lite.c:2006 (compress_task), task compress_task vv:1 mode:3
srm_malloc_suite(1)...
srm buffer #1: 48600064
srm_malloc_suite(f1408)
Resolution changed: 2080x1587 -> 2080x961
Black level: 2047
Compressed size higher than uncompressed - corrupted frame?
Please reboot, then decrease vertical resolution in crop_rec menu.
ML ASSERT: 0 at mlv_lite.c:2006 (compress_task), task compress_task vv:1 mode:3
I didn't change any resolution... In crop_rec menu it remains 1920x960.
I'm posting all my troubles... I have no idea if it is useful or not. If not, I will stop posting, just let me know ;)
@A1ex
Am I right in saying the key.press(KEY.HALFSHUTTER) wont (sic) work in the experimental 4K at the moment? It doesn't seem to.
I'm trying to explore triggering the experimental FRSP(4K version) in Lua script.
Or am I jumping the gun ;)
Quote
Am I right in saying the key.press(KEY.HALFSHUTTER) wont (sic) work in the experimental 4K at the moment? It doesn't seem to.
Right - will take care of this (as I want to merge lua_fix in regular nightlies anyway).
Quote
CTRL SRV Stack Overflow free=84 Used=163008176
Would be great if you can find a way to reproduce this.
@A1ex
:) Great :)
Does a 1:1 crop mean it's a 2x crop similar to a GH4?
The references to 1:1 mean that every pixel on the sensor becomes a pixel in the image. The "crop factor" depends on the resolution you choose.
A 1920 x 1080 1:1 central section of the sensor (5760 pixels wide) gives a crop factor of 3.
2880 x 1620 is a 2x crop factor.
@Kyle Kearns Naw.. 3K 1:1 I believe is 1.84 - UHD is 1.5 - True 4K is 1.4
QuoteQuote
CTRL SRV Stack Overflow free=84 Used=163008176
Would be great if you can find a way to reproduce this.
After 2 hours I'm not able to reproduce CTRL SRV...
QuoteSuddently get this message in top of screen (in red letters):
[103] COMPRES_TASK: TASK WARNING: Free=164 Used=3932
Camera off.
When I boot camera again, I get this message full screen:
ML ASSERT: 0 at mlv_lite.c:2006 (compress_task), task compress_task vv:1 mode:3
srm_malloc_suite(1)...
srm buffer #1: 48600064
srm_malloc_suite(f1408)
Resolution changed: 2080x1587 -> 2080x961
Black level: 2047
Compressed size higher than uncompressed - corrupted frame?
Please reboot, then decrease vertical resolution in crop_rec menu.
ML ASSERT: 0 at mlv_lite.c:2006 (compress_task), task compress_task vv:1 mode:3
This ML ASSERT message appears when in crop_rec mode 1920 50p 3x3, I change (in Canon menu) from 1280x720 50 All-I to 1920x1080 25 All-I before change crop_rec to OFF. I need to reboot 2 times and then all come back to normal operation...
I will try again to reproduce the CTRL SRV message :)
select 14bit lossless,and "frame and block size mismatch:0xcba800 0x20 0x88bc00" shows when play back.
ive tired many times,but the problem still exists.
what could i do?
I shot something like 12 takes.
I tried 14lossless and got a warning at the top of the screen (couldnt read due to age)
and a series of many warnings.
I removed the battery, start from scratch, default ML settings, load modules.
I set 10bit
Set the Canon video to 1280x720, NTSC.
Everything as expected.
In order to reach optimum performance, I set the Canon video back to 1920x1080, PAL.
The resolutions 3K, UHD and 4K had less height that the promised in ML forums.
e.g 3072x1728
In these resolutions, I couldnt record more than 3 secs maximum.
Surprisingly I managed to record 3568x1320 for much longer duration. <--- pretty strange not to be able to record longer for lower resolution
I was experimenting with the lossless lj92 compression in MLV files. I wanted to see if I can decode the image and reinterpret it as a raw cfa pattern. I came across a number of problems that could also be a problem in mlv_dump. I took the liblj92 implementation by Andreaw Baldwin found here:
https://bitbucket.org/hudson/magic-lantern/src/7a9b6805756c3b86f2174bac00433c544a976501/modules/mlv_rec/lj92.c?at=mlv_rec_lj92&fileviewer=file-view-default (https://bitbucket.org/hudson/magic-lantern/src/7a9b6805756c3b86f2174bac00433c544a976501/modules/mlv_rec/lj92.c?at=mlv_rec_lj92&fileviewer=file-view-default)
But I could not make it work. I had decoding problems due to the fact that the library is missing some support for multiple components. I then came across the implementation in tiny_dng_loader by Syoyo Fujita here:
https://github.com/syoyo/tinydngloader/blob/master/tiny_dng_loader.h (https://github.com/syoyo/tinydngloader/blob/master/tiny_dng_loader.h)
I then extracted his changes back into a simple standalone .c and .h file and tried that and viola, the decoding result was much better:
(http://i.imgur.com/5C88tC6.jpg)
I posted the extracted version of the library on GitHub:
https://github.com/martinhering/liblj92 (https://github.com/martinhering/liblj92)
Just saying, if you encounter any decoding problems in mlv_dump with the mlv_rec_lj92 branch, give my version a try.
QuoteI shot something like 12 takes.
I tried 14lossless and got a warning at the top of the screen (couldnt read due to age)
and a series of many warnings.
I removed the battery, start from scratch, defauly ML settings, load modules.
I set 10bit
Set the Canon video to 1280x720, NTSC.
Everything as expected.
Yes, same as me. Now message is:
[76] CEC: stack warning: free=128 used=3968I'm totally lost...
@A1ex
I'don't know what to do, so if you can point me to what to do to help, just let me know.
After the messages I can reboot and I can record continuous without corrupted frames (allways talking about crop_rec April 4th build, 1920 50p 3x3 mode)... so this is not a big problem, but I have no idea of code so no idea what all this messages mean and what to do... sorry :-[ I'm limited to explain what happens...
Did some changes that might help with memory corruption (new build posted).
In particular, I think I've found a way to limit compressed data size when it ends up larger than the allocated buffer. Previously, passing an arbitrary limit for buffer size simply locked up the camera at certain resolutions - it appears this limit must be aligned at 4096-byte multiples (maybe a bit less, but in any case, at least 1024).
If you still get stack overflow, please report even if you don't have a way to reproduce it. This kind of bug is usually very serious.
@A1ex
Just checking: I assume this build doesn't include the Lua fix yet? I doesn't look if it does.
Cheers
Garry
Just tested April 6th 113.
Can't get continuous recording like before with april4th, only 10 sec. before message "didn't make it to stimated record time" and then recording stops. (1920 50p 3x3 mode).
Quote from: a1ex on April 06, 2017, 10:32:12 AM
This kind of bug is usually very serious.
Is this mean we could get a semi brick or definitive brick ? What is the long-term risk ?
@pc_bel: nothing was changed on the recording speed side; I bet any difference is because of different scene complexity. Please test the stack overflow bug first.
@garry23: Lua changes are a bit too much when I'm troubleshooting a memory corruption bug. The changelog from the Experiments page is valid.
Quote from: GutterPump on April 06, 2017, 11:12:03 AM
Is this mean we could get a semi brick or definitive brick ? What is the long-term risk ?
Yes, as with all ML builds. Explained a little here (http://www.magiclantern.fm/forum/index.php?topic=12627.msg170133#msg170133) and also somewhere in the Tragic Lantern threads. Currently I'm diagnosing this report (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182395#msg182395), and it appears to be, indeed, caused by some bad settings reflashed by Canon code into ROM - possibly after memory corruption caused by ML.
The warnings from the download page are not there for fun - this kind of errors cannot be prevented on the current processor design, as it doesn't have a MMU (also valid for many other simple CPUs that use non-volatile storage updated frequently). Canon started to use a processor with MMU with DIGIC 7 (https://chdk.setepontos.com/index.php?topic=13014.30), but they don't seem to use its memory protection features (they simply do a flat memory mapping, except a 0x1000-byte block unique to each core).chd
Just in case, make sure you have a copy of your ROM in a safe place (any recent installer tells you how to do so).
QuoteJust tested April 6th 113.
Can't get continuous recording like before with april4th, only 10 sec. before message "didn't make it to stimated record time" and then recording stops. (1920 50p 3x3 mode).
After reboot, I get the same message as with the april4th build:
ML ASSERT: 0 at mlv_lite.c:2006 (compress_task), task compress_task vv:1 mode:3
srm_malloc_suite(1)...
srm buffer #1: 48600064
srm_malloc_suite(f1408)
Resolution changed: 2080x1587 -> 2080x961
Black level: 2047
Compressed size higher than uncompressed - corrupted frame?
Please reboot, then decrease vertical resolution in crop_rec menu.
ML ASSERT: 0 at mlv_lite.c:2006 (compress_task), task compress_task vv:1 mode:3
Quote from: pc_bel on April 06, 2017, 11:23:05 AM
ML ASSERT: 0 at mlv_lite.c:2006 (compress_task), task compress_task vv:1 mode:3
Fortunately, that's no longer causing memory corruption (so it's a minor bug for now). It's simply invalid image data being compressed (and output size resulting higher than initial size). The extra data is no longer written to RAM on latest build (and before, it was written in the place reserved for future frames, not on DryOS data structures or persistent setting areas - to my knowledge).
The stack overflows are worrying me, though (as they do indicate memory corruption in the middle of DryOS).
Thanks for those groundbreaking efforts :) :) !!!!!!
Don't know if it helps. If I can remember...
I got just while moving in the menu but without CF card (only SD):
white flashing "[93] audio-common-task stack warning free=180 used=3016"
then red flashing "[91] clock-task st(don't remember the end of the word) free=36 used=8156"
ASSERT: this->VramNv12Param.hJpegMemSuite
at ./Epp/Vram/VramStage.c:745, task Epp
lv:1 mode:3
Magic Lantern version : crop_rec_4k.2017Apr04.5D3113
Mercurial changeset : e7e1fdb85f6b (crop_rec_4k) tip
Built on 2017-04-04 15:41:29 UTC by jenkins@nightly.
Free Memory : 146K + 3094K
ASSERT: 0
at ./PathDrvWrap/EekoYuvToVramPath.c:606, task Epp
lv:0 mode:3
Magic Lantern version : crop_rec_4k.2017Apr04.5D3113
Mercurial changeset : e7e1fdb85f6b (crop_rec_4k) tip
Built on 2017-04-04 15:41:29 UTC by jenkins@nightly.
Free Memory : 158K + 3072K
this was on 4th. didn't see there is a new version.
testing up to date version atm.
looks good so far.
I have little knowledge on how you managed to achiev this update.
But I assume, there will be no way to use an external monitor on this, right?
HDMI connectors are limeted to 1080p huh?
Without ML (SD card with ML removed) and only CF card (data card):
In Canon menu, the sensor cleaning gets disabled when the camera is switched on.
(I am with the 4th april build)
For now, all I could get are these error messages:
ASSERT: hLvJob->hJpegMemSuite
at ./Epp/Vram/VramStage.c:868, task Epp
lv:1 mode:3
Magic Lantern version : crop_rec_4k.2017Apr06.5D3113
Mercurial changeset : bc99f7502deb (crop_rec_4k) tip
Built on 2017-04-06 08:28:51 UTC by jenkins@nightly.
Free Memory : 172K + 3779K
ASSERT: IsSuiteSignature( hSuite )
at ./PackMemory/PackMem.c:599, task RscMgr
lv:0 mode:3
Magic Lantern version : crop_rec_4k.2017Apr06.5D3113
Mercurial changeset : bc99f7502deb (crop_rec_4k) tip
Built on 2017-04-06 08:28:51 UTC by jenkins@nightly.
Free Memory : 172K + 3778K
ASSERT: fInitialize == TRUE
at ./System/PostPostman/PostPostman.c:243, task Epp
lv:0 mode:3
Magic Lantern version : crop_rec_4k.2017Apr06.5D3113
Mercurial changeset : bc99f7502deb (crop_rec_4k) tip
Built on 2017-04-06 08:28:51 UTC by jenkins@nightly.
Free Memory : 172K + 3781K
Quote from: BBA on April 06, 2017, 12:40:10 PM
Without ML (SD card with ML removed) and only CF card (data card):
In Canon menu, the sensor cleaning gets disabled when the camera is switched on.
(I am with the 4th april build)
Does clearing Canon settings help? Otherwise, do you have a screenshot, so I know what to look for?
Day of testing!!! :)
I returned to crop_rec 113 april4th build due to the short recorded times I get with april6th (I don't know why...) :(
Really surprised for the extremely low rolling shutter in 1920 50p 3x3 mode... but panning, I get a cut line in the top part of the frame, only with panning! it dissapears in static frames.
This video explains better:
http://ovh.to/b2isZ5B
Edited: I can't see the cut line in live view, only in processed dng's.
Quote from: pc_bel on April 06, 2017, 06:04:04 PM
Really surprised for the extremely low rolling shutter in 1920 50p 3x3 mode... but panning, I get a cut line in the top part of the frame, only with panning! it dissapears in static frames.
Edited: I can't see the cut line in live view, only in processed dng's.
same here. got this strange line, when panning.
@A1ex
I've continued 'playing around' with the latest experiments in still photography and am a bit confused. ML seems to be creating 'time dilation in the space-time continuum' :)
I set up a long exposure test, 30s, and took a base .cr2 capture.
I then kept this exposure and used the full res LV in the experimental build.
As soon as I pressed the half shutter press, I got the saving message. I pull the card immediately.
I repeated and left the card in for a minute.
The two DNGs were the same, ie I seem to get a 30s silent in a couple of seconds.
The cr2 and the two DNGs were exposed OK, ie ETTR like and not overexposed.
I obviously don't understand how long exposures are working with the crop/silent stills.
When you have time, I would welcome any insight.
Cheers
Garry
Quote from: pc_bel on April 06, 2017, 06:04:04 PM
I returned to crop_rec 113 april4th build due to the short recorded times I get with april6th (I don't know why...) :(
Proof that it has shorter recording times, please! A static test scene (without varying light levels and with the same focus), 10 test runs with each build, starting from a formatted card, at the same resolution and ISO, should do the trick.
Quote from: garry23 on April 06, 2017, 06:23:38 PM
I set up a long exposure test, 30s, and took a base .cr2 capture.
I then kept this exposure and used the full res LV in the experimental build.
As soon as I pressed the half shutter press, I got the saving message. I pull the card immediately.
The two DNGs were the same, ie I seem to get a 30s silent in a couple of seconds.
The cr2 and the two DNGs were exposed OK, ie ETTR like and not overexposed.
I obviously don't understand how long exposures are working with the crop/silent stills.
Best guess: you have tried in photo mode (which I didn't try yet), with exposure simulation. So, if you have metered a 30s ISO 100, the LiveView is probably at 1/FPS and a very high ISO. An example, if FPS override was set to 5, the LiveView frame might have been exposed at 1/5 seconds, aperture wide open - even if you have selected something different in menu - and ISO as high as required to get the same brightness as with a 30-second exposure (considering the other two variables).
I'm curious to see the DNGs.
@A1ex
No EXIF so can't tell other than looking at them.
Canon and ML ISO settings remain at 100.
Tried with expo sim off. Saving straight away, i.e. Doesn't do a 30 sec capture.
Cheers
Garry
@A1ex
Sitting here trying to think this through.
As this is coming from a video development, I guess we will be limited, at ISO100, to shutter speeds that are consistent with video framing rates.
I guess this means this still photography approach will solve fast shutter speeds, but not give ISO100 slow shutter speeds.
Or have I got it wrong?
pc_bel said:
Quotebut panning, I get a cut line in the top part of the frame, only with panning! it dissapears in static frames.
In other contexts, this was theorized to be a symptom of overloading the processor. I used to get it with greyscale ml preview on my 7D in 5x crop mode.
@A1ex
Thanks for your assistance ! Don't hesitate to ask more if necessary ! I am not stuck ;) .
a) more infos about my problem:
a.1) without ML card (I use the SD card for ML)
a.2) CF card in camera (formatted in camera)
a.3) I go to Canon menu : yellow settings menu / SETUP3 / "sensor cleaning" and set "auto cleaning" to "enabled"
a.4) I switch off the camera : the sensor cleaning screen appears as before; the red bottom led is not lit.
a.5) I then switch on the camera : the sensor cleaning screen does Not appear; the bottom red led is lit.
a.6) if I go again to the Canon menu : yellow settings menu / SETUP3 / "sensor cleaning", the "auto cleaning" has been set to "disabled"
b) reinit all settings : no change
b.1) Canon menu / yellow settings menu / SETUP4 / "reinit all settings"
b.2) the "auto cleaning" feature is actually set to "enabled" if I verify in the Canon menu
b.3) then I get the same as above in a.4) , a.5) and a.6)
[EDITED 2017/04/07]
c) circumstances ? no CF card ?
Unknown
I think (not sure at all) it may be due to the fact that I tried to repeat the problem I got with no CF card in the camera (silly thing : should not be done without further investigation): the camera got stuck; no message ; I had to remove the battery.
Investigated by A1ex (thanks a lot) : no such problem when switched on with no CF card (see post #156).
@BBA: could not reproduce.
Side note: I've let the camera on for 10 hours, on a power adapter, with today's build, on the 3K video mode, 3072x1920 in mlv_lite menu, lossless compression, "framing" preview, in standby - so it would estimate the compression ratio every 2 seconds. Every 30 minutes, it closed LiveView for a second, and returned to LV (I could hear it when I came home).
Temperature after this experiment was 62 degrees C. The camera was enclosed in a drawer during the entire experiment.
I've now downloaded the previous build (Apr04) and will perform the same test overnight (or until it crashes).
edit: it crashed after 15 minutes or so of recording short test clips in 50p 3x3; back to Apr06, no crashes yet.
I'm doing these tests to check these reports:
Quote from: Quentin on April 06, 2017, 05:52:35 AM
I tried 14lossless and got a warning at the top of the screen (couldnt read due to age)
and a series of many warnings.
Quote from: pc_bel on April 05, 2017, 03:20:15 PM
Trying to reproduce the live view disconnection, I let camera on... This time get this message:
CTRL SRV Stack Overflow free=84 Used=163008176
[...]
Suddently get this message in top of screen (in red letters):
[103] COMPRES_TASK: TASK WARNING: Free=164 Used=3932
@A1ex
Ok I get it now ;)
As you say on the first post
QuoteThe last feature complements the well-known full-resolution silent pictures - the new implementation will be usable at fast shutter speeds, without the exposure gradient - but with rolling shutter (just like regular LiveView frames).
Photographers need to look to using the crop approach and the current FRSP, together.
I'm looking forward to trying to script these two techniques together in Lua :)
Really exciting and fun times.
Cheers
Garry
Please follow the same steps as ju64 and contact me on IRC.
Upper right corner:
IRC: #magiclantern (http://webchat.freenode.net/?channels=magiclantern)
Another case of bad setting flashed somehow into ROM. Reproduced the error in QEMU. Appears easy to fix, since these settings fortunately have backups (wear leveling?). I've already unbricked a similar case a while ago (also 5D3), and I've already booted the GUI in QEMU on the affected ROM.
Two failures on the same build are a bit too much to be a coincidence. For now, my advice would be to avoid the Apr04 build, even though some reports say it's faster (I'm willing to bet it's not, but I'll benchmark it, just in case).
Apr06 should be safe regarding this issue (see my tests) (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182552#msg182552).
Thanks for the heads up!
I'll update to Apr06.
@a1ex Thank you for all your hard work.. I tried the April 4th build but did not encounter such problems on 1.1.3. Will give April 6th a test drive tonight I'll post my findings.
Amazing work @a1ex as always!
I'm using MLV_Dump for the newest crop_rec module with higher resolutions (4K, 1080p48 etc) build - And I'm getting vertical stripes on the .dng exports.
Is this normal for now?...or am I missing something simple? I have not used MLV_Dump before now (and I'm a code ignorant idiot)
Test shots were lower ISO 100-400, various resolutions tested from April 4th build - all exhibited some vertical stripes (some more noticeable at lower ISO)
Running this code on Win 7 works for me to export the .dngs's, I just don't know if vertical stripe correction is an option yet in this version of MLV_Dump...or I simply need to somehow amend the .cmd code to 'switch on' the correction in MLV_Dump?
@echo off
md "%~n1"
"Q:\NEW MLV\mlv_dump.exe" -o %~n1\%~n1_ --dng %~n1.MLV
Link to screengrab of most prominent vertical stripes:
https://www.dropbox.com/s/md2inmxhaj604gv/M06-1750%20%2800123%29.jpg?dl=0 (https://www.dropbox.com/s/md2inmxhaj604gv/M06-1750%20%2800123%29.jpg?dl=0)
.Dng Sample files:
https://www.dropbox.com/sh/fe5nve001l2vqx5/AADpbW7eVOqafc7XP4XnhgD0a?dl=0 (https://www.dropbox.com/sh/fe5nve001l2vqx5/AADpbW7eVOqafc7XP4XnhgD0a?dl=0)
Upload a sample file?
Shortened file:
mlv_dump -f 5 -o Output.MLV Input.MLV
Input.MLV being your file with stripes. Do the command correct or you might loose the file.
@A1ex
I thought I would carry out a few experiments in photo mode looking at 'lowish' shutter speeds with the FRLV.
I decided to use an FPS twice the selected shutter speed.
It appears I can push the FPS down to 0.5 (ML says its 0.554) before it plateaus out.
I took the following combinations and all the DNGs looked OK in LR:
FPS = 2, ML:shutter = 1/10s
1, 1/4
1, 1/2
0.5(0.554), 1s
From my perspective this looks a healthy overlap with the 'normal' FRSP, which could use down to, say, 1/4s.
Cheers
Garry
Quote from: garry23 on April 07, 2017, 08:38:42 AM
From my perspective this looks a healthy overlap with the 'normal' FRSP, which could use down to, say, 1/4s.
Sounds reasonable - as the full-res implementation can be updated decide whether to use LiveView or the "long exposure" version, depending on the shutter speed selected in photo mode.
@A1ex
Yes: I've worked out how to do it in my Lua bracketing scripts and I will confirm things as soon as I can. ie when I can trigger with the half shutter key press.
This should give photographers a 30s to 'ultra fast' seamless capture capability.
:)
QuoteTwo failures on the same build are a bit too much to be a coincidence. For now, my advice would be to avoid the Apr04 build, even though some reports say it's faster (I'm willing to bet it's not, but I'll benchmark it, just in case).
Ok, back to april6th...
192050p3x3, 5D3 113,
ISO 100: 10 recordings 1min/each (camera didn't stop recording, I stoped recording assuming continuous).
192050p3x3, 5D3 113,
ISO 400: 10 recordings 1min/each (camera didn't stop recording, I stoped recording assuming continuous).
192050p3x3, 5D3 113,
ISO 800: 1 recording, camera stops recording after 10sec. (message:"didn't make it to stimated record time")
I turn off camera.
When I turn on camera, a full screen message appears like this one:
QuoteWhen I boot camera again, I get this message full screen:
ML ASSERT: 0 at mlv_lite.c:2006 (compress_task), task compress_task vv:1 mode:3
srm_malloc_suite(1)...
srm buffer #1: 48600064
srm_malloc_suite(f1408)
Resolution changed: 2080x1587 -> 2080x961
Black level: 2047
Compressed size higher than uncompressed - corrupted frame?
Please reboot, then decrease vertical resolution in crop_rec menu.
ML ASSERT: 0 at mlv_lite.c:2006 (compress_task), task compress_task vv:1 mode:3
So ISO was the culprit for not get continuous recording with april6th?... My fault. :-[
No STACK OVERFLOW error for now...
Edit: in this tests I got the same corrupted image that before when panning (top part of frame with motion cutted).
Can I ask if is easy to adress this, or not? That would make the recordings usable...!!!! (at least fro me).
Apr06 3104x1298 1600 ISO 93.4MB/s ~57%
5.26Gb
5.78Gb
Restarted
8.15Gb
25.06Gb (I got bored and stopped recording)
I was going do a comparison with Apr04 but the stack error appeared (for the 1st time) when I loaded the modules so I chickened out.
Quote from: pc_bel on April 07, 2017, 09:50:40 AM
Edit: in this tests I got the same corrupted image that before when panning (top part of frame with motion cutted).
Can I ask if is easy to adress this, or not? That would make the recordings usable...!!!! (at least fro me).
Here's a boring task: what's the number of pixels before the cut? (min/max/median would be great)
QuoteHere's a boring task: what's the number of pixels before the cut? (min/max/median would be great)
:-[ Sorry for that!!!... Its only thing that makes image unusuable.
I've measured px and the pattern is very strange... min distance I've found is 7px, max is 125px, but it doesn't follow a logic. At first it seems increassing with the panning velocity, but looking close, during panning, one frame have 98px (from top to cut) the next 112px, the next 18px, one frame (in between all of this) is correct, and for example, the next have 73px...
Min I found: 7px
Max I found: 125px
Average from random 17dng:83,8px
@A1ex
Is this what you are asking for?
Yes. I thought I'd get away with it by increasing the optical bar size, but it looks like I should re-enable double-buffering (at least for certain video modes).
If you need to do some "mechanical-tedious" work, if you explain me how to do it, I can do it...
@A1ex
Quote
Could not reproduce
A good thing for other users : if you agree, I will edit my #155 post and remove the unnecessary warning about not switching on a camera without CF card
Sensor cleaning issue:
Maybe something interesting in the Debug menu: red dot in front of "memory patches"
https://www.dropbox.com/s/ap2zant3kn9vy2e/VRAM3.PPM?dl=0
(https://www.dropbox.com/s/ap2zant3kn9vy2e/VRAM3.PPM?dl=0)
memory patches, with a red dot:
https://www.dropbox.com/s/cafc5h767gbqgdv/VRAM2.PPM?dl=0
(https://www.dropbox.com/s/cafc5h767gbqgdv/VRAM2.PPM?dl=0)
Could it be of any help for fixing the cleaning sensor issue of my camera ?
Quote from: BBA on April 07, 2017, 11:54:09 AM
Maybe something interesting in the Debug menu: red dot in front of "memory patches"
That's unrelated; it's simply telling you about some module (here, crop_rec) using ROM patches aka cache hacks (which are a bit fragile in the backend).
You can use this build to get a startup log with all the debug messages from Canon. Before the log gets saved, you should try to perform the sensor cleaning, so messages about it will get included:
https://builds.magiclantern.fm/jenkins/view/Experiments/job/startup-log/
@A1ex
Thanks a lot !
I will give it a try and report.
Quote from: Hans_Punk on April 07, 2017, 05:40:11 AM
And I'm getting vertical stripes on the .dng exports.
I also noticed this issue with the 14 bits lossless feature.
Even if i change the code in the
raw2dng.c file :
if (stripes_correction_needed)
{
printf("\n\nVertical stripes correction:\n");
for (j = 0; j < 8; j++)
{
if (stripes_coeffs[j])
printf(" %.5f", (double)stripes_coeffs[j] / FIXP_ONE);
else
printf(" 1 ");
}
printf("\n");
}
}to
printf("\n\nVertical stripes correction:\n");
for (j = 0; j < 8; j++)
{
if (stripes_coeffs[j])
printf(" %.5f", (double)stripes_coeffs[j] / FIXP_ONE);
else
printf(" 1 ");
}
printf("\n");
}For constant vertical stripe fixs with mlv_dump. (Thanks to Danne for this tip)
The fix don't appear in the command line.
Quote from: a1ex on April 01, 2017, 02:06:31 PM
To decode, you will need a custom mlv_dump (also included). It works by simply dumping the lossless JPEG payload to DNG (that means, processing options such as bad pixel or vertical stripe fixes are not working yet).
Sorry a1ex didnt saw this message ! Thanks for the remind.
Ah yes...my fault for not acknowledging that info. I should have noticed the custom mlv_dump disclaimer.
Cheers A1ex / Danne & GutterPump for reply x
@A1ex
I don't know how to save the debug msgs after a "manually started" cleaning process:
Quote
Before the log gets saved, you should try to perform the sensor cleaning, so messages about it will get included:
When I switch the camera on, the "auto sensor cleaning" is set to "disabled";
The bottom red led is lit a while and then a few times (blocks of data written to the card ?)
It ended at 14:42
I went to the Canon SETUP3 menu to start a manual immediate cleaning
The camera made some noise as if taking 2 pictures then this greyed menu item became yellow again.
I then went to "auto cleaning" item to set it to "enabled"
I then went in the ML DEBUG menu and pressed SET button with cursor on DebugMsgLog
The bottom red led was lit a very little while (but I saw no file on the card created at that time = 14:46).
Finally I switched the camera off.
The following file is generated just when the camera gets started (gets the creation time = 14:42):
https://www.dropbox.com/s/mjickmertwial3y/DM-0001.LOG?dl=0 (https://www.dropbox.com/s/mjickmertwial3y/DM-0001.LOG?dl=0)
Diagnosed the bricking issue - it was caused by a null pointer bug in ML :(
Only Apr04 is affected - both older and newer builds are safe.
Fortunately, it's easy to recover (at least in the two previous cases). One uint32_t at an arbitrary address (between 0 and around 60MB, with a non-uniform probability distribution) might be overwritten with 0xA5A5A5A5 (and eventually end up saved into ROM at camera shutdown, when Canon code saves camera settings).
@BBA: if you believe your camera might be affected by this bug, please send me a recent (bad) copy of your ROM, alongside with an older (good) copy.
If you don't have any backup, copy the oldest files from ML/LOGS you can find on a ML card (hopefully these are "last known good" configuration).
To get a fresh copy, simply delete ML/LOGS from a ML card, and it will be re-created at next startup.
Quote from: BBA on April 07, 2017, 03:30:40 PM
I went to the Canon SETUP3 menu to start a manual immediate cleaning
The camera made some noise as if taking 2 pictures then this greyed menu item became yellow again.
This is normal, it's the same for me and for all 5DIII users.
infos here (http://www.canonrumors.com/forum/index.php?topic=13188.0)
@ Gutterpump
Thanks for your feed !
I get a black DNG (((((
CMD the code is:
@echo off
for %%a in ( %* ) do ( md "%%~na" 2>nul )&( "c:\RAW\mlv_dump.exe" -o %%~na\%%~na_ --dng %%~na.MLV )
The code is wrong?
I have Windows 10 x64. I drag MLV to CMD, it immediately closes, creates a folder and it has black DNG
(https://s8.hostingkartinok.com/uploads/images/2017/04/7bfaabd168d09474f10bb311ced4c840.png)
@A1ex I got the nasty error explosion today using the April 6 build.
I think it was probably because I was using an odd resolution
3008x1280 or something like that. Pulled battery everything seem fine with the camera.
Tried again with 3072x what ever 2.35:1 is and it seemed to work ok.
Would you like me to report the bug.
Here is a link to what was on my card. Error codes and all.
https://www.dropbox.com/sh/8lmopv8z7p2i91f/AAA4saYTmLE7rFQ3t7nSUBCaa?dl=0
Quote from: Mefist on April 07, 2017, 06:11:09 PM
I get a black DNG (((((
Can you share a small part of your MLV file ? Or record a very short part of movie for get arround 10 frames.
Be sure you're using the good version of mlv_dump (https://builds.magiclantern.fm/jenkins/job/crop_rec_4k/7/artifact/modules/mlv_rec/mlv_dump.exe) too.
That's what I used and settings.
(https://s8.hostingkartinok.com/uploads/images/2017/04/a0f13f057a711f0fe713aad9f9b6ffe9.jpg)(https://s8.hostingkartinok.com/uploads/images/2017/04/e8bd24d101d2c3c5dd3d648fdd1a3288.jpg)
(http://poster.radikal.ru/1704/2a/12acbdda0c84.jpg) (http://radikal.ru/video/eNEQy5t1t25)
Quote@echo off
for %%a in ( %* ) do ( md "%%~na" 2>nul )&( "c:\RAW\mlv_dump.exe" -o %%~na\%%~na_ --dng %%~na.MLV )
Just testing April 6th build on 123 with HDMI.
First up, it's a bit sketchy getting signal from the HDMI - have to power on, go into Canon menu, back out again, screen flashes all kinds of weird, live view then appears. First time I tried the camera crashed.
Second up, only getting 7 seconds at 1920x800 60p, GD off, 14-bit lossless, ISO 100. Data rate is low - only about 57mb/s
Just pulling out the HDMI getting 15 seconds. Data rate is more like 67mb/s.
Strange how plugging in the HDMI seems to lower the data rate by 10mb/s?
Successfully unbricked the two cameras that had problems earlier in this thread! (ju64 and filipe), and also solved BBA's issue.
More cameras might be affected (actually up to 817); will add an automatic check to identify them in the next build. The side effects range from nothing at all (overwriting an unused memory location) to soft-bricking (recoverable) and anything in-between (e.g. BBA's issue). The bug can overwrite any setting (aka property in Canon code), or the data structure itself. My own 5D3 was also affected, without any obvious side effects though (as I don't know where the affected setting is used in Canon code).
I'll probably stop adding features for a while and will look into ways for preventing this class of bugs.
Quote from: beauchampy on April 07, 2017, 09:46:20 PM
Strange how plugging in the HDMI seems to lower the data rate by 10mb/s?
It's not strange at all, as the camera has to work 6 times as much (1920x1080 vs 720x480) to create the HDMI image (strictly talking about the LiveView buffer). Most of the work is done by the image processing chips (external to the main CPU) and the DMA (EDMAC), so most of this load is on the memory bus.
The "Frozen LV" (previously known as Hacked preview) works by stopping LiveView buffers from being updated (therefore freeing the memory bus and a bit of CPU time).
@beauchampy
QuoteFirst up, it's a bit sketchy getting signal from the HDMI - have to power on, go into Canon menu, back out again, screen flashes all kinds of weird, live view then appears
Same for me, but the same thing happens to me with older (and more stable) builds... To get hdmi signal I often go to play button and back out again. In my case using a Zacuto EVF.
I get continuous recording with april6th build with ISO100 1920x960 50p GD off 14bit losseless with or without hdmi in 5Dmk3 113.
Quote from: pc_bel on April 07, 2017, 10:21:40 PM
@beauchampy
Same for me, but the same thing happens to me with older (and more stable) builds... To get hdmi signal I often go to play button and back out again. In my case using a Zacuto EVF.
I get continuous recording with april6th build with ISO100 1920x960 50p GD off 14bit losseless with or without hdmi in 5Dmk3 113.
I didn't have much problem with previous builds. Might be because right now I'm using a Leica prime with no electronics, so that might be throwing things off a bit (camera probably thinking a lens not attached).
Just tried 50p and yes, getting continuous at a steady 79MB/s @ ISO 100, 14-bit lossless with HDMI, GD off.
Bit weird how the 60p mode seems to have a much lower MB/s (57MB/s with HDMI, 67MB/s without), so I'm guessing its something a little more complex than just write speed which is the issue there. a1ex / devs if I can help test in any way, please do let me know.
Off to Bermuda tomorrow, so I'll do some more testing of this mode there. Thanks for the suggestion :)
Fantastic!!!
Be aware with ISO. For me 800 is where problems start. I can get only few seconds with 800 ISO. And as I report in post #148, I get a "cutted" image in the top of frame with paning movements... So no usable for serious work...
@a1ex I've used the April 4th build so does that mean my 5D Mark III is affected as well? I remember when using it red numbers were flashing on the top of my screen about memory being low.
I still do not get a good DNG. If you shoot as before then the DNG is good.
Quote from: Mefist on April 07, 2017, 10:47:17 PM
I still do not get a good DNG. If you shoot as before then the DNG is good.
Are the frames black in any of the App's you use ? e.g. Resolve , A.E. ACR ? Or what ever you use
@A1ex is it only 5Diiis that used April 4 build or could it be in the April 3 build too?
Here's a Python script to check for sure (run it and follow the instructions):
# checks whether your camera might have been affected
# by a null pointer bug from the 2017Apr04 ML build.
# updated Apr10 2017 (ignore some false warnings)
from __future__ import print_function
import os, sys
from struct import unpack
def getLongLE(d, a):
return unpack('<L',d[a:a+4])[0]
try:
ROM = open(sys.argv[1], "rb").read()
except IndexError:
print("usage:")
print("1) Backup the contents of ML/LOGS from your card")
print("2) Debug -> Dump ROM and RAM (to get a new ROM copy)")
print("3) Run this command on the latest copy of your ROM:")
print(" python %s ROM1.BIN" % sys.argv[0])
raise SystemExit
# scan for the magic number 0xA5A5A5A5 that might have been
# written into ROM as a result of the null pointer bug
found = 0
ignored = 0
can_ignore = False
ignore_msg = ""
gaonisoy = False
for a in range(0, len(ROM)-4, 4):
v = getLongLE(ROM, a)
n = getLongLE(ROM, a+4)
block_name = ""
can_ignore = False
if a >= 0xF40000 and a < 0xF80000: # RING - that's where most errors were
a0 = a & ~0xFFF # header of this block
if getLongLE(ROM, a0) == 0xFFFF: # active block? only check this one
block_name = "active RING"
if a == a0:
print("Active RING block: %x-%x" % (a0, a0+0xFFF))
else:
block_name = "inactive RING"
can_ignore = True # ignore inactive RING blocks
if a >= 0xF80000 and a < 0xFA0000: # last debug log - ignore
block_name = "last debug log"
can_ignore = True
if v == 0xA5A5A5A5 and n != 0xE5E5E5E5:
print(hex(a), hex(v), block_name)
found += (0 if can_ignore else 1)
ignored += (1 if can_ignore else 0)
if v == 0x6E6F6167 and n == 0x796F7369:
assert a == 0xC0004
gaonisoy = True
print("\nErrors found: %d, ignored: %d." % (found, ignored))
if not gaonisoy:
print("Doesn't look like a valid ROM1 dump.")
elif found > 0:
print("Your camera might have been affected by the Apr04 null pointer bug.")
print("Please contact Magic Lantern developers on IRC (#magiclantern on freenode)")
print("to take a closer look.")
else:
print("Your camera appears OK, no need to worry.")
Great work to all involved! Amazing!
I'm testing this out on my 5D Mk III on 1.2.3. I'm really missing the "centered 5x zoom" option from the older crop_rec module. I was previously running crop3x.2017Jan13.5D3123 version and using the centered 5x zoom was killer for avoiding the warping from the corners of the frame.
Am I missing something, or is this something that I just have to wait on future release?
@a1ex What if you don't know how to run that script.. Can you point me to a detailed instruction on how to run that code, would be greatly appreciated.
Did various tests.
My interest was focused on 3K, UHD and 4K trying to reach peak performance.
Global Draw OFF
1:1 mode
(3:3 always records 30FPS)
I concluded that 14bit lossless performs better.
I also concluded that setting Canon Video to 1280x720 gives shorter vertical size and leaves room for 4K recording.
Changing Aspect Ration allows more recording time
Havent managed to playback my recordings on the camera yet, I get an innocent message.
Quote from: goldenchild9to5 on April 08, 2017, 04:18:37 AM
@a1ex What if you don't know how to run that script.. Can you point me to a detailed instruction on how to run that code, would be greatly appreciated.
This is what I just did to test it (for Windows):
1. Install python for your computer: https://www.python.org/downloads/ (https://www.python.org/downloads/)
2. Make sure to check add python x.x to PATH
3. Make a backup of your current camera's ROM (using magic lantern)
3. When done make a new folder and copy your ROM1.bin to that folder (i made E:/tmp)
4. In the same folder make a new .txt file and rename it to testrom.py --> make sure you have file extensions visible
5. Open the file in your favorite text editor (i like notepad++) and copy the code in a1ex's post
6. Open a command window in the folder you just created (press right click on the folder while holding shift key, select
Open command window here and type this:
py testrom.py ROM1.bin7. Read and follow the output on your screen.
Quote from: janvkem on April 08, 2017, 07:48:25 AM
This is what I just did to test it (for Windows):
1. Install python for your computer: https://www.python.org/downloads/ (https://www.python.org/downloads/)
2. Make sure to check add python x.x to PATH
3. Make a backup of your current camera's ROM (using magic lantern)
3. When done make a new folder and copy your ROM1.bin to that folder (i made E:/tmp)
4. In the same folder make a new .txt file and rename it to testrom.py --> make sure you have file extensions visible
5. Open the file in your favorite text editor (i like notepad++) and copy the code in a1ex's post
6. Open a command window in the folder you just created (press right click on the folder while holding shift key, select Open command window here and type this:
py testrom.py ROM1.bin
7. Read and follow the output on your screen.
Worked great, thank you. My camera appears to be ok :)
Having never done it before: how long should a ML ROM dump take?
Update: I have 'Saving 4f000000...' on the ML screen for 15mins.
Although I had doubts in the beginning, after draining a couple of batteries, I am sure now that current build (6April) is power hungry
Quote from: garry23 on April 08, 2017, 09:14:32 AM
Having never done it before: how long should a ML ROM dump take?
Near instant? I believe when you "update" firmware with ML it will backup the ROM then. I just formatted my SD card in PC, format in camera, load ML from PC onto card, update firmware to setup ML, grab the ROM1.BIN and follow instructions.
@hindra
Thanks: I'll stop the current dump and try again.
When I looked at the card, I had a copy of ROM1.bin, however, the ML screen kept the 'Saving 4f000000...' on the ML screen for 15mins, until I stopped the Cam.
Run the test: all looks ok :)
Sounds encouraging; I'll integrate this test in the next ML build, but I'm a bit busy this weekend.
(that also means you will need a bit of patience on IRC - I'm not online this weekend)
I have got this:
C:\Users\XXX\AppData\Local\Programs\Python\Python36>python testrom.py ROM1.bi
n
0xf5c33c 0xa5a5a5a5
0xf5cf0c 0xa5a5a5a5
0xf5d33c 0xa5a5a5a5
0xf5df0c 0xa5a5a5a5
0xf5e33c 0xa5a5a5a5
0xf5ef0c 0xa5a5a5a5
0xf5f33c 0xa5a5a5a5
0xf5ff0c 0xa5a5a5a5
0xf6033c 0xa5a5a5a5
0xf60f0c 0xa5a5a5a5
0xf6133c 0xa5a5a5a5
0xf61f0c 0xa5a5a5a5
0xf6233c 0xa5a5a5a5
0xf62f0c 0xa5a5a5a5
0xf6333c 0xa5a5a5a5
0xf63f0c 0xa5a5a5a5
Your camera might have been affected by the Apr04 null pointer bug.
Please contact Magic Lantern developers on IRC (#magiclantern on freenode)
to take a closer look.
I have good ROM1 from magiclantern-crop3x.2017Mar21.5D3113.
Already on IRC
I just put together a quick video demonstrating the new options .
https://youtu.be/9trjJKDdLO0
Pushed the saturation a bit during the grade because I wanted to see how it holds up. I'm impressed.
QuoteYour camera appears OK, no need to worry.
Happy camper :)
Quote from: Danne on April 08, 2017, 10:53:24 AM
Happy camper :)
Nice save, @Danne! :) (will have to check mine first thing in AM after I get back home from work) -- good thing I only used the April 4th build on the 'B' cam that I just scooped up from Craigslist earlier this week and not my original baby, Ha!
Quote from: pocketrubbish on April 08, 2017, 10:40:34 AM
Pushed the saturation a bit during the grade because I wanted to see how it holds up. I'm impressed.
Indeed, extremely impressive.
What were the highest ISO's you were able to use for the night scenes @pocketrubbish?My apologies for lack of paying attention to details in your clip as I was obviously all eyes on the static shots. Similar effect if one were to stare at a Television showing nothing but static and it just sucks you right in which means you got the job done well even if it wasnt intended for.
The reason why I asked (hope you're reading this @a1ex) is because I'm an avid fan of using DarkFraming average processes for high ISO shots and ever since this whole lossless compression 4k crop_rec stuff came out for the 5D3 -- haven't been able to figure out how or why when I apply DF avg process it just don't correspond properly and not sure if this is related to the modified mlv_dump spitting out different headers for each DNG or some sort?
I think because of this it's making it a bit trickier for the DF avg process to work properly (at least in @Danne's cr2hdr.app is what I use) so not sure if this at all possible due to the limitation factor of this new lossless compression codec in MLV 14-bit lossless or perhaps just a simply fix in the code would do the trick?
Quote from: janvkem on April 08, 2017, 07:48:25 AM
This is what I just did to test it (for Windows):
py testrom.py ROM1.bin
"python testrom.py ROM1.bin" works for me
Not sure if this would help anybody but I put in a little simplifying doubleclick command and a howto for mac users here.
https://bitbucket.org/Dannephoto/magic-lantern/downloads/ROM1test.zip
Basically the same for mac as for windows if running straight on the command line.
"python testrom.py ROM1.bin"
Your camera appears OK, no need to worry.
:) :) :)
Thanks Danne
I'm feel quiet 8) :
(http://image.noelshack.com/fichiers/2017/14/1491644829-py.jpg)
Thanks to @a1ex for the script and thanks to @janvkem for explanations.
Thx for the mini guide. "Your camera appears OK, no need to worry."
@alex:
i guess its a good thing to add this checker to the module itself.
so all users get informed who potentially don't realize there are broken properties.
e.g. not crashing due to the property being bad, but weird image metadata or correction curves.
would warn them on display, telling them that the old version had a bug and this camera
seems to have a property being flipped.
Quote from: pocketrubbish on April 08, 2017, 10:40:34 AM
I just put together a quick video demonstrating the new options .
https://youtu.be/9trjJKDdLO0
Pushed the saturation a bit during the grade because I wanted to see how it holds up. I'm impressed.
How did you manage to shoot in 3072x1308 24fps at 40 seconds? Did not change the aspect ratio? I do not write 5dm3 in 3k 14bit-lossless 16:9 for more than 2 seconds
@A1ex Ran the Python test and my camera was not affected!!!!!
Quote from: Mefist on April 08, 2017, 01:14:45 PM
How did you manage to shoot in 3072x1308 24fps at 40 seconds? Did not change the aspect ratio? I do not write 5dm3 in 3k 14bit-lossless 16:9 for more than 2 seconds
16x9 would be more like 3072x
1728. The best I've been able to do continuous is 3072x1536.
Quote from: ewinemiller on April 08, 2017, 01:52:47 PM
16x9 would be more like 3072x1728. The best I've been able to do continuous is 3072x1536.
I have 5dm3 firmware 1.1.3 CF SanDisk 64GB 160Mb/s
My settings: 3072x1308 with the aspect ratio of 2.35:1, Crop mode 3K 1:1, 14bit-lossless. If I put a resolution higher, the recording automatically stops.
I did several 4K recordings with 14bit lossless. However I noticed some fat blotchy areas of compression, easily detectable.
I was shooting with 50mm 1.8 and compression was applied naturally in out of focus areas.
I thought 14bit lossless was visually the same as the ordinary 14bit.
it definitely is the same. compression is lossless.
I was using 4K 1:1
Selection PAL/NTSC mode or 1080/720p through Canon Menu, changing Aspect Ratio, was giving me some more resources for more FPS.
Then I switched to 1920 50/60p, 1920x1080.
I pressed REC and got a warning "EDMAC timeout"
Brick camera
Did you take out the battery before you concluded the camera is bricked?
Quote from: Danne on April 08, 2017, 11:22:44 AM
Not sure if this would help anybody but I put in a little simplifying doubleclick command and a howto for mac users here.
https://bitbucket.org/Dannephoto/magic-lantern/downloads/ROM1test.zip
Downloaded & ran the ROM1test (Thanks for that @Danne) and I was even more
surprised by the result because I HAD USED the 2017-04-04 build.
Hmm... what gives?
(https://c1.staticflickr.com/3/2848/33914393675_a8a4fd7d52.jpg) (https://flic.kr/p/TEU8Fr)
By any chance could it be possible for some of us to have used this particular build and
NOT be affected by the null pointer bug at all @a1ex? :P
Also felt this was worth sharing after running the 2017-04-06 experimental build for 5D3.123 in case they give out hints or anything.
(https://c1.staticflickr.com/3/2894/33531220290_0eab0f61d3.jpg) (https://flic.kr/p/T63gzL) (https://c1.staticflickr.com/4/3684/33786501351_b00a979cb1.jpg) (https://flic.kr/p/TtADKn)
Some small bug I also ran into upon when selecting 45 in FPS override (w 1080p45/48 3x3 selected within crop_rec.mo) which then freezes the LiveView more than half of the time.
(https://c1.staticflickr.com/3/2810/33786952001_ec813f47b7.jpg) (https://flic.kr/p/TtCXHc)
After each battery pull I had to immediately go back into FPS override and either put it back into 48p or turn it off to keep LV from freezing up again. This is reproducible.
This might be a stupid question but should MLVFS work with the compressed 14bit files? im just getting pinkish static when trying to view them.
Tried the script on my camera and everything seems fine, i missed the April 4th update, tried all of the other builds ;)
@ Lars Steenhoff
If the camera doesnt respond, on/off and I need to take off the battery, is considered as brick ?
Sorry if I didnt specify clearly :D
@Quentin: ROM backup done, right? If not, save the contents of ML/LOGS from the oldest ML card.
Then, get a fresh copy of your (bad) ROM with the portable ROM dumper (https://www.magiclantern.fm/forum/index.php?topic=16534.0) and contact me on IRC (online for the next 2-3 hours, but busy, then tomorrow at about the same time).
@janvkem @Danne @a1ex Thank you guy's ran the test my camera seems to be Ok :D
Quote from: a1ex on April 08, 2017, 09:22:06 PM
@Quentin: ROM backup done, right? If not, save the contents of ML/LOGS from the oldest ML card.
Then, get a fresh copy of your (bad) ROM with the portable ROM dumper (https://www.magiclantern.fm/forum/index.php?topic=16534.0) and contact me on IRC (online for the next 2-3 hours, but busy, then tomorrow at about the same time).
a1ex, how to connect to you in IRC#magiclantern? I login in IRC#magiclantern with my nickname, but there is no yours nickname in the nickname list.
(http://i.imgur.com/J7DfURv.png)
1920x960 50fps 3x3 pixel binning gives me static on all footage. Am i doing something wrong here?
QuoteTo decode, you will need a custom mlv_dump (also included). It works by simply dumping the lossless JPEG payload to DNG (that means, processing options such as bad pixel or vertical stripe fixes are not working yet).
Quote from: Vegandelight on April 09, 2017, 12:27:49 AM
Am i doing something wrong here?
Yeah, you're using MLVFS, it hasn't been updated for these experimental builds. Use this: http://www.magiclantern.fm/forum/index.php?topic=15108.0 or mlv_dump.
I wanted to record h.264, my daughter with her new bicycle. No RAW, no anything.
Crop Mode and Raw Recording werent loaded.
The camera was recovered after a bad crash and removed the battery.
I never reset to ML defaults, or reload the modules as the usual procedure.
I shot 4 takes.
The two were MLVs.
I havent touched anything.
RAW recording enabled itself somehow
Hi Alex and every one
i am so sorry but my english very bad !!!
how i can upgrade my camera ?
is't free or i have to pay or what i am new here in this forum i don't get every thing ????
thank you
Thanks for the response. Finally got a set of DNG's, wish they worked directly in Premiere.
Keep up the good work.
Quote from: DeafEyeJedi on April 08, 2017, 06:56:59 PM(https://c1.staticflickr.com/3/2894/33531220290_0eab0f61d3.jpg) (https://flic.kr/p/T63gzL) (https://c1.staticflickr.com/4/3684/33786501351_b00a979cb1.jpg) (https://flic.kr/p/TtADKn)
hey @a1ex
is this any indicator for a bricked camera?
because this shows up to me, even though the script says, everthing is ok.
I have some troubles running those scripts.
not sure if I do it right. (I just take ROM1.bin from sd card and scan it via script.)
Quote from: Quentin on April 09, 2017, 11:06:20 AM
I wanted to record h.264, my daughter with her new bicycle. No RAW, no anything.
Crop Mode and Raw Recording werent loaded.
The camera was recovered after a bad crash and removed the battery.
I never reset to ML defaults, or reload the modules as the usual procedure.
I shot 4 takes.
The two were MLVs.
I havent touched anything.
RAW recording enabled itself somehow
Was ML still in the camera?
After you pull the battery, on the next power up the modules won't load. If you power down then power back up again, the modules will load again with your settings and all. Could this be what happened?
After removing the battery, its was an up normal shutdown.
I havent touched anything.
Next day I turn on the camera to shoot my daughter.
I saw that the modules didnt load. Τhe setting werent reset.
I shot h.264, (I thought)
When I load the CF card into the computer, I was surprised.
There were 4 videos, two MOV and two MLV.
@quentin Do you remember what the record symbol looked like for any of the clips? Was it the Canon rec icon or did the cam beep then record like when it does for RAW. Also did you try processing the MLVs ? And did you test your Rom with the Python script?
You could also try recording again. Making sure that the mods are disabled for sure.
Remember that this build is bleeding edge.
The recording icon was normal, h.264 and MLV however I didnt pay attention to the change.
I processed the MLVs, they were normal.
Nope, unfortunately I have no clue about python scripts, ROM processing etc.
I just checked the camera now.
Modules restored as the crash never happened.
Definitely I didnt reset to defaults and didnt select which modules to load.
I am familiar with this first-aid procedure and did it many many times.
I did a try too :
https://vimeo.com/212477560
3009 x 1280 + 14 bits lossless = continuous shooting.
Just don't shot in ETTR, keep an eye on the compression percentage and check the histogram and you're done.
I used the new amazing ew ML Cinelog-C to Ektar 100 LANDSCAPE lut of hyalinejim (http://www.magiclantern.fm/forum/index.php?topic=19338.msg182770;topicseen#msg182770)
Quote from: GutterPump on April 09, 2017, 07:33:15 PM
I did a try too :
@gutterpump you make art out of a camera test. Beautiful!
@aschille84 glad to know you like it !
If anyone is interested I think we have everything to do it for 500D (Full res LV others will be useless - too slow sensor).
3 years ago I increased the width, but I did not know how to increase the height.
I suppose these are the registers (my old notes) :
ADTG[2]1172: 0x3d5 (was 0x667)
ADTG[2]1173: 0x39b (was 0x6e6)
ADTG[2]1178: 0x3d5 (was 0x667)
ADTG[2]1179: 0x39b (was 0x6e6)
ADTG[2]1214: 0x3de (was 0x622)
So look at crop_rec.c - /* some modes require additional height adjustments */
I,m interested. Got a couple of 500D cams at work :) Could you prepare a build for testing?
Sorry it will not be possible.
It's just a hint for someone like reddeercity but with 500D.
I see. Thanks. Short on time over here.
Quote from: Greg on April 09, 2017, 11:04:39 PM
It's just a hint for someone like reddeercity but with 500D.
Quote from: Greg on April 09, 2017, 10:50:34 PM
If anyone is interested I think we have everything to do it for 500D (Full res LV others will be useless - too slow sensor).
3 years ago I increased the width, but I did not know how to increase the height.
I suppose these are the registers (my old notes) :
ADTG[2]1172: 0x3d5 (was 0x667)
ADTG[2]1173: 0x39b (was 0x6e6)
ADTG[2]1178: 0x3d5 (was 0x667)
ADTG[2]1179: 0x39b (was 0x6e6)
ADTG[2]1214: 0x3de (was 0x622)
So look at crop_rec.c - /* some modes require additional height adjustments */
Very useful , thanks :D I was wonder on this , I was going to pm you if you have any notes that could help.
I still can't reproduce you test in the 500d , registers are different on 5d2 but close thou.
my problems is the timer A can't lower it lock to 610 , and ever time I try agjusting the Headtimer
I get a frozen camera/lockup (battery pull) so if I can get timer A to lower I think I will have it
I have the width , but compressed to the left in a 2K crop window , anyways thanks it helps .
I'm afraid there may be a problem with raw buffer. Something like 10/12bit raw video with old cameras.
@greg think it will be possible on eos m? I know it won't do 4K or 3k but any resolution boost would be good.
This is where I am at with 700D so far, just trying to figure out how to change height without it getting all scrambled, was able to change width past 4096 but was getting borders so brought it down to an even number, am able to go to almost 1500 on height until it scrambles, so I am assuming other registers need to be adjusted to make it work.
Also, fps drops to 6.5fps with the register changes to make it work, was able to force it to 9fps with C0F0[6014], any higher caused some sort of overlap on image.
Here are some examples of resolutions I would get by adjusting C0F0[6804]. It seems as I adjust the 3 on the left, the height would change, the 3 on the right, the width would change, the middle 0 seems it has to stay 0. I can go much higher on the height, but as I said, it gets crazy looking, shifty and predator vision like.
4550204 1920 x 1080
4540298 1928 x 1144
4e40448 4248 x 1217
4cc0422 4096 x 1193
45b0422 4096 x 1080 current
50b0564 5384 x 1256
(https://s29.postimg.org/we3eb6yuv/4096x1080.jpg)
I managed to get height going by adjusting
6804 0xc3b0422 4096x3096
713c 0x4a7 to 0xca7
8172 0x6fd to 0x8fd
8178 0x6fd to 0x8fd
But I end up with this image, not sure how to fix this????
(https://s10.postimg.org/mbk2633ax/Untitled-1.jpg)
I was thinking about looking into the EOSM and 700D. Since there are others out there that are willing to fry their sensors to get 4k working I posted adtg_gui builds for these cameras.
https://bitbucket.org/daniel_fort/magic-lantern/downloads/
@mk11174 -- Did you already get 14-bit lossless compression working?
Quote from: jankrueck on April 09, 2017, 04:27:47 PM
is this any indicator for a bricked camera?
A bricked camera is easy to identify - it does not boot (in other words, it's as useful as a brick). So far, there were two cases, both successfully recovered.
However, the Apr04 null pointer bug might (or might not) have changed some of your settings somehow. Since you have not noticed any side effect yet, your camera is probably OK, or you may find out some strange behavior when using some functionality that you don't use very often. That doesn't count as "bricked", but still, those (persistent) changes that appeared because of that bug are not desirable.
Quote from: DeafEyeJedi on April 08, 2017, 06:56:59 PM
By any chance could it be possible for some of us to have used this particular build and NOT be affected by the null pointer bug at all @a1ex? :P
Of course - what happened was not deterministic.
I've just posted a new build that checks whether your camera was affected by this bug. All you have to do is to upgrade - if no message pops up, your camera was not affected by that bug at all. Otherwise, just follow the instructions.
For this particular bug, the persistent changes are easy to identify, because the value that could get written into ROM is always the same - 0xA5A5A5A5.
Note that any other code (from ML and even from Canon) can cause similar issues (and probably much harder to identify). We have patched one such issue many years ago - on most camera models, a bug in Canon's startup code overwrites one single (predictable) memory address with 0 (4 bytes). You can check it by looking for ARMLIB_OVERFLOWING_BUFFER and tracing the patched address in QEMU (e.g. set a value at startup and watch it getting overwritten). In some camera models, this address ended up right in the middle of ML code (thus causing unpredictable behavior that changed with... the amount of code before the affected spot - e.g. insert a printf and watch things working in a totally different way).
This is a limitation of the current CPU and operating system - any task is allowed to write at any memory address. On top of that, Canon code saves their settings to ROM (by reflashing), so all it takes to soft-brick a camera is a programming mistake that results in writing at the wrong memory address.
I can think of a run-time check that detects any writes to Canon's property data structures (some of these are the persistent settings that get flashed into ROM) from tasks other than (Canon's) PropMgr, but it will definitely have a noticeable overhead (impact on performance). I can also think about adapting g3gg0's mem_prot module (which, back then when g3gg0 wrote it, was way above my level of understanding), but this won't catch badly configured DMA transfers. I can also imagine a routine that would check the integrity of Canon's property data structure right before they get saved into ROM (and maybe lock up the camera if problems are detected).
nice work @a1ex. I tested new build and now message pops up.
I was/am curious about this error/bug saying cache is locked.
I also can't get 1920x1920 working, can't get any higher than 1728px.
Before the april6 I was able to record square footage.
Not sure if you changed something, or what happend.
(http://i.imgur.com/Q5dA739.jpg)
Also If I do 3k ISO400 I get a crash when there is to much detail moving.
(http://i.imgur.com/27HLwt7.jpg)
ML ASSERT:
0
at mlv_lite.c:2036 (compress_task), task compress_task
lv:1 mode:3
Magic Lantern version : crop_rec_4k.2017Apr10.5D3113
Mercurial changeset : 2bbf0bbbaa74 (crop_rec_4k) tip
Built on 2017-04-10 08:54:19 UTC by jenkins@nightly.
Free Memory : 164K + 3162K
hope my data is helping you in any way. if not, just tell me to stop it :D
cheers!
ps: Is there a chance to get some reference sound back in 3k mode, to have better syncing in post?
See this post (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182619#msg182619) for the red dot and the first post for resolution:
Quote
* 1920x1920 @ 24p (1:1 square crop) - continuous*)
...
/* 24p 25p 30p 50p 60p */
[CROP_PRESET_3X_TALL] = { 1920, 1728, 1536, 960, 800 }, /* 1920 */
Quote from: jankrueck on April 10, 2017, 12:15:21 PM
ps: Is there a chance to get some reference sound back in 3k mode, to have better syncing in post?
Quote from: a1ex on April 07, 2017, 10:19:14 PM
I'll probably stop adding features for a while and will look into ways for preventing this class of bugs.
uuuhm sorry for the timewaste.. I somwhow skipped page 8 completly.
It took me a while to understand the resolution"issue"
I probably switched to 25p whithout noticing this will affect something ;)
maybe I have to get a filmclap, till you find time to add soundrecording ;)
Cheers and thanks!!!!
QuotePosted by: dfort
@mk11174 -- Did you already get 14-bit lossless compression working?
I have only been working with resolution changes for fun just to see what happens and of course sharing what I find. :)
Tried some of the new experimental modes on Fw 123 today. Looked really good but i could not get 1920x1080 45p continious on my Komputerbay 1066x I got around 10-20sek rec time. Has anyone achieved 45p continuous on FW 123?
Quote from: a1ex on April 10, 2017, 11:12:36 AM
Of course - what happened was not deterministic.
I've just posted a new build that checks whether your camera was affected by this bug. All you have to do is to upgrade - if no message pops up, your camera was not affected by that bug at all. Otherwise, just follow the instructions.
Fortunately, no messages prompted up after upgrade w 2017-04-10 build though I am curious to see what the instructions would be for the affected ones.
Thanks for double checking @a1ex! ;)
Actually decided to try this (since it was reproducible last time) again w 1080p 45/48 3x3 under Crop mode while in 60p from Canon menu together with FPS override set to 45p and can now confirm that this bug has either probably gotten worse or I'm doing something incorrectly (which I doubt) but feel free to tell otherwise.
(https://c1.staticflickr.com/3/2934/33828393021_6e5e8b3f96.jpg) (https://flic.kr/p/TximFx)
Hence the reason why I had to use an iPhone to snap this screenshot because no matter how quick I can try getting back into ML menu upon restart (after each battery pull) and getting to scroll around (both Canon or ML menus) within 5 seconds more or less was impractical. It just freezes up the LV.
Quote from: Markus on April 10, 2017, 06:30:05 PM
Tried some of the new experimental modes on Fw 123 today. Looked really good but i could not get 1920x1080 45p continious on my Komputerbay 1066x I got around 10-20sek rec time. Has anyone achieved 45p continuous on FW 123?
How were you able to achieve 15-20 seconds without the LV display freezing up within 5 seconds more or less @Markus?
Meanwhile gonna try defaulting Canon settings to defaults and reload a fresh 2017-04-10 experimental build again just to double check.
For now, FPS override only works in Low Light mode for reducing the frame rate (known bug).
For 1920x1080 at 45 fps, set 50 fps in Canon menu.
@jankrueck I got the same thing Compression crash recording @ 3K running April 6th build.
@A1ex Gonna try April 10th build today :D Thank you once again for all your hard work..
No lockups here. Only hard to get cont rec on alot of the sweetspot resolutions.
Some settings of mine:
Global draw on, canon in 720p all-I 50p mode, croprec 3x3 binning 1920x1080 47/48p @ 45p. Lossless 14bit.
Without global draw i could not get a bw preview in this mode.
Anything that sticks out?
The file size estimation got confused some times when switching between modes. It also got the wrong fps when switching. I went to photo mode and back as a workaround when this happend.
Quote from: a1ex on April 10, 2017, 07:41:09 PM
For now, FPS override only works in Low Light mode for reducing the frame rate (known bug).
For 1920x1080 at 45 fps, set 50 fps in Canon menu.
That did the trick. Haven't use PAL in years (I'm on NTSC land). Should have thought of that to begin with and Thanks for the tips.
So far 1920x1080 @ 45p MLV-lossless 14-bit seems to be continuous even w HDMI out to an monitor (clean native feed :D) which is a PLUS for gimbal work because obviously the 5D3 will have the grey previews while mounted on the Ronin (which will go off from 'Turn off LCD' within 5 seconds during each recordings -- useful feature from 'Powersave in LiveView' under Prefs within ML menu) and can then focus on the external monitor instead with a peace of mind! 8)
@a1ex Just tested April 10th Build so far so good. One thing that I've noticed is that I can record at higher resolutions for longer now for example 3520x1498 - 2.35:1 I can get over 12 sec on a Lexar 128GB 1066x card before it would record for like 2 sec on the April 6th Build great work. Also one thing that I've noticed is LiveView in 3K mode when recording @ 2.35:1 is all grayscale now, on the April 6th Build it showed both color and grayscale.
Any differences between Apr10 and Apr06, except the null pointer check performed at camera startup, are your own imagination (review the changes, please).
Or, do a controlled test. Same static scene, same settings (!), run 10 test clips on both builds (alternating the builds), and then draw the conclusion.
@a1ex Cool no problem.. @ camera startup I don't see any test running am I missing something, or it just runs in the background and if anything is wrong it flashes a message.
Upgrade camera again via Canon menu and wait for either a pop up screen notifying instructions or none will be prompt (meaning it isn't affected) @goldenchild9to5 if you haven't already.
So far, I've got 3 users with warnings about the null pointer bug (besides the 2+1 ones already solved over the weekend). Two of the new users had a false alarm (the modified value from ROM not affecting the functionality at all, being in unused areas), and the third user had his camera successfully patched.
Updated the tests (both python script and ML build) to ignore the false warnings I've encountered.
Awesome!!!!! @a1ex!!!!!
Quote from: Markus on April 10, 2017, 06:30:05 PM
Tried some of the new experimental modes on Fw 123 today. Looked really good but i could not get 1920x1080 45p continious on my Komputerbay 1066x I got around 10-20sek rec time. Has anyone achieved 45p continuous on FW 123?
Yes. (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182320#msg182320)
Quote from: a1ex on April 10, 2017, 09:52:34 PM
Updated the tests (both python script and ML build) to ignore the false warnings I've encountered.
A1ex, where can i copy updated python script?
Many thanks.
@DeafEyeJedi Upgraded ML, or installed ML from the canon menu a second time Didn't see no pop-up screen all it did was install ML again. So it seems like my camera is ok than.
WOW.. Getting incredible images @ 3K :D
Using April 10th build. Testing 3072 x 1728
Got
ML ASSERT:
RAW_IS_IDLE
at mlv_lite.c:584 (measure_compression_ratio), task shoot_task
lv:1 mode:3
Magic Lantern version : crop_rec_4k.2017Apr10.5D3113
Mercurial changeset : 11f405b62b31 (crop_rec_4k) tip
Built on 2017-04-10 19:48:55 UTC by jenkins@nightly.
Free Memory : 163K + 3118K
Pulled Battery then restarted and reloaded mods.
The only things that I don't like are the current gray scale that it jumps to when it records (I know its purpose is to reserve resources)
The inconsistency of clip length due to changing compression.
And I find I cant get over 4 seconds on average with a 128 GB Lexar 1066x 160MB/s card.
I know this is very experimental I am just reporting my findings so far.
Ran ML April 10 4K test my 5D mk3 and is indicating the problem of null pointer bug from April 4 ML 4K experimental. Reverted to latest nightly 1.2.3 2017-3-30 build which seems to work fine. Casual user of ML & not sure if I need to do anything to address potential bug?
QuoteTried some of the new experimental modes on Fw 123 today. Looked really good but i could not get 1920x1080 45p continious on my Komputerbay 1066x I got around 10-20sek rec time. Has anyone achieved 45p continuous on FW 123?
I failed to recognize how noise at higher isos gives less effective compression. 1920 1:85 45p at iso 800 seemes to be max cont rec for me.
I was wondering what this means, this feature will be merged into the silent pictures module? How easy will it be to trigger the shots?
Quote* Full-resolution LiveView: 5796x3870 at 7.4 fps (128ms rolling shutter) - continuous*) at 5 FPS!
The last feature complements the well-known full-resolution silent pictures - the new implementation will be usable at fast shutter speeds, without the exposure gradient - but with rolling shutter (just like regular LiveView frames).
Shot a card full of 50p / 960 / 3x3, 113 at the beach and unfortunately every clip has a tearing across the top of the frame :-(
This was processed in cr2hdr 4k and it is the April 06th build. I have live view set to real time if that makes a difference..
https://vimeo.com/212739433
Edit: I also kept getting error messages. Recording with the error log still on the screen resulted in the old 1.67x stretch slow motion clips.
That footage is recorded unstretched right? So cr2hdr.app shouldn,t add the compensating destretching tag to the dng file. I could maybe put in a "not stretched footage" in the menu so it would leave the tag alone.
Maybe the info is already in the rawc metadata? Don,t think compressed_raw mlv_dump keep that code yet?
Can you upload asmall mlvv sample Beauchampy?
Quote from: Danne on April 11, 2017, 03:21:43 PM
That footage is recorded unstretched right? So cr2hdr.app shouldn,t add the compensating destretching tag to the dng file. I could maybe put in a "not stretched footage" in the menu so it would leave the tag alone.
Yes, it's unstretched 960 50p 3x3 binning.
Do you think this is cr2hdr rather than ML?
Edit; I also tried turning the preview to auto (switched to greyscale low fps) and the problem persisted.
Definately tag added not ML. Could you upload a short sample? In latest cr2hdr.app you can create a sample package in mlv_dump menu. Selection (12).
Latest version also combines both compressed and non compressed versions of mlv_dump so no need for any 4k version now.
Quote from: Danne on April 11, 2017, 03:26:57 PM
Definately tag added not ML. Could you upload a short sample? In latest cr2hdr.app you can create a sample package in mlv_dump menu. Selection (12).
Latest version also combines both compressed and non compressed versions of mlv_dump so no need for any 4k version now.
Here's a problematic sample MLV (sorry, didn't have an option 12 in mlv dump settings on my cr2hdr?)
https://drive.google.com/open?id=0B-GY59oq1k7eM2xtVVhkZFRFUEk
Is this where you upload the latest builds for mac?
https://bitbucket.org/Dannephoto/cr2hdr/downloads/
EDIT: Tried looking at the same sample file in 'Footage' and the same tearing occurs.. Not sure it is cr2hdr? :-/
Tag is added by mlv_dump(ml-dng version, dng.c code from dmilligan).
Could you upload the zip package? Much smaller in size. I have no good internet connection.
QuoteIs this where you upload the latest builds for mac?
yes.
https://bitbucket.org/Dannephoto/cr2hdr/downloads/
Quote from: Danne on April 11, 2017, 03:53:03 PM
Tag is added by mlv_dump(ml-dng version, dng.c code from dmilligan).
Could you upload the zip package? Much smaller in size. I have no good internet connection.
yes.
https://bitbucket.org/Dannephoto/cr2hdr/downloads/
Here you go, thanks for your help!
https://drive.google.com/open?id=0B-GY59oq1k7eb2tDNm9VcEJnTmc
Thanks. I take a dive into this later tonight.
Quote from: Danne on April 11, 2017, 04:21:34 PM
Thanks. I take a dive into this later tonight.
Thanks mate I really apprecite that. Just FYI I just tried 123 and the latest April 10th build and the issue persists.
Ok, just took a quick look
Quote...old 1.67x stretch slow motion
I took this as you were getting stretched footage? Anyway, the tearing issue is as you say something coming from the compress raw itself not from mlv_dump. About stretching issue it all looks ok to me.
Quote from: Danne on April 11, 2017, 04:29:10 PM
Ok, just took a quick lookI took this as you were getting stretched footage? Anyway, the tearing issue is as you say something coming from the compress raw itself not from mlv_dump. About stretching issue it all looks ok to me.
That was only happening when my screen suddenly populated with errors and logs. I think crop_rec had crashed. When it was working properly (non stretched 3x3 50p @ 960px) I get the line tearing.
Anyone able to get 50p 3x3 1920x960 working without the frame tearing issue I am experiencing? :-\
@beauchampy I got 1920 x 1080 48p 3x3 without tearing but rec time was inconsistent.
I also got 1920 2.35:1 @ 48p without tearing. That was on the very first build
However, on the 4-10-2017 build I took about 5 or 6 clips of 3072 x 1728 @ 23.976 last night and notice that 1 frame out of all the clips had a small horizontal row of pixels that seemed like it was offset from the rest. Ill upload if I get a chance.
I was using RAWFlow with an updated MLV_DUMP to convert 14 Bit lossless to DNGS.
@beauchampy: known bug, but right now, figuring out how prevent the recent soft-bricks has a higher priority.
Until then, my advice would be to try the Apr03 or older builds (which are double-buffered, which avoids tearing, but Frozen LV is not working well, so recording times may be shorter).
Do not use Apr04.
As far as the frame tearing issue, I saw it on nightly builds a while back on my 7D in crop mode and eliminated it by not using the accurate preview (ml greyscale). It was suggested that processor overload was responsible.
The 7D issue does not apply here - on 7D, the issue is caused by the preview backend (and for the moment, I don't have any solution other than disabling the previews completely).
Here, the issue is my single-buffering experiment (which gives a little more memory for high-res modes, but apparently it doesn't play nice with high-FPS modes).
@beauchampy
I'm really interested in 192050p 3x3 mode and while I wait for a solution for the tearing issue, for me the solution is 1080p45 3x3 mode. Not same slowmotion like 50p but no tearing issue and continuous recording. Tested at ISO3200 and it works for me continuous!!!. Only framing is a little bit offset...
5Dmk3 113 Lexar 64Gb 1066x
Tested:
April3th build 1920x960 50p 3x3 mode NO TEARING ISSUE!!!! :) :) :) Continuous at ISO100 and 400. At ISO800 shorter recording times (aprox 20sec. for me).
5Dmk3 113 Lexar64Gb1066x
So I can use a little education here. Sorry Im behind. If I get the:
ML ASSERT:
RAW_IS_IDLE
at mlv_lite.c:584 (measure_compression_ratio), task shoot_task
lv:1 mode:3
where the text appears on the screen, is this a crash? I have been assuming it is and pulling the battery. Should I be doing this?
Here's a 5D3.123 build from 2017-04-03 for those that want to try and confirm...
https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads/magiclantern-crop_rec_4k_2017April03.5D3123.zip
Hello @a1ex , I ran the ROM1 test, no error found.
I updated ML to the April 10th, in debug section show this error
(http://dcmakecreative.altervista.org/VRAM0.png)
There is a way to fix it?
Thanks!
Quote from: hjfilmspeed on April 11, 2017, 06:28:46 PM
is this a crash? I have been assuming it is and pulling the battery. Should I be doing this?
Yes, taking the battery out after a crash is what I always do.
Unfortunately, it doesn't appear to prevent Canon code from saving corrupted settings into the ROM. Still looking into it.
edit: opening the battery door seems to be interpreted as some sort of emergency shutdown, but a clean one - settings are saved into ROM, but ML shutdown hooks (e.g. saving ML config files) are not executed. But, if you somehow hold the battery door switch pressed, and just remove the battery, the ROM is not updated.
@JackDaniel412: asked (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182790#msg182790) and (http://www.rookiemoms.com/try-this-with-me-asked-and-answered-to-end-little-kid-nagging/) answered (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182619#msg182619) ;)
Re: tearing I noticed this happening after the 2017-04-06 build and still persist with 2017-04-10 build as well when shooting in 1080p 3x3 45fps (50p Canon) and could this strange pattern of corrupted (plus dropped?) frames in several MLV's that spat out be related to the tearing at all?
Because my instincts are thinking that it may have started it after I had a bunch of random corrupted footage similar to the one below (is this what you were seeing also @beauchampy?) otherwise correct me if I'm wrong.
https://vimeo.com/212794399
Re: 2017-04-03 build I am now getting corrupted frames similar to the one above when shooting in 1080p 3x3 45p until I reduce the resolution down to 1920x960p which all seems fine.
Though on the older build I seem to be able to get clean recording w full 1920x1080p in 3x3 45fps which is rather confusing. Gonna need to take a step back and check again with all of this before I lose it for good. literally.
Quote from: a1ex on April 11, 2017, 05:03:14 PM
@beauchampy: known bug, but right now, figuring out how prevent the recent soft-bricks has a higher priority.
Until then, my advice would be to try the Apr03 or older builds (which are double-buffered, which avoids tearing, but Frozen LV is not working well, so recording times may be shorter).
Do not use Apr04.
Can confirm Apr03 build is working correctly. Thanks a1ex!
No problem buddy, you take your time. Thanks for all your hard work, these updates are amazing.
Hey a1ex,
With the latest build from April 10th (the one to ignore false warnings), I am still getting a message about the null bug. I have not run into any major issues or soft bricking, but I also do not have a backup or unaffected copy of my ROM from before the 4th. Am I screwed, any solution to this without a clean ROM from before?
In many cases, I can infer the missing values from another ROMs (so it's not a very hard requirement).
BTW, your existing copy from ML/LOGS is likely to be a good one (as it's only re-created after a format).
Quote from: a1ex on April 11, 2017, 07:58:18 PM
Yes, taking the battery out after a crash is what I always do.
Unfortunately, it doesn't appear to prevent Canon code from saving corrupted settings into the ROM. Still looking into it.
edit: opening the battery door seems to be interpreted as some sort of emergency shutdown, but a clean one - settings are saved into ROM, but ML shutdown hooks (e.g. saving ML config files) are not executed. But, if you somehow hold the battery door switch pressed, and just remove the battery, the ROM is not updated.
@a1ex Hmmmmm So if I mod the switch with say.... tape, I can pull battery with out afflicted ROM being saved?
Should I do this as a extra precaution when heavy testing?
Would this have adverse affects on Canon code being that its an "unsafe" battery pull at that point?
So many questions sorry.
I suppose there is always a chance I might not even know that the ROM is affected and I could clean shutdown anyway.
@Jordanwalkert
Did you save the roms from when installing the actual april 4th build? Those should be the last uncorrepted ones if so...
looong shot
More testing with april3th. 5Dmk3 113
1920x960 50p 3x3.
Now I'm recording continuous at ISO3200.
I'm a little confused... Maybe it depends on the detail level of the image?...
I may have missed this but once you set the settings for 3k 1:1 crop, what's the diff between the regular record mode and then x5 crop mode? They seem the seem on preview but regular mode will record around 3072x1228 but x5 Crop just stops. Is it cranking the data rate or something?
And does FPS override make any difference in any of thi?
Now I've got two consecutive errors... Error 70, then I turn off camera. Next an Error 80. Camera does not show live view after both errors. After a new reboot all is apparently ok...
Another issue in 1920x960 50p 3x3 is framing.
Here there is a composition comparing real image vs live view framing:
http://ovh.to/TpLuMh (http://ovh.to/TpLuMh)
Can anyone else confirm that it seems when you go into Full-res LiveView (it becomes 2+ stops brighter according to the LV as well as the ML histogram) than if in any other settings within crop mode from the latest experimental build?
If so, is this normal or a bug?
Probably it's because of the shutter speed mapping from 1/4000 ... 1/30 to 1/15000 ... 1/FPS (also used on 4K half-fps).
Suggestions for a better (less confusing) implementation?
Here is a quick test I did using 5D Mark III in 3k - 3:1 / Build April 10th latest one. I just Love the images that I'm getting & specially the motion, the images speak for themselves even when youtube super compresses it. Let me know what you guy's think.
https://www.youtube.com/watch?v=OmxEjoruetw&feature=youtu.be
When I find it, I am glad.
When I need it, I cant find it :(
Where is the menu option for setting the Preview ?
Thanks for your help
I tried today with April10th build. Since I normally shoot 1.85:1 I tried 3K 1:1 (2928 × 1580), ISO 640, shutter 1/50. But the cam can only record for 5 sec with 14 bit lossless. Is that normal? Or how to be able to record constantly with this resolution, if possible? CF card is Sandisk 160mb r/w. Thanks for al the heavy work and fun to reach new heights!
DNG: https://www.dropbox.com/s/tz79q88ozc8f5p4/M12-0944_000004.dng?dl=0
Bandwidth required:
2928 x 1580 x 24 (?) x 14 / 8 = 185 MByte/s
Quote from: Walter Schulz on April 12, 2017, 11:06:53 AM
Bandwidth required:
2928 x 1580 x 24 (?) x 14 / 8 = 185 MByte/s
Correct 23.976. Normal I watch ML recommendation in menu, and with these settings says 101.8MB/s at 23.976.
Try recording at ISO100
Quote from: mageye on April 12, 2017, 11:13:17 AM
Try recording at ISO100
Positive results, now 25 sec. Thanks! So reaching (for example) 5 min, lower resolution seems to be the only way?
At the moment yes. But the noise from the ISO causes problems for the compression algorithm. I am getting a (reasonably) reliable resolution of 3648x1216 but still not completely continuous!
Quote from: mageye on April 12, 2017, 11:37:12 AM
At the moment yes. But the noise from the ISO causes problems for the compression algorithm. I am getting a (reasonably) reliable resolution of 3648x1216 but still not completely continuous!
I see, do you mind sharing your ML settings? Thanks!
Crop mode UHD 1:1
RAW video ON, 3648x1216
Aspect 3:1
14-bit lossless
Within Canon menu:
Movie rec. size 1920 @ 24 ALL-I
Video System set to: PAL
That's it!
Today I was doing a studio shooting for 4-5 hours, several shots.
I had my camera plugged on electricity as well as the monitor.
5d mk3, 1.1.3
Experimental Build 10 April
I used 1920x800 60FPS 16:9 14bit lossless
All manual, no AutoETTR
While I was working on the shot, to adjust lights etc, Camera crashed and saw several messages.
Turn it OFF/ON fixed the problem.
The recording was not continuous but sufficient enough to record the useful action.
I noticed that the Aspect Ratio of the monitoring wasnt right.
Later, I saw that the monitoring the colours lost saturation and the Aspect Ratio was jumping from wrong to right.
I also noticed that although the ISO was set manually to 1250, on ML monitor was still 1600.
The temperature of the camera was 40 C
Monitor can't show the hacked bw preview on external monitor. And canons liveview does not give correct representation of ML enabled modes.
Iso in raw is only valid for real Iso values. 100,200,400,800 and so on. The values in between is just digital gain applied to h264 compressed video.
What kind of work flow are you guys using for the 1920x960 3x3 binning 14bit raw mode?
Quote from: Vegandelight on April 12, 2017, 05:18:27 PM
What kind of work flow are you guys using for the 1920x960 3x3 binning 14bit raw mode?
In camera? Or post processing?
For post I used the RAWFlow app and just replaced the MLV_DUMP with the newest version and it processed the 14Bit Lossless just as expected. Drag n Drop.
RAWFlow - http://www.magiclantern.fm/forum/index.php?topic=13338.0
MLV_DUMP Update - https://builds.magiclantern.fm/experiments.html
Quote from: hjfilmspeed on April 12, 2017, 05:55:28 PM
In camera? Or post processing?
For post I used the RAWFlow app and just replaced the MLV_DUMP with the newest version and it processed the 14Bit Lossless just as expected. Drag n Drop.
RAWFlow - http://www.magiclantern.fm/forum/index.php?topic=13338.0
MLV_DUMP Update - https://builds.magiclantern.fm/experiments.html
Nice, Premiere or DaVinci Resolve? The old DNG-files opeened right up in Premiere but these compressed ones does not.
Quote from: Vegandelight on April 12, 2017, 08:13:55 PM
Nice, Premiere or DaVinci Resolve? The old DNG-files opeened right up in Premiere but these compressed ones does not.
DaVinci opened them just fine. I don't know premiere.
Its and extra step but you can always export from resolve. Resolve + DNG = incredible color options!
Is there a chance we'll see 2k/3k/4k resolutions in 3x3? This is pretty amazing but without the massive crop factor it would be a dream come true.
Quote from: a1ex on April 01, 2017, 11:15:41 AM
Consider this being a huge leap towards our next mind boggling goal:
8K RAW Video Recording!
Sample DNG from 5D Mark III, to show that our proof of concept is working:
8k.dng (http://a1ex.magiclantern.fm/2017Apr01/8k.dng)
All that's left is to divide the resolution by 3 on each axis ;)
Has anyone notice an issue re: Dual-ISO 14-bit lossless MLV seems to be spitting out with correct black levels but incorrect for original 14-bit MLV's. Could this be related to the new mlv_dump not corresponding properly to cr2hdr or at least confused with each other atm?
(https://c1.staticflickr.com/4/3938/33870168051_2a8406905b.jpg) (https://flic.kr/p/TAZsWF) (https://c1.staticflickr.com/3/2926/33842109492_50757d012f.jpg) (https://flic.kr/p/TyvE7q)
Anyway here are some short samples in two of each (138 MB spat out w ease from latest cr2hdr.app -- Thanks @Danne!) and all shot within crop_rec @ 3.5k: https://mega.nz/#!OwV2DTLC!rb39cVOrAlXXuSer2M6fw3vSuvMfiSG508hewIQJvmg
New build posted.
Found a way to prevent Canon code from saving some of the settings at shutdown, including the settings block that caused issues earlier (reverse engineering notes here (http://www.magiclantern.fm/forum/index.php?topic=19369.msg182959;topicseen#msg182959)). While not perfect, if Apr04 had this safeguard, all the cases affected by the null pointer bug would have been caught. Still, there are ways to brick the camera, just a bit less likely to do so accidentally (and, as long as the bootloader is not erased, recoverable).
How it works: whenever a crash is identified, or whenever you open the battery door, Canon code no longer saves the usual setting groups at shutdown (RING and RASEN, if you look in the above link). There are still setting groups not covered by this safeguard (still looking into it, but so far, all the null pointer errors were in the RING group).
To test - the PAL/NTSC setting is in the RING group (and, to my knowledge, only saved at shutdown). You also have a dummy crash under Don't click me.
After a bit of battle-testing, I think this should be back-ported to all models.
Changes related to raw recording:
- double buffering is back (except at very high resolutions); didn't test much
- pre-recording indicator: mm:ss.f + ss.f
- full-res LV increases FPS when lowering vertical resolution (up to 5796x400 @ 48 fps)
- in 10x zoom, FPS override can be disabled by half-shutter to allow focusing
- minor fixes here and there (full list on the download page)
For preview with correct framing on external monitors, try Force HDMI VGA (not tested, but IIRC it used to help).
WOW!!!!! @a1ex this is amazing! Cant wait to test!
@a1ex Thanx for all the new updates.. Gonna give the new build a try tonight will post my results
Going to update to new build and found this on card from April 10 Build
ML ASSERT:
RAW_IS_IDLE
at mlv_lite.c:584 (measure_compression_ratio), task shoot_task
lv:1 mode:3
Magic Lantern version : crop_rec_4k.2017Apr10.5D3113
Mercurial changeset : 11f405b62b31 (crop_rec_4k) tip
Built on 2017-04-10 19:48:55 UTC by jenkins@nightly.
Free Memory : 163K + 3118K
Not sure if this needs to be reported.
Testing April 12 build now
Wait..... what did I do wrong here. I couldn't even get one solid rec out of the April 12 build so far. I had to pull the battery every time.
@a1ex Im sending you a message to a zip of my card its peppered with ASSERT messages.
Hi everyone! I just saw there is a new version, in case it helps I'm sharing this 4 CRASH.LOG files from crop_rec_4k.2017Apr04.5D3123 version.
ASSERT: hLvJob->hJpegMemSuite
at ./Epp/Vram/VramStage.c:891, task Epp
lv:1 mode:3
Magic Lantern version : crop_rec_4k.2017Apr04.5D3123
Mercurial changeset : e7e1fdb85f6b (crop_rec_4k) tip
Built on 2017-04-04 15:42:08 UTC by jenkins@nightly.
Free Memory : 163K + 3108K
ASSERT: IsSuiteSignature( hSuite )
at ./PackMemory/PackMem.c:599, task Epp
lv:0 mode:3
Magic Lantern version : crop_rec_4k.2017Apr04.5D3123
Mercurial changeset : e7e1fdb85f6b (crop_rec_4k) tip
Built on 2017-04-04 15:42:08 UTC by jenkins@nightly.
Free Memory : 163K + 3107K
ASSERT: IsSuiteSignature( hSuite )
at ./PackMemory/PackMem.c:599, task RscMgr
lv:0 mode:3
Magic Lantern version : crop_rec_4k.2017Apr04.5D3123
Mercurial changeset : e7e1fdb85f6b (crop_rec_4k) tip
Built on 2017-04-04 15:42:08 UTC by jenkins@nightly.
Free Memory : 163K + 3107K
ASSERT: IsSuiteSignature( hSuite )
at ./PackMemory/PackMem.c:983, task RscMgr
lv:0 mode:3
Magic Lantern version : crop_rec_4k.2017Apr04.5D3123
Mercurial changeset : e7e1fdb85f6b (crop_rec_4k) tip
Built on 2017-04-04 15:42:08 UTC by jenkins@nightly.
Free Memory : 163K + 3108K
@JADURCA Dont use that build it's flawed.. make sure you back-up your ROM.
@a1ex Just tried the April 12th build it's solid great work. The only thing that I've noticed is when recording the minute / second green counter stays @ 00.00 it doen't show the actual recording time.
Just wanted to give heads up to those who want most of resolution with lossless, but high ISO prevents you from it. You can sacrifice dynamic range but have a lower bandwidth requirement by using lower ISO and doing the pull in the post: the noise will be below the original gain line. Sure enough, it is worse than we'd like it, but...
Unfortunately, ISO 50 doesn't work with video, that would've basically removed one bit out of equation. Maybe there is another cheap (performance-wise) trick to lower effective number of bits, so lossless is more effective?
Here's a question: is there a way to have GD when I'm focusing, etc., but have it auto turned off (with everything else that could be turned off) during the recording?
Just tested newest build from Latest Build (2017-04-12 20:31).
I got the red dot in the memory patches again, no big deal right? It seemed like most of my shots using crop_rec had a bad first frame regardless of resolution. Shooting at 3504x1536 at 24fps I only got 157 frames with a Lexar 128gb 1066x card.
Also, something odd kept happening and it kept reverting back to 19fps with FPS override off.
Quote from: KMikhail on April 13, 2017, 08:00:36 AM
Maybe there is another cheap (performance-wise) trick to lower effective number of bits, so lossless is more effective?
Yes - adjusting the digital gain in the raw backend appears to help (as it reduces the range of the data and cuts off the noise bits). Added a (somewhat fake) lossless compression at reduced bit depths, based on this concept (theory explained here (http://www.magiclantern.fm/forum/index.php?topic=18443.msg181620#msg181620)).
Also added some suggestions about what ISO range to use with each option. See the commit for more details on how I've chosen the recommendations.
Quote from: goldenchild9to5 on April 13, 2017, 04:47:07 AM
The only thing that I've noticed is when recording the minute / second green counter stays @ 00.00 it doen't show the actual recording time.
Fixed - was broken by pre-recording updates.
Quote from: hjfilmspeed on April 13, 2017, 02:12:43 AM
@a1ex Im sending you a message to a zip of my card its peppered with ASSERT messages.
The issue seems to be in 1920x1080 48p. Hopefully fixed, but I'm seriously thinking to reduce the default resolution to something like 1040 (thus breaking the promise of 1080p48). Any objections?
Quote from: hindra on April 13, 2017, 10:28:30 AM
Also, something odd kept happening and it kept reverting back to 19fps with FPS override off.
How to reproduce this one?
Anyway, new build posted (minor fixes + experimental 8...12-bit lossless).
Be sure to read the changelog before reporting issues ;)
Edit: just noticed the lower bit depth lossless modes only update the raw buffer every other frame, for reasons not yet understood... if you want to try, compile the sources, but you'll get files with duplicate frames (good for estimating recording times, but nothing more). If you look at a static scene (as I did before posting the build), everything appears fine.
Edit2: looks like all raw types (http://www.magiclantern.fm/forum/index.php?topic=18393.0) that include digital gain are created every other frame :( (that means, I need to find some other method for darkening the image...)
Edit3: progress narrowing down (this used to work before merging 5D3-123 into main builds)Narrowed down to changeset 38e78de (https://bitbucket.org/hudson/magic-lantern/commits/38e78de). This change fixed it! (https://bitbucket.org/hudson/magic-lantern/commits/4cce3d2adafff9ac942fcf93fa3c4c3e7c4a84f6?at=crop_rec_4k)
Congrats, lossless 8bit support. 8)
Any easter egg planned to support other cams ;)?
Quote from: nikfreak on April 13, 2017, 12:17:00 PM
Any easter egg planned to support other cams ;)?
hint, hint (http://www.magiclantern.fm/forum/index.php?topic=18443.msg182989#msg182989) ;)
Quote from: a1ex on April 13, 2017, 11:07:58 AM
The issue seems to be in 1920x1080 48p. Hopefully fixed, but I'm seriously thinking to reduce the default resolution to something like 1040 (thus breaking the promise of 1080p48). Any objections?
Ouch, so no more 16:9 slow motion? This is kinda a big deal for someone like me to *has* to deliver in 16:9 to some clients..
You still have the option to fine-tune the internal parameters until you get continuous recording with no errors or corrupted frames. No programming knowledge is required (just fiddle with numbers from the menu), but it can easily take hours of tinkering until you find a sweet spot.
First of all, thank you so much for the tremendous work and the joy that you give us.
At me a following problem - on all last experimental insertions in a сropmode the screen becomes pink and there are diagonal strips. In the raw mode this does not happen
Okay so I went in for a second test. Over all here is what I am finding
3k 2.35:1 14 Bit lossless Preview auto is very stable
1920x1080 48p, 14 Bit lossless, Preview auto, out of focus scene - not stable lots of crashes even when not Rec out of focus scene
1920 2.35:1 48p, 14 bit lossless, preview auto, out of focus scene - not stable lots of crashes even when not Rec
1920 2.35:1 60p, 14 bit lossless, preview auto - seemed more stable did not crash. Pressed half shutter and it crashed
1920 2.35:1 60p, 14 bit lossless, preview real time - seamed more stable no crash
1920 2.35:1 48p, 14 bit lossless, preview real time - seemed more stable
1920 2.35:1 48p, 14 bit lossless, preview - framing crashed
There were more rec attempts in there but that is all I can remember for now. Unfortunately Ill have to stop testing for a bit. Here are my asserts
ML ASSERT:
0
at mlv_lite.c:2111 (compress_task), task compress_task
lv:1 mode:3
ML ASSERT:
0
at mlv_lite.c:2111 (compress_task), task compress_task
lv:1 mode:3
ML ASSERT:
0
at mlv_lite.c:2118 (compress_task), task compress_task
lv:1 mode:3
ML ASSERT:
0
at mlv_lite.c:2118 (compress_task), task compress_task
lv:1 mode:3
ML ASSERT:
0
at mlv_lite.c:2118 (compress_task), task compress_task
lv:1 mode:3
ML ASSERT:
0
at mlv_lite.c:2118 (compress_task), task compress_task
lv:1 mode:3
ML ASSERT:
0
at mlv_lite.c:2118 (compress_task), task compress_task
lv:1 mode:3
ML ASSERT:
0
at mlv_lite.c:2111 (compress_task), task compress_task
lv:1 mode:3
ML ASSERT:
0
at mlv_lite.c:2111 (compress_task), task compress_task
lv:1 mode:3
ML ASSERT:
0
at mlv_lite.c:2111 (compress_task), task compress_task
lv:1 mode:3
Magic Lantern version : crop_rec_4k.2017Apr12.5D3113
Mercurial changeset : dcdf7432df74 (crop_rec_4k) tip
Built on 2017-04-12 20:31:48 UTC by jenkins@nightly.
Free Memory : 163K + 3116K
48p might be a little better in Apr13.
Otherwise, see the last 2-3 messages. Decreasing vertical resolution in the crop_rec submenu is the only way I know to reduce the frequency of these "crashes" (actually corrupted frames).
Quote from: a1ex on April 13, 2017, 11:07:58 AM
The issue seems to be in 1920x1080 48p. Hopefully fixed, but I'm seriously thinking to reduce the default resolution to something like 1040 (thus breaking the promise of 1080p48). Any objections?
Yes. Let's keep tinkering down on this one please. 1080p 48p is crucial. Especially for HDR video kind of work.
Upscaling in post is a thing. I've shot stuff in 720p crop mode that I'd be comfortable projecting on a 40 foot screen. I can put nitrous in an old Honda, that doesn't make it smart. Reliability trumps (for want of a better word) a few more pixels every time.
I havent tried the latest build but i tried running a manual lens on the camera today, the screen doesnt turn on when in video mode. Works in photo mode.
HAs anyone experienced that?
Yes.
On Samyang 35mm (manual)
I get: 'Ensure a lens is attached. Press START STOP button to activate movie shooting.'
I think it's just a lens communication thing
Quote from: mageye on April 13, 2017, 05:44:39 PM
Yes.
On Samyang 35mm (manual)
I get: 'Ensure a lens is attached. Press START STOP button to activate movie shooting.'
I think it's just a lens communication thing
Thats a normal error, but when you press START STOP the screen goes black and you can hear the mirror locking up like it should. But no image.
Oh. Well I get the image so who knows what's happening there?
When it happens here, pressing MENU twice solves it (see first post).
Back to the crop record.
I have noticed something which at first may appear bizarre:
Indoors I get significantly
longer record times at
higher resolutions than I do
outdoors. This has happened repeatedly over a few days now.
I am thinking it has to do with light levels, detail and compression.
eg.
I have been able to record over 16 seconds at
[email protected] inside and just about 2 - 4 seconds at the
same settings outside!
(14-bit lossless compression)
Testing April 13 build. Much more stable for sure! Compression seems to be tricky still. I'm not the biggest fan of how slow settings (shutter ISO ect) appear to change when in Framing LV but I know it serves a purpose to get proper preview with low processing power.
If we can some how cap the data rate that would be insane. But then I guess it would be lossless at that point.
Higher detail low contrast scene, F1.6 1/125 (1/98) ISO 800
1920 2.35:1 14 Bit lossless 1920 48p preview auto - fairly long rec
16:9 Did not make estimated rec time
1920 2.35:1 14 Bit lossless 1920 60p preview auto - Did not make estimated rec time
3072 2.35:1 14 Bit lossless 3k 1:1 23.976 preview auto seamed good.
16:9 Did not make estimated rec time
On same scene I changed the shutter speed F1.6 1/50 (1/98) ISO 800 then none of the above modes made there estimated rec time.
It seems that if more detail is brought out of low exposure zone, less info gets tossed which means higher data rate.
No crashes/bad frame warnings until I lowered the shutter down to 1/30 in 3072 2.35:1 14 Bit lossless 3k 1:1 23.976 preview auto. Then it crashed/bad frame warning ML assert
ML ASSERT:
0
at mlv_lite.c:2120 (compress_task), task compress_task
lv:1 mode:3
ML ASSERT:
0
at mlv_lite.c:2120 (compress_task), task compress_task
lv:1 mode:3
Magic Lantern version : crop_rec_4k.2017Apr13.5D3113
Mercurial changeset : cf07f797b9c3 (crop_rec_4k)
Built on 2017-04-13 10:34:53 UTC by jenkins@nightly.
Free Memory : 163K + 3115K
Anyone else just try 1080p raw with the 1080p all-i proxie with the new april 13th build? Is it broken? Here is what happened to me today.
1080p raw w/ proxie broken, raw to CF and all-i to SD - Stops recording after a few seconds, camera locks up, preview still works, global draw on or off. After messing with proxie, 14bit lossless no longer working? recording at 90+mb/sec for 1080p as stated on my liveview monitor.
ML ASSERT:
0
at mlv_lite.c:2113 (compress_task), task compress_task
lv:1 mode:3
Magic Lantern version : crop_rec_4k.2017Apr13.5D3123
Mercurial changeset : cf07f797b9c3 (crop_rec_4k)
Built on 2017-04-13 10:35:29 UTC by jenkins@nightly.
Free Memory : 166K + 3785K
ML ASSERT:
RAW_IS_IDLE
at mlv_lite.c:592 (measure_compression_ratio), task shoot_task
lv:1 mode:3
Magic Lantern version : crop_rec_4k.2017Apr13.5D3123
Mercurial changeset : cf07f797b9c3 (crop_rec_4k)
Built on 2017-04-13 10:35:29 UTC by jenkins@nightly.
Free Memory : 167K + 3119K
I uninstall ML, format card in pc, then format in camera, then load ML software and reload firmware to set bootflag.
Still recording at 90mb/sec and in canon menu at format menu it shows no use of CF card. I am also now having issues recording with any of the crop_rec options. Recording stops very quickly. What did I break? :(
EDIT:
Lossless is still working, looking at footage the 1080p raw stills are only 1.90MB
Cant hide my excitement.
ML makes my camera as good as cameras costing $$$$$$
and still ML moves on ...
Gratitude!
@a1ex Tested the April 14th Build last night here are my results..
3K 2.35:1 14bit Lossless = Ok
12bit Lossless = Liveview splits in to two screens and image is not really viewable. Recording stops quickly and compression error pops up on screen
10bit Lossless = Same
8bit Lossless = Same
1080p 45/48fps 3x3 16:9 - 14bit Lossless = Records for a few seconds than crashes camera
12bit Lossless = Same
10bit Lossless = Same
8bit Lossless = Same
1080p 60fps 3x3 16:9 - 14bit Lossless = Records for longer time than 48fps, than after crashes camera
12bit Lossless = Same
10bit Lossless = Same
8bit Lossless = Same
Note after trying the slowmo functions camera turns on but ML Liveview only shows a black screen, without magic lantern sd in camera liveview works.
@Quentin Very exciting
Reproduced - looks like we've got some image processing module that has to be reconfigured for correct resolution. That means, solving this would be one tiny step closer towards fixing LiveView.
(I've only tested the 8...12-bit settings in plain 1080p, without crop_rec)
@a1ex Yeah LiveView seems to be very complicated wish I knew coding to help out. Do you think LiveView will ever be as smooth as 1080p in 3k - 4K mode? Gonna try them in 1080p today, and I'm gonna keep testing. All those new feature ad-ons makes this camera an elite camera. Thank you for your hard work.
12, 10, 8 bit Lossless!!!!!!!!!! Am I reading this correctly? Oh man. This is unreal!
Grazie a1ex! Magic!
Inviato dal mio HTC One utilizzando Tapatalk
10 bit, 8 bit compressed raw? Am I dreaming? Thanks for your ingenious work!
Just tested the new 14 april build.
I'am able to record in crop mod with 8 10 and 12 bits lossless. (but now in crop mod, i can't use more than the 2:35 ratio "eg : 3104x1320" and FPS are locked to 29 if we dont use FPS override. Even if we are in PAL)
This is just crazy in what the 5D mark III is become.
This mean we will really record 3.5k 4k continue.
---
Here a zip with DNG if some want check how look 8/10/12 bits lossless
https://www.dropbox.com/s/pj7db1izytkj3zj/8_10_12bitslossless_samples.zip?dl=0
Three samples from 3284x1320@24 clip
QuoteThis mean we will really record 3.5k 4k continue.
:o
Does april 14th build run the test for the april 4th build?
My connection is so incredibly slow, if i dont have to dowload apr10 just to test for a bug and then download 14th, well that would be swell :)
@GutterPump
You remind me of myself when I shoot lens cap :D
@quentin
Haha it could have been worse, it's really the mess on my desk. But for magic lantern I make an effort.
Hey guys, here's a quick 23 second shot of the 8 bit lossless 3520x1322 at 60 fps with the latest April 14th 1.2.3 build. My hands are really shaky, I'm not a pro and I had my shutter at 1/80th.
ISO 200, F8, 1/80th - SHAKY HANDS - straight out of camera no corrections/lut - 60mbit vbr
Its working :)
https://vimeo.com/213302351
EDIT: 30 fps
thanks for posting the video thats pretty crazy
Quote from: hindra on April 15, 2017, 05:37:19 AM
Hey guys, here's a quick 23 second shot of the 8 bit lossless ... ISO 200
Quote from: GutterPump on April 14, 2017, 04:43:57 PM
Here a zip with DNG if some want check how look 8/10/12 bits lossless
https://www.dropbox.com/s/pj7db1izytkj3zj/8_10_12bitslossless_samples.zip?dl=0
Have you noticed the ISO suggestions when selecting a lower bit depth from menu? (at the bottom of screen)
Please refer to this commit (https://bitbucket.org/hudson/magic-lantern/commits/2028d73f6f34dcdaaa4c241824e538d0e0e47267#Lmodules/mlv_lite/mlv_lite.cF2996T3033) for more details, and to this article (https://theory.uchicago.edu/%7Eejm/pix/20d/tests/noise/noise-p3.html) for some theory regarding bit depth, noise and posterization.
I've considered acceptable a quantization step of twice the noise stdev, which would be just a bit better than "gradient 3-bit tonality" from Fig. 3 in the linked article (log2(256/24) = 3.3), which gives the lowest ISO usable at each bit depth. For the highest ISO, I've considered "wasteful" a quantization step smaller than noise stdev / 2.5.
Quote from: hindra on April 15, 2017, 05:37:19 AM
3520x1322 at 60 fps
Impossible (please check the metadata with mlv_dump -v).
Filled a 32GB card with 3K / 2.35:1 / 23.97fps / 14bit lossless - Completely stable with zero bad frames! (I used cr2hdr v1.2 to extract DNG's on OSX Sierra and it worked perfect)
You guys are amazing!
@a1ex
I saw ISO suggestions.
8 bit digests noise generated by high ISO ? (I read that raw is divided by 64)
Thanks in advance for the enlightenment
and your hard work
Yes, at high ISOs, the lowest bits (trimmed for reducing the bit depth) are mostly noise.
At low ISOs, some of them are useful data (especially in shadows).
The last two bits are mostly noise on current sensors (so you can safely use 12-bit at low ISOs).
The posterization from ML previews is real (it matches what I'm getting when opening the DNG in ufraw, for example), so you can actually preview the loss of shadow detail.
For a visual explanation, don't forget to move the mouse over the images from the linked article (https://theory.uchicago.edu/%7Eejm/pix/20d/tests/noise/noise-p3.html) (see previous post).
Thats incredible and gives more room to film in low light, broadening the camera capabilities.
Knowing how ETTR works, I can understand what you describe
Quote from: a1ex on April 15, 2017, 06:04:02 AM
Have you noticed the ISO suggestions when selecting a lower bit depth from menu? (at the bottom of screen)
Please refer to this commit (https://bitbucket.org/hudson/magic-lantern/commits/2028d73f6f34dcdaaa4c241824e538d0e0e47267#Lmodules/mlv_lite/mlv_lite.cF2996T3033) for more details, and to this article (https://theory.uchicago.edu/%7Eejm/pix/20d/tests/noise/noise-p3.html) for some theory regarding bit depth, noise and posterization.
Impossible (please check the metadata with mlv_dump -v).
Yes, I did notice the ISO suggestions. I have no idea why I thought I was shooting at 60fps. I re-rendered at 30 fps, I also cant hit 60fps when attempting to re-create. but, 8 bit lossless works! :) I was trying to use the lowest bitrate to obtain the highest fps and I thought it was 60, I was mistaken. Thank you for those links.
Is there any chance MLVFS will start supporting these files without using mlv_dump?
What would be the reason the DNGs from the April 14th build would not open in After Effects? I'm getting can not parse errors. Worked fine with April 10th. Using Footage.app on mac. Shot 14bit lossless.
Quote from: giarcpnw on April 15, 2017, 10:17:26 AM
What would be the reason the DNGs from the April 14th build would not open in After Effects? I'm getting can not parse errors. Worked fine with April 10th. Using Footage.app on mac. Shot 14bit lossless.
I think I answered my own question. The first frame of the sequence in all the MLVs I shot tonight were black (I assume corrupted) I removed them from the sequences and then AE and Camera Raw could load the sequence.
Something to note for this build. April 14th.
Small MLV and steps to reproduce, please.
Quote from: a1ex on April 15, 2017, 06:04:02 AM
Have you noticed the ISO suggestions when selecting a lower bit depth from menu? (at the bottom of screen)
Of course i noticed these suggestions, now i read your article i'm able to understand better.
I will do differents tests in low light condition with different range of ISO. eg : 12800 ISO to 800.
So, if i understand well, in this way, the noise is our friend if we need less posterization because the 8bits depth ? So it means that if we film a scene very illuminated outside with the 8bits , we have all interest to use a variable filter ND.
Quote from: GutterPump on April 15, 2017, 11:47:57 AM
So, if i understand well, in this way, the noise is our friend if we need less posterization because the 8bits depth ?
Correct.
To be effective for reducing posterization, the noise must be applied before the quantization (therefore, analog ISO noise is fine). Adding it in post will not help.
Quote
So it means that if we film a scene very illuminated outside with the 8bits , we have all interest to use a variable filter ND.
I disagree here. Lower bit depths were added to reduce the data rate at higher ISOs (where the lowest bits are mostly noise, and because of them, the image doesn't compress well).
I don't think filming with 8 bits is actually your goal. More likely, I assume you actually want to minimize the data rate. With uncompressed data, it's easy (all frames have the same size: W x H x bpp/8). With lossless compression, you can't choose the compression ratio in advance (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182229#msg182229); this is given - roughly - by the entropy of the image.
These "fake bit depth" settings (actually dividing the signal by a power of 2) are one way to reduce the entropy (by removing the least significant bits, because of integer division). That's a
lossy operation (going from 14 bits to any lower value).
After this step, compression is lossless (that is, the uncompressed data will be identical to what was fed to the compressor).
(That means, 14-bit lossless will give raw data
identical to uncompressed 14-bit (100% identical, not one single bit different, contrary to some earlier report (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182741#msg182741)), while "12-bit lossless" should be interpreted as "14-bit to 12-bit conversion - lossy by definition - followed by lossless compression", so it should have the same number of useful levels as uncompressed 12-bit. Please note the 12-bit lossless is
not identical to 12-bit uncompressed - they differ by a constant value and possibly by some round-off error, and the same is true for lower bit depths.)
Back to entropy - increasing the ISO and underexposing (to keep the image brightness unchanged) will increase noise in the image (at all brightness levels). So, what you may gain by reducing the bit depth, you will lose from the added noise. Here's a quick test (static outdoor scene exposed to the right, plain 1080p, compression rates taken from ML menu):
Bits per pixel 14 12 11 10 9 8
ISO 100 1/100 61% 53% 50% 48% 46% 43%
ISO 200 1/200 62% 54% 51% 49% 47% 44%
ISO 400 1/400 63% 54% 51% 49% 47% 45%
ISO 800 1/800 65% 55% 52% 50% 48% 46%
ISO 1600 1/1600 67% 56% 53% 50% 48% 46%
ISO 3200 1/3200 70% 57% 53% 50% 49% 47%
ISO 6400 1/6250 76% 60% 55% 52% 50% 48%
ISO 12800 1/12500 79% 63% 57% 53% 50% 49%
=>
there's nothing to gain by using a lower bit depth at a higher ISO + underexposure.
Of course, if your goal is to record at 8 bits, no matter what, then of course, the best thing to do is to increase ISO at 12800 (or at least 6400) for half-decent results. It's also trivial to change the code to get 6 bits, 4 bits or even one single bit, if that's your goal ;)
My advice:
first expose properly (ETTR, zebras, raw histogram, whatever you prefer), and
afterwards choose the optimal bit depth. Not the other way.
Actually that post about compression needed to be in two parts
a) Why do I get these boxy big pixels compression-like in out of focus areas
b) Is 14bit and 14bit lossless visually identical
The question b) is answered already.
The question a), I guess was due to a different reason other than the bit depth
But I understand a1ex words about bit depth. Great explanation
Hi Guys,
I explain how work and how I setup my 5D for create 3,5K video, here :
https://www.youtube.com/watch?v=kPQMNHhV41M
:o
Now with these fake bit depth options, this is getting interesting for the SD card camera's :D
@Alex,
The red numbers in your bit depth / compression ratio table, do you mean that the serious loss of info/shadow detail is where the red numbers are ?
And if I understand correct, you're using digital gain, which cut's of the noisy bit, but the image technically stays 14 bit for the compression algorithm, but because of the noisy bit cut off, the compression rate goes even beyond the 50%.
Correct - the red numbers match my ISO recommendations (https://bitbucket.org/hudson/magic-lantern/commits/2028d73f6f34dcdaaa4c241824e538d0e0e47267#Lmodules/mlv_lite/mlv_lite.cF2996T3033) and are below "gradient 3-bit tonality" from the third figure (labeled Fig. 18) from the noise article (https://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html). Also added orange = wasteful (there's little or no additional image quality to gain at these settings).
If the theory is correct (which I didn't have the patience to check by pixel peeping), then it's probably best to stay near the transition line to red, on the black side (that is, 11-bit for ISO 100-800, and reduce the bit depth once you need ISO 1600 and beyond).
I believe the recommended settings are visually identical to regular 14-bit (while in 14-bit lossless, the raw data is 100% identical to uncompressed 14-bit). Again, this is not confirmed by pixel-peeping (it's just noise theory). There may be a small color cast in shadows, because the 0.5 LSB offset is not yet applied (but it's easy to fix once @martinhering helps us decode the JPEG stream (http://www.magiclantern.fm/forum/index.php?topic=18949.msg183230#msg183230)).
The estimations were done for 3x3 binning (plain 1080p); the 1:1 crop mode may require slightly different values.
At these (probably optimal) settings, the compression rate appears to be consistent (50%), at least on my test scene (it was some foliage exposed to the right, but didn't save it).
(my definition for compression rate is: saved frame size / size of a 14-bit uncompressed frame)
Quote from: a1ex on April 15, 2017, 11:43:21 AM
Small MLV and steps to reproduce, please.
Well, i could not duplicate it. I updated Martin's footage app this morning and they process fine. I do however remember it happened right after i tried to capture 12bit and 8 bit lossless and saw this on my screen.
This happens whenever I try to capture any lossless bit rate than other than 14bit lossless. Trying to record like this results in a ton of scrolling errors. (none of which I know what they mean) Have to power off to stop. Pressing x5 crop WILL ALLOW capture however in this mode. 3072x1308, FPS Override 23.976
(http://i65.tinypic.com/bhyh5j.jpg)
(http://i64.tinypic.com/2n7qul5.jpg)
@giarcpnw: already reported, and for the moment I don't know how to solve it. Will probably disable it in crop modes with modified resolution, until I'll understand what happens.
@Quentin: I don't see "boxy big pixels compression-like in out of focus areas" in your post - I only see black text on a blueish background...
Unfortunately I erased the videos.
But ... how do we attach an image on the post ?
Sorry for my ignorance ?
1080p48 at 12 and 11 bit lossless at 100 iso I get results with a split exposure. Global draw off
(http://i.imgur.com/dNVbq9P.jpg?1) (http://imgur.com/dNVbq9P)
Quote from: Quentin on April 15, 2017, 08:19:26 PM
Unfortunately I erased the videos.
But ... how do we attach an image on the post ?
Sorry for my ignorance ?
I upload to imgur.com and copy/paste the BBCcode link in the forum post.
Testing last build april14th.
Tearing issue in 1920x96050p3x3 is gone (I understand because modification of buffering). It is almost perfect for me. I'm really grateful, thanks A1ex!!!... It was a dream some time ago, and now...
Thank you very very much...
Now the only downside is recording time with hi isos in 14bit lossless and incorrect framing, but I can live with that.
Thanks again!!!!!!!
I successfully downloaded Apr14 build, but it did not do a ROM Test at start, not that I could see atleast. Do I need to download Apr10 specifically for the ROM test ?
Anyways, with this build I have correct/centered framing at 3072x1728 14Bit Lossless 16:9. Very nice! Will actually start using this thing now.
Quote from: Kharak on April 15, 2017, 11:00:41 PM
I successfully downloaded Apr14 build, but it did not do a ROM Test at start, not that I could see atleast. Do I need to download Apr10 specifically for the ROM test ?
Have you tried running firmware update again from Canon menu and restart camera?
If no messages prompts up afterwards then you haven't been affected from that April 4th null pointer bug.
Hey
Great job for the low bit depth it opens great perspective to play around the 3.5k.
Here is my bug encouters for low bit depth 8,9,10,11:
- the expo is darken in the realtime preview and hdmi out. realtimre preview for 8 bit totally unusable (this is bas for focusing) framing mode is working better but focus is still not possible from there.
- 1:1 Tall have the lower of the image not displayed correctly like mentionned for slow mode
- non working on 3K mode and higher
Can we expect a mlv sound coming back? post synchro is so burden with just the beep :P
Quote from: DeafEyeJedi on April 16, 2017, 04:57:43 AM
Have you tried running firmware update again from Canon menu and restart camera?
If no messages prompts up afterwards then you haven't been affected from that April 4th null pointer bug.
Thank you, Deafeye. I did as you said, update from Canon and restart. But could not see any test being done. Thought I saw a screenshot from someone doing the test and there was like this 0-100 Counter, maybe I am mixing it up with something else.
Its just a pain for me downloading builds cause I am on Satellite internet, I get 4-5 kb/s on a good day and then there are constant dropouts which which require me to start the download all over again... Just loading this forum page takes forever, and its only text more or less..
So its safe to assume that I need to download apr10 build because it has the Rom Test ?
https://bitbucket.org/hudson/magic-lantern/src/crop_rec_4k/src/debug.c?fileviewer=file-view-default#debug.c-288
If no errors are present, no obvious feedback is given (other than an extra LED blink, which is pretty obvious to me).
Quote from: Kharak on April 16, 2017, 03:31:17 PM
I get 4-5 kb/s on a good day and then there are constant dropouts which which require me to start the download all over again... Just loading this forum page takes forever, and its only text more or less..
Here's what helped me on a similar connection, besides obvious things such as disabling images or using a text-mode browser (mobile connection with poor signal, far away from the city, 4K/s at best):
http://lartc.org/wondershaper/
There are probably equivalents for other operating systems. The idea is to reduce download speed so you can load other pages (or use a chat client) while a long download is running.
Having a remote server accessible with VNC, with a very low image quality setting, can be also helpful (as loading a JPEG preview of a webpage can often have much lower bandwidth than its HTML+CSS+JS+images).
For a download with frequent dropouts, just leave a "wget" command open in background (as it doesn't give up easily, unlike most browsers).
There seems to be lots of interest getting crop_rec on steroids working on other cameras.
The EOSM can already do basic 3x3 720p crop_rec and I got it working a while ago on the 700D but was trying to get the other options available on the 5D3 working before doing a pull request. I haven't gotten very far but since other users are also experimenting with adtg_gui, dm-spy and other ML tools to get 4K working on their cameras, maybe we should learn to crawl before attempting to fly so I posted a pull request for limited support of the crop_rec module for the 700D.
https://bitbucket.org/hudson/magic-lantern/pull-requests/824/crop_rec-for-700d/diff
Uploaded a new test build in my bitbucket download area.
https://bitbucket.org/daniel_fort/magic-lantern/downloads/
Tested the new software update with my Canon 5D Mark III and I used this video mode: 1920x960 @ 50p (both 1:1 crop and full-frame - 3x3 pixel binning) - continuous*)
but al converted (with MLV Mystic) DNG"s are corrupt (pink pixels)
Check my MLV file here: https://we.tl/lvASYcNcRx
Some help please? :)
On the 700D, I finally found the reg needed to fix the darkened lower half of the image when trying for higher resolution than 2048x1152.
It was ADTG2[82B6].
Best fps I can get without a freeze at this res is 10.639fps.
Don't mind the crappy image quality, room was dark and was to lazy to turn on the light for testing, LOL, was shooting 3200ISO.
(https://s4.postimg.cc/xkgbmbgz1/4kfix.jpg)
Sorry for the double post!!! But I Just tested 4-14 build 12 bit 3072 2.35:1 at 23.976 ISO 3200 with the old 5x zoom method and it seemed like it was endless. Over 20 secs for sure!!!! DNGS are clean and processed fine and all worked in resolve.
So the this will work for sure!!!! Just need to get the crop rec working. Way easier said then done no doubt. this is crazy tho. 3k 12bit lossless 24p on a 2012 5D
@hjfilmspeed: already reported (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183180#msg183180) and answered (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183243#msg183243) a few (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183246#msg183246) times...
Framing preview works fine and shows what you are going to get.
Ahh sorry @a1ex missed that. But the concept is there. This is really awesome!
Great work...but how to convert the crop mode *.mlv file. I use MLVrawviewer but it crashes.
Any Tips or ideas?
Thnxs.
Can anyone post their advanced settings on the crop_rec on 5D3123 so that I can get the crop from the center? Currently it looks the crop start from the extreme right side of the sensor (3K).
Thanks for all your hero members for the advancement.
Admiration for many reasons
For revolutionary development of out of this world features , for so many cameras
Baby sitting the forums with so many users of so many experiences and variation of needs and knowledge
I surrender, I could not do that
Like in french : RESPECT !!!!
Quote from: zeus12 on April 17, 2017, 04:50:49 PM
Can anyone post their advanced settings on the crop_rec on 5D3123 so that I can get the crop from the center? Currently it looks the crop start from the extreme right side of the sensor (3K).
Thanks for all your hero members for the advancement.
Don't use 5x mode like before. If your in regular live view you will get proper framing. It's a bit faster than before also! If you go to 5x mode the framing is off. Took me a bit to figure this out. HDMI preview isn't working yet in this mode but it's amazing none the less. I'm standing by until it does so I can use the 5D on my Movi again.
@quentin
+1
Thank you!!!!!
Can ANYONE point where is a precise step by step instruction of using mlv_dump?
Yes, I know - noob! how da hell ... and so
But I'm using ML almost since it's first build, but thankfully chmee wrote his tool very soon...
I was always using RAW2CDNG, but now I have to deal with this rocket science - command line stuff ((
And there's no clear and simple TUTORIAL here on forum or youtube video tutorial. Everything I've found is either outdated or people being sarcastic (not helping) and in most cases for MAC (why mlv_dump discussions are always mac oriented?)
A1ex is so helping and responsive. I don't get it why didn't he created a normal software with UI ...or clear and simple tutorial or list of commands for us to just copy/paste?
Why not implement at least base and commonly used commands into mlv_dump somehow? Or enable drag'n'drop functionality? anything simple
:-\
who can help - what should I do with mlv_dump on PC to simply convert MLV to DNG?
Hey guys.. i'm kinda new to ML.. i've loaded this experimental build and started trying to record in UHD. I've loaded the Rec_crop module and tried the UHD setting. Now the 1080p/50fps setting works sortof(it's not actually "slow motion" since windows still recognizes it as 29fps) the 3K/UHD/4K setting won't work. There's a weird purple/pink artifact going on when i try to film.
Would that mean i have to record in RAW? If so, what are the appropiate modules i have to activate to make it work?
Thanks jn advance!
Is anyone getting continuous 3K raw in 4:3 aspect ratio (for 2x anamorphic shooting)? If somebody can confirm I'll pull the trigger on a used 5D3 body!
Quote from: child01 on April 18, 2017, 01:55:50 PM
A1ex is so helping and responsive. I don't get it why didn't he created a normal software with UI ...or clear and simple tutorial or list of commands for us to just copy/paste?
Why not implement at least base and commonly used commands into mlv_dump somehow? Or enable drag'n'drop functionality? anything simple
:-\
pay him 4k€ a month and he will probably do this additional work on saturday and sunday night - the only time when he usually sleeps.
ah and i forgot to mention that i implemented mlv_dump as reference implementation, tool for ML devs, users on nerd level 100 and fallback solution.
*if* there is a proper GUI frontend - go for it, else mlv_dump has to do the job :)
My opinion: Get rid of those experimental builds. It invites users not ready for experiments on this level.
Those wanting to go there may
a) have an environment to make their own builds
or
b) rely on a helping hand building it for them on demand.
Quote from: Walter Schulz on April 18, 2017, 08:15:47 PM
My opinion: Get rid of those experimental builds. It invites users not ready for experiments on this level.
Those wanting to go there may
a) have an environment to make their own builds
or
b) rely on a helping hand building it for them on demand.
Hmmm I not sure I agree when it comes to experimental builds. Although I guess I don't quite get what you mean by "rely on a helping hand building it for them on demand."
On one hand, the admins hold our hands and babysit us while testing. Time consuming to say the least.
On the other hand, some of us who have been using ML for years get a chance to learn about the inner workings of ML and learn how to become better testers.
Also, when it comes to working out the bugs, us inexperience ones are the best at breaking things because we don't listen.
However I do feel that people should not get upset or wine or cry or point fingers when they can't figure something out. I think people need to be more aware that this is all volunteer work. No pay for hours of endless coding. One should not complain especially when one has no clue what is involved with coding.
But that is my humble "I don't even know what I'm talking about but should since I have been using ML for so long and I am being grossly off topic and I should know better about that too run on quote" opinion. An opinion to which I feel I don't even deserve to give. Compared to Walter Schulz I am a dumb dumb head.
Not agree, experimentals builds help to understand better the universe of Magic Lantern, it's a big help for the community, the developpement is more fast, Alpha testers are more large.
Do not delete them, the simple warning message in the top of page is enough. People have been warned from the beginning with the risks of Magic Lantern.
Experimental builds are the soul of magic lantern right now, it allows people to test lit the latest and give very valuable feedback, @walter if someone dont like them, or are scared it may harm they are free not to use them, but i for one love them.
The experimental version of magic lantern is on the cutting edge, frustratingly cutting edge. It's part of the struggle for the code-blind mortals who's pulling their hairs out and ask the stupidest questions on the board. The struggles make magic lantern exhilarating, and inspires participants to make better picture, better videos. Most of us do not post anything on the board just for the fear of sounding really really..... stupid. However, we do really appreciate the "geniuses" and generosity of those immortals such as Alex. Please bear with us.
By the way, I finally get UHD to work as well as 1080p @45fps. The DNGs were generated with Rawflow (page 13 of this board). Somehow, the first frame is always bad and the rest are splendid.
Thanks for all the immortals on the board and hope you can improve magic lantern further and let the mortals to try the experimental versions. The 'code blind" mortals can't contribute much to the development of magic lantern. But we will do more volunteers to places such as "soup kitchen" to feed the homeless as our appreciation.
Quote from: Walter Schulz on April 18, 2017, 08:15:47 PM
My opinion: Get rid of those experimental builds. It invites users not ready for experiments on this level.
I was posting test builds for the 12-bit (and 10-bit) RAW video development discussion (http://www.magiclantern.fm/forum/index.php?topic=5601.0) before the Experiments (https://builds.magiclantern.fm/experiments.html) page went live so I feel partially responsible for rolling that snowball.
It would be great if more users would set up development environments or at least take a good look at the code. Even if you can't compile or understand the code, there are some great comments in there that explains what the developers are doing and perhaps more importantly, what areas still need some more testing. For example, in the crop_rec.c file of the crop_rec_4k branch (to stay on topic) you'll find this:
/* max resolution for each video mode (trial and error) */
/* it's usually possible to push the numbers a few pixels further,
* at the risk of corrupted frames */
static int max_resolutions[NUM_CROP_PRESETS][5] = {
/* 24p 25p 30p 50p 60p */
[CROP_PRESET_3X_TALL] = { 1920, 1728, 1536, 960, 800 },
[CROP_PRESET_3x3_1X] = { 1290, 1290, 1290, 960, 800 },
[CROP_PRESET_3x3_1X_48p] = { 1290, 1290, 1290, 1080, 1080 }, /* 1080p45/48 */
[CROP_PRESET_3K] = { 1920, 1728, 1504, 760, 680 },
[CROP_PRESET_UHD] = { 1536, 1472, 1120, 640, 540 },
[CROP_PRESET_4K_HFPS] = { 2560, 2560, 2500, 1440, 1200 },
[CROP_PRESET_FULLRES_LV] = { 3870, 3870, 3870, 3870, 3870 },
};So instead of simply reporting corrupt frames, indicate the resolution where you start getting corrupt frames and how far you had to back off to get consistently clean results. If you can compile, try pushing it a little further and report how far you can go before it breaks.
Quote from: dfort on April 19, 2017, 09:01:18 AM
So instead of simply reporting corrupt frames, indicate the resolution where you start getting corrupt frames and how far you had to back off to get consistently clean results. If you can compile, try pushing it a little further and report how far you can go before it breaks.
For this particular snippet, vertical resolution is actually adjustable from the crop_rec submenu (no need to compile). I don't remember anyone trying to change it, even though I've mentioned (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182320#msg182320) it a (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183060#msg183060) few (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183112#msg183112) times (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183115#msg183115) (including first post (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182052#msg182052)).
@dfort
I agree with your 'set up a development environment', and, as you know, I wish to do this.
However, and this is a big however, not all of us are competent to do that without very clear and simple instructions.
I have tried to set up such an environment, and although the instructions I was following were good, they still assumed some knowledge I don't have. I kept getting error messages and gave up :-(
From my perspective it would be good to reset/start an ML page specifically directed at getting your own compile environment up and running, and keep Win and Mac instructions separate. Also this page should be written from a nil knowledge perspective, i.e. for 'idiots' like me.
Of course, I appreciate I'm asking a lot, i.e. someone who knows what they are doing sitting down and writing that page.
But until I can follow a simple workflow for setting up a compile environment, I flummoxed.
Cheers
Garry
Just start out with what´s not working in one of the compiling threads and take it from there. It´s impossible to account for every single thing that comes up when compiling although dfort´s efforts made it much easier, especially for mac users.
I´m pretty confident the solution won´t be far when out in the open.
Here are two threads if anyone is interested. There are more efforts and threads out there.
Mac
http://www.magiclantern.fm/forum/index.php?topic=16012.0
Windows
http://www.magiclantern.fm/forum/index.php?topic=15894.0
I followed Mac instructions and have my own machine running for some time. Had to build a Hackintosh for that and I'm a Windows guy.
Didn't test Windows workflow, yet.
Having involved into beta testing of 3d software in the past, in my humble experience, its good to keep an order in information coming, sort them, classify them, confirm symptoms etc. Its very hard
Quote from: a1ex on April 19, 2017, 09:23:24 AM
For this particular snippet, vertical resolution is actually adjustable from the crop_rec submenu (no need to compile). I don't remember anyone trying to change it, even though I've mentioned (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182320#msg182320) it a (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183060#msg183060) few (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183112#msg183112) times (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183115#msg183115) (including first post (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182052#msg182052)).
@a1ex I have to say, I was super nervous about changing those settings since I didn't do enough research on what they mean but Ill see if I can figure it out. Has anyone one tried tweaking those? What did you find?
Simple example: if you often get corrupted frames in 1080p48, dial 1040 (or some other value lower than 1080) in the Target YRES field. Then refresh the LiveView manually (e.g. press MENU twice) and the resolution in RAW video menu should update.
Set 0 to disable the override and go back to default value.
Quote from: a1ex on April 19, 2017, 04:58:42 PM
Simple example: if you often get corrupted frames in 1080p48, dial 1040 (or some other value lower than 1080) in the Target YRES field. Then refresh the LiveView manually (e.g. press MENU twice) and the resolution in RAW video menu should update.
Set 0 to disable the override and go back to default value.
@a1ex Ohhh cool! Do you have to match the value in the MLV Lite options?
This sets the maximum vertical resolution available for the raw recorder (which records a cropped section from the full-size raw buffer).
It takes 5 minutes to dial proper number.
I would prefer a number to subtract rather than setting absolute vertical resolution.
Its faster and practical.
I'd actually like to try and raise the vertical resolution and shorten the width to try and get the closest to 16:9 at highest resolution possible.
@Quentin: have you tried using SET, left/right and scrollwheel?
The same controls are in many other places in ML menu, whenever you need to input a large number (check e.g. intervalometer).
Thanks a1ex, too many things to remember/learn
Its hard to carry a manual with you :D
Whats the best Card to use?
Quote from: tifose on April 20, 2017, 01:07:33 PM
Whats the best Card to use?
I'm not sure on the best card, but I have had great success with Lexar 32gb and 128gb 1066x CF cards.
same for the fastest sandisk cf 160 mb 128 gb they work great for me.
Thanks guys will get one my CF card stop working in camera
If your card is working in cardreader but not in cam you might have a defective cam.
New build:
- centered x5 zoom is back (3.5K with 8...12-bit lossless compression)
- disabled 8...12-bit lossless in incompatible modes (sorry, still couldn't figure out how to fix them)
- for bit depths between 8 and 11, the choice is now automatic, depending on ISO (see this post (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183231#msg183231))
- fixed silent pics (https://bitbucket.org/hudson/magic-lantern/commits/a9c8ad412af5c282bbad6889ce5f65d974ca3ada) (a bit sad to see them broken for 3 weeks with no bugs reported)
- minor UI fine-tunings.
Quote from: a1ex on April 21, 2017, 11:55:05 PM
- fixed silent pics (https://bitbucket.org/hudson/magic-lantern/commits/a9c8ad412af5c282bbad6889ce5f65d974ca3ada) (a bit sad to see them broken for 3 weeks with no bugs reported)
Sorry I noticed that while trying to get lossless compression working on other cameras but didn't create a proper bug report--though I did point it out. I'm working on the compressed_raw branch but maybe the crop_rec_4k branch has the latest changes? Noted that lossless.c and silent.c aren't in sync (https://bitbucket.org/hudson/magic-lantern/branches/compare/crop_rec_4k%0Dcompressed_raw#chg-modules/silent/lossless.c) between these two branches.
@Walter Schulz
it is working in my Reader and 5Ds
i use to use my 5DMk3 for Raw just didn't use it in over a year not sure why this is happening any idea how this can be fix?
Great work @a1ex you tried your best thank you for all the updates.
@tifose You should try a different card with your camera. Also use a flash light or light source look inside the card bay to see if any of your pins are bent, if not you may have a problem.
I have the null pointer bug warning coming up with April 21 build. Message says to contact ML developers on IRC Magic Lantern freenode. Tried but not sure I was at correct place to do this - Alex can you supply URL please. Unfortunately don't have old logs.
I've been experimenting with the 3K crop and 1080p slow motion, etc from the experimental build. Amazing stuff. It's hard to say, of course, how long it will be before usability is nailed down. Wondering if there might be a shorter term move to combine the "simple" 1080p24/30 lossless recording that's been developed along with the MLV Sound module? I would love to incorporate something like that to my everyday run and gun interviews and b-roll without the occasional random breaks in recording and the extra large files that quickly fill up my cards.
(this was posted in reverse engineering but it was recommended that I delete it and post here)
I just tested and this is my favorite build yet!!!
Scene is well exposed fine detail @ ISO 3200
3072 x 1308 (or what ever 2.35:1) Is working extremely well in 5x zoom 12 bit lossless fps override 24.006 and framing preview. I disable the crop rec mod while recording in 5x (not sure if I should use crop rec and 5x at the same time) converting DNGs as I type.
Edit: Out of about 10,000 frames not one bad frame!!!!!!
@hjfilmspeed: the centered x5 preset from crop_rec is for x5 zoom; all the others are for non-zoomed modes. The help at the bottom should tell you. You can also choose 23.976 for FPS override.
@csound: type IRC in your browser's search box (CTRL-F) ;)
@dfort: yes, I've looked into it because of your report, but it was my explicit request to try the bad commit, not a regular user trying it for e.g. timelapse or whatever. Also, this branch has a lot of stuff that can be broken into smaller branches and back-ported to all other models, in the main builds (already started with this PR (https://bitbucket.org/hudson/magic-lantern/pull-requests/825/prevent-canon-settings-from-being-saved/diff)).
@lostfeliz: I was also thinking to add mlv_sound compatibility to mlv_lite, just didn't look into it yet (since, for my own use, only the full-res and 4K half-FPS modes are interesting - for extracting still frames from a burst sequence, or for timelapse). That's the reason you see weird changes such as single-buffering, ability to record with only 2 buffers, half-shutter trigger etc.
Quote@csound: type IRC in your browser's search box (CTRL-F) ;)
A1ex, apologies for being a newbe clutz - but what do I do when I get to freenode?
Alex sound would be great, and if the manual lens info could be merged into it, that would be nice too.
Quote from: a1ex on April 22, 2017, 08:51:49 AM
@hjfilmspeed: the centered x5 preset from crop_rec is for x5 zoom; all the others are for non-zoomed modes. The help at the bottom should tell you. You can also choose 23.976 for FPS override.
@a1ex OHHHHHH I didn't even see that option! That mode also lets you use 12bit and at 3200 iso im getting what seems to be continuous 3072x1308 24 FPS!!!! And its centered!!!!! and you can use the zoomed real time preview if you don't want to use the framing preview!!! No crashes so far.
The only thing I noticed is the shutter speed in at the bottom didn't seem to be registering the right number. Unless something else is going on here. The cameras shutter speed was 1/50 but when zoomed + fps override it showed 1/27 or 1/37 or something like that.
But wow .... This is impressive!!!! THANK YOU FOR THIS!!!!
This is much more exciting then the C- log announcement.
The shutter speed issue appears only when using the 5x preset (it appears fine without crop_rec); can you confirm?
Thanks Guys it was a bad pin. back up and running :)
Quote from: a1ex on April 22, 2017, 10:00:03 AM
The shutter speed issue appears only when using the 5x preset (it appears fine without crop_rec); can you confirm?
Yup confirmed. Just crop mode 5x preset @a1ex just tested now. 5x without crop mode fine.
I also started seeing these stuck pixels pop up in 1:1 5x liveview real time iso3200 1/50 that wouldn't be there in normal 5x zoom real time. I think its Just in the liveview. I didn't try to see if it would appear in RAW file. Focus box was in was centered in both cases. I feel like this has been covered and I appollowgize if it was.
Thanks a lot for the groundbreaking work !!! Keep going !!!!
[Deleted].
I don't know if it can help
...
Wow FYI everyone I just did a 5div 4k vs 5diii 3k and 5d3 3k is sooooo much sharper and much more cinematic.
Same for 1080 5diii looks a lot better! I can't believe 2012 5diii has better video quality.
I guess the real test will be when 5div gets RAW video.
Very nice !
Restarted everything from new download.
Loaded only the mlv_lite.mo module
Everything seems OK !
@a1ex Tested the latest build it's solid.. Only got one crash when I was using slowmo 45p / 48p, but when I switch to 50p / 60p mode it seems to work fine. What's the difference between those two slowmo modes other than fps because on the 50p / 60p mode I was able to record @ 48p?
@hjfilmspeed Awesome didn't think the 5D III could be sharper than the Mark iv in 3k, and definitely the motion in 3K is very cinematic I posted about that. All I'm gonna say is a1ex got the keys.. keys.. keys.. Can you share a short clip of each camera? Did you zoom in 200% - 400% to pixel peep.
@goldenchild9to5 I'll post my quick test but before I hard conclude this I'll have to give the 5div a more suitable test.
I zoomed in and its really hard to to make a conclusion here but in both cases I gave the 5div the advantage especially with autofocus. First 2 shots are 1080 shots 3 and 4 are the higher resolution.
I tried to grade the RAW fairly.
See if you can tell what's what. I like the RAW better for sure.
If you download it it's a Cineform AVI in UHD
Also keep in mind I can only export UHD from Resolve.
https://vimeo.com/214307580
@hjfilmspeed You are definitely right the 5D III images looks better especially in 3K..
Quote from: goldenchild9to5 on April 22, 2017, 06:13:38 PM
What's the difference between those two slowmo modes [slowmo 45p / 48p and 50p / 60p] other than fps because on the 50p / 60p mode I was able to record @ 48p?
FPS and vertical resolution. The 48p mode is really pushed to the limit (already discussed), while the 45p should have a little more headroom.
Quote from: hjfilmspeed on April 22, 2017, 09:24:43 AM
The cameras shutter speed was 1/50 but when zoomed + fps override it showed 1/27 or 1/37 or something like that.
Solved, it was just an oversight from my side.
Also fine-tuned the preview behavior: you can use a long half-shutter press to change between framing and real-time (e.g. if you normally use the centered real-time preview in x5 zoom, but you also want to check the framing every now and then, just press half-shutter for 1 second or so).
@a1ex Thanks got it, didn't read up about that.
Quote from: a1ex on April 22, 2017, 09:41:08 PM
Solved, it was just an oversight from my side.
Also fine-tuned the preview behavior: you can use a long half-shutter press to change between framing and real-time (e.g. if you normally use the centered real-time preview in x5 zoom, but you also want to check the framing every now and then, just press half-shutter for 1 second or so).
This is incredible! Soooo impressed!
Also I just replaced my test with a much better one
https://vimeo.com/214307580
This means Either Canons 5Div 4K is actually 3K
or
That we get to call 5D3 ML 3K ... 4K
4 Clips
2 are 5div 1/100 f11 ISO 400 Sigma Art 35mm 4k
2 are 5Diii 1/50 f11 ISO 400 Sigma Art 35mm 3k ML
14bit lossless using 4-21-2017 4K Crop Rec build
(I underexposed the 5Div because I was clipping highlights at 1/50 shutter)
Both rendered to Cineform UHD and both were precisely focussed on the crosshatch on the color chart.
Can you tell which is which.
Download and zoom in. It's a UHD Cineform AVI
5Diii was quickly graded to match Canon's neutral picture style of the 5div
Sorry to be a party pooper but can anyone else confirm that this is reproducible (it seems to be on my end w build 2017-04-22 for 5D3.123) if one were to record in 3504x1400 resolution (and only in that resolution which is 2.50:1)?
It seems to produce a pattern of corruption after each 36th DNG frame from 14-bit lossless shot in 24p.
(https://c1.staticflickr.com/5/4183/33820054770_a4d51abbce_n.jpg) (https://flic.kr/p/TwyC1A) (https://c1.staticflickr.com/3/2934/34204130195_ca8604c418_n.jpg) (https://flic.kr/p/U7v7fF)
(https://c1.staticflickr.com/5/4172/33361447354_46a53aea0c.jpg) (https://flic.kr/p/SQ391b)
Here's an original corrupted 36th DNG: https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads/5D3.123_3504x1400_corruption%3F.dng
Settings used in ML:
(https://c1.staticflickr.com/3/2834/33393407053_c0143f495d.jpg) (https://flic.kr/p/SSRWvt)
SO far in other resolutions seems to be getting better results other than the random first DNG's being either black (blank) or half screen corruptions, etc from each certain MLV's. Gotta troubleshoot some more on this to narrow down the culprit. Don't think it's random at all.
@hjfilmspeed Great test.. This one was pretty hard to tell the difference. Is the 5D III scenes 2 & 4?
Quote from: goldenchild9to5 on April 22, 2017, 11:02:01 PM
@hjfilmspeed Great test.. This one was pretty hard to tell the difference. Is the 5D III scenes 2 & 4?
@ goldenchild9to5 did you download and zoom in? Which looks sharper or which do you prefer?
@DeafEyeJedi: does it help if you reduce the vertical resolution from the crop_rec submenu a bit? (e.g. instead of 1536, try 1500, 1450, 1400)
If you can reproduce it consistently without the resolution override, try the following patches
edit: nevermind; had some hypotheses, but rejected all of them.
Reproduced.
I was able to replicate the 19 FPS bug in the menu I reported a while baclk. If I forget to turn FPS Override off, turn on Crop Mode module for UHD recording, enter canon menu then go back into ML menu, when I turn off FPS Override it defaults back to 19 FPS. If FPS Override is off when I turn on Crop Mode, everything is fine.
(http://i.imgur.com/XvzW1N6.jpg)
Quote from: hjfilmspeed on April 22, 2017, 10:20:08 PM
This is incredible! Soooo impressed!
Also I just replaced my test with a much better one
https://vimeo.com/214307580
This means Either Canons 5Div 4K is actually 3K
or
That we get to call 5D3 ML 3K ... 4K
4 Clips
2 are 5div 1/100 f11 ISO 400 Sigma Art 35mm 4k
2 are 5Diii 1/50 f11 ISO 400 Sigma Art 35mm 3k ML
14bit lossless using 4-21-2017 4K Crop Rec build
(I underexposed the 5Div because I was clipping highlights at 1/50 shutter)
Both rendered to Cineform UHD and both were precisely focussed on the crosshatch on the color chart.
Can you tell which is which.
Download and zoom in. It's a UHD Cineform AVI
5Diii was quickly graded to match Canon's neutral picture style of the 5div
1 and 3 are much sharper when viewed at 200%
Quote from: DeafEyeJedi on April 22, 2017, 10:54:27 PM
Sorry to be a party pooper but can anyone else confirm that this is reproducible (it seems to be on my end w build 2017-04-22 for 5D3.123) if one were to record in 3504x1400 resolution (and only in that resolution which is 2.50:1)?
Narrowed down and solved; excellent catch @DeafEyeJedi!
That's how a proper bug report looks like - it contains all the details to allow others to reproduce it. This bug was quite hard to trigger, given all those reports with no corrupted frames.
Also got 4096x3072 working at 12 and 12.5 FPS (24/25 in Canon menu) and did some more fixes for very large resolutions. Even if you only get a few frames, I still find these modes useful with half-shutter trigger and pre-recording (e.g. for taking photos of really unpredictable subjects).
Quote from: a1ex on April 22, 2017, 08:51:49 PM
@dfort: yes, I've looked into it because of your report, but it was my explicit request to try the bad commit, not a regular user trying it for e.g. timelapse or whatever. Also, this branch has a lot of stuff that can be broken into smaller branches and back-ported to all other models, in the main builds (already started with this PR).
Oops, sorry. I bit off more than I could chew when I came across that bug and it was just easier for me to switch to an earlier (smaller) branch. I'd like to get lossless compression working on other cameras because that seems doable while 4k on an EOSM is pie in the sky. Though I wouldn't rule it out considering how quickly ML development has been progressing lately.
Thanks for the wonderful work. I Got some problem when update to the new buuld, don't know how to fix it.
My camera is 5D3, 1.2.3
I have installed the Apr-04 build before, it works, and then update to the newest Apr-23 build, then I got this scene on top.
(http://i.imgur.com/czeNgzz.jpg)
I tried to Uninstall, and format CF card, Install again, disable all modules, but got the same scene. then I tried degrade to the earlier build, the same, But the Apr-04 and Apr-06 build works fine.
Can any body know how to fix this? Thanks a lot.
sorry for beeing an asshole. But did you read the screenshot or any page of this tread, yet? :D
I'm getting frame and block size mismatch errors. I saw a few other people were getting various versions of this error as well but no responses with a possible solution. I know it's related to the 14 bit - lossless format as I don't get the message when I turn lossless off.
Testing April 23th build 5D3 113.
When I change to a 192050p3x3 mode with fps override on, and try to switch fps override to off, camera freezes and I get Error80. Need to pull battery.
@hjfilmspeed Check it out Zoomed in to 200% & 400% Scenes 1 & 3 is a tiny tiny bit sharper than Scenes 2 & 4. But here is the thing the tree behind the color checker seem a bit sharper in Scenes 2 & 4.
@lanhl Follow what it says on the screen. There was a dangerous bug in April 04 build so DON'T go back to it, and the new ones are detecting if that bug has affected your camera. It's Magic Lantern looking out for you ;)
Hi,
I used mlv_dump in mac for creating dngs. but unable to open them in After Effects CC 2017. Photoshop opens files with no problem. but AE gives me this error: "after effects error: the file formay module not parse that file. (45::35)"
@Ali Oliya - is the first DNG in your folder blank or corrupted in any way?
If so, try deleting it and try again which should open up fine in AE. Have had this happened to me in the past.
I was away for some time. Trying latest build.
Way much solid!
The exclusions of many combinations makes things more targeted.
Thank you for the hard work and constructive development
@DeafEyeJedi yes it was corrupted. i deleted it. still doesn't open. but the error message changed to: After Effects error: Photoshop file format error (-1). (45 :: 35)
Hmm would you mind using @Danne's latest cr2hdr.app to spit out original samples (#12 under mlv_dump menu) and upload the zip file for us to check out?
Quote from: goldenchild9to5 on April 23, 2017, 08:24:27 PM
@hjfilmspeed Check it out Zoomed in to 200% & 400% Scenes 1 & 3 is a tiny tiny bit sharper than Scenes 2 & 4. But here is the thing the tree behind the color checker seem a bit sharper in Scenes 2 & 4.
1 and 3 are mark iii
Hmmm Ill double check my lens details to see if f-stop was different. It could be possible that focal planes where not exact. I focused both on the crosshatch with autofocus. but in any event at 400% we should be seeing a HUGE difference in sharpness here. 5dIv has 1000 more pixels. Any way I don't want to go more off topic then I have.
Any dev think if there is any possibility to get some signal through hdmi in the new full res live view mode?... just for framing and focussing, don't mind if res is low (VGA)...
@DeafEyeJedi - i used cr2hdr app from Danne. But it doesnt let me open the MLV file at all! .mlv file is grayed out....(same as rawmagic)
ps. i used crop rec 3k-4k 14 April expermiment build btw. so far the only software that is able to open the .mlv files from this build is MLV_Dump (for mac). (raw2cdng opens but creates pink dngs!)
Greyed out? Could you share a screenrecording or a screenshot? Maybe the LOG.txt file?
Maybe post about it here.
http://www.magiclantern.fm/forum/index.php?topic=15108.100
@Danne - yes, i can not choose the file. it can't be selected:
(https://preview.ibb.co/ebuBmQ/Screen_Shot_2017_04_24_at_5_33_34_PM.png) (https://ibb.co/j0zcRQ)
You don´t choose files you choose the folder containing files. If you press choose in this folder a menu will pop up and you take it from there.
@Danne - Thank you! Sorry, it was the 1st time using this app! done. fixed. now i can open .dngs in AE without problem.
@DeafEyeJedi Thanks for your help
On 12 April 2017 I took several shots.
I was recording with that days latest build, 1920x800 60FPS 14bit lossless with 5dmk3.
On some of the shots I took, on the top of the frame there is a horizontal sharp edge that flickers, about 100 pixels away from top
Here is a link
https://drive.google.com/open?id=0B-d8ARtc7xwWT3BwbkdQYjdaLUE
(I am not aware if this is fixed already)
Quote from: a1ex
Narrowed down and solved; excellent catch @DeafEyeJedi!
That's how a proper bug report looks like - it contains all the details to allow others to reproduce it. This bug was quite hard to trigger, given all those reports with no corrupted frames.
Great to hear. Thanks for the acknowledgement!
Quote from: a1ex
Also got 4096x3072 working at 12 and 12.5 FPS (24/25 in Canon menu) and did some more fixes for very large resolutions. Even if you only get a few frames, I still find these modes useful with half-shutter trigger and pre-recording (e.g. for taking photos of really unpredictable subjects).
Oh absolutely this will be useful. Especially for birding, sports photography & whatnot. This is incredible and should be a bit more
forgiving than Canon's stock full burst mode which is only 6 frames per sec for 5D3. :P
Quote from: hjfilmspeed on April 24, 2017, 11:34:55 AM
1 and 3 are mark iii
Hmmm Ill double check my lens details to see if f-stop was different. It could be possible that focal planes where not exact. I focused both on the crosshatch with autofocus. but in any event at 400% we should be seeing a HUGE difference in sharpness here. 5dIv has 1000 more pixels. Any way I don't want to go more off topic then I have.
For those who havant seen them up-close w comparisons yet in between 200% & 400% (Thanks @hjfilmspeed for sharing them!):
#1) 5D3 RAW 3K 1.85:1 (upscaled to 4K in post)(https://c1.staticflickr.com/3/2827/34244929975_b78ee13b4d.jpg) (https://flic.kr/p/Ub7dBa) (https://c1.staticflickr.com/3/2868/34244928415_5c1066c961.jpg) (https://flic.kr/p/Ub7d9g)
#2) 5D4 MJPEG 4K (https://c1.staticflickr.com/5/4169/34114706071_ae086da248.jpg) (https://flic.kr/p/TYAMzr) (https://c1.staticflickr.com/5/4178/34244928795_8f2fa0d461.jpg) (https://flic.kr/p/Ub7dfP)
#3) 5D3 RAW 3K 1.85:1 (upscaled to 4K in post)(https://c1.staticflickr.com/3/2806/33403755734_c358e665fa.jpg) (https://flic.kr/p/STLYP1) (https://c1.staticflickr.com/3/2857/33403754414_df65c4273c.jpg) (https://flic.kr/p/STLYqf)
#4) 5D4 MJPEG 4K(https://c1.staticflickr.com/3/2876/34244931825_d2e4cb6740.jpg) (https://flic.kr/p/Ub7ea4) (https://c1.staticflickr.com/5/4166/33403754644_21c9204576.jpg) (https://flic.kr/p/STLYud)
@DeafEyeJedi the 5diii was actually 3k 3072 1.85:1 but technically upscaled at export.
Thanks @hjfilmspeed -- fixed! (It's even more flabbergasting to know that the 5D3 still looks sharper even when upscaled to 4K) :o
Quote from: DeafEyeJedi on April 24, 2017, 08:12:14 PM
Thanks @hjfilmspeed -- fixed! (It's even more flabbergasting to know that the 5D3 still looks sharper even when upscaled to 4K) :o
DeafEyeJedi Thank you for posting those crops!
Actually, you might as well call the the 5DIII 3K - 4K. Why not canon apparently seems to think it is okay. I thought to myself, maybe its the lens.. then I was like no way, the Sigma Art prime is design to resolve way high resolution.
It could be the 422 vs the .... 444 or what ever MLV 14Bit lossless is.
Regardless, this build gets better every time. Can't wait to see what the future holds!
Oh I meant to ask this, should we be using 123 vs 113 for 5d3? It was my understanding that 113 was fine unless you need hdmi features and AF @ f8. Is this still true?
Or should we be testing 123?
Hey all, how are you guys addressing the Vertical Lines or other pixel issues with these new builds? I'm on OSX. Martin Hering had created a great mlv to cng converter app called Footage that addressed them but he seems to be letting it time bomb in the 15th of May to pursue a revenue generating path. So now we're back to square one.
MLVDump does not fix them. Is there another app that i'm not aware of that does?
Thanks,
Craig
Quote from: hjfilmspeed on April 24, 2017, 08:31:48 PM
...Oh I meant to ask this, should we be using 123 vs 113 for 5d3? It was my understanding that 113 was fine unless you need hdmi features and AF @ f8. Is this still true?
Or should we be testing 123?
AFAIK 113/123 should be identical ever since @a1ex merged these 'two firmwares under one branch' though correct me if I'm wrong since this is experimental.
Quote from: giarcpnw on April 24, 2017, 08:33:59 PM
...he seems to be letting it time bomb in the 15th of May to pursue a revenue generating path. So now we're back to square one.
That's some hogshit, eh?
Quote from: giarcpnw on April 24, 2017, 08:33:59 PM
MLVDump does not fix them. Is there another app that i'm not aware of that does?
Care to share some original short samples @giarcpnw? (use @Danne's
cr2hdr.app (https://www.magiclantern.fm/forum/index.php?topic=15108.0cr2hdr.app) if you haven't already)
@DeafEyeJedi thank you!!!
Also has anyone tried these settings? 4/23/2017 build
Crop Rec with 5x zoom 3.5k enabled
And 12bit lossless with 3072 2.35:1 ?
It's very stable to me! Continuous with a good card!
And you can hit the DOF preview button or half shutter to zoom in with real time to focus!
I saw this mentioned but didn't try it yet. This is awesome!
I feel like real time framing preview of 3k is so close now!
(Coding wise it's probably not)
Care to share some original short samples @giarcpnw? (use @Danne's cr2hdr.app if you haven't already)
cr2hdr didn't seem to fix when i tried last week. unless it's been upgraded since.
The are DNGs from MLVdump, you can see the banding/lines in the mids. They get worse once you adjust exposure and sharpness. These were shot 14bit lossless with the April 10th build I believe. My understanding is this is inherent in the raw data and the converters are coded to adjust for it. I know there was an early post in this thread that said most of the apps or command line converters had not been adjusted to address this with the new 14bit 4k build's mlvs. Just wondering if they had yet or I missed one. Martin added it to his at my request last week, but...
https://www.dropbox.com/sh/a4158kqm85rvime/AADYDd6H8PyjYyGE1fXqKpeza?dl=0
Thanks,
C
Actually could you please just use cr2hdr.app (press 'm' to get in mlv_dump menu then press '12' to start sample processing). Need the originals.
Be sure to use a temporary Folder and include MLV inside since the app can only process from within Folders and not directly on the MLV file itself.
Also try to use the latest experimental builds whenever possible since it's always evolving for greatness.
Quote from: DeafEyeJedi on April 24, 2017, 09:12:03 PM
Actually could you please just use cr2hdr.app (press 'm' to get in mlv_dump menu then press '12' to start sample processing). Need the originals.
Be sure to use a temporary Folder and include MLV inside since the app can only process from within Folders and not directly on the MLV file itself.
Also try to use the latest experimental builds whenever possible since it's always evolving for greatness.
Same folder: you'll see the zip pack.
https://www.dropbox.com/sh/a4158kqm85rvime/AADYDd6H8PyjYyGE1fXqKpeza?dl=0
I have the newest April 23rd build. I just haven't shot any new footage since the April 19th. It was there in the 19th too. These are the MLVs I have with me at the moment.
Thanks,
C
Quote from: hjfilmspeed on April 24, 2017, 08:31:48 PM
Oh I meant to ask this, should we be using 123 vs 113 for 5d3? It was my understanding that 113 was fine unless you need hdmi features and AF @ f8. Is this still true?
Correct.
I'm currently using 1.2.3, but that's because it has more quirks (usually, if something works on 1.2.3, it's likely OK on 1.1.3, but in the other direction, there are a lot of exceptions...)
The notes from 1.1.3 thread (http://www.magiclantern.fm/forum/index.php?topic=14704) are still valid.
Quote from: DeafEyeJedi on April 24, 2017, 08:44:32 PM
AFAIK 113/123 should be identical ever since @a1ex merged these 'two firmwares under one branch' though
That means both ML versions are compiled from the same codebase (but each one is still tweaked to account for Canon firmware quirks - just like all other ML ports).
Quote from: giarcpnw on April 24, 2017, 08:33:59 PM
MLVDump does not fix them [vertical lines]. Is there another app that i'm not aware of that does?
I've explained here (http://www.magiclantern.fm/forum/index.php?topic=18443.msg183653#msg183653) what needs to be done for other converters (including mlv_dump). I'm not really familiar with compressed formats, and there are people in the ML community who can do this faster than me (hint, hint).
Quote from: hjfilmspeed on April 24, 2017, 09:02:52 PM
Also has anyone tried these settings? 4/23/2017 build
Crop Rec with 5x zoom 3.5k enabled
And 12bit lossless with 3072 2.35:1 ?
Confirmed after recording both takes for over 9 minutes each now @ ISO 3200 in 24p. Green icon all the way. Though it felt a bit toasty @ 53-54C towards the end, ha! :P
QuoteI've explained here (http://www.magiclantern.fm/forum/index.php?topic=18443.msg183653#msg183653) what needs to be done for other converters (including mlv_dump). I'm not really familiar with compressed formats, and there are people in the ML community who can do this faster than me (hint, hint).
If only i had studied computer science in college instead of theater, I might be able to fix this myself. Instead, I'll have to recite shakespeare to myself while I wait for someone else with programming knowledge.
C
@hjfilmspeed 5D III is really a force to be reckon with, can't wait for liveview to have a smooth feed @DeafEyeJedi great screen capture..
Can someone please point me to information on procedure to fix null pointer bug from April 4 build. Screen alert on camera indicates to prepare logs and go to freenode on IRC - what happens from there? Sorry, unfamiliar with all this.
Just tried the new Davinci Resolve 14 Beta with 5D III 3K footage man this new update is smooth. I've noticed as well that It might be processing dng's better, seems to resolve images detail a lot better. If anybody else used it let me know if you are experiencing the same thing.
@csound Contact a1ex, he will guide you.
Quote from: goldenchild9to5Just tried the new Davinci Resolve 14 Beta with 5D III 3K footage man this new update is smooth. I've noticed as well that It might be processing dng's better, seems to resolve images detail a lot better. If anybody else used it let me know if you are experiencing the same thing.
Same experience here - realtime playback on my 3k 2.39:1 14bit CDNG files, up to a few nodes. Extremely pleased with the new Resolve so far.
@Oedipax I'm glad that you are getting the same results so I'm not the only one. @a1ex new build is awesome thank you will be testing more tonight.
QuoteQuote from: a1ex
Also got 4096x3072 working at 12 and 12.5 FPS (24/25 in Canon menu) and did some more fixes for very large resolutions. Even if you only get a few frames, I still find these modes useful with half-shutter trigger and pre-recording (e.g. for taking photos of really unpredictable subjects).
Quote from: DeafEyeJedi
Oh absolutely this will be useful. Especially for birding, sports photography & whatnot. This is incredible and should be a bit more forgiving than Canon's stock full burst mode which is only 6 frames per sec for 5D3. :P
A dumb question maybe... but... someone are getting realtime preview in this mode? How can you focus and framing for sports and birds?... I think I'm missing something...
Quote from: pc_bel on April 25, 2017, 11:05:13 AM
A dumb question maybe... but... someone are getting realtime preview in this mode? How can you focus and framing for sports and birds?... I think I'm missing something...
Want to help? Grab ADTG_GUI and QEMU and start investigating!
All you need to do is to understand how Canon configures the image processing path for LiveView. Here's a starting point. (http://www.magiclantern.fm/forum/index.php?topic=2388.msg183168#msg183168)
I'm taking a look to the Starting Point... and I don't know what to do... I have no knoweledge in any coding languaje. For me is like cheneese :( :-[ I'm sorry but if I can do something more just let me know.
The difficult part is simply understanding what Canon code does (and what we have to change to fix the preview).
Those numbers from the logs and adtg_gui are like Chinese to all of us; the trick is to fiddle with them until a tiny part of them starts to make some sort of sense. That's how we figured out how to reconfigure the sensor to 4K in the first place ;)
Some useful tricks: comparing all those numbers in different video modes (adtg_gui has a simple register diff tool), recognizing values that could look like resolutions, logging the I/O activity in QEMU, or analyzing the still photo mode first (it's simpler).
And, of course, don't give up after reading a single post. Make yourself comfortable with what's happening in the Reverse Engineering area. Some relevant threads are linked in the first post; make sure you can follow them.
Of course, you will need some basic programming knowledge.
Bottom line: the real-time preview is not implemented because... I have little or no idea about how Canon configures the image processing path.
Sorry don't be pissed at me, but what is the best solution to convert 14 bits lossless .mlv files?
I've red we can use cr2hdr.app_4K, but i don't seem to find a link to it, only the cr2hdr.app version. Is this application compatible with all the .mlv flavours?
Is there better way to do so? I've always used MLVMystic which is simple and elegant to use...
Thanks,
S/.
mlv_dump is your friend
@A1lex
Thank you very much for the explanation!!!... I feel really dumb but I guess not everyone will know about everything. Coding is a unknown world for me. The most obvious things for you are absolutely unknown for me. :-[
I would like to do more than only download the builds, but outside testing that builds I don't know what can I do...
Quote from: motionSOUL on April 25, 2017, 02:44:35 PM
Sorry don't be pissed at me, but what is the best solution to convert 14 bits lossless .mlv files?
I've red we can use cr2hdr.app_4K, but i don't seem to find a link to it, only the cr2hdr.app version. Is this application compatible with all the .mlv flavours?
Is there better way to do so? I've always used MLVMystic which is simple and elegant to use...
Thanks,
S/.
cr2hdr has been updated, so no need for the '4k' version anymore. Just grab the latest from here - https://bitbucket.org/Dannephoto/cr2hdr/downloads/
sorry , big confusion here...2 questions please.
1) for decode dng files of this exp.release , must be used only with last release of mlv_dump (line command without interface) ? because I try anothers programs, but are not compatible, only pink and black dng.
2) any one can please tell me, the right setting for recording in 1920 50p ? must be select also in menu canon 1280x50p or can be 1920 x 25 all-i) ? must be set FPS over to 50 ?
As long as you use the Experimental Build, you have to use mlv_dump. Its located in the same page to download.
From the Camera-Icon menu / Crop / 1920 50/60 3x3
You have three options.
If you select Canon's 1920x1080 25FPS, you get 1920x1080 25FPS.
If you select Canon's 1280x720, you get 1920x960 50FPS.
If you select Canon's Video System as NTSC instead of PAL, you get 1920x800 60FPS.
thanks, now is ok.
for a recording 1920x960 50p continued , can be set 14bit loss ore better 12 ?
I'v read 1920 50p can be continued...but to me stoped later few seconds.
CF cbay 128 x 1066
Read the little help text at the bottom of your screen.
I've tested 1920x960 50p 14 bits lossless, convert the .mlv file with mlv_dump, all .dng are black for the moment, don't know why. I had no error message during the recording and stopped it myself.
I've also tried 4K at 24p, but I haven't found the way to get the 4096 x 1440 ratio it is written everywhere?
In 4K, preview isn't working, right?
Thanks and many thanks to all the team which is doing so much work to fill the gap Canon has let behind.
I'm currently finishing my second short with the help of Magic Lantern and it is great to be able to get such video quality!
S/.
Quote from: motionSOUL on April 25, 2017, 05:31:05 PM
I've tested 1920x960 50p 14 bits lossless, convert the .mlv file with mlv_dump, all .dng are black for the moment, don't know why. I had no error message during the recording and stopped it myself.
I've also tried 4K at 24p, but I haven't found the way to get the 4096 x 1440 ratio it is written everywhere?
In 4K, preview isn't working, right?
Thanks and many thanks to all the team which is doing so much work to fill the gap Canon has let behind.
I'm currently finishing my second short with the help of Magic Lantern and it is great to be able to get such video quality!
S/.
Are they black if you look at them in After Effects or Resolve?
In After Effects / Photoshop / OSX Preview...
Which converter did you use @motionSOUL? Latest cr2hdr.app (https://www.magiclantern.fm/forum/index.php?topic=15108.0cr2hdr.app) has been working quite flawlessly w lossless MLV's as of late.
No, I've used mlv_dump from the Terminal. I will test cr2hdr.app, which has just converted a 4096x1366 test file to .dng flawlessly.
Quote from: jimiz on April 25, 2017, 04:59:17 PM
thanks, now is ok.
for a recording 1920x960 50p continued , can be set 14bit loss ore better 12 ?
I'v read 1920 50p can be continued...but to me stoped later few seconds.
CF cbay 128 x 1066
What is better 12bit or 14bit lossless depends on an image content. It is a pity that 12bit (10-8) lossless is not available for this mode. For 1920x960 @ 50p (both 1:1 crop and full-frame - 3x3 pixel binning) recording length depends on an image content. Sometimes 40 sec, sometimes 10. But for 40 fps (FPS override, Exact fps) I got continues rec. SanDisk, Extrime pro 160MB/s, 256GB.
BTW could not set 4096x1440 @ 25p (1:1 crop) mode. How?
Quote from: vstrglv on April 25, 2017, 09:31:24 PM
BTW could not set 4096x1440 @ 25p (1:1 crop) mode. How?
The preset is named "4K 1:1 half (https://en.wikipedia.org/wiki/One_half)-FPS". See also the first post.
Quote from: a1ex on April 25, 2017, 09:45:36 PM
The preset is named "4K 1:1 half (https://en.wikipedia.org/wiki/One_half)-FPS". See also the first post.
Thank you A1ex! Canon menu 50fps.
Hello A1lex,
Do we have to do something when the Memory patches is turned red, red on Code:FF28CC3C?
It seems to have appear after my first test with the Apr23.5D3113 build when testing 1920x960 50p, I had the screen filled with error alerts, then this...
Thanks,
S/.
Can someone write tutorial about how to rec on these modes? Why do I get this http://joxi.ru/YmEqBpZF06jjnm.jpg instead of DNG? What mlv_dump for?
Also I don't get how card spanning is working (it seems it doesn't) for when I enable this mode then at 3200x1200 it record 4 seconds (I checked it with various superfast CF-cards), but with only one CF card it also rec 4 seconds only!
QuoteCan someone write tutorial about how to rec on these modes? Why do I get this http://joxi.ru/YmEqBpZF06jjnm.jpg instead of DNG? What mlv_dump for?
mlv_dump is lovely!! make sure you use the mlv_dump in experiment page. otherwise you end up with noisy pink DNGs. been there before. or you can try the amazing cr2hdr.app. (note: dont judge the frames by their thumbnails. open them in photoshop to see if they are converted right. and not the first frame also!)
I see you have opened the first frame which is usually pink and corrupted with this mode. have you tried other frames too?!
this is MLV_Lite with this build which is different than the old MLV. MLVRawviewer and raw2cdng can't open it or they deliver pink DNGs instead.
@motionSOUL to help
Nothing to worry about : see my post #174 and A1ex reply #175 Question asked many times...
Quote from: Quentin on April 25, 2017, 05:12:04 PM
Read the little help text at the bottom of your screen.
yes , but also is write green 80 mb/s CONTINUE rec.
after 10-12 seconds a sequence of BEEP and stop the recording...
so I try anothers crope mode...like 3k , I crope like I have 70/80 mb max writing and compare continue recording...
preview is slow and gray , recording number stay green....(think normal) but after 10-15 seconds ...STOP !
CF is perfect (x1066) and work very well without stop with normal raw to 90/100 mbs
I don't know why many anothers here say can recorded continued in same situations....
What brand CF cards are you using @jimiz? Are they formatted into exFAT at the very least?
Also were you using '3.5k 1:1 center x5' within crop_rec.mo from ML menu and more importantly did you enabled FPS override to 23.976 while in x5 zoom?
Here's a screenshot from MLVFS webGUI to show that I was able to achieve continuous w 12-bit lossless in both 3072x1308 & 3072x1320 in 24p @ ISO 3200 until pressed stop.
(https://c1.staticflickr.com/3/2809/33444364344_551229871d_c.jpg) (https://flic.kr/p/SXn7ks)
This was used w KomputerBay 128GB 1066x and it was formatted to exFAT via Disk Utility on OS X prior to recording (256GB cards can be formatted in cam while keeping the exFAT).
Settings used under Movie mode within ML:
(https://c1.staticflickr.com/3/2811/33902403100_12478ee69e.jpg) (https://flic.kr/p/TDQFio) (https://c1.staticflickr.com/5/4185/33902403380_4bf81ce3fd.jpg) (https://flic.kr/p/TDQFod)
@DeafEyeJedi has the magic settings here. I can verify most of this. But you need to make sure ML is on your SD card and you are recording to your CF card that is formatted exfat from your computer. DeafEyeJedi settings are exactly what I have been using. It you want a smidge extra headroom you can set it to 2.35:1 which brings your vertical resolution down a tiny bit ... I think.
Edit: actually... I just got these settings to spike!
ISO 100 at some very bright trees! It went from like 89 MBs to 114 MBs I'll upload a dng
At the 11-8 bit Lossless with 3008x1280 I was able to get the same scene down to 90ish A1ex was right! (Not hard to believe) that you can get seemingly continuous settings to spike high! Very high
Quote@Ali Oliya
mlv_dump is lovely!! make sure you use the mlv_dump in experiment page. otherwise you end up with noisy pink DNGs. been there before. or you can try the amazing cr2hdr.app. (note: dont judge the frames by their thumbnails. open them in photoshop to see if they are converted right. and not the first frame also!)
I see you have opened the first frame which is usually pink and corrupted with this mode. have you tried other frames too?!
this is MLV_Lite with this build which is different than the old MLV. MLVRawviewer and raw2cdng can't open it or they deliver pink DNGs instead.
But where is this experiment page? I just download build pack and mlv_dump.exe but what should I do with this exe file? I actually can't understand why it so difficult to write tutorial about how to use these.
http://www.magiclantern.fm/forum/index.php?topic=19300.msg183363#msg183363
A user makes fun of this site and calls it "shark bay". If that is true you are pretty much bathing in Piranha infested waters right now.
Quote from: Savely on April 26, 2017, 08:55:15 PM
But where is this experiment page? I just download build pack and mlv_dump.exe but what should I do with this exe file? I actually can't understand why it so difficult to write tutorial about how to use these
@Savely
What modes exactly? This tutorial would be a lot to go through. I assume you read DeafEyeJedi settings and you are up to speed on the new build.
I have some that would offer even more stability depending on what you are filming.
3008x1280 11-8 bit lossless MLV Lite with preview set to auto.
Crop rec mode set to 5x zoom 3.5 k mode
Also it looks like you are having post processing issues.
If so it's easy.
For windows:
1) Download RawFlow v0.2 http://www.magiclantern.fm/forum/index.php?topic=13338.0
2) Download new MLV_dump https://builds.magiclantern.fm/experiments.html
3) Open up the Rawflow folder and delete current the MLV_Dump that came with Rawflow and replace it with the new MLV_dump you downloaded from here https://builds.magiclantern.fm/experiments.html leave this folder open for now.
4.) download your MLVs off your card to a new folder on your computer.
5.) Open up the RAWFlow folder if you closed it.
6.) Select all your MLVs and drag and drop them right on the Rawflow.exe application icon in the Rawflow folder (note you do not have to open the exe or run it. It will do that on its own after you drag and drop mlvs on it)
7.) The app will open a command prompt window. Let it do its thing and it will place new folders in the same folder as you MLVs with DNGs for you to play with.
Hope this helps and hope this doesn't break your camera or computer. I think this workflow is similar on MAC but not sure.
@DeafEyeJedi
sorry to double post but it appears that even 3072 x 1308 12bit lossless 23.976 fps ISO 100 under 5x zoom can cause a spike in data rate. Like I think it said 117 MB/s
@A1ex told me that if you try hard enough you can find a scene that can do this. It appears that it's not just high iso that can cause a data rate spike. Even at iso 100 12bit with a brightly exposed detailed scene that data rate can shoot through the roof. I assume if the details are bright enough to stay out of compression then this can happen. That's my guess anyway. I was able to get the scene back to 90MBs by using 11-8 bit at 3008x1280 here is the DNG that went off the charts.
https://www.dropbox.com/sh/8lmopv8z7p2i91f/AAA4saYTmLE7rFQ3t7nSUBCaa?dl=0
1920x1080 14bit lossless 24p was still okay. Not even close to danger zone.
EDIT!!!!!!! I actually tried to rec this scene at 1920x1080 14bit lossless 24p even tho MB/s was good but it gave me an ML assert
ML ASSERT:
0
at mlv_lite.c:2344 (compress_task), task compress_task
lv:1 mode:3
Then I notice that the compression ration was at 106% or something like that. @a1ex I found the scene you were talking about!
all frames are here:
https://www.dropbox.com/sh/8lmopv8z7p2i91f/AAA4saYTmLE7rFQ3t7nSUBCaa?dl=0
It will be nice to have an option for manual bit lossless setting in order to reduce data rate.
Quote from: vstrglv on April 26, 2017, 10:36:04 PM
It will be nice to have an option for manual bit lossless setting in order to reduce data rate.
Not after seeing samples like (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183187#msg183187) these (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183208#msg183208), but feel free to prove me wrong (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183231#msg183231).
The automatic choice is on the gradient 3-bit tonality (https://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html) border (any lower bit depth would result in obvious posterization).
Quote from: Savely on April 26, 2017, 01:40:21 AM
Can someone write tutorial about how to rec on these modes?
You're reading it; page 23.
WOW........ @a1ex you did it. I just tried the new build, it recorded @ 3.5K 12bit Lossless 2.35:1 continuously.. Zero Hick-ups so far. Converted the images with no problem they came out beautiful :D
Have you actually read the change log? There were *no* changes on the camera side since latest build; only mlv_dump was updated.
Not really was too excited to try it out :D
Quote from: DeafEyeJedi on April 26, 2017, 07:08:42 PM
What brand CF cards are you using @jimiz? Are they formatted into exFAT at the very least?
Also were you using '3.5k 1:1 center x5' within crop_rec.mo from ML menu and more importantly did you enabled FPS override to 23.976 while in x5 zoom?
Here's a screenshot from MLVFS webGUI to show that I was able to achieve continuous w 12-bit lossless in both 3072x1308 & 3072x1320 in 24p @ ISO 3200 until pressed stop.
(https://c1.staticflickr.com/3/2809/33444364344_551229871d_c.jpg) (https://flic.kr/p/SXn7ks)
This was used w KomputerBay 128GB 1066x and it was formatted to exFAT via Disk Utility on OS X prior to recording (256GB cards can be formatted in cam while keeping the exFAT).
Settings used under Movie mode within ML:
(https://c1.staticflickr.com/3/2811/33902403100_12478ee69e.jpg) (https://flic.kr/p/TDQFio) (https://c1.staticflickr.com/5/4185/33902403380_4bf81ce3fd.jpg) (https://flic.kr/p/TDQFod)
Thanls for your really important info about!
Cf is a CB 128 GB x1066 tested ,formated exFAT, and seem to work very well with normal raw 14 bit 1920 25p + audio , no stops.
In the setting you talk I become automatic stops , sometime with few beeps and error....BUT I think is some problem her of many another people....
also if you read the setting you do, work on continued recording.... iso and LIGHT of sky example...give a big input data that overwriting buffer and STOP the recording.
With 1920x960 50 P 14, 12 LOSS bit....begine well and suddenly break the recoding, why ? I just do few test before....if I begin recordin in the dark area/room ALL ok....when I turn to the windows light....BREAK recording, so also if was around 60/70 Mb...the light jump over 100.... this do in all the LOSS bit.
i think this is the bottleneck which does not hold recording on some resolution, though not exaggerated and low frames. or not?
Quote from: a1ex on April 26, 2017, 10:52:31 PM
Not after seeing samples like (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183187#msg183187) these (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183208#msg183208), but feel free to prove me wrong (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183231#msg183231).
The automatic choice is on the gradient 3-bit tonality (https://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html) border (any lower bit depth would result in obvious posterization).
Yes, I understand. But this build is experimental one. A person who is going to use it must understand or try to understand what's going on. But it's not easy to set a proper exposure and sometimes the exposure is set with some shift intentionally. In these cases the relation between ISO and bit depth is not strictly defined. So manual bit depth option will be usefull.
BTW, is there a probability to get 12-8bit lossless for other modes, espesially 45/50/60fps?
Quote
In these cases the relation between ISO and bit depth is not strictly defined. So manual bit depth option will be usefull.
Please show me one (1) non-trivial example where reducing the bit depth below the recommended value is useful.
Quote
BTW, is there a probability to get 12-8bit lossless for other modes, espesially 45/50/60fps?
Probability to get lower bit depths in other modes is zero with current method (I'm not aware of any situation when it could work by chance, no matter how many times you would try).
If you ask about future possibilities, I'm unable to answer (because it's something that, at the moment of writing, is beyond my understanding). Whether I'll be able to figure it out tomorrow, or next month, or within the next 10 years, I have absolutely no idea. Same for real-time previews with 4K or other similar questions.
If you ask whether the hardware can do this, I'm pretty sure it can. What I don't know is how to program it to do so. You (http://www.magiclantern.fm/forum/index.php?topic=14656.0) can (http://magiclantern.wikia.com/wiki/Register_Map/Brute_Force) help (http://www.magiclantern.fm/forum/index.php?topic=2864.125) with (http://www.magiclantern.fm/forum/index.php?topic=6751.msg71720#msg71720) that (http://www.magiclantern.fm/forum/index.php?topic=10111.0).
Quote from: a1ex on April 27, 2017, 10:04:39 PM
Please show me one (1) non-trivial example where reducing the bit depth below the recommended value is useful.
Only for reducing data rate a little. But what is the problem with manual bit rate?
Quote from: a1ex on April 27, 2017, 10:04:39 PM
If you ask whether the hardware can do this, I'm pretty sure it can. What I don't know is how to program it to do so. You (http://www.magiclantern.fm/forum/index.php?topic=14656.0) can (http://magiclantern.wikia.com/wiki/Register_Map/Brute_Force) help (http://www.magiclantern.fm/forum/index.php?topic=2864.125) with (http://www.magiclantern.fm/forum/index.php?topic=6751.msg71720#msg71720) that (http://www.magiclantern.fm/forum/index.php?topic=10111.0).
Unfortunately I can not help, not enough knowledge...
Quote from: vstrglv on April 27, 2017, 10:25:41 PM
But what is the problem with manual bit rate?
Asked and answered. Have you actually clicked on my links?
Yes.
@vstrglv:
please find the requested download here: [this feature is only available for members having a gold subscription]
if that doesn't work, and the arguments of the developers isn't reason enough: feel free to implement it
So @a1ex April 27th Build has no new changes from the April 26th version.. Like before just mlv_dump changed.
Hey, installed latest build on 5D3 today as I was really hoping to record 24fps 1080p 14bit lossless on the CF and proxy files on the SD.
Unfortunately, the proxy files will only record for about 6 seconds or so before live-view locks and the battery needs to be pulled. Recording both the 14bitlossless and proxy on the CF card results in the same/similar result.
With the latest crop mode build recording 10bit 1080p on CF and proxy on SD works great!
Anychance I'm missing something or there is a quick fix for this? Cheers
Right, I had two commits in the raw-h264-branch that I thought were included, but looks like they were not (I've never actually tried this feature on this branch). One (https://bitbucket.org/hudson/magic-lantern/commits/d0b78dfc82) is a workaround to prevent hard lock-ups when the H.264 stream stops, and the other (https://bitbucket.org/hudson/magic-lantern/commits/9465a180dfe4) attempts to sync the file names.
Last time I've checked (a long time ago), I had good results with H.264 IPB on SD, but recording stopped very quickly with both on CF (don't remember much about ALL-I).
Build updated.
Thank you Alex!
Quick question, are there any real strong incentives to record 10bit raw instead of 14bit lossless besides possibly better stability with crop mode build?
Currently, the only reason I could think of would be in-camera playback.
I'd actually recommend lossless compression with 12-bit (high quality, as the last 2 bits are mostly noise (https://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html)) or less (to be tested). Keep in mind I'm not an active video user, so the recommendation is based on theory, not on practice.
Thank you for the recommendation and explanation why!
In theory and as you understand it, is there a possibility that the last 2 bits provide some type of head-room for difficult shots? Or would you say 99% not likely..
You can judge from these samples: http://www.eoshd.com/comments/topic/21004-12-or-10-bit-raw-magic-lantern/?page=3#comment-168058
At the very first: ML is very great stuff and is a great ingenious work! Its an overwhelming thing, can't say thanks much enough! It makes a lot of fun playing and working with it!
Right now I'm playing with ML on the high res end.
I confirm that 3.5K works continously at 2.67:1; 12bit lossless
Quote from: goldenchild9to5 on April 27, 2017, 07:56:48 AM
WOW........ @a1ex you did it. I just tried the new build, it recorded @ 3.5K 12bit Lossless 2.35:1 continuously.. Zero Hick-ups so far. Converted the images with no problem they came out beautiful :D
@goldenchild: are you sure that you managed real 2.35:1 with 3.5K?
3.5K @ 2.35:1 (3520x1498) only 14bit lossless is supported with this vertical resolution and here you end up at >105MB/s at 50% compression --> not continuous
Also 1320pixel is max vertical resolution for the 5x crop mode (3.5K 1:1 centered x5 as well as 5x live view crop) and this is currently the max. vert res for 12bit lossless and lower
By the way I tested 3.5K 2.39:1 at UHD crop mode to get a vert res >1320 but at 14bitlossless. Depending on the brightness of the image I manged to get 120 to nearly 200 frames in this resolution.
Here is a quick test:
https://1drv.ms/i/s!ArsIFng0eZ9dgYhqj9iycEK3sGsFSQ
3520x1472; 14bitlossless; 23.976; UHD crop mode --> ~105-110MB/s
ISO 400 --> 120 frames
ISO 320 --> 150 frames
ISO 200 --> 186 frames
as mentioned above 3520x1320 works continuous due to 12bit lossless capability, but is 2.67:1...
By the way there is a small bug:
when accidentally entering the UHD crop mode from 3.5K (where 5x lv zoom + fps override to 23.976 needs to be enabled) at a resolution of 3520+ the FPS override changes to an odd 33.. fps speed. When turning fpsoverride off it changes to 19.. fps. While turning off, the Picture Style window from Canon menu flashes up once.
To eliminate this you need to turn off crop mode und raw video, close ml menu and turn everything on again (without x5 live view zoom + fps or!)... Nothing substantial but a bit confusing.
Can anyone confirm this?
Quote from: a1ex on April 27, 2017, 11:44:01 PM
Right, I had two commits in the raw-h264-branch that I thought were included, but looks like they were not (I've never actually tried this feature on this branch). One (https://bitbucket.org/hudson/magic-lantern/commits/d0b78dfc82) is a workaround to prevent hard lock-ups when the H.264 stream stops, and the other (https://bitbucket.org/hudson/magic-lantern/commits/9465a180dfe4) attempts to sync the file names.
Last time I've checked (a long time ago), I had good results with H.264 IPB on SD, but recording stopped very quickly with both on CF (don't remember much about ALL-I).
Build updated.
Hey Alex, unfortunately those changes you made to the latest build (2017-04-27 21:51) didn't do it. It still only records 6 seconds ALL-I or IPB before I get an "emergency stop" notification on the live view. Tried 12bit lossless, 12bit raw, and 10bit raw, makes no difference. I made sure to set record to card 2 (SD) in the canon menu each time.
@Janosch Yep got 3.5K 12bit Lossless 2.35:1 to work continuously. Doing more testing but so far it's been stable and continuous.
How are you getting higher than 1320 vertical resolution to get 2.35 aspect ratio at 3.5k?
QuoteBy the way there is a small bug:
when accidentally entering the UHD crop mode from 3.5K (where 5x lv zoom + fps override to 23.976 needs to be enabled) at a resolution of 3520+ the FPS override changes to an odd 33.. fps speed. When turning fpsoverride off it changes to 19.. fps. While turning off, the Picture Style window from Canon menu flashes up once.
To eliminate this you need to turn off crop mode und raw video, close ml menu and turn everything on again (without x5 live view zoom + fps or!)... Nothing substantial but a bit confusing.
Can anyone confirm this?
can confirm!
Quote from: Janosch on April 28, 2017, 01:43:12 AM
By the way there is a small bug:
when accidentally entering the UHD crop mode from 3.5K (where 5x lv zoom + fps override to 23.976 needs to be enabled) at a resolution of 3520+ the FPS override changes to an odd 33.. fps speed. When turning fpsoverride off it changes to 19.. fps. While turning off, the Picture Style window from Canon menu flashes up once.
To eliminate this you need to turn off crop mode und raw video, close ml menu and turn everything on again (without x5 live view zoom + fps or!)... Nothing substantial but a bit confusing.
Can anyone confirm this?
Trying to reproduce this on my end to no avail. Was this done on 113 or 123?
Quote from: jankrueck on April 28, 2017, 05:02:17 PM
can confirm!
Really? ...How?
Quote from: hyalinejim on April 28, 2017, 03:57:18 PM
How are you getting higher than 1320 vertical resolution to get 2.35 aspect ratio at 3.5k?
Hmmm care to explain this @goldenchild9to5 or maybe you assumed incorrectly? Either way would you mind to elaborate a bit more or better yet share some samples?
Because atm it just stays within 2.65:1 even if you selected '2.35:1' from the ML menu:
(https://c1.staticflickr.com/5/4182/34322239625_b36e100648.jpg) (https://flic.kr/p/UhWs5a)
and yet it isn't continuous even if w FPS override enabled down to 20p.
Quote from: DeafEyeJedi on April 28, 2017, 05:18:38 PM
But how did you get 3504x1320? I can get only:
3072x1320 (~2,33:1)
3520X1320 (~2,67:1)
3584X1320 (~2,72:1)
@vstrglv -- perhaps you didn't press magnified zoom to get into x5 mode (which is required for this mode - read the notes on bottom of LCD LiveView) while having '3.5k 1:1 centered x5' enabled from crop-mode within ML menu and then select 3520x1320 in 2.50:1 w 12-bit lossless.
(https://c1.staticflickr.com/5/4156/34165732962_ff94fa00a8_n.jpg) (https://flic.kr/p/U47j6q) (https://c1.staticflickr.com/5/4186/34165732722_57361246f1_n.jpg) (https://flic.kr/p/U47j2h) (https://c1.staticflickr.com/3/2816/34165914172_d653796838_n.jpg) (https://flic.kr/p/U48eXJ)
Finally, while highlighted on the 'Resolution 3520x1320' use your shutter scroller (top right) to fine-tune it down 3504x1320 which is 2.65:1.
@DeafEyeJedi Thank you very much! I forgot about shutter scroller...
I'm liking the latest build, full res liveview is getting pretty darn reliable at 7.5fps, 2.35 crop. Almost continuous recording, I had a few shut off after 5 seconds. I drove up into the mountains to see how some of my older lenses handle 4k+ and made a short video.
I did notice that it would shut off right away if I was shooting right into the sun at ISO100 but I kind of expected that after reading through the thread. It's getting damn good though.
Amazing work so far!
Quote from: erikbaldwinson on April 28, 2017, 03:27:11 AM
Hey Alex, unfortunately those changes you made to the latest build (2017-04-27 21:51) didn't do it. It still only records 6 seconds ALL-I or IPB before I get an "emergency stop" notification on the live view. Tried 12bit lossless, 12bit raw, and 10bit raw, makes no difference. I made sure to set record to card 2 (SD) in the canon menu each time.
Confirmed - it was broken by an experimental change in this branch - the way we allocate memory for the main raw buffer (during standby) conflicts with H.264. I've assumed Canon's memory layout does not change as long as you are not leaving LiveView, but it looks like it does (so our raw stream ended up overwriting the memory areas used by H.264). Looking into it.
Quote from: pocketrubbish on April 28, 2017, 07:17:25 PM
I'm liking the latest build, full res liveview is getting pretty darn reliable at 7.5fps, 2.35 crop. Almost continuous recording, I had a few shut off after 5 seconds.
Nice to see this mode working! You should be able to reduce the memory workload (and hopefully free some resources to make them available to the card writing task) with some tricks:
- dial down the resolution in crop_rec submenu (target yres); note the FPS will increase
- re-center the image using CMOS[1] hi and lo (trial and error)
- enable FPS override in low light mode and dial it down to 7.5 fps or whatever you need
- try the "Frozen LV" preview mode for a little more writing speed.
Quote from: a1ex on April 28, 2017, 07:25:21 PM
Nice to see this mode working! You should be able to reduce the memory workload (and hopefully free some resources to make them available to the card writing task) with some tricks:
- dial down the resolution in crop_rec submenu (target yres); note the FPS will increase
- re-center the image using CMOS[1] hi and lo (trial and error)
- enable FPS override in low light mode and dial it down to 7.5 fps or whatever you need
- try the "Frozen LV" preview mode for a little more writing speed.
Thank you, I'll try all of that this weekend.
Quote from: DeafEyeJedi on April 28, 2017, 05:18:38 PM
Trying to reproduce this on my end to no avail. Was this done on 113 or 123?
113!
you may encounter it if you make 3.5K (w/ 5x zoom + fps override to 23.976 enabled) working and switch as described to the UHD crop mode...
Quote from: DeafEyeJedi on April 28, 2017, 05:18:38 PM
(https://c1.staticflickr.com/5/4182/34322239625_b36e100648.jpg) (https://flic.kr/p/UhWs5a)
and yet it isn't continuous even if w FPS override enabled down to 20p.
it seems there is a problem with your cf card as 49.3 MB/s is too slow. 77.1MB/s should be no problem for continuous recording.
Maybe try to run a new cf card benchmark...
So I decided to test the 5k mode at 6fps 10bit (not lossless) 2.35:1 and ..... Continuous. This is going to be wild for timelapsing!
Quote from: hjfilmspeed on April 29, 2017, 12:09:14 AM
So I decided to test the 5k mode at 6fps 10bit (not lossless) 2.35:1 and ..... Continuous. This is going to be wild for timelapsing!
Just curious. Why would you choose to shoot a timelapse this way compared to shooting raw stills using the intervalometer? I don't shoot a lot of timelapse, so genuinely curious.
@beauchampy No shutter actuations and a slightly nicer workflow IMO. The only issue would be battery life for really long lapses. The Mark iv shuts off the LCD. But for 1fps or maybe .5 fps it would be awesome. Havent tried it yet.
Wich continous maximum resolution can I use for my new full length movie?
(I promess next Cannes festival take a picture in the end of the movie with the logo of ML)
;)
Off topic: I dreamed recording 4k watching by the optical view, not the viewfinder :D
Quote from: a1ex on April 27, 2017, 11:44:01 PM
Hey Alex, unfortunately those changes you made to the latest build (2017-04-27 21:51) didn't do it. It still only records 6 seconds ALL-I or IPB before I get an "emergency stop" notification on the live view.
Solved. The changes were non-trivial, but since it was a memory management issue, it got higher priority.
At 1920x1080 24p 12-bit lossless, and a bit of luck (because of the variable bitrate) recording can be continuous even with both RAW and H.264 on the same card. With H.264 on SD and RAW on CF, both recorders have *much* more headroom.
Careful when the card gets full, as this event is not handled very well (and it requires more reverse engineering to figure it out).
Quote from: jimiz on April 26, 2017, 06:50:13 PM
preview is slow and gray , recording number stay green....(think normal) but after 10-15 seconds ...STOP !
Solved that too (although I wish I had a proper bug report). The real-time estimations (and the indicator color) were completely messed up in lossless recording modes, and just OK-ish in uncompressed mode (with estimations being far away from actual recording times).
@budafilms I would say to stick with 3K..
@a1ex the change log for 29th build looks very nice!
@a1ex Thank you for the new build, will give it a spin today.
Wow, the latest build is very stable and much more fool proof! Got 48p 1080 working flawlessly and the full res live view puts up no fuss at all. I really love the total frame count at the end, very useful.
Now I'm trying to decide if I should drive out somewhere this evening and do another test... maybe I'll go to the beach.
I've cherry-picked some changes from this branch, to be included in the main builds (for which I need some help with code review). First round:
https://bitbucket.org/hudson/magic-lantern/pull-requests/827
https://bitbucket.org/hudson/magic-lantern/pull-requests/828
https://bitbucket.org/hudson/magic-lantern/pull-requests/819 (last few commits)
https://bitbucket.org/hudson/magic-lantern/pull-requests/825 (older, but no reviews yet)
Next on the list:
- compressed_raw (without all the changes required for 4K, which are mostly 5D3-specific), so you can start porting it on other cameras
- compressed_raw with lower bit depth (I'd like to keep them separate, so you can focus on plain 14-bit lossless and then move on to more complicated stuff)
- H.264 proxy (there are few changes, but currently entangled in the middle of other 4K experiments)
If you are wondering why: this branch became pretty complex (includes many changes that are not exactly related) and many of these things are not portable to other cameras, or just highly experimental. Splitting it into small logical chunks should make it a bit more manageable.
Awesome @a1ex
https://youtu.be/ZdvKALy58Es
Bunch of 3520x1320 12bit shot last night and this morning. Sadly, youtube compresses the shit out of it but, man, you should see it in prores at full size. Glorious!!
Slightly graded in AE with VisionColor Osiris LUTs.
Looks great, giarcpnw. The shot of your dog looks out of a feature film. I was just at that spot yesterday. Then you freaked me out because I also have the same hummingbird feeder and hedge! :D
Quote from: lostfeliz on April 30, 2017, 02:46:35 AM
Looks great, giarcpnw. The shot of your dog looks out of a feature film. I was just at that spot yesterday. Then you freaked me out because I also have the same hummingbird feeder and hedge! :D
Doesn't every Angelino have an ivy hedge and HB feeder? ;-)
Thanks, each shot went well over 30 seconds, I just didn't want to bore everyone. I'll push it further another day but it seem pretty stable for all intents and purposes. Pretty amazing @a1ex!
Now, to shoot something with it!!
Because the preview in 3k and above is so laggy, I hadn't thought of plugging in my monitor to it. But tonight I tried it out. The field of view is about 40% of what's on camera screen and being recorded. But it's great for focusing and no lag. If you can get used to operating with two monitors (one for composition and one for focus), you can really shoot a 3k project with a 5diii. Hell, if you had a focus puller you'd be in even better shape.
The HDMI monitor output (when mirroring, at least) is completely centered. Short of being the correct frame without jaggy lag, I'm wondering if the HDMI mirrored output could be panned as desired to frame what you want to see to keep in focus.
The cropped panning control knob does something when you use it but the screen then pop back to how they were.
Quote from: lostfeliz on April 30, 2017, 08:00:53 AM
Because the preview in 3k and above is so laggy, I hadn't thought of plugging in my monitor to it. But tonight I tried it out. The field of view is about 40% of what's on camera screen and being recorded. But it's great for focusing and no lag. If you can get used to operating with two monitors (one for composition and one for focus), you can really shoot a 3k project with a 5diii. Hell, if you hades a focus puller you'd be in even better shape.
The HDMI monitor output (when mirroring, at least) is completely centered. Short of being the correct frame without jaggy lag, I'm wondering if the HDMI mirrored output could be panned as desired to frame what you want to see to keep in focus.
The cropped panning control knob does something when you use it but the screen then pop back to how they were.
Thanks for posting this info! Just curious. Are you using 1.2.3 firmware or 1.1.3? Thanks!
I was doing exposure tests and the exposure lowered dramatically when I switched from 14-bit lossless to 12-bit lossless. Settings: Crop 3.5, 3072x1320. Is this to be expected?
Rob, I'm running 1.2.3.
The exposure stays the same; only the real-time preview gets darker. That's how it works. The non-realtime preview should be correct.
Refer to the initial commit (https://bitbucket.org/hudson/magic-lantern/commits/2028d73) and the subsequent ones.
Hello :)
I wanted to ask, if, with all the new development and discoveries you guys made lately, there is a way to improve/shorten rolling shutter ms for normal 1080p recording.
Hello everyone!
Just a quick video shot in Roma last week to show that everything just works perfectly !
Thanks to the team !
For informations
//4k Settings //
- Resolution : 3072x1024 px
- Bits : 14 bpp Lossless
- Aspect Ratio : 3:1
I was able to shoot continuously.
Yes, I did some continuous flawless 3K shots. Incredibly awesome!
The 1st of April is ML special day.
I hope MLV_Sound returns some time and works in hand with video.
Wow!!!!!! 29th build!!!!!!
@baksh Great job.. awesome footage.
Quote from: rudimental on April 30, 2017, 06:58:11 PM
Hello :)
I wanted to ask, if, with all the new development and discoveries you guys made lately, there is a way to improve/shorten rolling shutter ms for normal 1080p recording.
Already done (http://www.magiclantern.fm/forum/index.php?topic=12656.0) a long time ago (actually since 2012 (https://groups.google.com/d/msg/ml-devel/tSg5v99I7lQ/uL-XTHQ8fBQJ); we just didn't know what we had done, back then).
It might be possible to push it a little lower. With adtg_gui and enough patience, you can probably figure it out. All you have to do is to reduce FPS timer A as much as you can, but you'll also have to reconfigure some other parameters to allow this (don't know which ones - that's what you have to find out).
Sorry guys, with the new MLV_Dump the DNG file sizes are coming out much larger, for 3k 12 bit they are 7mb each. I am using this script to run MLV_Dump, please help me make corrections. Also, it seems to be taking much longer as well.
Quote@echo off
for %%a in ( %* ) do ( md "%%~na" 2>nul )&( C:\MLVFS\mlv_dump.exe -o %%~na\%%~na_ --dng %%~na.MLV )
EDIT: MLV files are 16.6gb, dng output are closer to 30gb.
If you used compressed raw, the MLV files are always smaller. The dng files that are made by MLV_dump are not compressed, so bigger.
And as far as I know, the output in dng from MLV_dump is always in 14 bit.
So you could have a compressed 12 bit MLV file, but the dng's that come out mlv_dump are uncompressed 14 bit.
Quote from: Levas on May 01, 2017, 10:23:28 AM
If you used compressed raw, the MLV files are always smaller. The dng files that are made by MLV_dump are not compressed, so bigger.
And as far as I know, the output in dng from MLV_dump is always in 14 bit.
So you could have a compressed 12 bit MLV file, but the dng's that come out mlv_dump are uncompressed 14 bit.
That makes perfect sense! Thank you for clearing that up for me.
Hello!
I installed the firmware version magiclantern-crop_rec_4k.2017Apr26.5D3113. I shoot 1920x1080 30p 14bit losless, it works fine, but when I open the video on the computer, in all the videos I see on the left a black bar about two pixels wide. What is the problem?
(http://thumb.ibb.co/mJ0SVk/M01_0745_000008.jpg) (http://ibb.co/mJ0SVk)
kinda offtopic. just wanted to show some testfootage.
just wanted to thank Alex and team for their work!
Can't wait for final release and having sound recorded. keep it up!
3k 1:1 | 3072x1320 | 14bit lostless
Sandisk Pro 160mb/s 64GB card
ISO100-200 continous
3840x2160 (1650 w/o letterbox) upscaling, sharpening and some lightadjustments done in resolve 12.5.
very rough and fast, noch "real" grading or so.
Quote from: hindra on May 01, 2017, 03:50:21 AM
Sorry guys, with the new MLV_Dump the DNG file sizes are coming out much larger, for 3k 12 bit they are 7mb each. I am using this script to run MLV_Dump, please help me make corrections. Also, it seems to be taking much longer as well.
EDIT: MLV files are 16.6gb, dng output are closer to 30gb.
you now can choose between
a) output uncompressed DNGs
b) compress DNG content (bit slower bit same as uncompressed, just many tools have trouble with lossless compressed DNG)
c) pass through the original lossless data (which will be fast but skip all processing like stripe and hot pixel detection)
this can be done by adding "-c" multiple times to the commandline.
e.g.
./mlv_dump in.mlv --dng # outputs uncompressed DNGs
./mlv_dump in.mlv --dng -c # outputs compressed DNGs using mlv_dump's LJ92 algorithm
./mlv_dump in.mlv --dng -c -c # outputs uncompressed DNGs by passing through unmodified lossless data
Quote from: g3gg0 on May 02, 2017, 01:34:03 AM
you now can choose between
a) output uncompressed DNGs
b) compress DNG content (bit slower bit same as uncompressed, just many tools have trouble with lossless compressed DNG)
c) pass through the original lossless data (which will be fast but skip all processing like stripe and hot pixel detection)
this can be done by adding "-c" multiple times to the commandline.
e.g.
./mlv_dump in.mlv --dng # outputs uncompressed DNGs
./mlv_dump in.mlv --dng -c # outputs compressed DNGs using mlv_dump's LJ92 algorithm
./mlv_dump in.mlv --dng -c -c # outputs uncompressed DNGs by passing through unmodified lossless data
Thank you for that. It is nice having options.
@jankrueck Great job footage looks awesome.. You also captured great dynamic range, windows are well exposed & still having enough dynamic range for the interior superb.
g3gg0, thanks!
Here´s latest mlv_dump for mac which beautifully applies lossless compression to dng files!
https://bitbucket.org/Dannephoto/magic-lantern/downloads/mlv_dump_compress_dng.zip
try:
mlv_dump --dng -c INPUT.MLV
And watch dng files process lossles with less than half it´s size. This works with uncompressed footage as well.
@g3gg0
Quote from: g3gg0 on May 02, 2017, 01:34:03 AM
you now can choose between
a) output uncompressed DNGs
b) compress DNG content (bit slower bit same as uncompressed, just many tools have trouble with lossless compressed DNG)
c) pass through the original lossless data (which will be fast but skip all processing like stripe and hot pixel detection)
Haha, I was doing the same pass-trough stuff yesterday but you really did it a lot better (and faster) than me!!! :D plus there are also some more fixes/changes. Thank you so much!
There is a small question. When -c not specifyed e.g. 'mlv_dump -o result.mlv source.mlv', the standard output is:
Processing...
saving: 3628800 -> 1906112 (52.53% ratio)
saving: 3628800 -> 1906112 (52.53% ratio)
saving: 3628800 -> 1905600 (52.51% ratio)
In case of "!verbose":
if(frame_buffer_size != (uint32_t)frame_size && !verbose)
{
print_msg(MSG_INFO, " saving: "FMT_SIZE" -> "FMT_SIZE" (%2.2f%% ratio)\n", frame_size, frame_buffer_size, ((float)frame_buffer_size * 100.0f) / (float)frame_size);
}
Do we really need this info (calculated uncompressed size to already hw compressed size comparizon) during MLV->MLV pass-trough? Maybe make it somewhat optional.
bb
Quote from: Danne on May 02, 2017, 09:35:00 AM
g3gg0, thanks!
Here´s latest mlv_dump for mac which beautifully applies lossless compression to dng files!
https://bitbucket.org/Dannephoto/magic-lantern/downloads/mlv_dump_compress_dng.zip
try:
mlv_dump --dng -c INPUT.MLV
And watch dng files process lossles with less than half it´s size. This works with uncompressed footage as well.
Fantastic! Look forward to seeing this with cr2hdr!
Quote from: Danne on May 02, 2017, 09:35:00 AM
try:
mlv_dump --dng -c INPUT.MLV
And watch dng files process lossles with less than half it´s size. This works with uncompressed footage as well.
Produced (processed and recompressed) DNGs are even a bit smaller than passed-trough ones :) I guess also becase of there is no padding bytes.
bb
Quote from: goldenchild9to5 on May 02, 2017, 05:24:12 AM
@jankrueck Great job footage looks awesome.. You also captured great dynamic range, windows are well exposed & still having enough dynamic range for the interior superb.
thanks! my maingoal was to test around with DR, detail and cameramoving, as most of the test I saw where handheld or tripod landscape.
the Dynamic Range is just insane. Its out of Camera. Didnt lift shadows or saved highlights at all.
cherrio!
* 1920x960 @ 50p (both 1:1 crop and full-frame - 3x3 pixel binning) last exp.4k 29 april
crash (freeze) on ML menu options, in few seconds when push upd and down select.... and must put out battery ....
any one confirm ?
@jimiz
For me that mode is no more continuous recording with the last build, I don't know why... I return to April14th, the most reliable for me... :-[
If you want the future builds to improve...
http://www.chiark.greenend.org.uk/~sgtatham/bugs.html
Before reporting any differences in recording times, please make sure they are not caused by small variations in the data rate (unavoidable with lossless compression) or by different settings (e.g. global draw, preview options).
I'm currently benchmarking the two builds with this method (http://www.magiclantern.fm/forum/index.php?topic=17091) (this script (http://www.magiclantern.fm/forum/index.php?topic=17091.msg165766#msg165766)).
Just FYI, it's not the first time people are reporting speed differences between nearly identical builds (http://www.magiclantern.fm/forum/index.php?topic=17091.msg165688#msg165688) (where the only change was something like fixing a typo in XYZ language (https://bitbucket.org/hudson/magic-lantern/commits/2a1f972cdbd87e77059539622fd4308618fa9ae6)).
Quote from: pc_bel on May 02, 2017, 04:03:56 PM
@jimiz
For me that mode is no more continuous recording with the last build, I don't know why... I return to April14th, the most reliable for me... :-[
yes to me too , no possibile to rec. in 1920x920 50p , around 20 seconds not more...little bit better in x800....but not continued.
With the settings recommended in the first post, 1920x960 50p is continuous here (orange icon at first, green after ~30 seconds).
Benchmark results tomorrow (will let the script test the two builds overnight).
Quote from: bouncyball on May 02, 2017, 10:43:09 AM
Do we really need this info (calculated uncompressed size to already hw compressed size comparizon) during MLV->MLV pass-trough? Maybe make it somewhat optional.
no, good point.
reworked a lot of stuff (so errors may appear that weren't present before).
this is one thing. got to fix that, right.
Quote from: a1ex on May 02, 2017, 06:51:24 PM
With the settings recommended in the first post, 1920x960 50p is continuous here (orange icon at first, green after ~30 seconds).
Benchmark results tomorrow (will let the script test the two builds overnight).
Thanks A1ex
to me and a friend (we have same CF CBx1066 tested) with 27 and 29 april , vers. 123 , orange icon at first 20 seconds circa red after and stop! no way ...tryed 20 times...
if we set 1920x800 orange icon at first 10-15 seconds...green , orange, green.... but seem to be continued...(i try more of 2-3 minutes without stop)
so I don't know if 1920x960 50p was working better in old version...this I can't say....
Quote from: jimiz on May 02, 2017, 08:11:42 PM
to me and a friend (we have same CF CBx1066 tested) with 27 and 29 april , vers. 123 , orange icon at first 20 seconds circa red after and stop! no way ...tryed 20 times...
Screenshot of the raw recording menu, please.
I think is something related to compression and how both builds manage it. With April14th I get green icon from start to end of recording, no matter what image I'm recording. With last buid April29th, I get green icon if I'm recording a dark image with no detail, and when I point the camera towards a detailed and well iluminated image, icon changes to orange and quickly to red. Then stops.
But I can be wrong... It's just an assumption from my ignorance.
Hello!
Does anyone else besides me have a black bar two pixels wide on the left side of the frame on all the videos? Firmware version magiclantern-crop_rec_4k.2017Apr26.5D3113. I shoot 1920x1080 30p 14bit losless.
@ DLM
Yes!
When I use Handbrake to convert some videos to MP4, Handbrake detects 2 pixels wide column and crops it.
e.g if I shoot at 3072, it shows 3070, 2 pixels offset
Quote from: pc_bel on May 02, 2017, 10:14:47 PM
With April14th I get green icon from start to end of recording, no matter what image I'm recording.
... and stops with green icon when you do this:
QuoteWith last buid April29th, I get green icon if I'm recording a dark image with no detail, and when I point the camera towards a detailed and well iluminated image, icon changes to orange and quickly to red. Then stops.
As mentioned here (http://www.magiclantern.fm/forum/index.php?topic=19300.msg184055#msg184055), I've fixed the indicator, so it will actually warn you when it's going to stop.
That said, looks like the benchmark has some significant results:
mlv_lite version : ML/MODULES/RAWBENCH/APR14.MO
854,872,940,989,924,802,930,894
767,783,869,856,944,872,909,941
746,973,1004,840,912,986,1088,800
660,714,798,875,939,982,830,946
868,929,941,983,857,960,870,846
910,1072,929,1016,877,890,1021,924
840,929,951,840,894,928,928,874
651,720,926,837,850,812,1095,1078
840,1030,804,960,910,1017,971,963
1034,1032,961,806,894,1018,872,836
926,1011,1074,902,960,838,864,1123
783,949,871,890,746,760,796,853
mlv_lite version : ML/MODULES/RAWBENCH/APR29.MO
335,727,943,761,882,929,608,781
338,754,852,554,572,876,750,1017
608,683,592,689,765,844,713,652
834,733,574,804,638,560,927,853
423,747,775,943,745,745,588,797
505,687,1003,703,786,583,838,799
495,597,895,872,872,572,837,926
771,636,702,891,761,802,833,979
710,590,905,978,889,929,810,851
652,837,727,726,742,696,707,763
486,655,826,729,669,882,916,697
Quartile stats: median (Q1 ... Q3)
mlv_lite version : ML/MODULES/RAWBENCH/APR29.MO: 752 frames (654 ... 852)
mlv_lite version : ML/MODULES/RAWBENCH/APR14.MO: 906 frames (840 ... 960)
P =
1 1.3323e-15
1.3323e-15 1
T =
0 -8.7668
8.7668 0
Looks like there is, indeed, a small speed regression.
Each number from the "big matrix" shows how many frames were in each test clip, and a new line means the card was formatted (and possibly other build was tested meanwhile - they are chosen randomly after filling and formatting the card). 1920x960 50p, 14-bit lossless, dark scene with artificial light (not changing during the experiment), preview set to Framing (which is not continuous here).
So, I've added some intermediate versions to the test script to narrow it down. The scene was a bit different (moved the camera), so the numbers are not directly comparable to the first run (variable bit rate):
mlv_lite version : ML/MODULES/RAWBENCH/APR23.MO
1764,1691,1623,1565,1180,1299,1481,1307
1173,1284,1576,1439,1482,1578,1421,1330
1306,1110,1670,1184,1368,1439,1228,1568
922,1360,1603,1455,1208,1201,1528,1844
3200,3181,3199,3093,3028,3112,3215,3149
mlv_lite version : ML/MODULES/RAWBENCH/E93203.MO
827,1134,1099,1324,1399,1475,1515,1397
675,1121,1270,1271,921,893,1126,1088
mlv_lite version : ML/MODULES/RAWBENCH/B420A8.MO
1596,1296,1169,1322,1292,1475,1839,1443
1068,1895,1441,1363,1327,1360,1805,1444
1322,1516,1703,1672,1591,1311,1185,1558
mlv_lite version : ML/MODULES/RAWBENCH/APR27.MO
965,1490,1459,1667,1441,1234,1315,1481
1470,1644,1536,1536,1728,2007,1773,1777
mlv_lite version : ML/MODULES/RAWBENCH/3F48DF.MO
1349,1783,1441,1282,1444,1288,1624,1650
1083,1368,1543,1273,1498,1680,1615,1644
1402,1498,1477,1384,1211,1781,1110,1460
3153,3211,3245,3260,3223,3043,3027,3162
945,1436,1577,1595,2027,1466,1576,1322
mlv_lite version : ML/MODULES/RAWBENCH/E67FAC.MO
627,888,1233,825,710,644,1197,1098
531,721,833,1026,1357,1083,908,1784
544,1034,1056,1509,1380,1109,928,1157
892,956,985,1057,1270,1214,1147,1099
1084,1252,1087,746,901,1042,1025,970
1005,1235,1249,1076,1224,1305,1256,1384
889,967,909,1542,839,1152,517,1313
672,935,1002,1345,1249,1339,1327,1593
mlv_lite version : ML/MODULES/RAWBENCH/APR21.MO
1346,1688,1683,1588,2244,1988,2444,1745
1154,1398,1545,1725,1510,1630,1555,1337
3246,3107,3236,3153,3108,3189,3275,3145
mlv_lite version : ML/MODULES/RAWBENCH/APR14.MO
678,1752,862,1520,1394,1357,1025,1085
1047,1553,1034,1687,1284,1459,941,1520
1526,1709,1421,1398,1444,1701,1766,1691
1252,1777,1760,1637,1634,1483,1654,1406
695,1481,1520,1433,1814,1492,1360,1339
mlv_lite version : ML/MODULES/RAWBENCH/APR29.MO
895,1055,1099,1323,1362,1265,1261,1646
3143,3092,3185,3055,3208,3243,3207,3206
1066,1006,1230,1388,1078,777,1045,1198
861,1208,964,1137,968,1297,1002,1457
763,1032,1173,1192,1227,1242,1485,1456
Quartile stats: median (Q1 ... Q3)
mlv_lite version : ML/MODULES/RAWBENCH/E67FAC.MO: 1067 frames (905 ... 1249)
mlv_lite version : ML/MODULES/RAWBENCH/E93203.MO: 1130 frames (1005 ... 1361)
mlv_lite version : ML/MODULES/RAWBENCH/APR29.MO: 1229 frames (1050 ... 1471)
mlv_lite version : ML/MODULES/RAWBENCH/B420A8.MO: 1442 frames (1317 ... 1594)
mlv_lite version : ML/MODULES/RAWBENCH/APR14.MO: 1470 frames (1312 ... 1646)
mlv_lite version : ML/MODULES/RAWBENCH/APR23.MO: 1482 frames (1303 ... 1728)
mlv_lite version : ML/MODULES/RAWBENCH/APR27.MO: 1513 frames (1450 ... 1698)
mlv_lite version : ML/MODULES/RAWBENCH/3F48DF.MO: 1521 frames (1376 ... 1782)
mlv_lite version : ML/MODULES/RAWBENCH/APR21.MO: 1735 frames (1550 ... 3127)
P =
Columns 1 through 5:
1 0.20463 2.3393e-05 4.2457e-09 7.9956e-09
0.20463 1 0.063283 0.00020993 0.003231
2.3393e-05 0.063283 1 0.55111 0.29445
4.2457e-09 0.00020993 0.55111 1 0.53456
7.9956e-09 0.003231 0.29445 0.53456 1
3.5275e-10 0.0021903 0.26451 0.053254 0.0069184
8.3596e-09 0.00014102 0.88954 0.31542 0.16273
2.648e-11 0.00093461 0.16911 0.025252 0.0021659
4.4409e-16 1.3427e-05 0.0054545 8.9505e-05 7.4696e-07
Columns 6 through 9:
3.5275e-10 8.3596e-09 2.648e-11 5.2588e-16
0.0021903 0.00014102 0.00093461 1.3427e-05
0.26451 0.88954 0.16911 0.0054545
0.053254 0.31542 0.025252 8.9505e-05
0.0069184 0.16273 0.0021659 7.4696e-07
1 0.23071 0.7823 0.038059
0.23071 1 0.14709 0.0032367
0.7823 0.14709 1 0.061734
0.038059 0.0032367 0.061734 1
T =
0 -1.2792 -4.4339 -6.5382 -6.2902 -6.9502 -6.4563 -7.4827 -9.9692
1.2792 0 -1.8962 -4.0996 -3.0823 -3.2171 -4.3595 -3.5022 -4.9964
4.4339 1.8962 0 0.59938 1.0555 -1.1239 0.13954 -1.3879 -2.8799
6.5382 4.0996 -0.59938 0 0.62454 -1.9705 -1.0173 -2.293 -4.2946
6.2902 3.0823 -1.0555 -0.62454 0 -2.7744 -1.4153 -3.1719 -5.5061
6.9502 3.2171 1.1239 1.9705 2.7744 0 1.2122 -0.27728 -2.1195
6.4563 4.3595 -0.13954 1.0173 1.4153 -1.2122 0 -1.471 -3.1431
7.4827 3.5022 1.3879 2.293 3.1719 0.27728 1.471 0 -1.9026
9.9692 4.9964 2.8799 4.2946 5.5061 2.1195 3.1431 1.9026 0
D =
126 78 102 86 102 102 78 102 86
78 30 54 38 54 54 30 54 38
102 54 78 62 78 78 54 78 62
86 38 62 46 62 62 38 62 46
102 54 78 62 78 78 54 78 62
102 54 78 62 78 78 54 78 62
78 30 54 38 54 54 30 54 38
102 54 78 62 78 78 54 78 62
86 38 62 46 62 62 38 62 46
It looks like the first 3 builds (in the sorted list) have the regression; that means, it must have been introduced in the earliest changeset from this group (since the previous changeset was also tested): e67fac (https://bitbucket.org/hudson/magic-lantern/commits/e67facae488444a132aaa6107de0e3eb4e3129eb?at=crop_rec_4k).
That means:
- The regression
only affects the behavior when recording is about to stop (how many frames it manages to squeeze in the last moments)
-
It does not affect the ability to record continuously. This piece of code starts kicking in when the buffer is about 90% full.
Is the regression caused the overhead of printf (possibly delaying the decision by a non-negligible amount of time) or it's because of the change in thresholds (resulting in smaller blocks being written to the card)? Need to restart the experiment to find out.
To interpret the statistical tests, check Frank7D's posts (http://www.magiclantern.fm/forum/index.php?topic=17091.msg166137#msg166137) and the t_test_2 (https://octave.sourceforge.io/octave/function/t_test_2.html) function description.
I cannot find information about missing mlv sound recording missing in the experimental branch - why I cannot record sound ?
Experimental page -> Notes:
Quotemlv_lite only for now
Features page -> Modules -> mlv_lite.mo -> (link) ->
QuoteThere are some caveats to what you get compared to the full mlv_rec:
[...]
2. No audio
to make mlv_snd work with mlv_lite, the buffering structure must be changed to allow other modules to queue blocks.
that are only a few callbacks, but this might have a negative feedback on write speed.
currently mlv_lite is meant to be a rewrite from scratch to get the maximum write speed.
when all experimental stuff settles, maybe this will be added.
for now just let things settle.
@A1ex
Thank you very much for the explanation and for the tests!!!!!!!!!
I will continue testing.
dear devs, is it possible to make at least preview pictures of footages? I know now you try to make h264 proxy possible, but may it be more wise to make preview pictures option and then h264 proxy?
And please, is it possible to tag processed files in RawFlow not just frame001, frame002 etc. (nightmare for postproduction) but rather MLVfileName_001, MLVfileName_002 etc?
Previews are there depending on your NLE. I see thumbnails on mac just fine.
You can create any name you want in mlv_dump.
Try this for instance:
mlv_dump --dng -o MLVfileName_0000_C0000_ INPUT.MLV
just following :)
Quote from: extremelypoorfilmaker on May 03, 2017, 06:24:28 PM
just following :)
still waiting for the day Alex will explode because of people asking dump questions, same questions again and again or both compared into 1...
;D :) ;)
ps: not yours.
I LOVE mlv lite sooooo much! It's perfect. Sound does not matter to me. Just use a slate and a zoom and you'll be found. Much more like film!
Quote from: Danne on May 03, 2017, 03:24:49 PM
Previews are there depending on your NLE. I see thumbnails on mac just fine.
You can create any name you want in mlv_dump.
Try this for instance:
mlv_dump --dng -o MLVfileName_0000_C0000_ INPUT.MLV
on previews: but what if I have Windows(8)?
on file names: It's in mlv_dump.exe? How should I tape this code if it's window dissapeared almost instantly?
@Savely
In the directory where MLV_Dump.exe lies, make a .bat file and put this in it:
@echo off
for %%a in ( %* ) do ( md "%%~na" 2>nul )&( "mlv_dump.exe" -o %%~na\%%~na_ --dng %%~na.MLV )
Drag and Drop the MLV files onto the .bat file.
It will generate Folders with the frames
Quote from: Savely on May 04, 2017, 05:50:41 AM
on previews: but what if I have Windows(8)?
Take a look @ this: http://www.visualbakery.com/Tools/PhotoTools.aspx (http://www.visualbakery.com/Tools/PhotoTools.aspx)
Quote from: Quentin on May 04, 2017, 06:15:29 AM
@Savely
In the directory where MLV_Dump.exe lies, make a .bat file and put this in it:
@echo off
for %%a in ( %* ) do ( md "%%~na" 2>nul )&( "mlv_dump.exe" -o %%~na\%%~na_ --dng %%~na.MLV )
Drag and Drop the MLV files onto the .bat file.
It will generate Folders with the frames
Thanks man, it works! But you'll have to put mlv file directly into the folder where this .bat file is.
Its a harmless compromise
a good basic interface, resolve all the problems,and speed all the multi clips to convert
This test was moderately helpful. I wanted to see how the 3k and HD compared with a high ISO -- 1600 (high for a 5Diii, at least). Three different lenses all shot at f2.8 at 1600 ISO. The field of view for ML cropped 3k is about 1/2 of what it is for full frame HD. So for the HD version of the shot, the camera was moved from 48" away to 24" away. I down-rezzed the 3k to HD.
I didn't have the patience to get the scale exact. I kept image pretty neutral. I used ETTR for exposure. I think it helped lessen the shadow noise (I tend to underexpose ML RAW, not in a good way). Anyway, the noise isn't too awful especially since you'd most likely crush the blacks some if you were using this shot. I was expecting more noise at 1600 ISO.
https://youtu.be/qWsWEl7fb4A (https://youtu.be/qWsWEl7fb4A)
Hey guys, just curious. I shot some footage, about 4 takes of a local artist and its about 114gb. 3k 12bit lossless 3.5k centered mode and it all went very well. It flashed between orange and green but I also had the global draw on. Camera temp hit orange a few times but it all worked great and it is super sharp!!
My question has to do with MLV_dump.exe, I left it overnight to process the DNGs but I think it got stuck because it never got past the first set of mlv files. I had selected all the time, 32 in total. I am re-attempting to do them one clip at a time now, but I'm curious if there is limit on GB size or amount of files I can run at once.
Is this normal? Does it need to process the images more than once? Thanks for all your help guys!
Quote
MLV Dumper v1.0
-----------------
Mode of operation:
- Input MLV file: 'M02-2034.MLV'
- Enforcing 14bpp for DNG output
- Convert to DNG frames
- Output into 'M02-2034\M02-2034_'
File M02-2034.MLV opened
File M02-2034.M00 opened
File M02-2034.M01 opened
File M02-2034.M02 opened
File M02-2034.M03 opened
File M02-2034.M04 opened
File M02-2034.M05 opened
File M02-2034.M06 not existing.
Processing...
Vertical stripes correction:
1.00000 1.00781 1.00792 1.00864 1.00664 1.00841 1.00572 1.00777
Cold pixels : 36
Reached end of chunk 1/7 after 1235 blocks
Reached end of chunk 2/7 after 1232 blocks
Reached end of chunk 3/7 after 1230 blocks
Reached end of chunk 4/7 after 1232 blocks
Reached end of chunk 5/7 after 1237 blocks
Reached end of chunk 6/7 after 1238 blocks
Reached end of chunk 7/7 after 436 blocks
Processed 7776 video frames
Done
MLV Dumper v1.0
-----------------
Mode of operation:
- Input MLV file: 'M02-2034.MLV'
- Enforcing 14bpp for DNG output
- Convert to DNG frames
- Output into 'M02-2034\M02-2034_'
File M02-2034.MLV opened
File M02-2034.M00 opened
File M02-2034.M01 opened
File M02-2034.M02 opened
File M02-2034.M03 opened
File M02-2034.M04 opened
File M02-2034.M05 opened
File M02-2034.M06 not existing.
Processing...
Vertical stripes correction:
1.00000 1.00781 1.00792 1.00864 1.00664 1.00841 1.00572 1.00777
Cold pixels : 36
EDIT: It seems to be on an endless cycle just repeating and wont move on to the next clip. Even if I only select the mlv files for one clip its just in an endless loop.
Hmm, I've been using mlv-dump (and the dumper batch) a bunch lately and haven't had any issues. Do you have the latest version of mlv_dump.exe? And are you sure you have enough hard disk space?
Quote from: pocketrubbish on May 05, 2017, 05:39:01 AM
Hmm, I've been using mlv-dump (and the dumper batch) a bunch lately and haven't had any issues. Do you have the latest version of mlv_dump.exe? And are you sure you have enough hard disk space?
Yeah I am using the latest dump, and I have plenty of HD space. All the files get extracted, I can stop mlv-dump and my files are there for 1 clip. Ive been using it fine for the past week or so this just occurred the past few days. I havent changed anything so I'm not sure whats going on.
Quote from: a1ex on May 03, 2017, 10:14:19 AM
Is the regression caused the overhead of printf (possibly delaying the decision by a non-negligible amount of time) or it's because of the change in thresholds (resulting in smaller blocks being written to the card)?
Results of over 30 hours of benchmarking (Apr29 codebase with minor changes - included in the raw log - to check the effect of these changes on performance). During this time, the camera was under the control of this script (https://bitbucket.org/hudson/magic-lantern/src/raw_benchmark/scripts/extra/rawbench.lua), unattended.
mlv_lite version : ML/MODULES/RAWBENCH/ASSUMEFS.MO: 865 frames (772 ... 925)
mlv_lite version : ML/MODULES/RAWBENCH/APR29.MO: 866 frames (789 ... 900)
mlv_lite version : ML/MODULES/RAWBENCH/NOPRINTF.MO: 874 frames (795 ... 902)
mlv_lite version : ML/MODULES/RAWBENCH/ASSUME70.MO: 885 frames (840 ... 956)
mlv_lite version : ML/MODULES/RAWBENCH/ASSUME80.MO: 938 frames (866 ... 984)
mlv_lite version : ML/MODULES/RAWBENCH/ASSUME90.MO: 946 frames (846 ... 986)
The effect of printf is not significant (could
not prove the version without printf is any faster):
Quartile stats: median (Q1 ... Q3)
mlv_lite version : ML/MODULES/RAWBENCH/APR29.MO: 866 frames (789 ... 900)
mlv_lite version : ML/MODULES/RAWBENCH/NOPRINTF.MO: 874 frames (795 ... 902)
P =
1 0.63117
0.63117 1
T =
0 -0.48056
0.48056 0
But, the assumed write speed (when deciding to limit the number of frames saved in a single file write call, to avoid running out of space during the call) appears to have a sweet spot between 80 and 90%:
Quartile stats: median (Q1 ... Q3)
mlv_lite version : ML/MODULES/RAWBENCH/ASSUMEFS.MO: 865 frames (772 ... 925) ; ASSUME100
mlv_lite version : ML/MODULES/RAWBENCH/NOPRINTF.MO: 874 frames (795 ... 902) ; ASSUME50
mlv_lite version : ML/MODULES/RAWBENCH/ASSUME70.MO: 885 frames (840 ... 956)
mlv_lite version : ML/MODULES/RAWBENCH/ASSUME80.MO: 938 frames (866 ... 984)
mlv_lite version : ML/MODULES/RAWBENCH/ASSUME90.MO: 946 frames (846 ... 986)
P =
Columns 1 through 4:
1 0.4984 6.2291e-07 7.7958e-10
0.4984 1 6.194e-05 7.2981e-07
6.2291e-07 6.194e-05 1 0.06076
7.7958e-10 7.2981e-07 0.06076 1
1.7948e-08 4.1919e-06 0.11062 0.86482
Column 5:
1.7948e-08
4.1919e-06
0.11062
0.86482
1
T =
0 -0.67773 -5.0558 -6.2829 -5.7438
0.67773 0 -4.0546 -5.0451 -4.6852
5.0558 4.0546 0 -1.8798 -1.5988
6.2829 5.0451 1.8798 0 0.17034
5.7438 4.6852 1.5988 -0.17034 0
So, while the first place (which assumes 90% of the measured speed for the last file write calls) is clearly better (squeezes more frames) compared to last two places (which assume either 100% or 50% of the measured speed for the same calls), we cannot tell the same about the first two winners - they are tied (could not prove one is faster than the other). Same for the last two places. The sweet spot is probably somewhere near 85% (not tested, just guessed).
(http://a1ex.magiclantern.fm/bleeding-edge/raw/ovf-sweetspot.png)
However, the best thing to do for speed is to write blocks as large as we can (in other words, as few file write calls as we can), and free the buffers as soon as we know they have been written. Here's an experiment on this:
Quartile stats: median (Q1 ... Q3)
mlv_lite version : ML/MODULES/RAWBENCH/ASSUME80.MO: 938 frames (866 ... 984)
mlv_lite version : ML/MODULES/RAWBENCH/ASSUME90.MO: 946 frames (846 ... 986)
mlv_lite version : ML/MODULES/RAWBENCH/SPECFREE.MO: 1015 frames (921 ... 1090)
P =
1 0.86482 3.7192e-14
0.86482 1 2.8011e-13
3.7303e-14 2.8022e-13 1
T =
0 0.17034 -7.8429
-0.17034 0 -7.5833
7.8429 7.5833 0
With large blocks, and reusing image buffers *during* a file write call, performance is clearly better (possibly also helping with overall speed, as the large buffers are freed and can be reused quicker).
(this experiment is a litte risky in my opinion, as I have not tested it for data integrity, other than self-checks built in mlv_lite, so I'm not comfortable including it in the builds,
but you can build it from source - diffs available in the raw log)
Number of frames recorded by each build:
mlv_lite version : ML/MODULES/RAWBENCH/ASSUME70.MO
832,1060,989,959,869,953,898,895
741,837,839,827,921,767,905,985
732,703,836,944,954,949,830,800
739,864,926,961,956,949,866,866
840,983,870,747,856,848,972,896
762,870,987,853,978,953,876,852
647,880,876,917,984,895,1002,886
555,871,966,919,882,1028,976,791
651,939,852,868,1063,863,872,925
737,977,989,974,958,973,855,846
756,871,974,978,969,994,953,791
849,860,1023,974,997,979,993,943
765,758,964,1004,1013,942,986,824
823,835,864,955,969,832,847,877
744,997,844,839,1039,951,941,880
773,884,904,985,974,619,833,778
850,1018,947,857,942,868,1011,901
637,990,723,930,757,837,806,872
756,952,977,856,965,950,956,883
647,870,828,946,938,868,832,879
750,904,944,942,863,976,872,843
626,760,986,975,937,755,838,986
540,851,764,874,1036,1048,1052,836
655,856,878,747,928,763,843,905
626,829,953,912,966,822,997,952
804,893,884,840,902,939,933,937
754,943,905,988,874,962,897,952
885,966,885,971,950,984,872,874
843,955,848,887,871,953,861,950
mlv_lite version : ML/MODULES/RAWBENCH/ASSUME90.MO
665,1025,986,985,1000,882,993,698
683,992,993,918,946,1026,906,781
463,854,720,907,982,1040,1052,834
828,777,806,956,1001,967,879,800
555,851,876,859,807,986,881,944
589,986,997,979,962,1003,918,856
762,961,945,888,916,966,998,866
689,906,970,975,826,975,684,972
474,898,982,1008,982,982,990,978
582,889,963,1010,1007,1044,994,765
782,904,978,964,1002,965,841,928
710,966,1018,802,989,903,981,952
882,992,974,918,1013,961,1018,1039
762,922,963,1009,1024,961,971,970
765,903,922,948,1011,975,903,874
679,917,904,992,965,916,962,985
876,902,805,865,874,761,727,966
745,981,1061,890,737,1002,1095,783
831,875,837,992,711,940,1031,975
581,651,976,901,1055,1005,922,901
659,946,978,971,997,914,816,954
773,989,968,979,987,798,1010,964
858,841,685,793,993,812,987,980
654,997,983,891,1036,918,997,899
mlv_lite version : ML/MODULES/RAWBENCH/NOPRINTF.MO
572,818,859,843,691,827,898,805
880,920,893,799,896,794,826,981
781,973,995,916,877,777,874,893
776,891,907,896,799,914,894,697
768,796,918,890,902,715,810,792
680,824,899,901,916,860,896,718
466,641,900,848,907,836,870,959
678,576,891,917,698,889,921,813
862,850,797,943,871,892,969,882
801,879,898,898,779,905,936,768
581,930,916,1008,892,872,895,908
440,720,919,896,890,906,831,1016
571,797,883,861,985,875,845,793
686,784,906,891,877,974,869,742
772,874,857,916,968,775,801,786
mlv_lite version : ML/MODULES/RAWBENCH/ASSUMEFS.MO
690,1040,864,754,879,861,935,972
523,728,833,969,935,944,971,1054
404,819,957,836,778,845,881,514
796,886,779,776,917,957,892,760
361,763,1003,597,929,743,907,983
758,868,875,859,971,979,999,934
848,803,794,777,883,861,669,681
612,746,879,904,774,942,972,950
572,789,555,769,862,786,918,823
534,792,847,859,865,910,980,738
838,935,980,915,850,879,670,785
561,863,954,780,809,875,940,907
795,964,1003,891,955,870,636,813
460,733,980,974,1037,1072,874,801
873,908,937,838,855,801,909,891
753,834,877,881,948,910,935,1050
856,874,941,764,873,920,585,884
666,638,709,947,857,947,884,906
576,967,818,985,660,798,774,1001
711,889,799,900,869,837,810,836
667,790,886,871,798,872,837,886
602,887,922,721,929,706,949,936
358,882,724,824,882,689,876,700
438,693,928,789,860,888,869,867
606,826,748,782,870,908,877,858
551,523,933,760,930,711,911,965
674,847,732,957,958,765,888,957
665,731,881,1012,986,933,914,995
mlv_lite version : ML/MODULES/RAWBENCH/APR29.MO
532,631,813,903,892,905,876,1030
768,715,803,718,949,826,893,892
807,977,930,799,889,821,887,775
410,803,852,984,603,777,900,883
653,802,984,707,774,889,858,815
703,899,707,866,793,825,841,987
696,987,926,883,966,896,869,945
766,900,887,789,867,788,890,897
801,860,969,972,839,840,909,890
682,906,793,897,894,915,911,911
509,756,807,811,945,1003,984,780
678,808,716,809,806,827,925,932
501,770,852,905,951,878,632,887
737,899,870,981,959,939,888,798
662,886,919,760,782,684,808,947
664,884,719,685,882,804,822,884
861,824,795,693,991,868,814,888
686,886,895,880,870,712,914,861
662,888,804,828,910,966,947,817
667,820,961,784,904,842,900,884
561,795,873,784,949,739,756,768
557,896,880,804,903,921,802,911
695,865,905,863,885,911,865,885
670,972,868,908,850,877,868,881
mlv_lite version : ML/MODULES/RAWBENCH/ASSUME80.MO
490,483,716,909,1020,996,991,814
663,982,999,925,983,1004,974,898
890,923,822,1015,879,987,857,882
911,984,960,971,952,964,974,674
847,932,978,911,1003,985,980,998
783,986,984,925,981,955,995,871
872,970,871,970,788,913,1027,1086
678,872,873,843,980,980,995,894
766,919,971,758,806,813,1002,933
657,891,969,974,939,968,1023,963
650,996,989,995,975,984,982,899
780,703,997,1014,936,937,881,834
760,955,966,772,989,980,866,990
926,939,972,1013,981,960,1010,767
676,1014,925,987,1007,897,908,763
667,1027,801,890,985,767,1001,797
758,912,866,1015,876,976,891,770
466,910,772,941,981,755,979,742
889,973,941,1014,969,968,667,756
640,908,958,994,972,977,882,888
793,889,986,986,1004,1000,983,1069
752,991,1029,868,884,987,906,784
630,881,879,982,955,926,836,841
517,772,922,1023,1034,852,898,985
856,943,835,885,805,992,971,1006
684,952,975,1015,863,987,996,892
561,975,890,978,985,1019,904,975
559,954,909,922,902,1124,982,929
783,1019,935,1035,971,996,884,962
mlv_lite version : ML/MODULES/RAWBENCH/SPECFREE.MO
973,1083,1029,908,1121,1011,1110,718
794,1132,1023,853,908,1098,1114,1004
791,1066,899,920,958,1086,1197,811
857,1135,1034,1097,996,937,1006,921
819,1066,970,995,1063,1021,1113,1078
641,1095,1076,992,996,1078,1015,892
932,1111,1003,1109,1072,974,1007,1084
796,1009,1098,1114,1121,1000,717,881
910,928,1012,917,1055,1085,1073,820
617,970,1141,996,1031,1102,1022,1012
790,1010,1072,1017,1023,1043,1034,922
910,1140,920,1106,1116,1126,989,919
791,772,1016,1024,1110,1123,1096,832
802,1013,961,1097,1120,1057,1103,971
754,984,1081,1078,1176,907,908,602
848,1100,1115,1089,1111,1102,908,935
1016,1106,1017,1013,1104,1045,1007,982
713,1057,987,997,1037,954,1095,925
992,1101,1084,1076,1068,1024,1095,920
790,911,1128,1121,1096,1020,1010,600
808,938,1096,978,1042,919,1099,910
837,982,1100,1077,1080,1015,1039,917
769,1006,1083,1091,1102,1090,915,981
Raw log (http://a1ex.magiclantern.fm/bleeding-edge/raw/RAWBENCH-4K.LOG) (includes diffs)
Thank you very much for the test...
If somebody can compile the build that A1ex suggest, I will test it.
Does anyone have tried Dual ISO in crop_mode? It seems there are problems there. Frames (after Dual ISO Processor) comes with stripes like unprocessed dual ISO frames. At least with x5 (x3) zoom. Is there some solution?
Could you post a sample file?
Actually it comes with stripes with, and with no zoom. Here it is:
zoom: https://yadi.sk/i/9R1QvSSJ3Hm4HD
no zoom: https://yadi.sk/i/ZVhz4gHe3Hm3Nn
download button (russian interface)http://joxi.ru/J2bJWXZFXLXed2.jpg
Ok, trying to understand what to look for.
The non crop file seems ok. After running the file through cr2hdr I get this.
(https://s18.postimg.org/72li7s4jt/Screen_Shot_2017-05-06_at_16.46.00.png)
QuoteQuoteDoes anyone have tried Dual ISO in crop_mode? It seems there are problems there. Frames (after Dual ISO Processor) comes with stripes like unprocessed dual ISO frames. At least with x5 (x3) zoom. Is there some solution?
Checking your file
frame_000008frame_000024_crop_mode_with_zoom.dng it doesn´t look healthy. So you´re problems are only the crop mode recording with compressed 4k modes? If yes, could you share a small MLV sample instead of a single dng file?
Quote from: Danne on May 06, 2017, 04:51:09 PM
Ok, trying to understand what to look for.
The non crop file seems ok. After running the file through cr2hdr I get this.
(https://s18.postimg.org/72li7s4jt/Screen_Shot_2017-05-06_at_16.46.00.png)
Checking your file frame_000008frame_000024_crop_mode_with_zoom.dng it doesn´t look healthy. So you´re problems are only the crop mode recording with compressed 4k modes? If yes, could you share a small MLV sample instead of a single dng file?
Try to zoom it. Non crop I mean. And how did you running it through cr2hdr? I tried but it just flashing and nothing.
Thid mlv file with no zoom. https://yadi.sk/d/7nw0j1Fq3HmYpN (with zoom pretty big for half Gb mininum). As I see, problem with stripes is in unzoomed DNG's (RAW Dual ISO on crop_mode build just fine as well as Dual ISO MLV's on older builds).
The mlv file you just posted @savely converts just fine.
(https://s2.postimg.org/5d0vpzqrd/Screen_Shot_2017-05-06_at_21.04.29.png)
I´m on mac and using this. It´s a1ex cr2hdr binary working.
http://www.magiclantern.fm/forum/index.php?topic=15108.0
*update 3
Pull request made here
https://bitbucket.org/Dannephoto/magic-lantern/commits/642fb8c4d5bd7bd8169bc9b85bae11eb27495892
I´m already a big fan of the crop_rec4k mlv_dump version and the compression of dng and MLV files are great. Encountered an issue with eosm footage. Files won´t parse(open) in acr and dcraw reveals what´s wrong. Corrupted file. Other footage from eosm is working.
Testfile:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/M20-1113_samples_Issue_parsing.zip
dcraw output and terminal message:
Unexpected end of file
(https://s29.postimg.org/r72sssd6v/Screen_Shot_2017-05-07_at_20.27.28.png)
*update.
Maybe metadata related. Other random ml builds is working with eosm. Guess I was lucky with this corrupted footage 8). Wish I knew what build I was using when filming. Have a bunch of files om my computer which comes out like this. The transcode just fine with other builds of mlv_dump.
*update 2
It seems it´s because of being 10 or 12 bit footage. Guess that´s not working at the moment with the crop_rec4k mlv_dump build.
Quote from: Danne on May 06, 2017, 09:04:42 PM
The mlv file you just posted @savely converts just fine.
(https://s2.postimg.org/5d0vpzqrd/Screen_Shot_2017-05-06_at_21.04.29.png)
I´m on mac and using this. It´s a1ex cr2hdr binary working.
http://www.magiclantern.fm/forum/index.php?topic=15108.0
Can you please upload your "fine" result? Cause in crop mode build with Dual ISO I always end up with striped pics or this pattern:(http://joxi.ru/brRd9nNFJdY4z2.jpg)
I just shot a 3x crop of a hummingbird on a perch with a 70-200mm lens. This is just insane detail. 3072x1228 12bit lossless.
Download the movie and watch it in ProRes. Literally moving photographs.
https://www.dropbox.com/s/0vm8zsabe39vvf9/Humming3072.mov?dl=0
Quote from: giarcpnw on May 08, 2017, 07:03:33 AM
I just shot a 3x crop of a hummingbird on a perch with a 70-200mm lens. This is just insane detail. 3072x1228 12bit lossless.
Download the movie and watch it in ProRes. Literally moving photographs.
https://www.dropbox.com/s/0vm8zsabe39vvf9/Humming3072.mov?dl=0
wow. Can I ask a favor? Would you mind sharing a single dng frame of that hummingbird that I can look at in davinci? That's beautiful.
Quote from: ted ramasola on May 08, 2017, 08:03:34 AM
wow. Can I ask a favor? Would you mind sharing a single dng frame of that hummingbird that I can look at in davinci? That's beautiful.
Here ya go.
https://www.dropbox.com/s/ebxx9xrebn04451/M07-1434_C0000_00697.dng?dl=0
Quote from: giarcpnw on May 08, 2017, 05:30:06 PM
Here ya go.
https://www.dropbox.com/s/ebxx9xrebn04451/M07-1434_C0000_00697.dng?dl=0
Thank you, the image can still be zoomed to fill an UHD timeline in resolve. Nice.
Quote from: ted ramasola on May 08, 2017, 10:15:44 PM
Thank you, the image can still be zoomed to fill an UHD timeline in resolve. Nice.
I'm certain I can get 3520x1320. Not sure that offers a 3x crop option though, I just didn't dive into the settings that hard since i was trying to capture the little guy before he took off.
Quote from: giarcpnw on May 08, 2017, 10:31:57 PM
I'm certain I can get 3520x1320. Not sure that offers a 3x crop option though, I just didn't dive into the settings that hard since i was trying to capture the little guy before he took off.
3520x1320 works for me but only up to maybe 100 frames (KomputerBay 128gb 1066X CF card). But I have only tried it in the 14-bit lossless mode, 24fps. Is anyone getting longer recordings using 12-bit lossless instead?
Great looking footage, by the way!
I was getting continuous in 12bit...
all this...most of these scenes were in the 20-30 second range before I cut them shorter. I was using KomputerBay 64gig 1000X
https://www.youtube.com/watch?v=ZdvKALy58Es
I realized after posting yesterday that 12-bit lossless and 10-bit lossless aren't enabled yet for the 3K/4K crop modes anyway. So 3520x1320 was continuous for you in 12-bit uncompressed? I'll have to try that out! I just assumed the larger bitrate would be prohibitive, but maybe it's something else that inhibits the 14-bit lossless from doing it continuously at that res?
I'm not sure these are accurate rec times mentioned above. Possibly I guess but as far as my testing has shown, 3072 2.35:1 23.976fps override on the 5x mode in the crop rec menu @12bit lossless are continuous. Even with these settings there could be scenes that can spike the data rate so that it is not continuous. Above that resolution, I don't THINK you will see continuous. Remember, if you are using the 5x option under crop rec menu, you have to hit the zoom button to actually rec the resolution. If you don't it will just rec 1920. I'm thinking maybe that's where some confusion can happen. If your not zoomed in the 5x option in the crop mod you will not be recording higher then 1920 resolution. Also if you are using the 5x mode you will need to enable fps override since canon live view defaults to 30fps when zoomed in.
I shot 3520x1320 12bit continuous in the 5x centered mode continuous for 2 minutes last night. It made a 10gig (ish) mlv file but it did it. It's definitely doable.
Oh I didn't realize you were using 2.66:1 ratio @giarcpnw
And you mean 12bit lossless right?
Also did you hit the zoom button before recording?
If you are in 12bit lossless mode Try to ETTR really branchy tree tops with no highlight clipping on an over cast day and I bet your data rate will skyrocket with those settings. It did for me anyway.
Quote from: hjfilmspeed on May 09, 2017, 11:29:17 PM
Oh I didn't realize you were using 2.66:1 ratio @giarcpnw
And you mean 12bit lossless right?
Also did you hit the zoom button before recording?
If you are in 12bit lossless mode Try to ETTR really branchy tree tops with no highlight clipping on an over cast day and I bet your data rate will skyrocket with those settings. It did for me anyway.
Yeah sorry. lossless. Always lossless. Can't seem to get past 3072 if 14bit lossless. I've not seen a big diff between 14 and 12 bit to try to go higher.
Recording @ 3.5K 12bit Lossless is really tricky it all depends on your scene. 3072 - 12bit Lossless is working lovely.. Live View stay's full color and I'm able to focus properly. Not sure if anybody experienced spiked data rates using the Sigma 35mm ART lens, or any of the ART lens. I'm thinking that because those lens are soo sharp it's causing that issue.
@goldenchild9to5
Do you get real time liveview with 3072 - 12 bit lossless?. Full color but no realtime, isn't it?
Thanks.
@pc_bel no it's not realtime but I'm getting use to it.
@pc_bel, @goldenchild9to5:
Have a look at the preview options; the help text has some hints for your use cases (also discussed earlier in this thread (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183548#msg183548)).
Today I had crash during record:
CRASH00.LOG
ASSERT: IsSuiteSignature( hSuite )
at ./PackMemory/PackMem.c:983, task MoviePlay
lv:0 mode:3
Magic Lantern version : crop_rec_4k.2017Apr29.5D3123
Mercurial changeset : d55423f1e270 (crop_rec_4k) tip
Built on 2017-04-29 14:31:28 UTC by jenkins@nightly.
Free Memory : 164K + 3811K
ASSERT00.LOG
ML ASSERT:
fullsize_buffers[1] == UNCACHEABLE(raw_info.buffer)
at mlv_lite.c:1379 (free_buffers), task raw_rec_task
lv:1 mode:3
Magic Lantern version : crop_rec_4k.2017Apr29.5D3123
Mercurial changeset : d55423f1e270 (crop_rec_4k) tip
Built on 2017-04-29 14:31:28 UTC by jenkins@nightly.
Free Memory : 144K + 3821K
Info
I used H.264 proxy (H264->SD with ML; RAW 1920x1080 12-bit->CF)
Magic Yoom Large TL 3>1, Waveform small, Red Crossout, DIGIC peaking: Slightly sharper, Zoom with old button..
Quote from: a1ex on May 10, 2017, 10:07:47 PM
@pc_bel, @goldenchild9to5:
Have a look at the preview options; the help text has some hints for your use cases (also discussed earlier in this thread (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183548#msg183548)).
@a1ex this feature changed everything! So awesome!
Some fun stuff w DarkFraming Avg Process in 12-bit lossless @ ISO 6400 from 5D3.123 shot w Helios 44-2.
Cinelog_Rec709 has been applied for comparison viewing purpose. Full album can be found here: https://flic.kr/s/aHskVk85io
Pre DF (pushed exposure up to +2.50 in ACR):
(https://c1.staticflickr.com/5/4158/34450561991_2c584c7d1a.jpg) (https://flic.kr/p/Uuh8QK)
Post DF (pushed exposure up to +2.50 in ACR):
(https://c1.staticflickr.com/5/4168/33738123384_3ddba92881.jpg) (https://flic.kr/p/TpjGEu)
Pre DF (pushed exposure up to +2.50 in ACR):
(https://c1.staticflickr.com/5/4174/34450578201_a8ffa6712b.jpg) (https://flic.kr/p/UuhdEe)
Post DF (pushed exposure up to +2.50 in ACR):
(https://c1.staticflickr.com/5/4161/34450549851_0bec0a5d1f.jpg) (https://flic.kr/p/Uuh5er)
Pre DF (pushed exposure up to +2.50 in ACR):
(https://c1.staticflickr.com/5/4179/34450580751_e333e68295.jpg) (https://flic.kr/p/Uuheqc)
Post DF (pushed exposure up to +2.50 in ACR):
(https://c1.staticflickr.com/5/4183/33738094614_51049bfdb3.jpg) (https://flic.kr/p/Tpjy7s)
Pre DF (pushed exposure up to +2.50 in ACR):
(https://c1.staticflickr.com/5/4169/33738115394_d00f1a6282.jpg) (https://flic.kr/p/TpjEhJ)
Post DF (pushed exposure up to +2.50 in ACR):
(https://c1.staticflickr.com/5/4194/34194921370_04b217357d.jpg) (https://flic.kr/p/U6FUMQ)
Such a relief that @bouncyball, @Danne, @g3gg0 and the rest of you all have made this possible to bring back this useful feature in DarkFraming Average Processing. Especially for lossless compressions.
Yes, all of this was solely done with the latest and greatest cr2hdr.app (https://www.magiclantern.fm/forum/index.php?topic=15108.0cr2hdr.app) (Thanks @Danne for standing by with all of your love & support!)
Looking good DeafEyeJedi! Do you create the darkframe by shooting (with the same settings) with the lenscap attached? Can you do it anytime after the shot (like when i go on holiday and i make the darkframes when i come back home) or does it have to be close together in time? For windows users mlv_dump can do this right?
Quote (with pics removed):
Quote from: DeafEyeJedi on May 11, 2017, 07:54:00 AM
Some fun stuff w DarkFraming Avg Process in 12-bit lossless @ ISO 6400 from 5D3.123 shot w Helios 44-2.
Cinelog_Rec709 has been applied for comparison viewing purpose. Full album can be found here: https://flic.kr/s/aHskVk85io
Such a relief that @bouncyball, @Danne, @g3gg0 and the rest of you all have made this possible to bring back this useful feature in DarkFraming Average Processing. Especially for lossless.
Yes, all of this was solely done with the latest and greatest cr2hdr.app (https://www.magiclantern.fm/forum/index.php?topic=15108.0cr2hdr.app) (Thanks @Danne for standing by with all of your love & support!)
Quote from: janvkem on May 11, 2017, 10:24:20 AM
Do you create the darkframe by shooting (with the same settings) with the lenscap attached?
Correct.
Quote from: janvkem on May 11, 2017, 10:24:20 AM
Can you do it anytime after the shot (like when i go on holiday and i make the darkframes when i come back home) or does it have to be close together in time?
Anytime works.
Quote from: janvkem on May 11, 2017, 10:24:20 AM
For windows users mlv_dump can do this right?
I believe so, not sure how tho.
Thanks for the reply, going to implement this into my workflow!
QuoteQuote from: a1ex on Yesterday at 10:07:47 PM
@pc_bel, @goldenchild9to5:
Have a look at the preview options; the help text has some hints for your use cases (also discussed earlier in this thread).
Thank you!!! I didn't see that feature.
Hope it worked in 50p3x3 for correct framing.
@A1ex
It's easy to implement it in other button different from the shutter release?...
Thanks again.
There are a few other buttons that send the same event (it's hard to tell the difference between them, other than waiting about 0.5 seconds to see whether it starts autofocusing or not). Use any of them ;)
:D ;D
THANKS!!!!
Thanks @a1ex
Hello everyone,
Could someone explain, please, what i do wrong? My brain is just exploding of complexity.
I've been trying to shoot 3k or 4k or 5k for a few days using instructions here, but everything either work 5 sec or switches to 1920 automatically. Every day it behaves differently. One day i managed to work each resolution but time was different - 3 sec mostly. Couldn't do what i've seen others shot.
Right now i tried to shoot 3,5K crop according to instructions and it allowed me to shoot only 5 sec, also preview freezes. Maybe i've done something wrong with FPS override? I tried 23.976 (from 25), Exact FPS - 23.976 and Low light - 23.980.
My goal is to find the maximum resolution and aspect ratio with reasonable time (say, from 5 min of recording).
I have 5DM3 + 1.1.3 Firmware + Sandisk 30 mb/s (exFAT) with ML (crop from 29 April) + KomputerBay 64GB 1000x (exFAT).
Thank you!
@whysodifficult:
Are you choosing a lossless compression? 3.5k only seems to work continuously with 12bit lossless. 3k maybe with 14bit lossless. You'll have to lower frame rates down significantly to get 4 or 5k. They are not feasible options for traditional frame rates.
@whysodifficult I am currently making a video tutorial for 3k if you can give me to the end of the day
Quote from: hjfilmspeed on May 12, 2017, 06:30:50 PM
@whysodifficult I am currently making a video tutorial for 3k if you can give me to the end of the day
@hjfilmspeed
Really looking forward to your video tutorial! Thanks so much in advance!
Been fiddling around mlv_dump code in crop_rec_4k branch and by doing two minor changes it works a little better around 10bit12bit files.
https://bitbucket.org/Dannephoto/magic-lantern/commits/b1e226a565287ae563944cc035957bd213852855?at=crop_rec_4k_ver2
What seems to work now is following:
MLV files
- Compression(lj92) of all flavours of mlv files including 10bit12bit14bit, compressed raw, regular raw. Compression level is very efficient, almost 50%.
- Decompression of mlv files into decompressed dng files or back to decompressed mlv. This works with all MLV flavours going from compressed straight to decompressed dng files. This is really great since we do not need to decompress the MLV file once it´s compressed.
DNG files
- Compression of 14bit dng files, all flavours works very good. Efficient compression around 40-50%.
Not working
- Compression of 10bit12bit files. DNG output comes out all black but compressed and with metadata present.
I did some minort changes to get 10bit12bit working at all. There is also some testing with decompressing code since the files will come out with incosistent sizes when decompressing. Depends on if coming from mlv_block or outputting dng files. Check code here.
https://bitbucket.org/Dannephoto/magic-lantern/commits/b1e226a565287ae563944cc035957bd213852855?at=crop_rec_4k_ver2
I´m trying to see what is missing around 10bit12bit now. One funny note is that if erasing bitdepth = 14; mlv_dump will output compressed 10bit12bit files fine into original 10bit12bit. Of course this will not work with vetical stripes code since it´s written in 14bit but then again it means it´s halfway working when run_compressor is set.
If any hints to these findings I´d be greatful since I hit a wall now.
Here are some testfiles by the way.
https://bitbucket.org/Dannephoto/magic-lantern/downloads/10_12_14_bit.zip
Hi everyone,
I spent a few days trying to shoot with the crop mode 3.5K but nothing conclusive.
When I use crop mode 3.5K 1:1 centered (resolution 3504x1320 and fps overide 23.976) if I use 12 bit lossless I have the overexposed zone pink, with 14 bits lossless no problem.
Any idea where is my problem ? Did I miss something ?
Thank you !
Quote from: giarcpnw on May 12, 2017, 05:50:23 PM
@whysodifficult:
Are you choosing a lossless compression? 3.5k only seems to work continuously with 12bit lossless. 3k maybe with 14bit lossless. You'll have to lower frame rates down significantly to get 4 or 5k. They are not feasible options for traditional frame rates.
If lossless compression means 10-12 bit instead of 14 bit, then yes, i tried it as well. Thank you for the tip!
I wonder can there be a difference between 1.1.3 and 1.2.3 firmware versions.... I'm using 1.1.3.
Quote from: hjfilmspeed on May 12, 2017, 06:30:50 PM
@whysodifficult I am currently making a video tutorial for 3k if you can give me to the end of the day
Thank you very much! I am looking forward to watch it, so as other people, i'm sure!
QuoteBeen fiddling around mlv_dump code in crop_rec_4k branch and by doing two minor changes it works a little better around 10bit12bit files.
@danne Thank you very much!!!
@whysodifficult @rob_6
This video is not for those new to ML. Try this at your own risk. I am not responsible for any damage this video may bring if your camera explodes or bricks. Magic Lantern is not responsible either. It will probably just brick not explode though.
Magic Lantern 3072 2.35:1 12 bit Lossless RAW Recording from the CROP_REC_4K build from 4-29-2017
Once you have ML installed and the correct build on your SD card, this video should help you set up your 5DIII to record 3K almost continuously depending on the scene. Remember ML on SD and Rec to CF.
Remember this build is experimental and not for those that are new to ML. Try at your own risk.
Quote from: Danne on May 12, 2017, 08:09:28 PM
Been fiddling around mlv_dump code in crop_rec_4k branch and by doing two minor changes it works a little better around 10bit12bit files.
https://bitbucket.org/Dannephoto/magic-lantern/commits/b1e226a565287ae563944cc035957bd213852855?at=crop_rec_4k_ver2
What seems to work now is following:
hi danne
thanks for your feedback and testing!
does that mean with that changes you managed to compress 10/12 bit images correctly?
made test cases for all combinations of compression/decompression/recompression/pass-thru-unprocessed-lossless to check mlv_dump behavior.
though i didn't complete that to cover 10/12 bits, just made some sloppy dev tests.
here is the test case i had for these tests:
mlv_dump_test: mlv_dump
rm *.dng
./mlv_dump -v 14bit_lossless.mlv -o mlv_dump_test_original.mlv
./mlv_dump -v 14bit_lossless.mlv -o mlv_dump_test_decompressed.mlv -d
./mlv_dump -v 14bit_lossless.mlv -o mlv_dump_test_recompressed.mlv -c
./mlv_dump -v 14bit_lossless.mlv -o mlv_dump_test_recompressed_double.mlv -c -c
./mlv_dump -v mlv_dump_test_original.mlv --dng -o mlv_dump_test_0_exp_0
./mlv_dump -v mlv_dump_test_decompressed.mlv --dng -o mlv_dump_test_1_exp_0
./mlv_dump -v mlv_dump_test_recompressed.mlv --dng -o mlv_dump_test_2_exp_0
./mlv_dump -v mlv_dump_test_recompressed_double.mlv --dng -o mlv_dump_test_3_exp_0
./mlv_dump -v mlv_dump_test_original.mlv --dng -o mlv_dump_test_4_exp_1 -c
./mlv_dump -v mlv_dump_test_decompressed.mlv --dng -o mlv_dump_test_5_exp_1 -c
./mlv_dump -v mlv_dump_test_recompressed.mlv --dng -o mlv_dump_test_6_exp_1 -c
./mlv_dump -v mlv_dump_test_recompressed_double.mlv --dng -o mlv_dump_test_7_exp_1 -c
./mlv_dump -v mlv_dump_test_original.mlv --dng -o mlv_dump_test_8_exp_2 -c -c
./mlv_dump -v mlv_dump_test_decompressed.mlv --dng -o mlv_dump_test_9_exp_e -c -c
./mlv_dump -v mlv_dump_test_recompressed.mlv --dng -o mlv_dump_test_a_exp_3 -c -c
./mlv_dump -v mlv_dump_test_recompressed_double.mlv --dng -o mlv_dump_test_b_exp_3 -c -c
md5sum mlv_dump_test_*.dng
the result then i manually checked if that is what i expected.
all with exp_0 should have the same hash, same for exp_1, exp_2 and exp_3.
the exp_e is expected to throw an error.
Hi g3gg0. Thanks for getting back on this. I'm out at the moment but will describe from memory.
As you describe 14bit compression(-c --dng) is working nicely with dng exports. When exporting 10bit12bit the dng files comes out all black.
What is working is compression of MLV files and that works for all flavours, 10bit12bit14bit.
What is adressed in my code changes is to firstly get regular uncompressed 10bit12bit working again at all since it was broken in this build. Next was to try -c with 10bit12bit but always comes out black.
My last code changes made it possible to export to uncompressed dng files coming from compressed MLV. That works with all bits and flavours.
Sorry for these messy allround tests.
If I would suggest for one priority fix is to get -c --dng option working with 10bit12bit files if possble at all?
@hjfilmspeed
Thank you very much for the video, i did everything you said. It works now!! I really appreciate it!
Anyone else's experienced that 12-but is unavailable to choose in latest build 1.1.3? It says in mine that it's only available in photo mode.
Quote from: D_Odell on May 13, 2017, 11:58:15 PM
It says in mine that it's only available in photo mode.
There's no code path that could display this message in the raw video menu, and there are no lower bit depths available in photo mode.
http://www.chiark.greenend.org.uk/~sgtatham/bugs.html
Quote from: ju64 on May 12, 2017, 09:02:47 PM
When I use crop mode 3.5K 1:1 centered (resolution 3504x1320 and fps overide 23.976) if I use 12 bit lossless I have the overexposed zone pink, with 14 bits lossless no problem.
See commit messages from 2028d73 (https://bitbucket.org/hudson/magic-lantern/commits/2028d73f6f34dcdaaa4c241824e538d0e0e47267), 945aadb (https://bitbucket.org/hudson/magic-lantern/commits/945aadbefad03916a40f11762b79a82127d2a6f2) and 2aee5f8 (https://bitbucket.org/hudson/magic-lantern/commits/2aee5f851ab17becb16dfcb00bf3b41e7d510d0f).
@hjfilmspeed
Amazing! Thank you so much for the tutorial video. Everything makes sense now! So seriously cool you made this for us! I love magic lantern! You guys are all awesome!
Rob
(http://thumb.ibb.co/jKTf5k/7678_re.jpg) (http://ibb.co/jKTf5k)
My 3K continuous video record settings.
HDMI Output is check focusing preview monitor (real time fps), and DSLR monitor is check frame position preview monitor (only 7 fps slow, but no crop fully size preview)
DMStorage DM38 128GB CF Card (exFAT)
HDMI output monitor will help you.
Sick!!!! @pjs9505! Are you on 1.2.3 canon firmware?
Here are a couple quick edits I did last night. First one no real color work I was chasing my dogs around testing my 3d printed electronic follow focus I made for my Movi.
https://youtu.be/pRROVuxevuU
And one I shot yesterday at Red Bull Hart Lines in Detroit yesterday. No press pass so it was hard to get where I needed to be.
https://youtu.be/Iag4strSaEg
Thanks so much to the ML team! The raw advancements made me fall in love with my camera again. Everything was shot near 3k in 5x centered crop mode as this is the best for shooting with an external monitor on my Movi m10.
Quote from: Danne on May 12, 2017, 08:09:28 PM
Not working
- Compression of 10bit12bit files. DNG output comes out all black but compressed and with metadata present.
when using windows tools for preview, or lightroom i also get black images.
but when using dcraw to convert to TIF, i get a valid image.
a1ex said that adobes tool do currently not support all types of DNG images properly, so this might be due to their libraries.
Hm, dcraw outputs a valid one. Thought I tried that before. Will test on my mac soon.
Fyi if compress a 10bit file(--dng -c) and export it with bit_depth = 10; I get a crisp, compressed and fully valid 10bit dng file. Makes me speculate if upres to 14bit could be applied later after compression? Of course I do not know fully how those bits behave going up and down.
Read a number of posts, and saw a 700D reference but no real explanation for any aps-c port.
I am a 70D user, and was wondering what the difference is here? to get to 4k?
The 70D even at 3X zoom mode, would be hard pressed to get 2k for any length of time.
Is there something I am missing, and what would be a 70D (card write is still 40 MB/s) max?
Quote from: ShootMeAlready on May 15, 2017, 03:11:09 AM
saw a 700D reference
Yeah, that was me. It seems that all cameras can do the ProcessTwoInTwoOut (http://www.magiclantern.fm/forum/index.php?topic=18443.msg184464#new) thing. Basically the CR2 files are run through that lossless compression process. Then it is another thing to figure out how to tweak the sensor using adtg_gui in order to squeeze every pixel you can out of the camera. I looked into it a while ago but was in way over my head. Some guys are still trying and eventually someone should come up with something that works on the aps-c cameras.
You're right though--that maximum SD card write speed is going to keep these cameras from reaching 4K, except maybe the 7D because it has a faster CF card?
Quote from: hjfilmspeed on May 14, 2017, 05:31:52 PM
Sick!!!! @pjs9505! Are you on 1.2.3 canon firmware?
1.2.3 Firmware.
Hello everyone, someone could guide me about the workflow with crop_rec under Windows, thank you very much. 8)
Hi!
Does crop mode have snd module available? Or impossible?
Best,
David
http://www.magiclantern.fm/forum/index.php?topic=19300.msg184232#msg184232
@hjfilmspeed Awesome video.. This will help the Newbies big time.
@pjs9505 Great setup..
Hello guys.
Since I've tried April crop mode module, my shutter-release button has been acting up - it has reset itself twice to AE lock, instead of AFing. Plus microadjustment has switched itself once to a diff value as well. It has never happened to me before and at the time of malfunction the standard ML was installed. No warnings of any kind were shown upong booting with any of the April versions of the ML.
Thanks
The half-shutter function is changed by ML and can be modified without notice if you take the battery out in the middle of taking a picture initiated from ML, for example. This is documented (http://wiki.magiclantern.fm/faq#does_ml_do_any_persistent_changes_to_my_camera).
If it's not that, find a way to reproduce (http://www.chiark.greenend.org.uk/~sgtatham/bugs.html). I'd expect some similar behavior to happen after an assertion, a crash (e.g. err70) or after taking the battery out - in these cases, ML actively prevents Canon code from saving their settings at shutdown. This is a safeguard to prevent invalid settings (that could be related to the crash) from being saved into ROM (more info (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183060;topicseen#msg183060)).
Note: I've reviewed the calls related to changing half-shutter functionality (cfn_set_af_button and its wrappers) and could not find any instance where the original setting is not restored as soon as it's no longer required. So, unless you (have to) take the battery out in the middle of some photo operation (e.g. bracketing sequence), I don't see how it would happen. Nothing on the video side changes this setting.
Thanks, much appreciated. I'll look into provided docs.
Mikhail
Hi,
I suppose it has been covered, but I haven't been able to find a direct answer to my issue, so don't hit me too hard :)
In short - the dng files that have lossless compression aren't supported natively in the latest version of Adobe premiere (at least on my PC). I wouldn't even bother with that because Resolve does a great job, but with the latest version of Premiere i found that it handles canon raw quite good (it can be absolutely my feeling, as didn't find anything in the release notes)
Is there any way to import the dngs directly into premiere? The simple uncompressed dng files work just fine.
If on mac you get compatiblity with a mlv_dump version used in cr2hdr.app (ml-dng branch + several addons)
Thx Danne, I'm on PC and using the latest availabe mlv_dump from the experiments download section.
If you can compile mlv_dump from this branch it should work @RTI
https://bitbucket.org/Dannephoto/magic-lantern/branch/ml-dng-unified_8_-Mr_B_compress_ver2
@Danne, thx for the input. But i have zero knowledge in coding and compiling stuff :) For now it will be easier for me to install a hackintosh.
Or someone with a windows setup could help you compile a version.
I've been messing around with the 1.5x Iscorama shooting 4K scope 24p anamorphic (2304x1382 with a 25% upscale). With a 24mm Leica-r it's equal to 40mm full frame. Shot at 3200 ISO with ACR chroma noise reduction. Filmconvert Fuji 8553 LUT.
(http://i.imgur.com/KG169dqh.jpg)
I have 16 MLV files that fail to decompress. They almost 1GB average, each.
I was shooting with 5d mk3, 1.1.3. 1920x1080/50/60 2.35:1, 14bit lossless.
I use latest mlv_dump.
I get a message
LJ92 : failed (-1)
The mlv files were random among 70 files and their order do not satisfy any logic.
The last 10 shots were faulty. I assume the problem rested there.
I can upload a mlv file on my GoogleDrive for public access.
Any clue ?
Thanks in advance
Please upload a sample file. If you dig round mlv_dump code it says following.
Error related to int ret. Frame size?
/* this block will load an image from a MLV file, so use its reported frame size for future use */
if(subtract_mode)
{
printf("Loading subtract (dark) frame '%s'\n", subtract_filename);
int ret = load_frame(subtract_filename, &frame_sub_buffer, &subtract_frame_buffer_size);
Your whole error code snippet from mlv_dump.c
int ret = lj92_open(&handle, (uint8_t *)frame_buffer, frame_buffer_size, &lj92_width, &lj92_height, &lj92_bitdepth, &lj92_components);
/* this is the raw data size with 16 bit words. it's just temporary */
size_t out_size = lj92_width * lj92_height * sizeof(uint16_t) * lj92_components;
if(ret == LJ92_ERROR_NONE)
{
if(verbose)
{
print_msg(MSG_INFO, " LJ92: Decompressing\n");
print_msg(MSG_INFO, " LJ92: %dx%dx%d %d bpp (%d bytes buffer)\n", lj92_width, lj92_height, lj92_components, lj92_bitdepth, out_size);
}
}
else
{
print_msg(MSG_ERROR, " LJ92: Failed (%d)\n", ret);
goto abort;
}
Quote from: Danne on May 18, 2017, 10:31:07 AM
Or someone with a windows setup could help you compile a version.
Or you could cross compile a Windows version on your Mac. I compiled a Mac and Windows version from that branch, check if this works:
https://bitbucket.org/daniel_fort/magic-lantern/downloads/ml-dng-unified_8_-Mr_B_compress_ver2.zip
Quote from: RTI on May 18, 2017, 09:05:29 AM
For now it will be easier for me to install a hackintosh.
You would still need to cross compile to get a Windows binary. In fact on a Windows system you need to set up either Cygwin or a Linux virtual machine and cross compile to get a binary that works on any Windows machine.
Quote from: Danne on May 18, 2017, 03:17:47 PM
Please upload a sample file. If you dig round mlv_dump code it says following.
Error related to int ret. Frame size?
The data is corrupted. See lj92 header:
LJ92_ERROR_CORRUPT = -1
@ dmilligan
The data is corrupted.
There is no way to retrieve anything ?
Just wanted to let you guys know that I have been recording without any crashing for quite some time with the latest build. I am using the settings recommended in hjfilmspeed's video at 3k 2.35 and it works so well! I am using a 256GB Lexar 1066x compact flash card and can record about 44-46 minutes in 12 bit 3k reliably and it even closes the file once the card is full without crashing! Amazing work A1ex and everyone else that helped make this possible. I literally feel like I have a new camera! The only thing to be careful of is the compression and not letting the camera overexpose which seems to stop the recording due to the increased recording bitrate.
I have one question. As we know, if I half press the shutter while in the 5x crop mode it zooms in and allows me to focus with a normal live view playback. I usually half press the shutter before hitting the record button to get the focus right. But, I have been successful pressing the shutter half way while recording to get focus during the recording, too. Does pressing the shutter half way while recording put extra work on the camera which could cause the recording to stop early? So far, it seems to work fine. Just curious if anyone else has tried this technique or has any insight!
Thanks again to all!
Rob
Quote from: Quentin on May 18, 2017, 03:59:03 PM
@ dmilligan
The data is corrupted.
There is no way to retrieve anything ?
without a sample we could only clearly answer: "don't know"
you could also apply this (totally untested) patch - maybe it was only one frame that was faulty.
diff -r 8b31c98380d5 modules/mlv_rec/mlv_dump.c
--- a/modules/mlv_rec/mlv_dump.c Tue May 02 01:28:48 2017 +0200
+++ b/modules/mlv_rec/mlv_dump.c Thu May 18 19:26:05 2017 +0200
@@ -2958,7 +2958,7 @@
else
{
print_msg(MSG_INFO, " LJ92: Failed (%d)\n", ret);
- goto abort;
+ goto skip_block;
}
free(compressed);
@@ -3763,6 +3763,7 @@
lua_handle_hdr(lua_state, buf.blockType, "", 0);
}
+skip_block:
file_set_pos(in_file, position + buf.blockSize, SEEK_SET);
}
Quote from: rob_6 on May 18, 2017, 05:10:54 PM
...Does pressing the shutter half way while recording put extra work on the camera which could cause the recording to stop early? So far, it seems to work fine. Just curious if anyone else has tried this technique or has any insight!
I had similar thoughts. It seems to work fine on my end but
sometimes I suspect that it
may cause the recording to stop though not as often. Still trying to reproduce this but it could very well be related to the compression scheme from being slightly overexposed.
I have no clue how to modify mlv_dump.
Here is one of the files
https://drive.google.com/open?id=0B-d8ARtc7xwWWkltLUpUcTJmams
(4K + dual_iso)/2 = 1920x800 @ 14+ stops DR w/o aliasing?
Not without aliasing, dual ISO introduces aliasing in the areas where the two ISOs don't overlap (highlights and shadows).
A sensel cannot be read out at two different ISOs at the same time (so you are going to have some missing data in either spatial or temporal domain). In the first case you'll get aliasing; in the second one, you'll get motion artifacts.
This might be doable though: http://www.magiclantern.fm/forum/index.php?topic=19315.0
Quote from: dmilligan on May 19, 2017, 08:05:45 PM
Not without aliasing, dual ISO introduces aliasing in the areas where the two ISOs don't overlap (highlights and shadows).
I assumed that halving resolution would eliminate/reduce aliasing, since the highlight/shadow lines would no longer be doubled.
Thanks for this module it's awesome !
Has anyone succeeded in getting 25 FPS 2.35 4K ? I'm stucked at 12.5 FPS (I have a good CF card)..
Here's what I've done with it https://www.magiclantern.fm/forum/index.php?topic=19568 (https://www.magiclantern.fm/forum/index.php?topic=19568)
Hi everyone.
I have problems with sound record on 1.2.3 firmware version, experimental build.
Got 4 modules on - crop rec, dual iso, mlv lite, mlv play
What am i doing wrong and how to solve it?
Kind regards, Anna
Is there any way for Win users do decode MLVs from crop_mode to DNG sequence except RawFlow?
In Dual ISO mode in crop_mode processed through RawFlow and Dual ISO Processor files sometimes corrupt. I did some tests but it seems randomly with this pattern in resulted files: http://joxi.ru/Dr8oB9wh4Eol8A.jpg and it's RawFlow.exe bug cause DNGs from MLVP (on old, not crop_mode build though) processed through Dual ISO Processor seems good.
So again, is there some way for Windows users to decode mlv files of crop_mode to DNGs except RawFlow? (MLVP doesn't work with crop_mode MLVs)
There is no sound in crop_rec experimental builds. mlv lite does not support it. Use an external recorder. The Zooms are nice.
Quote from: Anntria on May 21, 2017, 04:10:09 PM
Hi everyone.
I have problems with sound record on 1.2.3 firmware version, experimental build.
Got 4 modules on - crop rec, dual iso, mlv lite, mlv play
What am i doing wrong and how to solve it?
Kind regards, Anna
Quote from: ewinemiller on May 21, 2017, 11:04:57 PM
There is no sound in crop_rec experimental builds. mlv lite does not support it. Use an external recorder. The Zooms are nice.
Do i understand correct that there are no time codes for crop_rec experimental builds as well?
So mixing of external sound with footage is a sort of challenge as well, or am i wrong?
Quote from: Anntria on May 22, 2017, 12:43:49 AM
Do i understand correct that there are no time codes for crop_rec experimental builds as well?
So mixing of external sound with footage is a sort of challenge as well, or am i wrong?
You can use a slate. or a simple clap your hands to sync it with ext recorder.
Quote from: g3gg0 on May 18, 2017, 07:27:08 PM
you could also apply this (totally untested) patch - maybe it was only one frame that was faulty.
diff -r 8b31c98380d5 modules/mlv_rec/mlv_dump.c
--- a/modules/mlv_rec/mlv_dump.c Tue May 02 01:28:48 2017 +0200
+++ b/modules/mlv_rec/mlv_dump.c Thu May 18 19:26:05 2017 +0200
@@ -2958,7 +2958,7 @@
else
{
print_msg(MSG_INFO, " LJ92: Failed (%d)\n", ret);
- goto abort;
+ goto skip_block;
}
free(compressed);
@@ -3763,6 +3763,7 @@
lua_handle_hdr(lua_state, buf.blockType, "", 0);
}
+skip_block:
file_set_pos(in_file, position + buf.blockSize, SEEK_SET);
}
Any chance to review the corrupted mlv file I placed on my Google Drive ?
Thanks in advance
download latest mlv_dump (https://builds.magiclantern.fm/jenkins/view/Utilities/job/mlv_dump_4k/lastSuccessfulBuild/artifact/src/modules/mlv_rec/mlv_dump.exe) and add option "--skip-block 9" to your commandline
Awesome !!!
It heals the problem.
I tried it on other corrupted mlv and it works.
Thanks for your wizardry ;)
Great with all the latest "autopsy" settings @g3gg0.
--relaxed 8)
glad to hear it helps :)
any further suggestions are welcome.
Is there any way to merge DNG-sequence like AviSynth with RAW.mov do? I'm trying to shoot HDR video with crop_mode but AviSynth work just with mov files. Anyone now How to merge DNG frames with different expo to sequence with one corrected sequence?
*I'm on Windows
This is a bit of topic but regarding HDR there is one open source tool that merges dng files
https://github.com/jcelaya/hdrmerge
I tried this for a period of time but the caveat back when testing was that the tonemapping algorithms are mainly for stills. When merging movie sequences you get flicker since tonemapping will vary from image to the next.
What is not that much of topic is to elaborate about mlv_dump being able to merge hdr files coming from MLV footage. I tried some crude examples with averaging code working with consecutive frames. Bouncyball helped me out here with coding but simple averaging would produce pinkish highlights. Now averaging(tblend filter) code in ffmpeg is in another ballpark when looking at the merged footage that I played around with in MLP and cr2hdr.app. No pinks, just a good looking merged file.
Of course any of such changes would surely take a lot of time to accomplish and put into mlv_dump but since the question arised about HDR I thought I just put it out there.
crop_rec_4k Apr29 not working here. I have followed exactly the video tutorial, but it result in random multicolored pixels (viewed in Mlrawviewer and raw2cdng1.7.9). also tested with and without fps overide.
dng sample:
https://mega.nz/#!DocmzRJJ!-A96kskYpm7ZqzFynl_4MZH9yd0cfc9O9bNCcrPlM5k
Edit: Ok sorry, I succeed with mlv_dump.exe
I would like to focus on 10bit12bit files in recent mlv_dump again. Seems the fix is very close but if it the problem passes unnoticed now it will probably be buried into other functions adding up in time.
I test mlv_dump with the following command(latest crop_rec_4k code)
mlv-dump --dng INPUT.MLV
12bit test file
https://bitbucket.org/Dannephoto/magic-lantern/downloads/12bit.MLV
The last commit that will export 10bit12bit files correctly is this
1738cb0
https://bitbucket.org/hudson/magic-lantern/commits/1738cb06e2652d9751f9407490de507fe30ef520?at=crop_rec_4k
Next commit is where things are getting wrong with 10bit12bit footage. Compiling from this commit will cause segmentation fault 11, however the next commit partly fix this but will export frames with missing information(see examples below)
92d1b4d
https://bitbucket.org/hudson/magic-lantern/commits/92d1b4d2958975f50ba428853545f64db8a61205?at=crop_rec_4k
Following commit, "re-adding 14bpp enforcement for DNG" produces black 10bit/12bit dng files cause it won´t fill the frame entirely(see examples). Dcraw exports the file and it shows where the image information is missing.
8b31c98
https://bitbucket.org/hudson/magic-lantern/commits/8b31c98380d5dc27573886d0f84ac7ed0c5d0f2b?at=crop_rec_4k
Looking at the code changes I see a lot of changes around frame_buffer_size, read, prev_frame_size so I guess somewhere in there this could be fixed. Unfortunately I couldn´t get this right with the example file provided but hopefully a dev could take a look or give a hint on how to fix it.
dcraw -T INPUT.dng gives:
Working commit
1738cb0
(https://s4.postimg.org/cc9w4w10d/Screen_Shot_2017-05-24_at_14.21.46.png)
Not working
(https://s4.postimg.org/wide3rwnx/Screen_Shot_2017-05-24_at_14.19.39.png)
Can I update from 113 to 123 Canon firmware, if I install Apr. 29 Build? Or do I have to update firmware via EOS utility? I am currently on 113... Its 2 years since I did firmware update :-\
You can install *Canon* firmware 1.2.3 using Canon menu. If you do this you have to run ML builds for *Canon* firmware 1.2.3.
ML is not firmware.
All builds should be available for 1.1.3 and 1.2.3. There is no need to upgrade Canon firmware if you don't have to for other reasons.
I meant if the 123 firmware is also included in the Apr 29 build. As in i could update to 123 from 113 by updating the build and simaltaneously update the firmware?
Best way to semi-brick/stalling your cam.
@Danne thanks, you clarify issue pretty well. Just to remind you and those who develop some software solutions within magic lantern - currently mlv_dump didn't work properly with Dual ISO MLVs from crop_mode. Sometimes frames corrupted with looking-like-labirynth pattern and it seems randomly.
So it seems that currently there are just one reliable (but long) workflow for HDR-video when you shoot in crop_mode - MLV to DNG-sequence then DNG-sequence to MOV then MOV with Avysinth (in HDR-workflow pack) processed to A, B and C jpg-sequence then this C-sequence again assembled in some final video format.
Quote from: Walter Schulz on May 24, 2017, 04:40:26 PM
Best way to semi-brick/stalling your cam.
So the simple answer is: No. Thank you
Quote from: Danne on May 24, 2017, 02:21:02 PM
12bit test file
https://bitbucket.org/Dannephoto/magic-lantern/downloads/12bit.MLV
where does that file come from? mlv_lite?
reason for asking is this:
Block: RAWI
Offset: 0x00000034
Number: 1
Size: 180
Time: 212.444000 ms
Res: 960x540
raw_info:
api_version 0x00000001
height 1181
width 1888
pitch 2832
frame_size 0x003B8A48
bits_per_pixel 12
black_level 512
white_level 4050
active_area.y1 26
active_area.x1 152
active_area.y2 1181
active_area.x2 1886
exposure_bias 0, 0
cfa_pattern 0x02010100
calibration_ill 1
Block: VIDF
Offset: 0x0000027c
Number: 9
Size: 3344624
Time: 550.688000 ms
Frame: #0000
Crop: 536x332
Pan: 536x333
Space: 0
depth: 12 -> 14, size: 777600 -> 907200 (116.67%)
black: 512 -> 2048
white: 4050 -> 16200
Res: 960x540 -> 777600 bytes frame size.
but the frame seems 1181x1888 -> 3344592 bytes.
looks like the whole raw buffer was written instead of the specified frame size.
patching the fields results in proper frames.
any idea where this file came from?
It's an early build put into a 600D. Wonder if it was called raw_rec(mlv_lite). Stupid of me not to download a later build when testing. Right now I don't have the camera nor the firmware version until monday.
On another note. Is recent 10bit12bit files working with latest crop_rec_4k like it should? Last I tried it wouldn't work. If so, I guess there is no issue. Don't have any cam here unfortunately to verify.
By the way. That particular file will process with earlier builds. Guess there is some other buffer detectors in previous code?
Just tested with mlv files on my eos m coming from mlv_lite and regular mlv_rec and both transcodes 10bit12bit with wrong buffer(black bottom). Downloaded experimental raw_video_10bit_12bit 30 april build. Ran latest mlv_dump (crop_rec_4k) code.
Here is the thing. If I replace
if(read_size != (int)frame_buffer_size)
with
if(read_size > (int)frame_buffer_size)
later 10bit12bit files will process correctly. Is this the fix we could use? It still won´t work to compress 10bit12bit files(mlv_dump -c setting produces black output) but I guess they are already compressed being 10bit12bit so maybe compression for lower bits should be excluded?
Here's my first 3072x1320 24 fps recording:
https://www.youtube.com/watch?v=B6mEqWalnJY
so with 14-bit lossless 3K i am seeing about 6.5 minutes of record time on a 64GB CF card... does this seem accurate? also what is a recommended Aspect Ratio?
Quote from: Danne on May 25, 2017, 12:12:34 AM
Here is the thing. If I replace
then you will get crashes :)
but thanks for your testing. just pushed a fix (https://bitbucket.org/hudson/magic-lantern/commits/9b214f11098a3be752ca8a5cfe3a17584b3c23b8) that should fix that issue.
after converting bit depth, frame size was updated, but not data size.
reason for having both is obvious when compression is used. frame size wont change with or without compression, whereas the resulting binary data varies in size.
Quote from: erek on May 25, 2017, 09:30:09 AM
Here's my first 3072x1320 24 fps recording:
so with 14-bit lossless 3K i am seeing about 6.5 minutes of record time on a 64GB CF card... does this seem accurate? also what is a recommended Aspect Ratio?
3072 * 1320 * 24 * 60 * 6.5 * 14/8 * 0.58
gives ~38GiB
@g3gg0
frame_buffer_size = new_size;
:D
Tested, working. Thanks a million.
@erek, your shadows look a little noisy. maybe you underexposed or maybe you had a high ISO?
What program are you guys using post production (after exporting via mlv_dump). I would normally use Premier pro cc 2015 but it doesn't recognize the .dngs whenever I use any new feature settings...although I can view the individual .dngs fine in photoshop or any image viewer.
Mlvfs or cr2hdr.app will produce cdng files
I tried cr2hdr.app before and still got the same message as using mlv_dump. I'm prob missing a setting that allows for the proper export. Regardless just tried Mlvfs and it worked great!
Thanks for the help
I see. What settings did you use for it not to work? I would like to reproduce this issue. Do you have a sample file to test with?
@g3gg0
mlv_dump -c -c is not working healthy right now. dng files comes out with 0 black and white level for instance. I narrowed the issue down to this change.
if(new_depth != old_depth)
reverting back to this and it works again.
if(new_depth)
There is also one of these beauties left
LJ92: Failed
which could go into nifty --relaxed to let mlv_dump produce compressed dng files.
/* set new compressed size and copy buffers */
frame_buffer = realloc(frame_buffer, compressed_size);
assert(frame_buffer);
memcpy(frame_buffer, compressed, compressed_size);
frame_buffer_size = compressed_size;
}
else
{
print_msg(MSG_ERROR, " LJ92: Failed (%d)\n", ret);
goto abort;
}
to
/* set new compressed size and copy buffers */
frame_buffer = realloc(frame_buffer, compressed_size);
assert(frame_buffer);
memcpy(frame_buffer, compressed, compressed_size);
frame_buffer_size = compressed_size;
}
else
{
print_msg(MSG_ERROR, " LJ92: Failed (%d)\n", ret);
if(relaxed)
{
goto skip_block;
}
goto abort;
}
first one done, really have to rework some code to make such changes happen just in one point.
the second thing, does this ever happen?
Quotethe second thing, does this ever happen?
You´re right. I´m working a version which turns on the compression switch when selecting -c -c. Not a problem in crop_rec_4k version.
@g3gg0 @Danne Great job guy's.. ya'll working.
Do I missing something? My preview like:
https://gyazo.com/ed2429dc5904467c229dab961b1c4e6d
Hi everyone,
I was fiddling around with the UHD 1:1 setting of the crop_rec module
The data I was recording was 14bit lossless, iso 1600.
It was recording continuously until I pointed the camera at the PC screen and this happened! :D
(http://i64.tinypic.com/eb2zk3.jpg)
I still have the file if any of you would like to inspect it.
p.s.
I am sure it happened because of the high ISO and the sharp change from a dark corner of my room and the PC screen.
I read that 14bit lossless works best at low iso and with not too bright scenes, therefore I believe that was what made magic lantern to do this. I restarted the camera removing the battery and all come back to normal.
p.p.s.
When that happens, what button do I press to make it go away without having to restart the camera? :D
Been testing out these option in the crop_rec_4k
* 1920x960 @ 50p (both 1:1 crop and full-frame - 3x3 pixel binning) - continuous*)
* 1920x800 @ 60p (same as above) - continuous*)
* 1920x1080 @ 45p and 48p (3x3 binning) - continuous at 45p
Works promising with HDR footage.
I noticed compressed raw and lossless 10/12 bits can´t be selected only 14bits lossless. Searched around but I couldn´t find out the reason why other than from a live view message and some posts over here http://www.magiclantern.fm/forum/index.php?topic=19300.msg183763#msg183763
Is it simply not possible for 10/12bit and compressed raw with the binning modes?
A potential bug. Can somebody else but me reproduce following issue.
- Set camera to mlv_lite and select 1920x960 @ 50p in the crop_rec menu. Camera set to PAL.
- Record a short sequence and stop filming. Now exit liveview closing it the normal way.
- Now try to switch into liveview again.
From now on all I get is a black screen. Only way to get back into liveview is to set 1920x960 @ 50p in crop_rec menu to off, then open up liveview again and reselect 1920x960 @ 50p again.
5D mark III (1.1.3)
crop_rec_4k.2017Apr29.5D113
*related to 720p movie mode. When in 1920 no more black screen.
@Danne it happened to me as well.. can't remember which build exactly, but it was one from April. What I did was to format my SD card again and reinstall Magic Lantern again. Sometimes when you use 50p 60p mode something breaks in the RAW modules I think not sure but ask @a1ex.
Yes, it´s related to 720p movie mode. I updated my question. Thanks for getting back.
Quote from: Danne on May 28, 2017, 03:17:02 PM
Been testing out these option in the crop_rec_4k
* 1920x960 @ 50p (both 1:1 crop and full-frame - 3x3 pixel binning) - continuous*)
* 1920x800 @ 60p (same as above) - continuous*)
* 1920x1080 @ 45p and 48p (3x3 binning) - continuous at 45p
Works promising with HDR footage.
I noticed compressed raw and lossless 10/12 bits can´t be selected only 14bits lossless. Searched around but I couldn´t find out the reason why other than from a live view message and some posts over here http://www.magiclantern.fm/forum/index.php?topic=19300.msg183763#msg183763
Is it simply not possible for 10/12bit and compressed raw with the binning modes?
What build are you running? I cant get continues in 1920x800 60p. Neither on 113 or 123. especially when bright and lots of details, it will even crash at times saying "compressed size larger than uncompressed" or something like that. Been using the 60p mode a lot this last week.
And a question for @A1ex . Is there possibility of getting bit higher resolutions in the 60p mode in the future? 800 vertical pixel is a bit of a squeeze, i love the look, but not every project deserves the cinemascope ;) Perhaps if you get spanning to work aswell?
1920 x 960 50fps is continuous. Early stops with other ones. Did not have the time to test extensively. I'm pretty amazed about the 1920x50 fps option.
I confirm the bug with the black screen when 1920x800 @ 60FPS
It takes some patience and some assistance to recover Liveview back.
I press Canon Menu a couple of times and then ML button.
What are the options for Windows users, to batch convert these MLV's?
I was using this Code:
@echo off
md "%~n1"
"C:\XXXXXX\Video Program\MLV_Dump on Steroids\mlv_dump.exe" -o %~n1\%~n1_ --dng %~n1.MLV
To Batch convert Selected MLV files in a folder. But after replacing the mlv_dump with latest, I can not get it to Batch Convert anymore, it will only convert one and then exit.
I don't know of any other converter for these new MLV's?
Please help, I have so incredibly many MLV's to convert.
MLVFS, does not work for me, as AE will only take in to account the changes of the First Frame from ACR.
Was testing out the most recent build and had a slew of errors when I tried recording video (there is one possible mistake I made, reinstalling ml now to see if I can reproduce error). I tried to start recording, and it did so for a few seconds, but then said card full. I didn't realize at the time, but it didn't switch over from cf to sd card, so I couldn't test to see if it would still have worked if I had switched manually. That said, since that happened I have been experiencing two kinds of errors. One, when I go to record video if I am using anything above 3k resolution a "script" will run indefinitely, saying that "mean is incorrect" and something about black level, and a range of numbers in the 2000-3000s. At the moment, I am able to reproduce this, but am unsure if reinstalling ML will fix. I have crash logs, etc. as well. The main thing that is also happening, is that I kept seeing text saying "video end not working". When I ran the self test, this message popped up quite a bit. Any thoughts? Should I provide any of the stuff I've got, crash logs and such?
QuoteMLVFS, does not work for me, as AE will only take in to account the changes of the First Frame from ACR.
Same for me... Searched for a solution but unable to find it.... :(
Magic Lantern updates may generate incompatibilities with existing supporting tools. This is natural.
Developers of ML and ML tools are not gathered under the same roof and develop in synchronization.
ML developers specified in the first page that the only way to get DNG files is using mlv_dump
that is available in the download / experiments page.
Insisting to use last years tools, or older, together with today's ML build, is a waste of time.
QuoteMLVFS, does not work for me, as AE will only take in to account the changes of the First Frame from ACR.
You could build xmp sidecar files of every dng and after effects will treat every dng as unique despite being a sequence.
The problem is not mlvfs but AE.
Quote from: Danne on June 01, 2017, 04:44:54 PM
You could build xmp sidecar files of every dng and after effects will treat every dng as unique despite being a sequence.
The problem is not mlvfs but AE.
Hey Danne,
How do i create xmp sidecars in AE? And wont that take a lot of time? Not sure how it works, but i am imagining that ACR has to go through each and every DNG and save its xmp.. I currently have about 200k frames to process.
You can build xmp from lightroom or even faster would probably be with exiftool through scripting.
Was affraid you'd bring something like that up. I don't know how to do scripts or coding of any sorts. And incorporating yet another Adobe product in to my workflow is not what I want to do. I had a slow, but steadily working workflow "Send to Mlv_dump" batch convert - Transcode to Log intermediate in AE - Edit and Colour Grade in Resolve - Done.
Danne, you are good at coding. The code I posted above, it is the same I used before with the older version of mlv_dump. I assume the code is right, as it worked before. Could it be something with the mlv_dump version that is preventing it from opening the next MLV ?
I have tried deleting all MLV_dump, change locations of the mlv_dump, tried having no Blanks in the Location to Avoid the "", which I understood was for Blanks in locations. But nothing works. I have not tried going back to the older mlv_dump, as i really like that I now can play the DNG's in MLRawViewer, its actually a Must-Have that I can do that.
I have no idea. Don't you're command prompt tell you why it breaks?
Suggestion.
1 - try an older mlv_dump file and see if that works
2 - upload a problematic file if any.
3 - check with one of the windows threads
4 - test with latest mlv_dump and specify --relaxed which raises tolerance levels regarding broken/missing blocks.
Hello! Here is some test of 3k raw and slow-mo RAW!
https://youtu.be/Ei3g_-YxtUc (https://youtu.be/Ei3g_-YxtUc)
I noticed, that in Experiments/"4K raw video recording" appeared builds "magiclantern-startup-log.2017May31....zip" for a lot of cameras, what is the purpose of these?
Hm... looks like Jenkins found the cellar keys on the build server, and noticing that nobody was looking at him, he had a little too much wine :)
Thanks for reporting.
Well, early christmas came to mind :D
Quote from: a1ex on June 02, 2017, 09:09:14 PM
Hm... looks like Jenkins found the cellar keys on the build server, and noticing that nobody was looking at him, he had a little too much wine :)
Thought this was either a joke or a typo by Jenkins... :P
(https://c1.staticflickr.com/5/4224/35018847766_78eea78879.jpg) (https://flic.kr/p/VmuKpm)
Anyway I went ahead and downloaded a build for 7D.203 just because...
Here are the start up logs in case they are of any help or give out hints:
https://mega.nz/#!qhNF1Q7I!qmnhGAKHLktvQ-2FiT692qjpLL9wDvji1Jv68XcGbV4
worst tease ever. I'll go back to being depressed now.
There was no teaser, to begin with. It was a bug (in the build server configuration).
Hey there.
I know you have plenty to do sir, but do you think it could be possible to throw in a very experimental sound build?
Don't care if it will be bulky to use. I have a projekt where its impossible to use a clap every time ;(
I thought about something like proxxy wave files on SD card.
sorry to annoy you again @A1ex :P
I'd be happy to review a pull request with this feature ;)
haha yeah. But when I say "I thought of" it literaly means "I thought of"
I have zero progamming skills beside html and css :D :D :D
Understandable, but I also have good reasons to believe the ~5000 downloads of this build were not done by you alone :)
Quote from: a1ex on June 03, 2017, 03:39:18 PM
I'd be happy to review a pull request with this feature ;)
having patches on my disk.
wont take too long.
Quote from: a1ex on June 03, 2017, 05:33:47 PM
Understandable, but I also have good reasons to believe the ~5000 downloads of this build were not done by you alone :)
please help me to understand your sarcasm, as I'm not a native speaker ;)
I guess he mean (not native speaker though too) ir's not just you who asked something. I did too but dropped almost instantly since nobody responded. Aaand... A miracle! Now I know almost everything that once puzzled me. Actually thanks Danne for Dual ISO problem with crop_mode since it's he's tests with my footage showed me that it is not bad camera settings problem but rather flawed Dual ISO processing software
Quote from: jankrueck on June 04, 2017, 12:32:30 AM
please help me to understand your sarcasm, as I'm not a native speaker ;)
What makes me laugh is, I suspect both you and A1ex have a language in common, but out of kindness, write in english so everyone can understand :D
Nah. Their native languages are quite different.
If they added sound to this build I would probably buy another 5D! Image quality is incredible! Sound is my only wish. So that's 2 of us out of 5000! :)
I am the german one if you refer to this language ;)
I thought A1ex was german as well! :O :D
In the end we are all german and everybody is just writing english, because they think the others don't understand :D
Guys I have on quick question. Most likely im bringing something up, that has been answered somewhere else, but I couldn't find it. Why is it not possible to shoot 12bit or 10 bit raw in modes with higher vertical resolution like 3k mode or 1920 60 fps mode? Is this a limitation in anyway or has it just not yet been implemented?
Anyways you are doing an awesome job here. Thank you very much!
Na wenn das so ist, dann schreib ich mal in meiner Muttersprache :D
Also G3gg0 wie erstell ich ne pullrequest damit Alex glücklich wird?
Meinste wave als "proy" auf die SD zu schreiben hat Potential?
* 4096x1440 @ 25p (1:1 crop)
Is anyone could find this mode? I've check all modes and just can't find it. Is it exist?
Quote from: Savely on June 05, 2017, 05:12:32 AM
* 4096x1440 @ 25p (1:1 crop)
Is anyone could find ершы mode? I've check all modes and just can't find it. Is it exist?
Try changing Canon Video System to PAL/NTSC
or try changing 1080/720 and see how it affects your vertical size.
Quote from: Quentin on June 05, 2017, 06:15:03 AM
Try changing Canon Video System to PAL/NTSC
or try changing 1080/720 and see how it affects your vertical size.
Thanks, changing 1080/720 does help.
As I understand card_spanning doesn't work on experimental crop_mode build so we are restricted with ~100mb/s so there are no hopes for sustainable 4k at 24fps (half-fps or redused vertical resolution is not an option). Or it is possible to add SDcard speed and up a bit writing speed for sustainable 4k >5sec rec?
+1 for the sound also i currently have a work around which is really cumbersome but that works with Full HD defintion.
I activate proxy record
i choose the SD card in ML menu as the prefered card for writing.
In canon menu I set the IPB as codec (all-i gives me more crashes but this is probably due to a weak SD card). SD mode 640 does works aswell.
In the End you get 2 files : 1 proxy video with sound on SD and 1 Mute MLV on CF.
The problem with this techniques is that the 2 files does not start in SYNC (h264 always start first) so you need to resync you audio with the MLV and run an export of the audio to match the MLV sequence.
It's like a clap sync without the clap but as heavy to deal with though. The benefits i that you get a proxy file you can start working with and find a little hand to do the dirty work of exporting matching audio.
The problem is that it does not work with 3K Mode as there is no preview to make image match
So if getting sound into the MLV is still hard to manage maybe enabling any kind of poor preview to the 3k mode would help to do the trick in meanwhile. or even better a way to have the proxy start in sync with the MLV
Quote from: etienne on June 05, 2017, 02:59:38 PM
+1 for the sound also i currently have a work around which is really cumbersome but that works with Full HD defintion.
I activate proxy record
i choose the SD card in ML menu as the prefered card for writing.
In canon menu I set the IPB as codec (all-i gives me more crashes but this is probably due to a weak SD card). SD mode 640 does works aswell.
In the End you get 2 files : 1 proxy video with sound on SD and 1 Mute MLV on CF.
The problem with this techniques is that the 2 files does not start in SYNC (h264 always start first) so you need to resync you audio with the MLV and run an export of the audio to match the MLV sequence.
It's like a clap sync without the clap but as heavy to deal with though. The benefits i that you get a proxy file you can start working with and find a little hand to do the dirty work of exporting matching audio.
The problem is that it does not work with 3K Mode as there is no preview to make image match
So if getting sound into the MLV is still hard to manage maybe enabling any kind of poor preview to the 3k mode would help to do the trick in meanwhile. or even better a way to have the proxy start in sync with the MLV
yeah nice idea. but this realy "sounds" (haha) like too much work.
getting a clap with timecode and external recorder will make it a lot easier. (even though it is more expensive)
looking forward to any solution wich brings sound to 3k
Sound is good.... But understanding live view and compression are probably more important I think.
And no one care about card spanning idea to make (or at least, to try) crop_mode sustainable with 4096 24fps and reasonable aspect ratio? Because now we have essentially half 4k. If it's 24fps so it have some adsurd aspect ratio - like, just band throughout black screen. If we have descent aspect ratio - it's just 12fps... If we have both - it's just 3 seconds of record... 4k, fps, aspect ratio, continuance - choose one. What one can shoot with it?
Okay, I'm not pressing, but can we at least discuss this idea? It is even possible?
Hi there, I am in the middle of a little film project (although I have some time to complete it), and using 10 bit (2017-01-12 / crop3x.2017Jan.5D113) to focus on a water droplet (around 3K). In my setup it is good to make the final film 1080 (also to have the water droplet as big as possible). It is shot inside, and at 1600 iso there is already quite some noise (but I think manageable). I would be very happy if somebody could tell me if it is worth to put the new crop version on it, since I might have to change the setup of my project and lose some time.
- Will the lossless 12 bit (or 14 bit lossless) have less noise and better image quality compared to 10 bit (nonlossless), and more than 3K at 24fps? (there is a lot of movement in the imagery so that might limit the record time)
- Will the new version also have the option to choose the 3x crop mode (in addition to the around 1.6x crop mode)?
- Will the (older) 3x crop mode have less image quality considering (I assume) it is using a smaller portion of the sensor?
I might have to do these experiments myself, but it would be great if someone already knew some of the answers and help me safe some time. Thanks!
@MartijnR the new 4k crop rec mod is way nicer. Your crop area will be centered, and you can't get better preview options. It might be a little different to use though. Also it is 1:1 resolution like before. Your crop factor depends on the mode/resolution you choose.
@hjfilmspeed Thanks for letting me know! Due to the nature of the project, this will save me a lot of time :) I have seen your tutorial (thanks) and was wondering if you would recommend the 29 april built, or just the latest built?
I should probably update the tutorial. I would say try the newest build but make sure you use the newest MLV_Dump too.
@MartijnR
Wich lenses are you going to use?
Take in mind the crop factor because it's huge.
@MartijnR If you're shooting indoors, can't you adjust your lighting in order to lower you ISO. You should get less noise.
@hjfilmspeed Thanks for the advice!
@budafilms Probably Zeiss 35/1.4 or 50/2 (or somewhat wider, 28/2 or 25/2 if the crop requires that), and the 100/2 for the droplet. That the crop factor is huge is a great thing, considering I need to get the droplet as big as possible in my frame. I also tried a reverse ring, but that's a bit too risky with all the water involved. I will soon start to do some testing. Cheers
@lostfeliz Thanks for your reply. I use a very nice lamp (just the one in my bathroom), with great intensity and nice character. For my film idea, I need this lamp, and to replace it with another bulb is actually a bit risky (been broken before). The intensity is more than sufficient for most shots, but when going as close as possible with the 100/2, you loose quite some light, and 1600 iso seems inevitable. I am curious how the imagery will differ with the new crop and lossless 12 bit compaired to the 10 bit nonlossless raw. Thanks.
Gentlemen, I've been searching everywhere but haven't found instruction on usage of MLV_DUMP on Mac. Can you kindly help me?
I'm using Footage for Mac app, it's good, but whites on video are pink there. I suppose it will not be a problem using MLV_DUMP(?)...
Thank you!
That could be related to white level settings. What settings are you using in cam? Upload a sample mlv file?
You could develop with cr2hdr.app. It contains latest mlv_dump code. Mlvfs as well. Or mlv_dump binary itself of course.
Danne, thank you for quick reply! Is the DNG file of the video enough? https://drive.google.com/file/d/0B47FS5gpVSwOU3NxanJDaS1TNUE/view?usp=sharing
I have MLV now of 1,19 Gb.
What a beautiful shot. Where is it from?
Usually white balance should be around 15000 but I see yours is at around 5000. Anyway seems a tricky shot, very bright so 15000 won´t work either. I keep a mlv_dump menu section in cr2hdr.app for manually set new white balance and also to create sample file packages so if you want to share a sample instead of the whole 1gb file you could go down the cr2hdr.app route
http://www.magiclantern.fm/forum/index.php?topic=15108.0
Once installed head over to the mlv_dump menu. There is a setting for samples and one for white balance.
You could also share the whole 1gb file if you like.
@Danne:
exiftool M30-1836_C0000_00001.dng -WhiteLevel -BlackLevel
White Level : 5586
Black Level : 2047
(5586-2048)*4 + 2048 = 16200
Which means, this is a 12-bit lossless image - that's how the bit depth reduction works. The container is still 14-bit, but the usable range is restricted to 12 bits, with black level equal to whatever Canon firmware assumes (if it's 2048, that would give a valid range of 1536 ... 5586, which means 4050 levels - just below 4096).
The DNG looks just fine here, without adjusting its metadata. It's overexposed, but the highlights are white.
BTW, white balance != white level.
I was fooled by the container. Thanks for getting back. Darn good view in that dng and still a lot of dynamic range to pull back but overexposed.
Danne, Alex,
Thank you! It's Yosemite National Park in California (rock, called Half Dome) a few days ago, i was lucky with a cloudy weather. It was much more beautiful when i drove to that point (Glacier Point), nothing but clouds around and under you and then they slowly started to disappear, you could see clear spots of the valley down there, of rocks around, and i went to parking for my camera but it was too late when i came back :)
I've just opened this DNG in Photoshop and Lightroom and it looks overexposed (so as google drive shows), but earlier i only looked at all photos in LilyView image viewer on Mac, that shows this photo NOT overexposed, it shows the same how i saw the photo on Live View screen on camera, but the whites are pink: https://drive.google.com/file/d/0B47FS5gpVSwOS1h3b214UmZILTA/view?usp=sharing
So as Finder app (native file browser in Mac) shows — pink. So as Footage for Mac app shows it in the app — https://drive.google.com/file/d/0B47FS5gpVSwOeHptZmxkNWtGQkU/view?usp=sharing , and how it converts MLV to MOV video — pink. https://drive.google.com/file/d/0B47FS5gpVSwONG9WSWlRV3kyUE0/view?usp=sharing
So i didn't even think to open DNG in Photoshop and Lightroom because everywhere else whites are pink.
I read somewhere on this forum, A1ex replied to someone on this pinkness, but i didn't understand that, so i figured that this is the temporarily downside of 3.5K. By the way, i suppose pinkness happens only in high-resolution video (crop mode)... As i recall. Also i thought that this Footage for Mac app doesn't remove this pinkness, but i read somewhere that MLV_dump does. Maybe i got it wrong. But wanted to try.
Maybe over-exposition is my fault during shooting, i'm not a professional photographer or videographer, but that's how i saw it on camera screen.
Adjusting in Lightroom shows that this photo is not fatally overexposed, all details are kept. So i don't know what to think about it (what is the reason):
https://drive.google.com/file/d/0B47FS5gpVSwOWENZWW5sZU9VV0E/view?usp=sharing
This is the MLV, video is 11 sec of static view, but you can see the clouds are slowly moving:
https://drive.google.com/file/d/0B47FS5gpVSwOOEpDN19xeXIxM0E/view?usp=sharing
It was 3.5K according to instructions in this thread above from @HJfilmspeed https://vimeo.com/217313287 so it's 12 bits but the Footage app says it's 14 bits.
Thank you.
Here is a sample package of the 1gb provided from whysodifficult if anybody wants to check.
https://bitbucket.org/Dannephoto/magic-lantern/downloads/M30-1838_samples.zip
The quick look previewer in mac doesn´t get the white balance level at all. See the last example. The other two is how DaVinci resolve and adobe camera raw interprets the dng when opened up in each program.
(Added dcraw -T -w output as well).
DaVinci Resolve
(https://s13.postimg.org/kp7pzy3uv/image.png)
Adobe camera raw
(https://s13.postimg.org/ckzlv7hfr/image.png)
dcraw(tiff)
(https://s2.postimg.org/wdofc4dh5/image.png)
Quick look previewer
(https://s13.postimg.org/l4izsyps7/image.png)
@Danne: the white balance looks fine to me in all 3 images. It's the white level tag that's misinterpreted.
Just for kicks: ufraw-mod with highlight recovery in HSV, EC -0.8 (linear) and +1 (soft-film):
(http://a1ex.magiclantern.fm/bleeding-edge/ufraw-mod/M30-1836_C0000_00001-08lin.jpg)
(http://a1ex.magiclantern.fm/bleeding-edge/ufraw-mod/M30-1836_C0000_00001+1.jpg)
Regular rendering (EC 0 without highlight recovery) looks pretty much identical to dcraw.
Oops, word slip. I corrected my post. Meant white level. Added dcraw tiff output as well.
Quote from: a1ex on June 09, 2017, 11:18:56 PM
@Danne: the white balance looks fine to me in all 3 images. It's the white level tag that's misinterpreted.
Just for kicks: ufraw-mod with highlight recovery in HSV, EC -0.8 (linear) and +1 (soft-film):
(http://a1ex.magiclantern.fm/bleeding-edge/ufraw-mod/M30-1836_C0000_00001-08lin.jpg)
(http://a1ex.magiclantern.fm/bleeding-edge/ufraw-mod/M30-1836_C0000_00001+1.jpg)
Regular rendering (EC 0 without highlight recovery) looks pretty much identical to dcraw.
Wow, that ufraw mod is doing quite a good job pulling back information. The code should be inside dcraw if you ask me. Seems to be pulling back information better than ACR. Is this possible.
Ex. ACR pulling down highlight recovery and whites to the bottom
(https://s10.postimg.org/6bq3yvdpl/image.png)
dcraw -T -H 2 -w
(https://s10.postimg.org/4xyh3keg9/image.png)
i noticed this too with the footage for mac app. theres pink all over my footage. not so with cr2hdr...so I've been using that lately.
I check out the 3.5k file that @whysodifficult posted on my PC Windows 7 Pro desktop for only WB issue .
I extracted the dng's with the latest mlv_dump.exe from the experimental download page and every thing looks right .
(https://c1.staticflickr.com/5/4218/35044374322_1fbbd1c299.jpg) (https://flic.kr/p/VoKzyo)
Test_3.5k_5d3_WIN7_FILE_EXPLORER_from_mlv_dump (https://flic.kr/p/VoKzyo) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr
No adjustments , just loaded in to A.E. CS6 PC/Win7 . The WB seems to be 5500K , looks ok to me
(https://c1.staticflickr.com/5/4257/34822932030_12fe396ce3.jpg) (https://flic.kr/p/V4bCtQ)
Test_3.5k_5d3_A.E.CS6 (https://flic.kr/p/V4bCtQ) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr
I was very impressed with the image , I started to push the image around with ACR and the image really pop out .
(https://c1.staticflickr.com/5/4205/35169952406_54b7316995.jpg) (https://flic.kr/p/VzRcxL)
m30-1838_00000.tiff (https://flic.kr/p/VzRcxL) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr
I started with "Camera Neutral" & 7250K WB , the .XMP from A.E. is below
I also exported 50Mb Mp4 H264 @ 35Mb/s link--> https://drive.google.com/file/d/0B3rFQHNAb91MQWxGUWNxM05WX2s/view?usp=sharing
I was amazed how much sharpness it could handle , really made the fine detail come out (very nice) :D
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.3-c007 1.136881, 2010/06/10-18:11:35 ">
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<rdf:Description rdf:about=""
xmlns:crs="http://ns.adobe.com/camera-raw-settings/1.0/"
crs:Version="7.0"
crs:ProcessVersion="6.7"
crs:WhiteBalance="Custom"
crs:Temperature="7250"
crs:Tint="+10"
crs:Saturation="+44"
crs:Sharpness="58"
crs:LuminanceSmoothing="40"
crs:ColorNoiseReduction="40"
crs:VignetteAmount="0"
crs:ShadowTint="0"
crs:RedHue="0"
crs:RedSaturation="0"
crs:GreenHue="0"
crs:GreenSaturation="0"
crs:BlueHue="0"
crs:BlueSaturation="0"
crs:Vibrance="+11"
crs:Defringe="0"
crs:HueAdjustmentRed="+17"
crs:HueAdjustmentOrange="+30"
crs:HueAdjustmentYellow="+8"
crs:HueAdjustmentGreen="0"
crs:HueAdjustmentAqua="0"
crs:HueAdjustmentBlue="+1"
crs:HueAdjustmentPurple="+18"
crs:HueAdjustmentMagenta="+1"
crs:SaturationAdjustmentRed="-10"
crs:SaturationAdjustmentOrange="-6"
crs:SaturationAdjustmentYellow="+89"
crs:SaturationAdjustmentGreen="0"
crs:SaturationAdjustmentAqua="-9"
crs:SaturationAdjustmentBlue="+13"
crs:SaturationAdjustmentPurple="-6"
crs:SaturationAdjustmentMagenta="0"
crs:LuminanceAdjustmentRed="-43"
crs:LuminanceAdjustmentOrange="+18"
crs:LuminanceAdjustmentYellow="+28"
crs:LuminanceAdjustmentGreen="-4"
crs:LuminanceAdjustmentAqua="-3"
crs:LuminanceAdjustmentBlue="-100"
crs:LuminanceAdjustmentPurple="-8"
crs:LuminanceAdjustmentMagenta="0"
crs:SplitToningShadowHue="0"
crs:SplitToningShadowSaturation="0"
crs:SplitToningHighlightHue="0"
crs:SplitToningHighlightSaturation="0"
crs:SplitToningBalance="0"
crs:ParametricShadows="-48"
crs:ParametricDarks="-100"
crs:ParametricLights="-15"
crs:ParametricHighlights="-75"
crs:ParametricShadowSplit="10"
crs:ParametricMidtoneSplit="20"
crs:ParametricHighlightSplit="90"
crs:SharpenRadius="+1.0"
crs:SharpenDetail="25"
crs:SharpenEdgeMasking="0"
crs:PostCropVignetteAmount="0"
crs:GrainAmount="0"
crs:LuminanceNoiseReductionDetail="38"
crs:ColorNoiseReductionDetail="50"
crs:LuminanceNoiseReductionContrast="40"
crs:LensProfileEnable="1"
crs:LensManualDistortionAmount="0"
crs:PerspectiveVertical="0"
crs:PerspectiveHorizontal="0"
crs:PerspectiveRotate="0.0"
crs:PerspectiveScale="100"
crs:AutoLateralCA="0"
crs:Exposure2012="-0.55"
crs:Contrast2012="-47"
crs:Highlights2012="0"
crs:Shadows2012="0"
crs:Whites2012="0"
crs:Blacks2012="0"
crs:Clarity2012="+45"
crs:ConvertToGrayscale="False"
crs:ToneCurveName2012="Custom"
crs:CameraProfile="Camera Neutral"
crs:CameraProfileDigest="AF004EBA47092ECF696D06CE24881DE8"
crs:LensProfileSetup="LensDefaults"
crs:HasSettings="True">
<crs:ToneCurvePV2012>
<rdf:Seq>
<rdf:li>0, 0</rdf:li>
<rdf:li>50, 38</rdf:li>
<rdf:li>73, 68</rdf:li>
<rdf:li>107, 106</rdf:li>
<rdf:li>130, 126</rdf:li>
<rdf:li>176, 173</rdf:li>
<rdf:li>209, 206</rdf:li>
<rdf:li>255, 255</rdf:li>
</rdf:Seq>
</crs:ToneCurvePV2012>
<crs:ToneCurvePV2012Red>
<rdf:Seq>
<rdf:li>0, 0</rdf:li>
<rdf:li>255, 255</rdf:li>
</rdf:Seq>
</crs:ToneCurvePV2012Red>
<crs:ToneCurvePV2012Green>
<rdf:Seq>
<rdf:li>0, 0</rdf:li>
<rdf:li>255, 255</rdf:li>
</rdf:Seq>
</crs:ToneCurvePV2012Green>
<crs:ToneCurvePV2012Blue>
<rdf:Seq>
<rdf:li>0, 0</rdf:li>
<rdf:li>255, 255</rdf:li>
</rdf:Seq>
</crs:ToneCurvePV2012Blue>
</rdf:Description>
</rdf:RDF>
</x:xmpmeta>
A1ex, Danne,
I didn't understand what you were writing above, of course, though it wasn't intended for me obviously.
BTW, thank you for such a huge work on making shooting video on Canon a pleasure. All the best to you!
@reddeercity, beautiful edit.
In case anybody would like to download my other 3.5K MLV's of Yosemite, i've uploaded them here:
Waterfalls — 1,3 Gb — 11 sec https://drive.google.com/file/d/0B47FS5gpVSwObWxhTTNiT0k1TW8/view?usp=sharing
(https://thumb.ibb.co/cAuJ8F/Screen_Shot_2017_06_10_at_4_31_51_PM.png) (https://ibb.co/cAuJ8F)
Mountain river — 2,2 Gb — 20 sec
https://drive.google.com/file/d/0B47FS5gpVSwOSkZYZU8tSHZ6aTQ/view?usp=sharing
(https://thumb.ibb.co/eLT6vv/Screen_Shot_2017_06_10_at_4_32_09_PM.png) (https://ibb.co/eLT6vv)
Thanks , I'll check out the other Yosemite shots .
For all win users -- I been looking at different ways to use mlv_dump with out the cmd line interface as some have a issue with that .
So in a nut shell any of the old PC app that use mlv_dump , just replace it with the latest ver. from the experimental 4k crop rec download page.
It works with MLVBrowseSharp & MLVViewSharp MLV Viewer to view .mlv (and .raw) files on windows (C#) (http://www.magiclantern.fm/forum/index.php?topic=8447.msg77759#msg77759) FYI this is cross platform (PC & MAC) .
I really like this old app it just never got finished being develop , it very powerful you can browses to any file location and extract dng's or export as legacy .Raw .
The reason I being that up is if you do convert to .Raw app's like MLVProducer (http://www.magiclantern.fm/forum/index.php?topic=15271.msg148538#msg148538), ML Raw Viewer 1_3_4 (http://www.magiclantern.fm/forum/index.php?topic=9560.msg132429#msg132429), can then work with the new UHD/4k resolution's when shot with compressed
raw (it will be a bigger filer thou with uncompressed14bit) . I should mention with MLVBrowseSharp there's no thumbnail preview but does extract the dng's correctly by a right click
of the mouse hovering over the file and MLVViewSharp there is pink highlight (works with converted .Raw , .MLV compressed raw error out) the great thing about MLVViewSharp
is you can change the raw debayering down to 1/8 of the resolution for quick playback plus many other image processing , LUTs etc. .... .
The other one I check out was MLV Converter 1.9.2 (http://www.magiclantern.fm/forum/index.php?topic=10198.msg98284#msg98284) , replaced mlv_dump and I could export dng's , 1080p proxy all at the same time , I can even export with Lossy compression
which gave me a 1.35MB 3.5k file verse a 8MB uncompressed 14bit frame . A.E. ACR see them and can be imported but not in to Resolve 12.5 .
I tested the Lossy dng compressed to 1.35 MB pre-frame in A.E. and I could not tell the different between the compressed or uncompressed raw file.
Hope that helps a little.
Thank you very much for this info reddeercity.
I was searching for a way to preview 14bit lossless mlv's without converting to dng's, and had no success.
Thanks!!!!!
:(
Tryed but no success.
I can't view 14bit lossless mlv's in MLVviewSharp.
mlv_dump replaced for the last version from experimental builds in same directory.
Did I miss something?...
I did not know the existence of MLVViewSharp, thanks you reddeercity.
But do you know how extract DNG into separate folders ? All the DNG are exported in the same directory.
Quote from: pc_bel on June 11, 2017, 12:39:16 PM
Tryed but no success. I can't view 14bit lossless mlv's in MLVviewSharp. Did I miss something?...
Quote from: reddeercity on June 11, 2017, 05:44:35 AM
..... MLVViewSharp there is pink highlight (works with converted .Raw , .MLV compressed raw error out)
@pc_bel you need to convert .mlv to .raw legacy , unless the developer "g3gg0" is willing to update to support compressed raw
that's the workaround at the moment . FYI you could request an update in his thread (http://www.magiclantern.fm/forum/index.php?topic=8447.msg77759#msg77759) ;)
One workaround previewing any MLV footage is going through MLVFS. Once the virtual dng files are created MlRawViever will be able to read the dng sequence. Just right click a dng file and select MlRawViewer.
Quote from: GutterPump on June 11, 2017, 01:04:14 PM
I did not know the existence of MLVViewSharp, thanks you reddeercity.
But do you know how extract DNG into separate folders ? All the DNG are exported in the same directory.
Just use MLVBrowersSharp , right click on DNG+wave then it open to a window to save , (see next image)
(https://c1.staticflickr.com/5/4197/34863292500_20ff8c061b_n.jpg) (https://flic.kr/p/V7Kuf1)
1 (https://flic.kr/p/V7Kuf1) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr
Select which drive or folder you want.
(https://c1.staticflickr.com/5/4195/34441432383_18e40fb1e8_n.jpg) (https://flic.kr/p/UttkVH)
2 (https://flic.kr/p/UttkVH) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr
FYI: 10-12bit uncompressed support with metadata , this is how it should work :D
(https://c1.staticflickr.com/5/4259/34863459760_e1bcd26f30_n.jpg) (https://flic.kr/p/V7LkXN)
10_12bit_5D2+metadata (https://flic.kr/p/V7LkXN) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr
Quote from: Danne on June 12, 2017, 04:23:53 AM
One workaround previewing any MLV footage is going through MLVFS. Once the virtual dng files are created MlRawViever will be able to read the dng sequence. Just right click a dng file and select MlRawViewer.
Yea I know , but way to slow especially with UHD/4K it even lags in full HD with mlvfs .
On PC's best option for compressed raw is to convert to .raw legacy and use the existing PC apps .
There's more reasons to use MLVViewSharp then just for viewing the files .
MLVProducer developer was thinking of supporting compressed raw in his app , until then work arounds
@Danne
@reddeercity
Thank you very much!!!!.
Quote from: reddeercity on June 12, 2017, 05:19:58 AM
Just use MLVBrowersSharp , right click on DNG+wave then it open to a window to save , (see next image)
(https://c1.staticflickr.com/5/4197/34863292500_20ff8c061b_n.jpg) (https://flic.kr/p/V7Kuf1)
1 (https://flic.kr/p/V7Kuf1) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr
Select which drive or folder you want.
(https://c1.staticflickr.com/5/4195/34441432383_18e40fb1e8_n.jpg) (https://flic.kr/p/UttkVH)
2 (https://flic.kr/p/UttkVH) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr
Thanks for your answer, but as you can see in my picture :
(https://image.noelshack.com/fichiers/2017/24/1/1497262305-batch.jpg)
all my DNG sequence are mixed in a same folder; i see you are using windows 7, maybe this issue is only about windows 10.
Quote from: reddeercity on June 12, 2017, 05:31:01 AM
Yea I know , but way to slow especially with UHD/4K it even lags in full HD with mlvfs .
On PC's best option for compressed raw is to convert to .raw legacy and use the existing PC apps .
There's more reasons to use MLVViewSharp then just for viewing the files .
MLVProducer developer was thinking of supporting compressed raw in his app , until then work arounds
On a decent hardware there is no lag on PC (w/o antivirus installed) and there is no need to use legacy RAW, you can just uncompress (with appropriate mlv_dump) to MLV again and use whatever tool you desire (mlrawviewer etc)
Eh, I totally forgot about decompression of MLV files although I made use of it in cr2hdr.app 8).
Thanks for the reminder.
Sorry to annoy you,
I don't know what I'have done with the latest build (pretty sure a bad choice in bit depth in the hurry) but half of my converted mlv's are just blacks dng (latest mlv dump).
I'm asking just to be sure, but I think this shots are lost?
A sample of the problem:
https://mega.nz/#!vxkkGQJT!8GjEWu1dzUq3yVL7siRQI_u7TuPnFIm-vDmFCELu594
Black or just darker? You´re uploaded file has a dark preview image but opens just fine in acr or when converted to a tiff with dcraw. I get those darker files as well. Not sure why but it seems related to 10/12bit and only preview.
Really, it open fine in acr??I was pretty upset by the black preview I come here too quickly...
(You're right, all open fine in acr, it's my viewer, picasa viewer the culprit)!
Thanks for the quick reply Danne!
@g3gg0
Yesterday I did some shooting with latest build on 5dmk3
1920x800 @ 60FPS
https://drive.google.com/open?id=0B-d8ARtc7xwWQU9kVkktTW9NMEk
I cannot decompress some MLVs, here is a sample.
Use your wizardry
Thanks in advance
mlv_dump --dng m14-0847.mlv
works here.
Do this.
mlv_dump --dng --relaxed
Fixes the issue on mac.
If you use cr2hdr.app you find the relaxed switch under (14) autopsy settings
as i am on low bandwidth, what is the problem with the file?
which block fails?
can you extract the troublesome block using:
mlv_dump m14-0847.mlv --extract <blocknumber> --autopsy-file bad.bin
and then upload the bad.bin
Here´s a 7mb sample of the MLV file.
https://bitbucket.org/Dannephoto/magic-lantern/downloads/M14-0847_samples.zip
It´s the LJ92: Failed (-1) error.
thanks!
FYI
when fetching the latest mlv_dump (https://builds.magiclantern.fm/jenkins/view/Experiments/job/crop_rec_4k/) and some recent mlv_lite / mlv_rec, you can use this to determine the exact build version:
./mlv_dump <mlv_file> --extract-type VERS --payload-only --ascii
Very nice indeed.
@Walter
@g3gg0
I get -1 error.
I use the mlv_dump that is in the Experiments page.
:(
mlv_dump --dng --relaxed
processes 381 out of 400 frames.
Quote from: Walter Schulz on June 15, 2017, 09:34:53 PM
mlv_dump --dng --relaxed
processes 381 out of 400 frames.
Yes, I was about to report that.
The relaxed switch does the trick.
Gratitude
@a1ex / @g3gg0
Could you look in to the mlv_dump from experimental. When I convert 60P 3x3 Footage I still have a lot of Vertical Stripes in highlights, even though mlv_dump is doing Vertical Stripe Fix, I am wondering if the code is not fitting for the 60p 3x3 mode or maybe there is an offset which enhances vertical stripes?, Maybe because of the 4 dead vertical rows on the left of the frame (also a bug btw). Especially if I have lens flare or direct sunlight flaring in the lens, then the Vertical Stripes are more visible than anything I've recorded with normal 1920x1080 MLV 3x3. What could be the cause of it?
When I play the footage in mlvrawviewer and turn on Vertical stripe fix in that program, sometimes that will get completely rid of the stripes and other times it will make it worse. Its really weird to work with.
The heavier the grade, the more visible the stripes. To make MLV footage look really good, you have to push footage so much that it crosses that threshold where the stripes become really visible. Temporal Noise reduction will also at times enhance it even more, as the algorithms "think" its important information to keep.
Here is a cropped screenshot from mlvrawviewer with a LUT pushing the image from a log curve, to show you the severity of the stripes.
https://mega.nz/#!8YIHGIJB!hzewH2JBX55vELsy2MPdp3gPSo2mTzlNKe_CQIVqu_o
As you can see the, flaring enhances the vertical stripes a lot and in this shot its only part of the image, sometimes the entire image will be engulfed in flare and everything turns to Stripes.. I could not find my shot where I had a full flare, aber its egal, same thing.
Also, it would be really nice if you could add Allow Global Draw - OFF during recording to MLV lite as in the vanilla MLV. Because when I am pressing half shutter to remove overlays, I am getting vastly improved recording times, almost double the time and sometimes continues (depending on scene).
and another bug to report from 60p 3x3, when selecting ISO 100 the Raw Zebras become disabled and the Histogram maxes out at about 90% the way to the right, the channels don't go beyond it, leading the operator to believe that one is well within ETTR range. The non-raw spotmeter also maxes out at 96% (not sure if also at other iso's).
Also a general question, why is it when hitting record button, that the recording starts so "sluggish", like 1-2 secs after, does it need to write information to a buffer or something, is it not something that could be done before hitting record, I mean just by entering the mode and keeping it Armed for recording? Also when recording stops, it takes about 4-6 seconds before I can hit record again, maybe longer.. It is quite counter productive, when one wants to shoot a lot of bursts at times and having to wait these combined almost 10 seconds per clip. Is it something one can expect some improvements on in the future ?
123 Fir
June 3rd build (I think it was, or the one thereabouts)
Thank you for your time.
Quotehttps://mega.nz/#!8YIHGIJB!hzewH2JBX55vELsy2MPdp3gPSo2mTzlNKe_CQIVqu_o
In this particular case, I believe the vertical stripes are there because - just a guess - the first frame probably did not have relevant highlights to be analyzed.
Another approach that I didn't implement, but only documented the backend support for it (http://www.magiclantern.fm/forum/index.php?topic=17795.msg171595#msg171595): it would require pointing the camera to something like a gray wall and running a calibration routine that would adjust the column gain registers (8 multipliers) until the artifact disappears. Might be worth trying.
Quote
Allow Global Draw - OFF during recording to MLV lite as in the vanilla MLV
This one was a major source of confusion, so my previous answer (http://www.magiclantern.fm/forum/index.php?topic=5533.msg93763#msg93763) is still valid.
Quoterecording starts so "sluggish", like 1-2 secs after
Sorry, not sure what you are talking about. Keep in mind I'm not a regular user of this mode (actually I don't remember touching the camera in the last few weeks...)
Is it the delay between you pressing the record button and the first recorded frame? Or the lack of responsiveness on the user interface?
QuoteI mean just by entering the mode and keeping it Armed for recording?
Pre-recording does exactly this.
Of course, the preparations can be done in background, with extra care when switching video modes, or when turning off raw recording. It's doable, but will increase the code complexity (although, probably not a bad idea, since some of the preparations are done in background in order to estimate recording times).
Quoteit takes about 4-6 seconds before I can hit record again, maybe longer
Set "Show graph: Buffers" to see what happens. All the frames that were recorded to RAM before you pressed Stop are now being saved to card (about 300MB iirc).
Quote
Is it something one can expect some improvements on in the future ?
If, in the future, you will select a resolution that gives continuous recording, then yes.
If we find some way to allocate more memory (the 5D3 has 512MB), this duration will increase (along with the recording time).
One can get a minor improvement (maybe speeding this up by 1-2 seconds) by pausing LiveView during this process. The silent picture module uses this trick, so it's an extremely easy coding task for those interested.
Quoteanother bug to report from 60p 3x3, when selecting ISO 100 the Raw Zebras become disabled
Will look into this one.
Quote from: a1ex on June 16, 2017, 09:38:25 PM
In this particular case, I believe the vertical stripes are there because - just a guess - the first frame probably did not have relevant highlights to be analyzed.
Another approach that I didn't implement, but only documented the backend support for it (http://www.magiclantern.fm/forum/index.php?topic=17795.msg171595#msg171595): it would require pointing the camera to something like a gray wall and running a calibration routine that would adjust the column gain registers (8 multipliers) until the artifact disappears. Might be worth trying.
You are exactly right, first few seconds of that clip has no flaring.
I'll take a look in to the rest of your reply tomorrow and the calibration.
I keep trying to install to firmware 1.2.3- 5d mark 3 however I cannot get the 4K tool. Can you please give the installation details? We tried on 2 different cameras but its not working at all :(
Any chance to get a flashing warning when card is almost full ?
Not during recording but when idle
QuoteAnother approach that I didn't implement, but only documented the backend support for it (http://www.magiclantern.fm/forum/index.php?topic=17795.msg171595#msg171595): it would require pointing the camera to something like a gray wall and running a calibration routine that would adjust the column gain registers (8 multipliers) until the artifact disappears. Might be worth trying.
Read the thread again and I've been so interested in Darkframe Averaging and flatframe since the talk started here a long time ago, but as a Windows user there is no GUI converter with these options. Unfortunately MLVFS does not work properly with AE and AE is at the core of my workflow. Can you link me to a thread that explains how to type in the different codes/commands for Darkframing and Flatframing in mlv_dump, I cannot find any good relevant thread or post, there are so many search results of mlv_dump, but if you know where I should look, that would be nice, if not I'll keep looking.
QuoteThis one was a major source of confusion, so my previous answer (http://www.magiclantern.fm/forum/index.php?topic=5533.msg93763#msg93763) is still valid.
Yeah, I just thought this 'Allow Global Draw - Off' was a specific "component" easy to port. I know what "Allow Global Draw - Off" means because I know what it does, but the wording of it makes no sense to me and even more confusing for newcomers.
QuoteSorry, not sure what you are talking about. Keep in mind I'm not a regular user of this mode (actually I don't remember touching the camera in the last few weeks...)
Is it the delay between you pressing the record button and the first recorded frame? Or the lack of responsiveness on the user interface?
Pre-recording does exactly this.
Of course, the preparations can be done in background, with extra care when switching video modes, or when turning off raw recording. It's doable, but will increase the code complexity (although, probably not a bad idea, since some of the preparations are done in background in order to estimate recording times).
Yes, recording starts 1-2 seconds after pressing record. I've tried with Pre-Recording and it does counter it but only after it has made the Pre-Recording which in it self takes 1-2 seconds to start recording aswell. On top of that, I have to activate the Pre-recording and then I have to press Record again, something that in the "heat of the battle" can get in to kind of an opposite effect where I record when I think I stopped recording or I stop a recording when I want to start it (happened a bit too much for me in 24p Mode.. and I only found out when I checked the footage ;D ) I don't know, maybe I just need to get more used to it.
But with Pre-Recording I am getting less recording time, I usually get around 400/500 to 700 Frames and with pre-record 1 sec I get 300-450's. I use the lowest amount 1 sec pre-record (60 Frames), maybe you could add 1 frame pre-recording specific for the 60/50p mode as a standard, just so that recording starts immediately as an "easy" workaround? Not sure if easy or not. The Background thing you mentioned, sounds even better.
I can't give any technical explanation on why the recording times are lower, maybe I've just been unlucky with the compression, I must add that I did not test it extensively. Just done a few shots with pre-record in scenes where I was expecting longer recording times than the 300 to 400s.
EDIT: Just did a few more tests, switching between Pre-Recording and no Pre-Recording. The recording time is almost halved with Pre-Recording 1 Sec.
And you not touching your camera or this mode, you really have to try out the 60p mode, it is absolutely amazing! It has given my camera a new spirit. I thought I would use it more professionally but for filming family and those speedy little kids, its amazing.
QuoteOne can get a minor improvement (maybe speeding this up by 1-2 seconds) by pausing LiveView during this process. The silent picture module uses this trick, so it's an extremely easy coding task for those interested.
I don't think I would mind the LV freezing during that period, I am unable to do anything in that time anyway, so if it shortens the wait time it sounds good to me. Though I would suggest a small text appearing right before the freeze saying "Writing from Memory" or something like that, so users wont experience frustration over their camera freezing after every take and not knowing why if they didn't read this thread or pull.
Thanks again for your time.
@Kharak
Darkframe processing
1 - Film a few seconds with caps lock on your lens(totally black). Let´s call it your input_darkfootage.MLV.
2 - Perform averaging on this darkframe file like so:
mlv_dump -o averaged.MLV -a input_darkfootage.MLV
3 - Now use your averaged.MLV with corresponding normal footage.( Note! Use same settings when filming darkframe as corresponding footage). in command prompt do:
mlv_dump --dng -o OUTPUT INPUT.MLV -s averaged.MLV
INPUT.MLV is your nomal filmed footage
OUTPUT is your end result(dng files).
*note
If you leave out --dng you´ll get a MLV file instead of dng´s.
mlv_dump -o OUTPUT.MLV INPUT.MLV -s averaged.MLV
Good point about flat frames - they are yet another option for correcting the vertical stripes (alongside with other defects, such as vignette or dirt spots on the sensor or on the lens), and the calibration can be done afterwards.
About Global Draw while recording - the difference I've got on a quick test is pretty small: about 450 vs 500 frames with Preview set to Frozen LV, or about 300 vs 350 frames with Preview Auto, tested in 10-bit uncompressed 1920x800 60p, on 1.2.3, with default overlay configuration - which includes CPU-intensive zebras. That's only one extra second with global draw off, and I'd expect a lower difference on 1.1.3, where zebras are hardware-accelerated.
@Danne,
Thank you very much for the codes.
But the averaging does not work with the 60P Mode files, I tried it on 24p 3x3 Vanilla MLV files and it worked. When I input a 60P file I get "windows has encountered an error" but Command Prompt does not exit or crash. It just generates an empty averaged.mlv
I have tried with short and longer recording times, 1 sec and 6 secs in case it was too little, but its the same. I had the range from 100 - 3200 ISO, 6 MLV files and none of them worked, not the 1 sec or 6 sec MLV's.
I have not tried Normal Lossless 24p 3x3 MLV's, to see if it has something to do with the Lossless mode, I will do that next.
I think the mlv_dump I used is the one from the 3rd or 4th of june (I dont know where to see older builds anymore with the changes on download page) , the one that is 759 KB in size.
@a1ex
I always shoot 14 Bit Lossless 60P 1920x800 with Real-Time Preview and I get about 500-700 Frames and with Pre-Recording ON it was down on 350 Max. Pressing and holding Half-Shutter almost always increase the recording time by atleast 2 seconds (real time). I say "almost" because sometimes but rarely the compression goes crazy, I have many times gotten 10-15 Seconds (real time) recording this way, where I just stop the recording because I probably got the burst shot I was aiming for. Like earlier when I was shooting Lens Cap on for the DF Averaging, first frame it crashed saying Compressed size was bigger than uncompressed. A completely black picture on ISO 100 and it could not compress it.
I have never tried 10 bit mode, but what I see from all the examples it is extremely noisy, perhaps that is why your recording times are so low, maybe the compression goes crazy from all the colour casts 10 bit footage introduces. But yeah.. 10 bit vs 14 bit, mathematically you should have higher record times than I am getting. no?
And why would the difference be lower on 113? I thought fir 113 always had higher performance than 123 because of the extra buffer.
QuoteBut the averaging does not work with the 60P Mode files, I tried it on 24p 3x3 Vanilla MLV files and it worked. When I input a 60P file I get "windows has encountered an error" but Command Prompt does not exit or crash. It just generates an empty averaged.mlv
I can confirm lossles 60p 1920x800 yields segmentation fault 11 when trying to create an averaged darkframe.
Workaround:Decompress your compressed footage prior to darkframe processing. Just tested and working.
mlv_dump -d -o OUTPUT.MLV INPUT.MLV
Quote from: Kharak on June 18, 2017, 05:04:55 AM
when I was shooting Lens Cap on for the DF Averaging, first frame it crashed saying Compressed size was bigger than uncompressed. A completely black picture on ISO 100 and it could not compress it.
That was a corrupted frame.
http://www.magiclantern.fm/forum/index.php?topic=19300.msg183115#msg183115
Quote
I have never tried 10 bit mode, but what I see from all the examples it is extremely noisy, perhaps that is why your recording times are so low
I've only used it in order to get constant bitrate. With lossless compression, the recording times will vary a lot with scene contents (so it's harder to benchmark this way; a static scene or a dark frame will make things a bit more repeatable in this case, but I didn't want the extra trouble.)
Quotemaybe the compression goes crazy from all the colour casts 10 bit footage introduces.
I thought I've mentioned "uncompressed"...
Quote
But yeah.. 10 bit vs 14 bit, mathematically you should have higher record times than I am getting. no?
No.
10 < 14
10 > 14 * 60%
Quote
And why would the difference be lower on 113? I thought fir 113 always had higher performance than 123 because of the extra buffer.
The cause is lower CPU usage on both Canon's side (regardless of ML settings), and on ML's side (only when Luma-based zebras are used). Memory bandwidth usage might also be a bit higher on 1.2.3 on Canon's side (didn't check this one, as it's not straightforward to measure). Of course, absolute performance can be higher both with and without global draw, but the difference between these numbers is likely lower (that is, global draw even faster on 1.1.3) iff you use Luma-based zebras (which is the default). I didn't benchmark these cases; it's just what I'd expect to happen.
@Danne,
Yes, now it worked generating averaged.mlv and outputting DNG but it is a very tedious process, having to decompress every file individually, rename it, move it to the appropriate folder... So the next big question, how do I batch process this ? Some command I should know? :)
Or atleast batch the decompression part. After decompression I see all the Metadata, ISO, fstop and so on in the MLV's. That way I can sort the MLV's with the same ISO's to their appropriate folders and do the DFA from there.
After DNG conversion, the metadata is lost again. a bug ?
Quote from: a1ex on June 18, 2017, 08:32:15 AM
I thought I've mentioned "uncompressed"...
Apologies, that slipped through.
QuoteNo.
10 < 14
10 > 14 * 60%
I don't understand this sorry, 10bit is smaller than 14bit, 10bit is bigger than 14bit at 60% do you mean with the lossless compression? Is there less to compress from 10 bits ?
Sorry for my silly questions all the time, but I do appreciate very much you taking your time explaining these things, so that I know what is going under the hood.
Cr2hdr.app batch processes both decompression and darkframe processing. Mac. Someone with an interest in windows scripting can do the same.
Quote from: Kharak on June 18, 2017, 05:04:59 PM
Is there less to compress from 10 bits ?
http://www.magiclantern.fm/forum/index.php?topic=19300.msg183231#msg183231
By the way. With latest version of mlv_dump on steroids(Bouncyball) the -b switch will work exporting different bits. Good for comparing let's say 14bit file into 8bit dng's.
mlv_dump -b 8 --dng Input.MLV
QuoteGood point about flat frames
What was the point about flat frames? It's just some coding stuff that didn't yet exist as some executable for Windows users? I'm really interesting about vertical stripes problem cause I have horrible ones (at the sky usually)
(http://image.ibb.co/cnh3Xk/M30_0921_000001.jpg)
is there a way to get rid of it for Win users?
Please use search function. Flat frame or flatframe.
The function exists in mlv_dump which is the same for both windows and mac.
Flat frame workflow isn't trivial. I experimented with it some a while ago but there is more to test out.
Here is one link
http://www.magiclantern.fm/forum/index.php?topic=17795.msg175189#msg175189
1. take a "flat field" video
a) put a white shirt over lens
b) point to the bright sky
c) record using the same settings as before (same exposure time not strictly needed, but would keep the ISO)
2. average that video using the -a option (e.g. mlv_dump movie.mlv -a -o flat.mlv)
3. process the problematic footage and supply this flat.mlv as flatfield (e.g. mlv_dump in.mlv -t flat.mlv -o out.mlv)
Danne, g3gg0 thanks a lot, I will try and will make some tests
btw.
to only pick the vertical stripes instead of lens speckles and vignetting also, you can add --avg-vertical when creating the flat.mlv
didn't try any of those on my own :)
but that could also introduce some weird left/right bar due to vignetting.
probably not a good thing when you have the lens on
Anyone who have footage of 1920x1080 @ 45p and 48p (3x3 binning)? I have tried to pull it of but images results in black blocks or other faults. Anyone?
Edit: tried in Canon menu All-I 50p and 48p in FPS ovettige and reciewed this massages:
(http://thumb.ibb.co/is8hSk/IMG_0550.jpg) (http://ibb.co/is8hSk)
Edit 2: Changed to 45p now worked a bit better. Got 7xx frames before stoped.
Edit 3: You who managed to record 45p and used it in a NLE, how much did you slow it down? Since 48p is really simple with 24p timeline. I tried to slow it down with 53% and it seemed ok. Any thoughts?
Quote from: D_Odell on June 19, 2017, 02:29:30 AMEdit 3: You who managed to record 45p and used it in a NLE, how much did you slow it down? Since 48p is really simple with 24p timeline. I tried to slow it down with 53% and it seemed ok. Any thoughts?
I think the easiest (in Premiere / After Effects) would be to conform the clip (using the Interpret/Modify command) to 23.976 or 25p depending on your edit needs. Then you won't have to figure out the exact right clip slow-down rate and it will probably be cleaner looking than slowing the clip back down with the NLE, which runs the risk of blending frames or having to generate new ones if it's not dead-on. Hope that made sense :)
Quote from: Oedipax on June 19, 2017, 10:00:17 PM
I think the easiest (in Premiere / After Effects) would be to conform the clip (using the Interpret/Modify command) to 23.976 or 25p depending on your edit needs. Then you won't have to figure out the exact right clip slow-down rate and it will probably be cleaner looking than slowing the clip back down with the NLE, which runs the risk of blending frames or having to generate new ones if it's not dead-on. Hope that made sense :)
That made excellent help! Works good in ae also. What shutter speed do you use? At first I left it at 94, but will probably go up to 180 next time? As one can see its quite blurry, also shot from a train...:
(http://thumb.ibb.co/doMb55/Comp_1_00884.png) (http://ibb.co/doMb55)
(http://thumb.ibb.co/k5a8JQ/Comp_1_03840.png) (http://ibb.co/k5a8JQ)
Also, anyone notised the black pixel line in bottom and to the left? If so anyone made it correct?
@a1ex, @g3gg0, @danne
I am thinking if the 60p mode is stressing the sensor so much that it is making the Vertical Stripes more prevalent than 24p 3x3? Is that a possibility? I am saying this because normally I didn't have issues with Vertical Stripes when I was shooting 24p, sometimes I would get some in highlights after heavy grade, but after getting addicted to 60p I am seeing a lot of shots with Vertical Stripes.
And I am not seeing Vertical Stripe fix working properly with 60p footage. These shots have the same exposure and lighting across the entire recording and yet the Vertical Stripes are still there. In the first screenshot its harder to see the the lines, but when playing the recording as this is a Slow pan shot, it becomes very visible. in shot 2 its much more visible, I also zoomed 200%.
Shot 1
https://mega.nz/#!QdpETRZR!_y6mdX7ZWGkVisIamJLLAeyss9uzdsEdMtn3TgETlTc - Screenshot from AE - ISO 200
The jpeg compression is hiding it in this shot, look furthest to the left, some remnants of the stripes are there.
Shot 2
https://mega.nz/#!wAhwzawJ!jirFfeb5WrvW6YdZ6S9ubqsXbdKTItaw1oGboUQKDh4 - Cropped - ISO 200. Here you can see them.
FYI: In this case, activating Stripe removal on the DNG's in MLRawViewer removed the stripes, but introduced big chunky vertical lines from all the shadow parts of the image.
This same shot without Darkframe Averaging makes the Vertical Stripes less prevalent, but still there, its as if the deep blacks from Darkframe averaging is enhancing the stripes or the contrast of the stripes.
My point with this post is that I think that mlv_dump stripe removal might need some optimization for 60p mode, I might be wrong and doing something wrong in the process introducing this problem myself, I'd love to be proven so. But its a big issue for the 60p mode.
Just a guess here, but when decompressing, is it also doing vertical stripe fix? and then converting to DNG the same process is done again introducing the stripes again? I don't see any text saying so when decompressing, but just a thought.
Anyone else shooting 60p having issues with Vertical Stripes?
Thank you for your time.
Could you post an example MLV with stripes?
Are you sure vertical stripe code is running on all clips?
Quote from: GutterPump on June 12, 2017, 12:16:53 PM
Thanks for your answer, but as you can see in my picture :
all my DNG sequence are mixed in a same folder; i see you are using windows 7, maybe this issue is only about windows 10.
Ok --I see the issue .
So to anyone that want to use the new mlv_dump.exe on Windows platform to batch your UHD/4K/10-12bit & compressed14bit mlv's in to separated file folder.
First I thought of making a .bat (win7) & or .cmd (win10) (really the same thing) but I had problems with importing more one file at a time
(to much code and getting too complex) . Then I considered building a simple .exe batch program , then it dawn on me why I'm I trying to
re-invent the wheel here ::) , so after a quick search I found a Older Win's app that uses mlv_dump.exe . MLVConverter1.9.2.zip (https://www.dropbox.com/s/7qhncp7pi4t268o/MLV%20Converter%201.9.2.zip)
MLV Converter 1.9.2 for WINDOWS (https://www.magiclantern.fm/forum/index.php?topic=10198.msg98284#msg98284) You will need some other tool if you want to have it fully functional --Adobe DNG Converter , mlrawviewer (only to view file not working work compressed raw) & IrfanView .
All I did was to replace the mlv_dump that there with the new mlv_dump from the experimental download page , installed the latest Adobe DNG Converter (to output compressed dng's)
and IrfanView . I tested 14 mlv's files which included 10-12bit , 14bit, UHD(3520x1320) compressed raw 14bit and all exported fine in to the respected folders . The dng's import ok in to
After Effect CS6 (Wins) didn't try any other app . *Note* in the tools folder you have (3) mlv_dump's (mlv_dump , mlv_dump1 & mlv_dump2) just copy the new mlv_dump and re-name it
mlv_dump1 etc. .... that's it should work fully from there with 1080p proxy you can add a customs .xmp file from A.E. and have it export to ffmpeg prores4444 if you wish .
If you want to export Dual ISO you need to update cr2hdr.exe to the latest windows version.
One thing windows will ask permission to run mlv_dump on every file unless you go the properties and in the general tab check "UnBlock" at least on Win7 , win10 should be the same
Hope that helps :D
Edit: update exiftool.exe also
(https://c1.staticflickr.com/5/4228/35290258021_e3e801ee01.jpg) (https://flic.kr/p/VLtNdv)
mlv raw converter1 (https://flic.kr/p/VLtNdv) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr
(https://c1.staticflickr.com/5/4282/34610387673_b85bb228b0.jpg) (https://flic.kr/p/UJphrX)
mlv raw converter2 (https://flic.kr/p/UJphrX) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr
(https://c1.staticflickr.com/5/4260/35290258091_b3a956b54a.jpg) (https://flic.kr/p/VLtNeH)
mlv raw converter3 (https://flic.kr/p/VLtNeH) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr
@reddeercity
Thank you very much!!!
But... I'm on windows 10 now and mlv_dump processes mlv files in separate folders when I select all and use the "send to" comand found here http://www.magiclantern.fm/forum/index.php?topic=10526.msg102123#msg102123 (http://www.magiclantern.fm/forum/index.php?topic=10526.msg102123#msg102123)
Anyway,
I think that using the workflow reddeercity suggest, you have a lot of advantages. Now I'm using it!!
Thanks!!!!!!!!!!!
Quote from: D_Odell on June 19, 2017, 11:52:52 PM
That made excellent help! Works good in ae also. What shutter speed do you use? At first I left it at 94, but will probably go up to 180 next time? As one can see its quite blurry, also shot from a train...:
(http://thumb.ibb.co/doMb55/Comp_1_00884.png) (http://ibb.co/doMb55)
(http://thumb.ibb.co/k5a8JQ/Comp_1_03840.png) (http://ibb.co/k5a8JQ)
I usually go by the 180-degree shutter rule of thumb, where the shutter speed is 2x that of your framerate. 24fps = 1/48 (or 1/50), and so on. So if you are shooting 45P, 1/90 (or 1/94 if that's what is available) on the shutter should be a good starting place. If it's still too blurry for your liking, you could keep pushing it further like 1/120 or 1/250 and see if you like that additional clarity.
Quote from: Danne on June 20, 2017, 05:50:01 AM
Could you post an example MLV with stripes?
Are you sure vertical stripe code is running on all clips?
Perhaps tomorrow or the day after I'll have access to high speed internet. I will send you a MLV.
Yes, mlv_dump says Vertical Stripe Correction and those numbers appear aswell.
Just saw that Bouncyball made a new version of mlv_dump on steroids --force stripes could be a solution?
tried the flat (https://www.magiclantern.fm/forum/index.php?topic=19300.msg185962#msg185962) field (http://www.magiclantern.fm/forum/index.php?topic=17795.msg175189#msg175189) videos to get rid of stripes?
tend to be the most reliable ones, as they are not just guessed from one frame and also care for speckles etc
No I have not tried flat field yet.
Current conditions are not ideal to make a flat frame. But I will try it out, for sure.
On another note, the --force-stripes has done a terrific job at removing the stripes, there are stripes, but virtually invisible. But it is slow.. Very slow.. Also with no batch process I am seeing a lot of issues ahead. I have a music video coming up, that I wanted to shoot all 60p, but with so many manual steps, decompressing and sorting, converting and having to stay glued at the screen to start next process when one is done, I am feeling like this is way too much to handle. Blaaah..
Screenshot with diagonal split -
https://mega.nz/#!QFYSwZRC!cpK0HV0lpmVNR5JAtETDqA7e5rGMb7LEieT8JGeYGQs
I hope the jpeg compression will do it enough justice for you to see. I did not do darkframe subtraction on the --forced-stripe footage. Did not look in to how I could do both with the Steroids_dump
Do you think the flat frame would be a quicker step ? It will also require sorting, even more so I assume, each lens to its own? each f-stop to its own ? Darkframe+Flatframe sounds like the ultimate quality we can get with our Canons, but it seems overly complicated for a project shot with multiple lenses at various f-stops and having some manual lenses, it will be hard matching f-stops to the right flatframe and on top of that having to write in the commands for each shot is very tedious. For photography, it seems like a no-brainer though.
@danne
I'll have 3G access tomorrow, I'll send you the mlv.
Flatframe for each lens is needed. Not sure of the ideal f-stop when it comes to flat frames. I had good results with around f5.6. I used light from a strong in door lamp. Pointing to the sky is probably better.
Since you are getting better results with --force stripes with the steroid version it indicates that mlv_dump is either using different code or isn't always on.
There are newer and older vertical stripes code so it could be an idea to compare those maybe? I think steroid version is using what is used in mlvfs(older mlv_dump code), havn't checked.
If someone had and found a solution to crop rec: cache is locked down? How do you free it?
Thanks!
Quotecache is locked down?
What do you mean?
Quote from: Danne on June 22, 2017, 09:25:46 AM
What do you mean?
(http://thumb.ibb.co/k5wbOQ/IMG_0571.jpg) (http://ibb.co/k5wbOQ)
(http://thumb.ibb.co/gLOO3Q/IMG_0572.jpg) (http://ibb.co/gLOO3Q)
Or is it nothing to worry about?
Also the colour green is now yellow or orange in Live view. Will update how the dng:s are. Weird, when removing a perfectly balanced ND 9 filter the colours are good again.
Here´s some more testing around HDR filmed like this:
Added darkframe averaged file to cleanest image possible(decompressed). Note that adding darkframe in this case probably won´t make much difference. Already got plenty light to work with.
Tried combining with an averaged flatframe file but got weird results. Didn´t test with flatframe solely.
Settings in Crop mode:1920 50/60 3x3 (PAL) 50fps
RAW video settings: 1720x960 50fps(continuous)
Processed with after effects HDR finished result 25fps
With some trickery I use a HDR AE template in cr2hdr.app which goes through aerender(command line render engine in after effects). In main menu select (p) then (23) for a template. Your MLV files have to be transcoded into folders with dng files through cr2hdr.app.
There is a bug with 1920 50/60 3x3 setting. Once you exit liveview after filming you can´t open liveview again without getting a completely black screen. Exiting the selected mode opening up liveview and reselecting the mode will make it work again.
Some examples:
(https://s8.postimg.org/kl5e8lfid/01a.png)
(https://s8.postimg.org/l0go1m1fp/02a.png)
(https://s8.postimg.org/gfuhmohqd/03a.png)
(https://s8.postimg.org/ey91b4czp/01b.png)
(https://s8.postimg.org/dzso8tznp/02b.png)
(https://s8.postimg.org/x65vc0g5h/03b.png)
Movie sequence checking movement and ghosting related to 50fps
Tested the 3.5k crop mode 3584x1320 on the lowest compression setting (11-8bit) Lossless, to see how it holds up, pretty pleased with how stable it is, no dropped frames or automatic stoppage's despite shooting some highly detailed scenes, it also preserved excellent colour detail too, fantastic job Alex.
As usual liveview is a challenge to work with, but workable with some extra attention, also there one clip that is standard 1920x804 cuts very well with the crop clips, see if any of you can guess which one it is?
EF70-200
FCPX, Colour Finale
https://www.youtube.com/watch?v=Ax0BL_Ci9po&t=5s
Hi folks - recent convert from GH4 and love it. I noticed your comment below @g3gg0
Quote from: g3gg0 on June 04, 2017, 12:26:31 AM
having patches on my disk.
wont take too long.
I assume that maps to this PR: https://bitbucket.org/hudson/magic-lantern/pull-requests/837/mlv_snd-and-dual_iso-support-for-mlv_lite ? If so, would have some testing done help at all? Happy compiling and checking it out on the 5d3 if that's of use.
John
@A1ex
Just tested the June 19th Build, it fixed the ISO 100 Raw Zebras bug and Histogram bug. Thank you very much!
Not that I use the h264 Spotmeter for exposure, but just tested it aswell. Instead of maxing out at 96%, it maxes at 97%. Don't think it matters for the 3x3 60p mode, its Raw only right? just so you know. Personally I don't use h264, ever.
Quote from: bastonford on June 26, 2017, 02:30:48 PM
Happy compiling and checking it out on the 5d3 if that's of use.
Yes, please.
Quote from: Kharak on June 26, 2017, 05:11:36 PM
Not that I use the h264 Spotmeter for exposure, but just tested it aswell. Instead of maxing out at 96%, it maxes at 97%. Don't think it matters for the 3x3 60p mode, its Raw only right?
If it's a percentage or RGB, it's YUV-based. If it's EV, it's RAW-based.
(Side note: I think I'd also like a Kelvin spotmeter)
Quote from: Wayne H on June 24, 2017, 06:49:45 PM
Tested the 3.5k crop mode 3584x1320 on the lowest compression setting (11-8bit) Lossless, to see how it holds up, pretty pleased with how stable it is, no dropped frames or automatic stoppage's despite shooting some highly detailed scenes, it also preserved excellent colour detail too, fantastic job Alex.
As usual liveview is a challenge to work with, but workable with some extra attention, also there one clip that is standard 1920x804 cuts very well with the crop clips, see if any of you can guess which one it is?
EF70-200
FCPX, Colour Finale
https://www.youtube.com/watch?v=Ax0BL_Ci9po&t=5s
what was you post workflow on this?
@a1ex
June 19th Build:
fir 123
I can not playback 14 bpp lossless - 3x3 60p in camera. Just a lot of noise and flickering.
Quote from: a1ex on June 26, 2017, 05:20:47 PM
(Side note: I think I'd also like a Kelvin spotmeter)
That sounds really cool, so if I understand correctly I could measure the Kelvin Temperature instead of exposure? Does any camera do that?, I never even thought about that possibility. Would it measure on a RAW basis or depending on the liveview "colours", could a Picture Profile skew it?
That would be a tremendous help in mixed lighting.
Can I ask some questions here about 5D3's experimental build?
I don't know where to ask. If it is wrong place, I apologize.
1.
I always think that why there is no standard resolution such as below?
Are these size so difficult to build?
Probably I think almost people using ML on 5D3 wish for them below.
2880x1620(3K HD 16:9)
3072x1620(3K 1.9:1)
3840x2160(4K UHD 16:9)
4096x2160(4K 1.9:1)
Why is it only incomplete resolution now, such as 3072x1920 or 3840x1600?
2.
I want to use 4096x2560 for the stabilized footage of 3840x2160.
Is it impossible to record at 24p?
(8bit, 10bit or 12bit raw is fine. H.264 is also fine.)
Can someone please make it?
3.
I don't want to push the crop button.
Is it impossible to be crop mode from the beginning?
I think it seems save time. What do you think?
4.
I want to participate in ML development, but I don't have the knowledge of it.
What should I study for ML development?
Always thank you for development of ML.
You saved my old 5D3 camera.
I want to use it on the 4k field.
Hope your reply soon.
Quote from: a1ex on June 26, 2017, 05:20:47 PM
Yes, please.
So compiling the PR for audio (with the 11 commits to crop_rec_4k since it was opened), gives me the following on 5D3.113:
- Modules enable fine and I can enable mlv_sound in the sound menu
- Sound records in MLV videos in full-sensor mode
- Sometimes when hitting stop, there is an error message on screen stating that "sound did not stop: state 5", however I can't reliably recreate it.
- There seems to be significantly more write speed required than "advertised" when sound is on - my slower Sandisk Extreme 120 card can't keep up for more than 400 frames with sound, when it can go continuous compressed raw without. Lexar Pro 1066x card is fine at 3k 2.39 though.
I'm a bit unsure of what the outcome should be when recording sound in the new crop_rec mode. There is a WAV output at the end of the run with mlv_dump, but it is only 44 bytes on crop_rec videos. Still new around here - is mlv_dump another compilation target within the magic lantern repo? I'll get googling.
Will report back as I come across more.
Quote from: nikki on June 26, 2017, 09:09:43 PM
what was you post workflow on this?
Hi nikki, i imported all MLV files into martin herings great mac app called FOOTAGE, corrected colour balance and then applied C Log, exported to prores 4444.
Setup a 4k timeline within FCPX, imported the 3584x1320 MLVs and used Color Finale Pro to add alexa LOG C LUT, from lut utility within color finale, then used the colour wheels and vectors to colour grade.
Exported as a 4K H.264 file and uploaded to youtube as a 4K file.
@a1ex
I was reading in the MLV_Play thread that disabling raw_twk should make playback work, I tried that and it did not work on 60p mode, "No decompression in this mode".
http://www.magiclantern.fm/forum/index.php?topic=9062.msg186484;topicseen#msg186484 the thread.
Does raw_twk affect recording in any way? I've just had it on all the time since lossless was released, I forgot about it, does it have any other purpose than speeding up playback? (which was it's original intent, AFAIK)
Since I only use older lenses (nFD glas), around 30-40 years of age. Would it be better to choose a mode that uses the center parts of the sensor more than the higher part of lens glas? In my experience the UHD options render only from the top part of the sensor while center, yeah is center.
Any thoughts on which options is best to use, I mean in terms of lens qualities and DNGs to use?
Option 1: UHD 1:1 3232 x 1376
Center crop 3,5k: 3232 x 1376
From a sequence I shot today I experienced a lot of CA and bleeding. Maybe it's just the lens quality?
DNG: https://www.dropbox.com/s/ffr4mgcz0argy7o/M28-1359_001359.dng?dl=0
This one uses UHD 1:1 option.
All lenses normally have most sharpness in the Center, so it should always be the best option unless you want CA.
Realised that crop-rec 3,5k doesn't cooperate as well as UHD at higher resolution. Anyone experienced the same?
@bastonford, so glad you're compiling this! Good luck with it. For either run & gun with a Rode VidPro or for dual system audio having the WAV recording in the 5Diii is huge help. It'll add years of usefulness to my camera.
Great to hear you're getting continuous 3k and sound.
I stopped playing with card spanning a long time ago but I always thought it'd be great if the audio could write to the SD card while the video wrote to the CF card. I'm guessing there are underlying Canon issues keeping that from happening or that it doesn't help write speeds at all.
Quote from: bastonford on June 27, 2017, 07:43:13 PM
So compiling the PR for audio (with the 11 commits to crop_rec_4k since it was opened), gives me the following on 5D3.113:
- Modules enable fine and I can enable mlv_sound in the sound menu
- Sound records in MLV videos in full-sensor mode
- Sometimes when hitting stop, there is an error message on screen stating that "sound did not stop: state 5", however I can't reliably recreate it.
- There seems to be significantly more write speed required than "advertised" when sound is on - my slower Sandisk Extreme 120 card can't keep up for more than 400 frames with sound, when it can go continuous compressed raw without. Lexar Pro 1066x card is fine at 3k 2.39 though.
I'm a bit unsure of what the outcome should be when recording sound in the new crop_rec mode. There is a WAV output at the end of the run with mlv_dump, but it is only 44 bytes on crop_rec videos. Still new around here - is mlv_dump another compilation target within the magic lantern repo? I'll get googling.
Will report back as I come across more.
I'm still a little stuck on this - Footage and mlv_dump seem pretty convinced there is sound in there for higher resolutions looking at the metadata, but when exported it's just an empty WAV file. I'm still trying to work out if this an in-camera thing or mlv_dump that isn't doing the right thing.
I have to say though, on the 5d3, the FF 1080p 14bit is stunning quality if you de-bayer it onto a 4k timeline in Resolve, rather than upscaling the video at the end of mastering it. Obviously 4k is way slower to work with, so I tend to edit in on a DCI2k 2.39 timeline, and then just flick the timeline resolution up to the 4k option before delivery. Not saying that higher resolutions aren't necessary, but the most exciting thing for me is 14 bit compressed RAW with audio - a ~45% space saving makes a huge difference on bigger projects!
Compiling the latest state of the "crop_rec_4k_mlv_snd" branch gives me this error on all takes using the 3k 1:1 mode.
ML ASSERT:
slots[slot_index].size < max_frame_size
at mlv_lite.c:3335 (raw_video_rec_task), task raw_rec_task
lv:1 mode:3
Magic Lantern version : Nightly.2017Jul03.5D3113
Mercurial changeset : 1ed473161338 (crop_rec_4k_mlv_snd)
Built on 2017-07-03 20:05:26 UTC by [email protected].
Free Memory : 134K + 3091K
This is with a Lexar 1066 card, and it seems to run fine for about 8-10 seconds and then bug out with that error dumped to the card. This is mlv_sound module enabled and the module turned on in the audio menu. On the normal crop_rec branch the 3k is continuous without any issues.
okay interesting.
can you tell me the exact settings so i can try to reproduce?
Of course - thanks for looking into it.
5D3.113 build of commit 1ed47316133855e7f459c98ec52f4e79e4f27952
Modules enabled:
- bench
- crop_rec
- mlv_lite
- mlv_play
- mlv_sound
Sound enabled in the Audio menu
Canon settings:
1080/24P All-I
ML Movie settings:
FPS override off or exact 23.976 (doesn't seem to matter which)
Image Fine Tuning -> Shutter +0.84ms (1/48)
Crop mode either 3k or UHD 1:1
Resolution 3072 in 2.39 crop
SRM memory enabled (disabling it shortens record time for me)
14bit lossless
Edit: Global Draw in auto mode
Happy to try anything to help - just let me know! Does this work for you? What are you using to convert the MLV files?
Thanks :)
John
Can someone give a clue how to make camera to rec h264 proxy on SD and not on CF card?
Hi there,
I tested this experimental firmware and that's amazing the video I can get from my "old" 5d mark iii, a big thanks to the community and THE developers.
My only concern is about missing mlv_snd module which is for me the reason I won't keep this firmware on my camera right now but I've seen that you are working on a subbranch which add this module, is it already somehow working or blocking issues are still to be fixed ?
I would have love to help (being myself a developer but more web/mobile oriented), I dont have the level I guess to dig into C/Perl code base.
Thanks again and if you need testers I am here.
Quote from: Savely on July 06, 2017, 11:53:26 AM
Can someone give a clue how to make camera to rec h264 proxy on SD and not on CF card?
In ML Menu under the tool icon you will find "Card Settings" in that menu you will find Preferred Card choose SD Reboot and your done.
Quote from: etienne on July 08, 2017, 04:42:48 AM
In ML Menu under the tool icon you will find "Card Settings" in that menu you will find Preferred Card choose SD Reboot and your done.
Thanks, I did, but it still doesn't rec h264. So I assume this option doesn't support any crop_rec mode?
Anyone know why I get blank screen playbacks for anything other 14bit? Anything recording in 10 or 12 bit at any frame rate on playback its just black. Thanks
Quote from: Savely on July 08, 2017, 02:06:30 PM
Thanks, I did, but it still doesn't rec h264. So I assume this option doesn't support any crop_rec mode?
Exact it works only when realtime liveview is availaible it does work however with the crop mode enabled with the zoom (not the one inside ML)
Can you direct me where I can download the update to record sound on the lastest build? I dont know how to work around bitbucket.
I basically want to record sound in regular 24p 1080p but I want the 50p 3x3 option from the latest update (I dont need sound for this option).
I just dont want to change cards each time I need sound.
@The_bald_guy https://builds.magiclantern.fm/jenkins/job/crop_rec_4k/36/artifact/platform/5D3.113/magiclantern-crop_rec_4k.2017Jun19.5D3113.zip (https://builds.magiclantern.fm/jenkins/job/crop_rec_4k/36/artifact/platform/5D3.113/magiclantern-crop_rec_4k.2017Jun19.5D3113.zip)
This is for 5D3 fir 113
Oh wait sorry I'm not sure if this version records sound. I missed that you needed that.
I think you have to compile it if you want sound.
Hi everyone
I have been using the Jun 19 crop_rec on 5d iii with kombuterbay 1106 card.
Most of the higher res crop modes stop recording almost immediately, I get maybe a second.
If there something I'm missing here? Something I haven't set up right.
Any advice would be greatly appreciated.
Thanks
You probably forgot to set FPS override.
Quote from: Boygetslost on July 17, 2017, 04:06:07 AM
Hi everyone
I have been using the Jun 19 crop_rec on 5d iii with kombuterbay 1106 card.
Most of the higher res crop modes stop recording almost immediately, I get maybe a second.
If there something I'm missing here? Something I haven't set up right.
Any advice would be greatly appreciated.
Thanks
Quote from: Boygetslost on July 17, 2017, 04:06:07 AM
Hi everyone
I have been using the Jun 19 crop_rec on 5d iii with kombuterbay 1106 card.
Most of the higher res crop modes stop recording almost immediately, I get maybe a second.
If there something I'm missing here? Something I haven't set up right.
Any advice would be greatly appreciated.
Thanks
Which settings do you use?
This is what I have been using. On the crop modes It just starts recording then stops.
The normal modes like 1920 1080 or 1920 1220, seems to work fine.
Any ideas? Am I missing a setting? Or using a wrong build?
Thanks
Chris
(http://thumb.ibb.co/ey8bxv/IMG_6810.jpg) (http://ibb.co/ey8bxv)
(http://thumb.ibb.co/kvJ9Hv/IMG_6811.jpg) (http://ibb.co/kvJ9Hv)
(http://thumb.ibb.co/eeqBVF/IMG_6812.jpg) (http://ibb.co/eeqBVF)
photos to upload in fb (http://imgbb.com/)
The settings you're showing are far from being sufficient. Compare to first post of this thread and read a1ex' remarks about this mode.
-> Frozen LV, low ISO and scene allowing good compression rate.
@boygetslost,
Check out this video created by hjfilmspeed! It was a huge help for me! Thanks again hjfilmspeed!
Rob
Quote from: hjfilmspeed on May 13, 2017, 08:31:59 PM
@whysodifficult @rob_6
This video is not for those new to ML. Try this at your own risk. I am not responsible for any damage this video may bring if your camera explodes or bricks. Magic Lantern is not responsible either. It will probably just brick not explode though.
Magic Lantern 3072 2.35:1 12 bit Lossless RAW Recording from the CROP_REC_4K build from 4-29-2017
Once you have ML installed and the correct build on your SD card, this video should help you set up your 5DIII to record 3K almost continuously depending on the scene. Remember ML on SD and Rec to CF.
Remember this build is experimental and not for those that are new to ML. Try at your own risk.
Thanks Rob!
Just what I needed, I have a much better understanding of it now.
Getting much better results.
Has anyone noticed bug on x5 crop_mode (3,5k) - sometimes center adjusted to the right. I mean you have an object almost perfectly at center and when you see a footage, it's on the right side of picture. Yet can't figure out what it depend of.
Quote from: hjfilmspeed on July 14, 2017, 06:31:25 PM
@The_bald_guy https://builds.magiclantern.fm/jenkins/job/crop_rec_4k/36/artifact/platform/5D3.113/magiclantern-crop_rec_4k.2017Jun19.5D3113.zip (https://builds.magiclantern.fm/jenkins/job/crop_rec_4k/36/artifact/platform/5D3.113/magiclantern-crop_rec_4k.2017Jun19.5D3113.zip)
This is for 5D3 fir 113
Oh wait sorry I'm not sure if this version records sound. I missed that you needed that.
I think you have to compile it if you want sound.
hey @hjfilmspeed is there any thread which shows how to compile a module? I'm looking for sound on the crop rec as well, happy to test out. cheers.
Sound, when you shoot is a good idea.
Please, share the build.
Is possible to use these 1920@60fps without RAW mode? In ALL-I maybe? Thank for your atention!!
Quote from: Rcercer on July 28, 2017, 07:13:49 AM
Is possible to use these 1920@60fps without RAW mode? In ALL-I maybe? Thank for your atention!!
If that'd be possible there wouldn't be needing for Magic Lantern at all.
Any news on bringing the MLV_SOUND to work with that branch?
Is anybody know how to achieve 3072x1920, I can't find the good set up in the camera to go up to 3072x1320?
Quote from: etienne on August 03, 2017, 12:16:26 AM
Any news on bringing the MLV_SOUND to work with that branch?
sorry, not yet. quite busy with having a life ;D
Quote from: sylvain_c on August 03, 2017, 06:29:43 AM
Is anybody know how to achieve 3072x1920, I can't find the good set up in the camera to go up to 3072x1320?
This could possibly help:
Totally dumb question (but to better understand what is going on): what are the differences between 3.5K 1:1 centered x5 and 3K 1:1 for the end result? Why using the first one over the second or vice versa?
I have managed to record 3072x1286 - 2.39:1 files with the two with correct LiveView Preview (Frame mode / lagged) and very close framing (just few pixels differences)...
Thanks,
S/.
Quote from: g3gg0 on August 03, 2017, 09:24:48 PM
sorry, not yet. quite busy with having a life ;D
this is totally unaccaptable!!! So just to get an idea is it something that just needs works to be done or is it something that needs some massive issue to be fixed and is yet still uncertain?
Stop trolling around, etienne.
This project is done by highly skilled people in their spare time not consumed by work, family, friends, household duties and other hobbies. For free, given to you free (as in beer). There is no schedule, time table, milestone to be fulfilled.
Top of page -> User Guide -> FAQ -> Troll questions sections
If you can't live with that you are free not to use ML but professional solutions more suitable to your needs.
Ceterum censeo experimental builds page esse delendam.
Hi All
Thanks for this new enhancements. I will give this a go this weekend ie crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
Has anyone tried the "Full-resolution LiveView: 5796x3870 at 7.4 fps (128ms rolling shutter) - continuous*) at 5 FPS!", if so, are you able to share the settings.
I may be doing something wrong as when I select this option the liveview on the camera is showing a pink scrambled screen. Any ideas or suggestions?
cheers
Denis
Hi what CF card speed needed at 5d3 to make 4k video? can use SD cards ?mine is CF 90mb 64gb only and i don't see any crop_rec_mo inside my latest magic lantern? my version is build on 2017-08-07, anyone can anwer me tqvm
@denishogan im not sure but it sounds like your preview might be set to canon mode or real time. It should be set to framing. I would have to double check that tho.
Quote from: Walter Schulz on August 09, 2017, 07:17:24 AM
Stop trolling around, etienne.
There is no schedule, time table, milestone to be fulfilled.
Top of page -> User Guide -> FAQ -> Troll questions sections
Hi walter,
just read that faq didn't saw mine :)
Ok Sorry if i did not make myself serious enough. Just want to assure you that my question was definitly not about the when?
but just asking (to whom have taken a look into it) is this a reasonable hope to have? or do there is evidence already that it could turn out to be something like the RT preview with a big question mark on if this is even doabble...
but rest assure i can definitly live without just as i can definitly live without even getting that answer.
Have fun & peace.
D.
Thanks hjfilmspeed
I followed your video with the exact settings but encountered 2 errors described below. I also then tried Full-resolution LiveView: 5796x3870 at 7.4 fps and got the same errors. The attached images are the settings for "full-resolution LiveView: 5796x3870 at 7.4 fps"
MY SUMMARY:
I tried the following :
1) video settings by Jonathan DeNicholas (https://vimeo.com/217313287#comment_15952088) How ro Record 3k on 5dmkIII with Magic Lantern
2) Full-resolution LiveView: 5796x3870 at 7.4 fps (128ms rolling shutter) - continuous*) at 5 FPS! - seeting at 14-bit lossless compression mode (LJ92), preview set to Frozen LV (previously known as Hacked Preview)
For both scenarios above the movie recorded and I extracted the DNGS using MLV convertor but could not open the DNGS in Photoshop CC 2017.
-The error was "Could not complete your request because the file-format module cannot parse the file." , see attachments
- In bridge I can see each DNG file has many dots of different colors. ." , attachments
OTHER DETAILS
Canon 5dmkIII
Card 64gb 1066 compact flash
Using latest crop_rec_4k.2017 Jun 19.5d3123.zip
Using MLV convertor to extract DNGs
Photoshop error reading the DNG
(http://thumb.ibb.co/kKY6sF/8_12_2017_2_07_11_PM.jpg) (http://ibb.co/kKY6sF)
What I see in BRIDGE- colored dots
(http://thumb.ibb.co/bDFKCF/8_12_2017_2_53_04_PM.jpg) (http://ibb.co/bDFKCF)
(http://thumb.ibb.co/hQPUCF/8_12_2017_2_54_06_PM.jpg) (http://ibb.co/hQPUCF)
(http://thumb.ibb.co/mv7MRa/006.jpg) (http://ibb.co/mv7MRa) (http://thumb.ibb.co/gKtHzv/005.jpg) (http://ibb.co/gKtHzv) (http://thumb.ibb.co/n9Co6a/IMG_9999.jpg) (http://ibb.co/n9Co6a) (http://thumb.ibb.co/kthSzv/IMG_0005.jpg) (http://ibb.co/kthSzv) (http://thumb.ibb.co/cLmWRa/IMG_0003.jpg) (http://ibb.co/cLmWRa) (http://thumb.ibb.co/bZF2XF/IMG_0001.jpg) (http://ibb.co/bZF2XF)
For developing dng files on windows try this for lossless.
http://www.magiclantern.fm/forum/index.php?topic=10526.msg186929#msg186929
QuoteThanks hjfilmspeed
I followed your video with the exact settings but encountered 2 errors described below. I also then tried Full-resolution LiveView: 5796x3870 at 7.4 fps and got the same errors. The attached images are the settings for "full-resolution LiveView: 5796x3870 at 7.4 fps"
MY SUMMARY:
I tried the following :
1) video settings by Jonathan DeNicholas (https://vimeo.com/217313287#comment_15952088) How ro Record 3k on 5dmkIII with Magic Lantern
2) Full-resolution LiveView: 5796x3870 at 7.4 fps (128ms rolling shutter) - continuous*) at 5 FPS! - seeting at 14-bit lossless compression mode (LJ92), preview set to Frozen LV (previously known as Hacked Preview)
For both scenarios above the movie recorded and I extracted the DNGS using MLV convertor but could not open the DNGS in Photoshop CC 2017.
-The error was "Could not complete your request because the file-format module cannot parse the file." , see attachments
- In bridge I can see each DNG file has many dots of different colors. ." , attachments
Try to turn off fps override, i've never used it when i shoot in full resolution
Thanks baccinoluigi for your prompt response. I will try that now.
I normally use Full RES DNG photo mode for extreme macro to save on my shutter count as some of these stacks take 100's to 1,000's of pics.
I am trying to do the same but using 4k video to save time. The example below is a reduced size macro. The full Res version is 10,467 pics Panorama (16,424 X 18,674 ) using magic lantern and perhaps a few actuations only. So my desire to get the 4k video Full Res working as that took nearly 2 weeks elapsed time in photo mode.
(http://thumb.ibb.co/dvOEUv/18556810_10155334927359810_1236929900883014327_o.jpg) (http://ibb.co/dvOEUv)
baccinoluigi ,
I tried turning FPS off but same issue. DNG has coloured spots and not recognised by CC 2017
I believe it has to be on as in FULL RES we must sate the FPS as shown in the documentation ie 7.4 fps , I believe
cheers
Denis
Thanks Danne for the link, I will try it, cheers
Hi Danne,
I did as you suggested but nothing occured ie no DNGs created but not sure what I did was correct with your utility.
I copied the utility into the folder with the MLVs
I am not sure what to do but at the cmd prompt, I selected 2 numbers, #6 then pressed return, then selected 11 and pressed return key. See the attached black screen image. Then I pressed "R" key and pressed enter... It seemed to do something. See the second screen capture. I copied the original MLVs into a folder called Originals and created 7 folders in the red box that I number as #3 however there are no files in it.
(http://thumb.ibb.co/j62Y6a/8_12_2017_10_26_22_PM.jpg) (http://ibb.co/j62Y6a)
(http://thumb.ibb.co/msSpev/8_12_2017_10_27_52_PM.jpg) (http://ibb.co/msSpev)
Just put the MLV files inside the batch_mlv folder and then double click batch_mlv.cmd :).
Then do additional settings from the menu and when finished select (r).
The automation process as follows:
MLV files are processed into folders with dng files and then the original MLV fileas are moved into a folder called A_ORIGINALS
The process is automated so user don´t need to worry about creating any folders.
Should look like this.
(https://s23.postimg.org/g3oql07rv/Screen_Shot_2017-08-12_at_16.55.27.png)
Danne
I did do that twice and it create folders but there are no DNGS in the folders
Hm, I see. Could you upload a sample file? If I can narrow down any issue I will fix it.
These are my MLV files that I put in the batch_mlv folder
(http://thumb.ibb.co/dcXQma/8_12_2017_10_52_01_PM.jpg) (http://ibb.co/dcXQma)
(http://thumb.ibb.co/fTJnzv/8_12_2017_10_53_39_PM.jpg) (http://ibb.co/fTJnzv)
Danne, how do I upload a sample MLV. I clicked attachments and nothing happend
You need a server host, google drive, dropbox, sprend or similar will do.
Thanks Danne, I will upload to dropbox, may take .
wow, its very slow here, saying 1 hour left
In the meantime, base on the settings I used, is the Aspect ratio correct for the Full Res video of 7.4FPS. Alos I used Frozen LV
When I hit the play button I cannot see anything in Liveview on the camera, just pink scrambled lines
Is that supposed to be the behaviour.
Are you able to get the Full RES video of 7.4FPS. working?
cheers
Denis
BTW I also tried RawFlow.exe and it extracted the DNGS from the MLV but same issue.
I am beginning to suspect perhaps my settings are incorrect in Magic Lantern so my MLV files are getting corrupted maybe,
Could be some corrupted MLV file. Let's see. Important though that you're using the latest versions of mlv_dump when processing your files.
Gonna be away from my computer for a while. Checking in tonight.
ok Thanks Danne, its 11.43pm in Perth Australi, I am going to bed but will upload the file tommorow or if I wake earlier
Much appreciated
cheers
Denis
Hi Danne
Here is a sample mlv as requested https://www.dropbox.com/s/lyjusbyn117t4xt/M12-1934.MLV?dl=0
Are there any Canon camera ( not ML) video settings I should set?
cheers
Denis
Downloaded the file and run the batch_mlv.cmd command. All files were exported into dng files.
Note! Try all very basic when things are cracking. For example. A more basic folder name might work instead of with spaces. Worth a shot. My script is very basic.
*Can confirm spaces in root folder is not working atm with the script. Will check into fixing this.
After double click on batch_mlv.cmd this is my view. Then I just print r and enter and files are sent to a folder.
(https://s30.postimg.org/g893xx91d/Screen_Shot_2017-08-13_at_02.35.35.png)
Here´s a version which might work with spaces. Can you try @denishogan?
https://bitbucket.org/Dannephoto/batch_mlv/downloads/batch_mlv.zip
OK I will try now, but can you please confirm if you open a DNG in photoshop?
If not then corruption perhaps
Opens fine, no corruption.
Corruption most probably happens because of when you´re using older versions of mlv_dump.
Wohoo Danne, you are a champ. it worked. So if I want to add parameters what is the syntax? at the prompt do I enter "6 11" without the quotes and press return to run parameter 6 and 11)
Once again thank you so much my friend. I am so happy.
With the Full RES video at 7.4fps, I notice that live view is scrambled with pick lines which make it hard for me to focus ( as can be seen in those example DNGs') or compose. What are your recommendations of the settings below for FUll RES? I think I am using 16.9 or 1:1 . Just charging my batteries.
(http://thumb.ibb.co/cLmWRa/IMG_0003.jpg) (http://ibb.co/cLmWRa)
img safe free image host (http://imgbb.com/)
Great :). Made me fix those damn folder space issues so all good.
You can add parameters one by one. For instance 1 then enter, then select next setting and so on. When done select r and enter.
Just reselect a setting number to erase it.
Happy shooting :)
Hi Danne,
Thank you so much for your help and patience. Thanks for the update program and effort, hugs
On a seperate note, I also found a link that there is a firmware update for the d800 that will provide 8k
Now to find the firmware. I go to Nikon site but no mention about it
Thanks once again and have a great day
cheers
Denis
I just tested H264 proxy with sound + 1080p compressed raw and it works perfectly. However h264 are longer. to sync h264 proxy + sound + 1080 raw, I tried to determine the offset. It is around 2s but not regular. Sometimes +1i or 2, sometimes -1i or two. Is there a way to have a stable offset ? If yes, a simple ffmpeg script could sync everything.
Hi All
I have shot some video on Full Res 7fps and able to extract DNGS ( thanks to Danne) however the liveView of the camera is showing pink and scrambled lines. Any way around this as I cannot focus or compose a shot?
(http://thumb.ibb.co/dbk7zv/IMG_0009.jpg) (http://ibb.co/dbk7zv)
(http://thumb.ibb.co/h0dSzv/IMG_0011.jpg) (http://ibb.co/h0dSzv)
(http://thumb.ibb.co/eug2XF/IMG_0012.jpg) (http://ibb.co/eug2XF)
Quote from: denishogan on August 13, 2017, 11:02:09 AM
Hi All
I have shot some video on Full Res 7fps and able to extract DNGS ( thanks to Danne) however the liveView of the camera is showing pink and scrambled lines. Any way around this as I cannot focus or compose a shot?
Tripod, compose, focus, shoot.
Or buy red weapon :)
Hi all,
Hope you're all well! I haven't checked in for a while. Last time I was here, devs were making leaps and bounds - all very exciting. I was just wondering where we're at with the 5D3? I think last time I tried the build 3.5k was almost continuous but without realtime live view or sound recording. Is this still the case?
Apologies, I did try and trawl back through this thread, but it gets quite busy!
Quote from: beauchampy on August 17, 2017, 10:16:06 AM
I think last time I tried the build 3.5k was almost continuous but without realtime live view or sound recording. Is this still the case?
Mostly. I can do 3.5k 2.67:1 continuous with a fast card in the zoom mode crop. Live view preview is responsive enough for framing that mode, but I think the other hires non-zoom modes still have problems with live view. No sound. Very usable though.
I have tested the 1920x960 @50fps & I came up with a continuous resolution.
1920x818 @ 50fps gives continuous shooting & when you convert the vertical resolution (818) to 1080 it give a film like anamorphic look.
Lots of thanks to the Magic Lantern team!!!
I'm soon shooting a professional work with it.
Short summary of latest updates:
- lossless compression for 700D and EOS M, thanks ErwinH (http://www.magiclantern.fm/forum/index.php?topic=18443.msg188163#msg188163) and dfort (http://www.magiclantern.fm/forum/index.php?topic=18443.msg188799#msg188799)
- included the 3x3 50/60p for EOS M and 700D from the original crop_rec, thanks rbrune (http://www.magiclantern.fm/forum/index.php?topic=16608.msg173854#msg173854) and dfort (http://www.magiclantern.fm/forum/index.php?topic=17021.msg179368#msg179368)
- compressed playback (mlv_play) also working on 5D3 1.2.3 (g3gg0 did the initial implementation on 1.1.3)
- lua_fix included ( cc @garry23 )
- dynamic my menu included (http://www.magiclantern.fm/forum/index.php?topic=18884) (already found a couple of bugs)
- full-res silent picture automation (https://bitbucket.org/hudson/magic-lantern/commits/812b4e5d5413f97cb5f5323ed6dab9876e8c1594) (still a bit rough, but it's one tiny step towards the "5D Mark III Mirrorless Edition")
- lossless DNG playback (with pic_view/file_man)
@a1ex Nice..
!!!!!!
Is there any possibilities shooting 5796x3870@24p in the near future?
(8bit, 10bit or 12bit, not raw video such as .mov or mpeg is ok)
I need some help reproducing a lossless compression bug.
Some days ago I took a few hundreds of pictures with the silent picture in Fullres LV mode, saved as lossless DNG. A few of them ended up corrupted - there were a few bytes of garbage data before the JPEG payload (as a result, these DNGs could not be decoded). Unfortunately, I wasn't able to reproduce the issue afterwards - that's where I need your help. The corruption may or may not be related to the silent picture module, or it may also happen with raw recording, or it may have something to do with quick button pressing and switching modes - no idea.
I've tried to record a timelapse of Fullres LV silent pictures - did not trigger the issue.
To spot the bug, I've added an assertion in 96c5204 (latest build). If you can trigger this assertion (easy to spot - it will fill the screen with a message about lossless.c:338), please try to find a way to reproduce.
Quote from: mr.smith on August 22, 2017, 04:35:57 AM
Is there any possibilities shooting 5796x3870@24p in the near future?
(8bit, 10bit or 12bit, not raw video such as .mov or mpeg is ok)
No, will never happen.
April 1st, 2018, for sure!
@Alex the latest build 22 AUG, gives me error 70 everytime i try to turn on raw video on 5d3 1.2.3, reverted back to older build and everything works ok again.
Same for me, tried to reinstall ML, error appears when I switch to video mode with RAW Video on or turn RAW Video on while in video mode. Crash logs say this:
ASSERT: IsSuiteSignature( hSuite )
at ./PackMemory/PackMem.c:599, task RscMgr
lv:0 mode:3
RscMgr stack: 151e68 [151f90-150f90]
0xUNKNOWN @ de48:151f88
0xUNKNOWN @ 17bbc:151f60
0x000178B4 @ ff0ed8c0:151f28
0xUNKNOWN @ 178e4:151f18
0xUNKNOWN @ 1796c:151ef8
0x0000AEB0 @ ff201904:151eb0
0x00001900 @ aef8:151ea0
0x00069868 @ 69b58:151e68
Magic Lantern version : crop_rec_4k.2017Aug22.5D3123
Mercurial changeset : 96c5204df97b (crop_rec_4k) tip
Built on 2017-08-22 08:40:04 UTC by jenkins@nightly.
Free Memory : 141K + 3732K
-----------------------------------------------------------------------------------------
ASSERT: IsSuiteSignature( hSuite )
at ./PackMemory/PackMem.c:983, task compress_task
lv:1 mode:3
compress_task stack: 1b08d0 [1b09d0-1af9d0]
0x00A295A0 @ a230c0:1b0970
0x0000B1D0 @ a296c8:1b0910
0x00001900 @ b1f4:1b0908
0x00069868 @ 69b58:1b08d0
Magic Lantern version : crop_rec_4k.2017Aug22.5D3123
Mercurial changeset : 96c5204df97b (crop_rec_4k) tip
Built on 2017-08-22 08:40:04 UTC by jenkins@nightly.
Free Memory : 141K + 3724K
-----------------------------------------------------------------------------------------
ASSERT: IsSuiteSignature( hSuite )
at ./PackMemory/PackMem.c:599, task Epp
lv:0 mode:3
Epp stack: 17dbc0 [17dca8-17d0a8]
0xUNKNOWN @ de48:17dca0
0xUNKNOWN @ 17bbc:17dc78
0x000178B4 @ ff0de67c:17dc58
0xUNKNOWN @ 178e4:17dc48
0xUNKNOWN @ 1796c:17dc28
0x0000AEB0 @ ff185a48:17dc08
0x00001900 @ aef8:17dbf8
0x00069868 @ 69b58:17dbc0
Magic Lantern version : crop_rec_4k.2017Aug22.5D3123
Mercurial changeset : 96c5204df97b (crop_rec_4k) tip
Built on 2017-08-22 08:40:04 UTC by jenkins@nightly.
Free Memory : 141K + 3731K
Solved.
The above bug hunting quest (http://www.magiclantern.fm/forum/index.php?topic=19300.msg188929#msg188929) remains valid.
Got some other bug. Havn´t been testing silent DNG almost at all but pretty impressive with lossless DNG. Scrolling between lossless DNG files with file manager is also great feature. Anyway. Tried producing errors and got this. A fake sized DNG. Assert.log included.
https://bitbucket.org/Dannephoto/magic-lantern/downloads/Lossless_DNG_error.zip
Can be reproduced like this:
1 - Activate silent.mo - open up live view in photo mode - shoot some lossless DNG files
2 - Switch over to movie mode, shoot a CR2
3 - Switch over to photo mode again and start shooting some lossless DNG files. After a while it should produce the error. Maybe after 3-4 lossless DNG files.
I also had mlv_lite and RAW video set to on, EXPO. Override - Global Draw OFF. The other error code mentioned I still havn´t found.
Got messages about lossless.c:338 everytime when trying to record at 1080x48p with crop_rec, after a couple seconds after starting recording. Three times this:
ML ASSERT:
0
at ../silent/lossless.c:338 (lossless_compress_raw_rectangle), task compress_task
lv:1 mode:3
compress_task stack: 1b08e0 [1b09d0-1af9d0]
0x00A29BF0 @ a23710:1b0970 <- difference
0x00069FA8 @ a29eec:1b0910 <- difference
0x00069868 @ 6a014:1b08e0
Magic Lantern version : crop_rec_4k.2017Aug22.5D3123
Mercurial changeset : 7c4a1262ac3e (crop_rec_4k) tip
Built on 2017-08-22 16:34:20 UTC by jenkins@nightly.
Free Memory : 135K + 3688K <- difference
And three times this:
ML ASSERT:
0
at ../silent/lossless.c:338 (lossless_compress_raw_rectangle), task compress_task
lv:1 mode:3
compress_task stack: 1b08e0 [1b09d0-1af9d0]
0x00A29C40 @ a23760:1b0970
0x00069FA8 @ a29f3c:1b0910
0x00069868 @ 6a014:1b08e0
Magic Lantern version : crop_rec_4k.2017Aug22.5D3123
Mercurial changeset : 7c4a1262ac3e (crop_rec_4k) tip
Built on 2017-08-22 16:34:20 UTC by jenkins@nightly.
Free Memory : 141K + 3724K
On a build before Aug 20 it failed too and told me to decrease vertical resolution.
Thanks Danne - taking a still photo (CR2) in movie mode was the key to reproducing the issue.
With silent pics (silent.mo):
- start the camera in movie mode, make sure silent pics are disabled
- take a still photo (CR2, not JPG)
- enable silent pics (simple, lossless DNG)
- take 2 silent pics using half-shutter
- the second one will trigger the error in about 90% of cases
The error does not happen if the above sequence is performed in photo mode.
With raw recording (mlv_lite.mo):
- start the camera in movie mode (any, even plain 1080p)
- take a still photo (CR2, not JPG)
- enable raw video if it isn't already, with lossless compression
- start recording => kaboom :)
Was the second sequence working in any of the previous builds? (back to April 1st 3rd - older builds here (https://builds.magiclantern.fm/jenkins/view/Experiments/job/crop_rec_4k/))
Reproduced the second sequence (raw recording opening up straight into movie mode) with the latest build.
Then jumped to test april 10th build and the same test is working without errors (lossless 14-bit 1920x1080) opening up straight into movie mode. Will try and see what build starts to fail.
Did the test wrong. Let me check again.
Ok, reproduced the error also on april 10th build.
Apr10 fails here after taking a CR2.
Yes, sorry, false positive, was doing the test wrongly. Updated my post. Only happens with lossless. Regular 14bits are dandy even after firing of a pre CR2 file...
August 20 fail in Emergency when have set 3,5K Crop, 12 bit lossless, and press rec without 5x zoom.
THis expected? Should work without zoom.
Battery out solve the issue.
Mlv play work very good in 14-12 bits lossless.
I'm looking for a build that has these functions:
MLV crop 3x
MLV sound
10/12/14 bit lossless
lossless in camera playback
I don't need the 3k, 4k. 1080 for all is good.
Is there such a build?
Quote from: Lars Steenhoff on August 24, 2017, 01:17:06 PM
I'm looking for a build that has these functions:
MLV crop 3x
MLV sound
10/12/14 bit lossless
lossless in camera playback
I don't need the 3k, 4k. 1080 for all is good.
Is there such a build?
No, the closest one is the crop rec + 10-12bit (without compression) build that was up on experiments until recently. It included sound module.
Quote from: a1ex on August 21, 2017, 12:08:18 AM
Short summary of latest updates:
- lossless compression for 700D and EOS M, thanks ErwinH (http://www.magiclantern.fm/forum/index.php?topic=18443.msg188163#msg188163) and dfort (http://www.magiclantern.fm/forum/index.php?topic=18443.msg188799#msg188799)
- included the 3x3 50/60p for EOS M and 700D from the original crop_rec, thanks rbrune (http://www.magiclantern.fm/forum/index.php?topic=16608.msg173854#msg173854) and dfort (http://www.magiclantern.fm/forum/index.php?topic=17021.msg179368#msg179368)
- compressed playback (mlv_play) also working on 5D3 1.2.3 (g3gg0 did the initial implementation on 1.1.3)
- lua_fix included ( cc @garry23 )
- dynamic my menu included (http://www.magiclantern.fm/forum/index.php?topic=18884) (already found a couple of bugs)
- full-res silent picture automation (https://bitbucket.org/hudson/magic-lantern/commits/812b4e5d5413f97cb5f5323ed6dab9876e8c1594) (still a bit rough, but it's one tiny step towards the "5D Mark III Mirrorless Edition")
- lossless DNG playback (with pic_view/file_man)
So Eos M could also record in 24fps 3x3 in full-resolution liveview without h-264 now?
Im not sure, is compressed dnd in 700D and Eos M are only in 14 bit or it could by also in 10 Bit with compression ?
Would be happy with that, do you still have the build somewhere? for 123 5d mk3 ?
2017-08-22 build,
How long can I shoot with 4096x2560@24p ?
I am considering to install the latest build.
why just consider and not try yourself? 5k even at 60fps no problem because the 1st post does exactly contain that info. :P
Quote
* 4096x2560 @ 12.5p (1:1 crop) - continuous*) at 8 FPS
24p should be a second - maybe!?
thanks for telling. but only a second? really? a second? at 24p?
Quote from: nikfreak on August 25, 2017, 02:57:59 PM
why just consider and not try yourself? 5k even at 60fps no problem because the 1st post does exactly contain that info. :P
24p should be a second - maybe!?
i recommend the calculator:
http://rawcalculator.bitballoon.com/calculator_desktop
Nikfreak, you are being so super nice! As soon as I have a free moment I will try to do some testing using an external monitor while shooting "4K" in my mind, using an external monitor should help while shooting with frozen liveview.. but again, I don't trust what my mind thinks
Took a closer look at the recent error messages. These problems were there in previous builds as well; they were just silent (in other words, I was unaware of the issues). Most of them appeared when estimating the compression ratio during standby - in this case, I've been calling the lossless compression routine in the so-called "dummy mode" (without image output - just to have it report the compression ratio). Turns out, when the planets were aligned, this method resulted in some left-overs on the EDMAC write channel (the one that writes the compressed output in memory); as a result, the first frame was sometimes invalid.
As I couldn't figure out how to clean the EDMAC channel, the easiest way was to avoid calling the lossless compression routines in "dummy mode" - that is, always provide a valid output buffer. If I go that far, I might as well refactor the memory management, so the raw recording buffers are always allocated during standby (and freed when exiting LiveView or when going to photo mode). It was a bit of work, but that's what I did.
You may ask, why?
Allocating memory for video buffers is slow (takes a few seconds to allocate the entire shooting memory) and complicated (see exmem.c and RscMgr thread). So, having the memory allocated in advance means recording can start right away (already discussed a few pages earlier (http://www.magiclantern.fm/forum/index.php?topic=19300.msg185903#msg185903)).
By contrast, creating the MLV file is very fast, so there's very little to gain by opening it in advance.
Minor: the status bar also shows how many buffers we have allocated, and their size.
Regarding 1080p48 - it was pushed a bit over the safe limit, so I've reduced its resolution to 1040. You still have the option to go back to 1080 in the crop_rec submenu (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183399#msg183399); however, you'll get a lot of errors. With some fiddling, you can probably get 1060 (e.g. set Target YRES to 1060 and Delta HEAD3 to -20), but I wouldn't trust it yet.
Also found a way to sync the H.264 proxy with the raw stream - by making the extra frames black. The first non-black frame from H.264 will match the first frame from MLV. Easy.
I've also integrated the thread-safety annotations (using clang) and fine-tuned the dynamic menus to make them halfway usable.
Overall, the build is still too buggy for my taste, but at least it's a small step forward.
Tremendous work, A1ex!
May we call it MLV 3.0 ?:)
Quotepoor man's sync between RAW and H.264 proxy
:P
I like it. Will experiment with some automation scripts around this.
Trying to build EOS M (toolchain 6.3.1-q2-2017) commit cffeb48 (https://bitbucket.org/hudson/magic-lantern/commits/cffeb4898221dbeab9848361d530db1ae9021cc8?at=crop_rec_4k) yields this error:
mlv_rec.c:4183:5: error: implicit declaration of function 'raw_force_aspect_ratio_1to1' [-Werror=implicit-function-declaration]
raw_force_aspect_ratio_1to1();
The *official-experimental build* has this too: https://builds.magiclantern.fm/jenkins/job/crop_rec_4k/42/consoleText
/LE: Fixed https://bitbucket.org/hudson/magic-lantern/pull-requests/851/
Nice find @Licaon_Kter -- so does that mean we can have a choice of the latest mlv_lite developments or the legacy mlv_rec with mlv_snd in the same build? I want it!
However -- not all is rainbows and unicorns in EOSM and 700D land.
The 700D saved a crash log and made a core dump when recording 10-bit lossless:
[2] raw_rec_task: NULL PTR (80c3e100,e1a00000)
pc= c288 lr= c288 stack=1abdd0+0x1000
entry=ac2fac(0)
e1a00000 e59ff014 e59ff014 e59ff014
e59ff014 e1a00000 e59ff010 e59ff010
Magic Lantern version : crop_rec_4k.2017Aug26.700D114
Mercurial changeset : cffeb4898221 (crop_rec_4k) tip
Built on 2017-08-26 21:27:15 UTC by jenkins@nightly.
Free Memory : 128K + 2524K
The EOSM didn't save a core dump but created this assert log when recording 10-bit lossless with H.264 Proxy (the only way to get mv1080 on the M):
ML ASSERT:
slots[slot_index].size < max_frame_size
at mlv_lite.c:3344 (raw_video_rec_task), task raw_rec_task
lv:1 mode:3
raw_rec_task stack: 1ded20 [1dede0-1ddde0]
0x0009EBBC @ ab9178:1ded50
0x0009E548 @ 9ec1c:1ded20
Magic Lantern version : crop_rec_4k.2017Aug26.EOSM202
Mercurial changeset : cffeb4898221 (crop_rec_4k) tip
Built on 2017-08-26 21:27:48 UTC by jenkins@nightly.
Free Memory : 148K + 2722K
Logs and core dump uploaded to dropbox (https://www.dropbox.com/sh/7souqw7s6xrmvx4/AAAP1AZnhRS_QHp1UWK3f9zwa?dl=0).
Had some issues too, indeed, too bad I deleted the logs/dmp by mistake though. :(
Aren´t mlv_rec and mlv_snd deliberately out of the equation on these builds? Wouldn´t mind them getting back of course :P.
Just recorded a few takes with H.264 proxy and 14bit compressed. The black frames seems to work as advertised which is friggin´ great. Back to script automation. Start by matching feature:
MLV:
Time: 08:29:45 (GMT+0)
H.264:
Create Date : 2017:08:27 08:29:43
Convert(imagemagick) will determine what is black or not:
convert test.jpg -colorspace hsb -resize 1x1 test.txt
# ImageMagick pixel enumeration: 1,1,255,hsb
0,0: ( 77,220, 1) #4DDC01 hsb(30.1915%,86.2547%,0.546273%)
Black 0.546273%
# ImageMagick pixel enumeration: 1,1,255,hsb
0,0: ( 25, 86, 96) #195660 hsb(9.99924%,33.7636%,37.4884%)
not black 37.4884%
Maybe someone knows how to read pixel brightness from ffmpeg by the way? Won't be needing convert if so.
*Found something interesting here myself:
http://www.ffmpeg.org/ffmpeg-filters.html#blackdetect
After some tweaking from this:
https://stackoverflow.com/questions/18722747/can-you-put-the-result-of-a-blackdetect-filter-in-a-textfile-using-ffmpeg
ffmpeg -i IMG_7621.MOV -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null -
Yields:
[blackdetect @ 0x7f8b08d1d600] black_start:0 black_end:1.37637 black_duration:1.37637
Output blackframes longer than 0.5 seconds and pixel threshold set to 0.01. If set to 0.00 it won´t work.
Next find a way to interpret numbers to frames and search only beginning of the file to save time.
Probably just cut off 0-1.37637 and export this example file to a new MOV should do the trick. Will check some more into this tonight.
@Danne, I dont know if its deliberate that mlv_rec and mlv_snd don't compile?
when I make a build they dont compile because of an error not because they are excluded as far as I can see.
I just did a build from the latest crop_rec4k with the aspect ratio fix from @Licaon_Kter.
All the modules compiled and I installed it on the 5d mk3 1.13.
:) mlv rec and mlv sound works
:) mlv rec + mlv crop + mlv sound works ( tested only 1080 )
:) mlv crop + mlv rec works
:) mlv crop + mlv lite works
:) mlv lite works
;) mlv lite + mlv snd does not work, but we know that.
:-\ mlv play does not work in combination with mlv_rec, check the screenshot for the error
(http://www.larssteenhoff.com/forum_pics/debug1.jpeg)
Very exciting progress!
Quote from: a1ex on August 26, 2017, 11:27:03 PM
Also found a way to sync the H.264 proxy with the raw stream - by making the extra frames black. The first non-black frame from H.264 will match the first frame from MLV. Easy.
Overall, the build is still too buggy for my taste, but at least it's a small step forward.
Thank you Alex, h264 proxy is a very good and complete feature now ! I use it to playback easily (of course) but also to use and sync the sound from H264 proxy with the raw stream. It solves temporarily the lack of mlv sound with comressed raw.
Quote from: Danne on August 27, 2017, 08:43:08 AM
Aren´t mlv_rec and mlv_snd deliberately out of the equation on these builds? Wouldn´t mind them getting back of course :P.
Just recorded a few takes with H.264 proxy and 14bit compressed. The black frames seems to work as advertised which is friggin´ great. Back to script automation. Start by matching feature:
MLV:
Time: 08:29:45 (GMT+0)
H.264:
Create Date : 2017:08:27 08:29:43
Convert(imagemagick) will determine what is black or not:
convert test.jpg -colorspace hsb -resize 1x1 test.txt
# ImageMagick pixel enumeration: 1,1,255,hsb
0,0: ( 77,220, 1) #4DDC01 hsb(30.1915%,86.2547%,0.546273%)
Black 0.546273%
# ImageMagick pixel enumeration: 1,1,255,hsb
0,0: ( 25, 86, 96) #195660 hsb(9.99924%,33.7636%,37.4884%)
not black 37.4884%
Maybe someone knows how to read pixel brightness from ffmpeg by the way? Won't be needing convert if so.
*Found something interesting here myself:
http://www.ffmpeg.org/ffmpeg-filters.html#blackdetect
After some tweaking from this:
https://stackoverflow.com/questions/18722747/can-you-put-the-result-of-a-blackdetect-filter-in-a-textfile-using-ffmpeg
ffmpeg -i IMG_7621.MOV -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null -
Yields:
[blackdetect @ 0x7f8b08d1d600] black_start:0 black_end:1.37637 black_duration:1.37637
Output blackframes longer than 0.5 seconds and pixel threshold set to 0.01. If set to 0.00 it won´t work.
Next find a way to interpret numbers to frames and search only beginning of the file to save time.
Probably just cut off 0-1.37637 and export this example file to a new MOV should do the trick. Will check some more into this tonight.
Great Danne !
Tested some more around ffmpeg and blackdetect filter. I´m sure there are more ways to go on about this but this seems to work as a oneliner in bash to cut of the black frames from the MOV file. MOV file should this way match start of MLV file and after this extracting the wav file is easy. Will work on automation details in Switch.
FFmpeg needs to be installed if anyone wants to test.
How this monkeywrench oneliner is working:
The INPUT.MOV is your H.264 proxy and OUTPUT.MOV will be the shortened MOV file without the blackframes. To speed up processing I do "killall ffmpeg" after one second of processing to get the the black frame detected. It then proceeds with copying out the "cleaned" OUTPUT.MOV file.
ffmpeg -i INPUT.MOV -ss $(sleep 1 && killall ffmpeg & ffmpeg -i INPUT.MOV -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2) -vcodec copy -acodec copy OUTPUT.MOV
Do a for loop inside a folder with MOV files and get them all processed:
for i in *.MOV; do
ffmpeg -i "$i" -ss $(sleep 1 && killall ffmpeg & ffmpeg -i "$i" -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2) -vcodec copy -acodec copy new_"$i"
done
Seems to be a few blackframes at the end as well. Not sure if we want to cut those out. Might check into it later.
Nice work
Yes there are also a few black frames at the end. for the most easy sync they could be cut too.
Also when using 1080 with x3 crop the frames turn out red in the h264. ( RAW lite + h254 )
https://youtu.be/JHYaozsP8OM
@danne
can ffmpeg also extract just the audio with the right length and save as a WAV file ?
ffmpeg -i INPUT.MOV -c copy -map 0:a OUTPUT.WAV
Extract from the "cleaned" proxy file
Working on detecting the blackframes at the end of the files...
Here is a for loop which will trim and "clean" beginning and end of the H.264 proxy file. It needs some testing so if anyone could check if the new files match the MLV file it would be nice to know. If running into issues please upload the LOG.txt file.
*Not really working with longer files regarding chopping of the ending. Start of files works with any file with the former for loop. Gonna check further when I have time...
cd into a folder with proxy MOV files and put following in terminal:
exec &> >(tee -a LOG.txt >&2 )
for i in *.MOV; do
start=$(sleep 1 && killall ffmpeg & ffmpeg -i "$i" -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2) ;
end1=$(sleep 10 && killall ffmpeg & ffmpeg -i "$i" -vf reverse -t 2 -f matroska pipe:1 | ffmpeg -i - -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2) ;
end=$(echo $start + $end1 | bc -l) ;
duration=$(ffmpeg -i "$i" 2>&1 | grep -o 'Duration: .*' | cut -d "," -f1 | cut -d ":" -f2,3,4 | tr -d ':')
ffmpeg -i "$i" -ss $start -t $(echo $duration - $end | bc -l) -vcodec copy -acodec copy new_"$i"
done
Hi,
do you think it might be possible to include resolution settings for anamorphic? I already experimented with the EOS 50D and anamorphic but it is limited to lower resolutions. With the 5D3 it might be possible to go higher and according to the first crop_rec on steroids posts it's not unrealistic that it might work at continous recording.
/* 24p 25p 30p 50p 60p */
[CROP_PRESET_3X_TALL] = { 1920, 1728, 1536, 960, 800 }, /* 1920 */
[CROP_PRESET_3x3_1X] = { 1290, 1290, 1290, 960, 800 }, /* 1920 */
[CROP_PRESET_3x3_1X_48p] = { 1290, 1290, 1290, 1080, 1080 }, /* 1920; 1080p45/48 <- 50/60p in menu */
[CROP_PRESET_3K] = { 1920, 1728, 1504, 760, 680 }, /* 3072 */
[CROP_PRESET_UHD] = { 1536, 1472, 1120, 640, 540 }, /* 3840 */
[CROP_PRESET_4K_HFPS] = { 2560, 2560, 2500, 1440, 1200 }, /* 4096 half-FPS */
[CROP_PRESET_FULLRES_LV] = { 3870, 3870, 3870, 3870, 3870 }, /* 5796 */
The aspect ratio does not necessarily have to be 4:3. The new Arri Alexa models allow for a new 6:5 ratio. With 2x anamorphics that leads to 2.4 : 1 and is perfect for anamorphic shooters. http://www.arri.com/de/camera/alexa/kameras/kamera_details/alexa-sxt-plus/subsection/new_sxt_features/
A ML resolution of 2048 horizontal and 1716 vertical would be the best option. With 2x anamorphics that would be 4096 : 1716 after desqueeze just like the new Alexa SXT 6:5 ProRes 4K cine anamorphic mode :). If you have the time it would be great if you could add some other resolution options in that area in order to find what works at continous recording. 1920 horizontal and 1600 vertical would also be great for 3840 : 1600 desqueezed.
@Danne: here's my attempt:
#!/bin/bash
# Trim black frames at start and end of a H.264 clip
readarray blacks < <(ffmpeg -i "$1" -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep "blackdetect")
first_black=${blacks[0]}
last_black=${blacks[-1]}
start=$(echo $first_black | grep -oP "(?<=black_end:)[^ ]*")
end=$(echo $(echo $last_black | grep -oP "(?<=black_start:)[^ ]*"))
echo "$1: cutting at $start and $end..."
# with -ss $start after -i, it also trims the first good frame, why?
# in this case, we have to use -ss start -i input.mov -to end-start
# with short clips, bc outputs .1234 rather than 0.1234, but ffmpeg doesn't like it
# also, without trickery, it may leave one extra black frame
trimmed=$(echo $end - $start - 0.01 | bc)
ffmpeg -ss 0$start -i "$1" -to 0$trimmed -vcodec copy -acodec copy new_"$1" -loglevel quiet
Only tested with 2 very short clips.
For long clips, you may want to scan only the first and the last 2-3 seconds. With some trickery, the analysis can probably be done in one single ffmpeg command (see e.g. https://superuser.com/a/898765 ), if you know the clip duration in advance ( http://ffmpeg.gusari.org/viewtopic.php?f=11&t=1284 ).
Nice. Works good on Ubuntu. Shortly tested on a virtual box set up with limited drive etc. Mac cries about grep syntax as usual but with some tweaking it will work on mac too.
The beginning frames are easier to remove. I have some ideas about the ending when it comes to longer files to avoid long scan delays.
*All filters(-vf) seems to require re-encoding which most likely will delay the trim filter solution(yet to be verified). That is why my reverse effect filter won´t work since it seems to encode audio first then video which will delay processing a lot. What works is -ss and -t with the lossless -copy function splitting files at the end on longer files for example. Also putting the -ss before the -i input file as suggested in above script.
ffmpeg -ss 01:19:00 -i Input.MOV -t 01:21:00 -c copy Output.MOV
Fetching the duration of the file is easy with ffmpeg:
ffmpeg -i Input.MOV 2>&1 | grep -o 'Duration: .*' | cut -d "," -f1 | cut -d ":" -f2,3,4
More digging/testing todo...
Also curious what the array from a1ex script will do with longer takes. Probably scan the whole file though which will slow down processing a lot.
readarray blacks < <(ffmpeg -i "$1" -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep "blackdetect")
I simply kill the process after one second of running and it would probably work with the array as well.
readarray blacks < <(sleep 1 && killall ffmpeg & ffmpeg -i "$1" -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep "blackdetect")
Then again it seems the array is fetching blackframe detection for both beginning and end parts so in this case we´ll have to wait for the whole process to finish.
Right - here's a version with two ffmpeg commands for analysis (and one ffprobe):
#!/bin/bash
# Trim black frames at start and end of a H.264 clip
duration=$(ffprobe -loglevel quiet -of compact=p=0:nk=1 -show_entries format=duration -i "$1")
echo -n "$1: duration=$duration; "
# analyze first and last 3 seconds
# clips shorter than 3 seconds require extra care
# bc outputs short values as .1234 rather than 0.1234, but ffmpeg doesn't like this; prepending a 0 does the trick
near_end=0$(echo "($duration > 3) * ($duration - 3)" | bc)
first_black=$(ffmpeg -i "$1" -to 3 -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep "blackdetect" | head -n1)
last_black=$(ffmpeg -ss $near_end -i "$1" -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep "blackdetect" | tail -n1)
# keep from first black_end (or 0, if the clip doesn't start with black frames)
# to last black_start (or to $duration, if the clip doesn't end with black frames)
start=0$(echo $first_black | grep -oP "(?<=black_end:)[^ ]*")
end0=0$(echo $last_black | grep -oP "(?<=black_start:)[^ ]*")
end=0$(echo "($end0 > 0) * (0$end0 + $near_end) + ($end0 == 0) * $duration" | bc)
if [ "$end0" == "00" ]; then
echo "please increase near_end"
exit
elif [[ "$start" == "0" && "$end0" == "0" ]]; then
[ $(echo "$end - $duration" | bc) == "0" ] || echo assert
echo "nothing to do"
exit
else
echo "cutting at $start and $end..."
fi
# with -ss $start after -i, it also trims the first good frame, why?
# in this case, we have to use -ss start -i input.mov -to end-start
# also, without trickery, it may leave one extra black frame
trimmed=0$(echo $end - $start - 0.01 | bc)
ffmpeg -ss $start -i "$1" -to $trimmed -vcodec copy -acodec copy new_"$1" -loglevel error -n
Usage:
for f in *.MOV; do ./cut_black.sh $f; done
Sample output:
3U3B6192.MOV: duration=3.128125; cutting at 01.29296 and 02.210545...
3U3B6194.MOV: duration=2.502500; cutting at 01.20954 and 01.626620...
BIG__3GB.MOV: duration=71.740000; nothing to do
BLACK_4GB.MOV: duration=90.300000; please increase near_end
MVI_4134.MOV: duration=46.946900; nothing to do
Shouldn't be affected by black frames in the middle of the clip, but might get confused by H.264 dark frames :)
The threshold may have to be increased at high ISOs and/or with low-contrast picture styles (didn't check).
Nice!
There goes my "killall" roundtripping. Thank god :P.
first_black=$(ffmpeg -i "$1" -to 3 -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep "blackdetect" | head -n1)
# clips shorter than 3 seconds require extra care
exactly.
Shouldn't be affected by black frames in the middle of the clip, but might get confused by H.264 dark frames
Detect this through ffmpeg as it will be treated as one long blackframe?
ffprobe -loglevel quiet -of compact=p=0:nk=1 -show_entries format=duration -i Input.MOV
yes, workable format. Can´t ffmpeg output this too. Gonna check. Nah, seems ffprobe is the way to go. Another dependency...
Static version of ffprobe here:
https://github.com/joshwnj/ffprobe-static
Here is my latest take on a proxy cleansing script after chewing a1ex refinements. Much faster than the initial script.
for i in *.MOV; do
duration=$(ffprobe -loglevel quiet -of compact=p=0:nk=1 -show_entries format=duration -i "$i")
first_black=$(ffmpeg -i "$i" -to 3 -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
near_end=$(echo $duration - 3 | bc -l)
last_black=$(echo $duration - $(ffmpeg -ss $near_end -i "$i" -vf "blackdetect=d=0.1:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2) | bc -l)
trimmed=$(echo $last_black - $first_black - 0.01 | bc -l)
ffmpeg -ss $first_black -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"
done
Note the change for detection of ending blackframes(Had a few issues with proxy files only having 0.495 blackframes):
-vf "blackdetect=d=0.1:pix_th=0.01"
Still seems to be one last black frame survivor at the end. Soon...
If we dont want to confuse black with a dark scene, perhaps it would be better to use another color instead of black?
I just tried Latest Build (2017-08-26 21:25)
On 5D3 123. Everytime I Q button (sub menu) anything in the menu, after that Shutter Scroll wheel and Apertures wheels are Yielding "Busy" when moved. The Aperture I can change in ML Menu, shutter speed will change in ML menu, but the busy sign comes on and it does not change to the shutter speed after all.
If I don't touch anything in the menus, I can keep using the Shtt wheel and Ap wheel.
Also Shutter Fine Tuning was behaving very weirdly, but that most likely has something to do with this.
Sorry to say, but I can't work with this build at all in this state.
I noticed the busy sign as well. Exiting and reentering liveview again seemed to get rid of it. Not a fix of course.
@Lars Steenhoff
I'd be surprised if ffmpeg would mix dark footage with blackframes. Do test against some very dark shots and tell us if the script breaks.
Quote from: Lars Steenhoff on August 28, 2017, 06:45:05 PM
If we dont want to confuse black with a dark scene, perhaps it would be better to use another color instead of black?
If you know how configure the sensor to output some other solid color, why not?
(pretty sure it's possible, some fiddling with adtg_gui should reveal how; for the current implementation, setting the exposure to 0 was the easiest way)
Quote from: Kharak on August 28, 2017, 07:00:30 PM
Everytime I Q button (sub menu) anything in the menu, after that Shutter Scroll wheel and Apertures wheels are Yielding "Busy" when moved. The Aperture I can change in ML Menu, shutter speed will change in ML menu, but the busy sign comes on and it does not change to the shutter speed after all.
Reproduced - the BUSY message appears when pressing half-shutter (e.g. to close ML menu), but it's not related to submenus. Workaround: press Q twice outside ML menu. Or, enter ML menu and exit it by pressing DELETE rather than half-shutter. Related bug: ETTR sometimes locks up for the same reason.
The "fix" would be to re-enable the SRM use-after-free (not straightforward, but not hard either). Reason: keeping SRM memory allocated causes Canon code to show the BUSY message (and I don't know where/how to patch it).
However, using unallocated memory is not exactly a good practice. Therefore, if one can find a solution to keep the SRM memory allocated (marked as used, so Canon won't overwrite it, no matter what), but without the BUSY message, that would be the best way to solve this.
Pressing Q twice "releases" the wheels again. Nice workaround.
Btw!! another tip for people with Manual Lenses. When starting the camera in crop mode with a manual lens attached, liveview wont start, the camera does not detect a lens and therefore does not turn on the liveview. Instead of entering photo to movie mode and turning off and On crop mode etc. A workaround: when you turn on Camera with a manual lens after 1-2 secs the message "Ensure a lens is attached..." It blinks quickly about 2-3 times, for the duration this message is blinking, you can press "Start/Stop" button and it will force in to liveview in Crop Mode :) boy did i save a couple thousand shutter actuations after learning this ;)
Sometimes you might have to go in to Canon Menu and back for the shutter speeds to sync.
Just press Start once, if you press twice h264 recording starts immediatly.
But also thinking if this is something a ML/Script could do?
With a manual lens, without pressing anything, LiveView starts directly into crop mode. With a Canon lens, it flickers a bit (starts in regular mode, then switches to crop mode after 1-2 seconds). Not sure what the issue is...
edit: sometimes the screen remains black when switching modes around (not sure how to reproduce); in this case, you don't need to go to photo mode; just press MENU twice (see first post).
Do you get an h264 with the black frames when recoding in crop mode? 3x crop 1080 the first option in the crop menu?
Yes, except they are red, just like in your report. The same happens in Canon's 5x zoom, so it's not a misconfiguration in crop_rec.
I can set exposure time to about 1/23000 (or maybe 1/46000? didn't measure it); however, that may confuse the script if the scene contains bright lights (so it won't be much better than red).
Ok I see, well its not a problem because its easy to manually cut the red frames.
just i was wondering if i did something wrong. thanks for comfirming its normal
Here's a quick and dirty patch for the BUSY issue:
--- a/src/exmem.c
+++ b/src/exmem.c
@@ -357,6 +357,11 @@
/* unlock the shutter button */
/* (only touch the shutter-related bits, just in case) */
gui_uilock(icu_uilock & ~UILOCK_SHUTTER);
+
+ /* 5D3 1.2.3 */
+ void (*SRM_ClearBusy)(void *, int) = (void *) 0xFF1F1F68;
+ void * RscMgr = 0x241C8;
+ SRM_ClearBusy(RscMgr, 1);
}
Press shutter fully to see why it's not committed ;)
Just tried some really dark shots and ffmpeg correctly did not detect them as blackframes. I think we´re cool with black.
Updated my script with some analysis. Having some difficulties comprehending A1ex advanced pieces but I think I got something similar in another way. bc causes parse errors as expected when missing a zero. It works anyway even if it looks shitty in terminal:
for i in *.MOV; do
duration=$(ffprobe -loglevel quiet -of compact=p=0:nk=1 -show_entries format=duration -i "$i")
first_black=$(ffmpeg -i "$i" -to 3 -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
near_end=$(echo $duration - 3 | bc -l)
last_black=$(echo $duration - $(ffmpeg -ss $near_end -i "$i" -vf "blackdetect=d=0.1:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2) | bc -l)
trimmed=$(echo $last_black - $first_black - 0.01 | bc -l)
if ! [ x"$first_black" = x ] && ! [ x"$last_black" = x ]; then
ffmpeg -ss $first_black -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"
elif ! [ x"$first_black" = x ] && [ x"$last_black" = x ]; then
ffmpeg -ss $first_black -i "$i" -t $duration -vcodec copy -acodec copy new_"$i"
elif [ x"$first_black" = x ] && ! [ x"$last_black" = x ]; then
ffmpeg -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"
fi
done
Still have to think something out with shorter than 3 s files.
The bc trickery goes like this:
echo "0.2+0.2" | bc
.4
echo "2.5+2.5" | bc
5.0
The first value is not accepted by ffmpeg (wherever it may appear); let's prepend a 0:
echo 0$(echo "0.2+0.2" | bc)
0.4
echo 0$(echo "2.5+2.5" | bc)
05.0
The leading zero from the second example is harmless - both values are accepted by ffmpeg.
Now the conditionals: I've looked through the man page of bc (it's the first time I use it) for some conditional operator, or min/max functions, but couldn't find any. As I didn't want to figure out the advanced syntax, I've stumbled upon the relational operators - for example (1 < 2) evaluates as 1 and (1 == 2) evaluates as 0. That makes it easy to build conditional expressions without knowing the if/then syntax.
After a second look over the man page, the if/then syntax no longer looks that hard:
echo "if (1 == 2) 5 else 7" | bc
BTW, a nice alternative to bc is pythonpy (https://github.com/Russell91/pythonpy).
Quote from: a1ex on August 28, 2017, 08:38:49 PM
With a manual lens, without pressing anything, LiveView starts directly into crop mode. With a Canon lens, it flickers a bit (starts in regular mode, then switches to crop mode after 1-2 seconds). Not sure what the issue is...
edit: sometimes the screen remains black when switching modes around (not sure how to reproduce); in this case, you don't need to go to photo mode; just press MENU twice (see first post).
Okey, my experience with manual lenses without electronics is that the liveview wont start. I'll say for the slow mo crop modes atleast. I have not used the high res modes for quite some time.
Edit: 2x menu works aswell! I retreat my bug. But Start/Stop button thing works too.
Tried the bc trickery but seems I´ve been building a different code nest so decided to go for some other roundtrippin´:
for i in *.MOV; do
duration=$(ffprobe -loglevel quiet -of compact=p=0:nk=1 -show_entries format=duration -i "$i")
if (( $(echo "$duration > 3" |bc -l) )); then
snippet=$(echo 2)
else
snippet=$(echo 3)
fi
first_black=$(ffmpeg -i "$i" -to 3 -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
near_end=$(echo $duration - $snippet | bc -l)
last_black=$(ffmpeg -ss $near_end -i "$i" -vf "blackdetect=d=0.1:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
if ! [ x"$last_black" = x ]; then
last_black=$(echo $duration - $last_black | bc -l)
fi
trimmed=$(echo $last_black - $first_black - 0.05 | bc -l)
if ! [ x"$first_black" = x ] && ! [ x"$last_black" = x ]; then
ffmpeg -ss $first_black -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"
elif ! [ x"$first_black" = x ] && [ x"$last_black" = x ]; then
ffmpeg -ss $first_black -i "$i" -t $duration -vcodec copy -acodec copy new_"$i"
elif [ x"$first_black" = x ] && ! [ x"$last_black" = x ]; then
ffmpeg -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"
fi
done
Too tired to explain stuff right now. Cutting the last frames with - 0.05 seems closer to perfect then - 0.01 from where I´m sitting.
Great script snippets today. Learnt a lot.
Did some minor changes which now seems to yield perfect cuts at start and ending in the script. Since we now how to delimit reencoding by using -to I tried the reverse filter again to achieve blackframe detection of the ending part.
this part:
near_end=$(echo $duration - $snippet | bc -l)
last_black=$(ffmpeg -ss $near_end -i "$i" -vf "blackdetect=d=0.1:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
becomes this:
last_black=$(ffmpeg -i "$i" $snippet -vf "reverse,""blackdetect=d=0.1:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
trimming section back to 0.01 from 0.05:
trimmed=$(echo $last_black - $first_black - 0.01 | bc -l)
Full script:(updated)
for i in *.MOV; do
duration=$(ffprobe -loglevel quiet -of compact=p=0:nk=1 -show_entries format=duration -i "$i")
if (( $(echo "$duration < 5" |bc -l) )); then
snippet=$(echo 2)
else
snippet=$(echo 5)
fi
first_black=$(ffmpeg -i "$i" -to $snippet -vf "blackdetect=d=0.1:pix_th=0.09" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
last_black=$(ffmpeg -i "$i" -to $snippet -vf "reverse,""blackdetect=d=0.1:pix_th=0.09" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
if ! [ x"$last_black" = x ]; then
last_black=$(echo $duration - $last_black | bc -l)
fi
trimmed=$(echo $last_black - $first_black - 0.01 | bc -l)
if ! [ x"$first_black" = x ] && ! [ x"$last_black" = x ]; then
ffmpeg -ss $first_black -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"
elif ! [ x"$first_black" = x ] && [ x"$last_black" = x ]; then
ffmpeg -ss $first_black -i "$i" -t $duration -vcodec copy -acodec copy new_"$i"
elif [ x"$first_black" = x ] && ! [ x"$last_black" = x ]; then
ffmpeg -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"
fi
done
https://www.dropbox.com/s/pjebqj2yf2bdf3t/YORAN800_S001_S001_T009.MOV?dl=0
I'm trying to run the script to remove the black frames and installed ffmpeg and it seems to process the file,
but the quicktime does not change.
I have attached the input quicktime file
Is there something that I need to change?
thunderbooksmbp:black thunderbook$ ls
YORAN800_S001_S001_T009.MOV
thunderbooksmbp:black thunderbook$ for i in *.MOV; do
> duration=$(ffprobe -loglevel quiet -of compact=p=0:nk=1 -show_entries format=duration -i "$i")
> if (( $(echo "$duration > 3" |bc -l) )); then
> snippet=$(echo 2)
> else
> snippet=$(echo 3)
> fi
> first_black=$(ffmpeg -i "$i" -to $snippet -vf "blackdetect=d=0.1:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
> last_black=$(ffmpeg -i "$i" -to $snippet -vf "reverse,""blackdetect=d=0.1:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
> if ! [ x"$last_black" = x ]; then
> last_black=$(echo $duration - $last_black | bc -l)
> fi
> trimmed=$(echo $last_black - $first_black - 0.01 | bc -l)
> if ! [ x"$first_black" = x ] && ! [ x"$last_black" = x ]; then
> ffmpeg -ss $first_black -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"
> elif ! [ x"$first_black" = x ] && [ x"$last_black" = x ]; then
> ffmpeg -ss $first_black -i "$i" -t $duration -vcodec copy -acodec copy new_"$i"
> elif [ x"$first_black" = x ] && ! [ x"$last_black" = x ]; then
> ffmpeg -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"
> fi
> done
thunderbooksmbp:black thunderbook$
You are probably missing ffprobe. Easiest install for mac is to do this in terminal:
brew install ffmpeg
Will install ffmpeg, ffplay, ffprobe.
If already installed try:
brew uninstall ffmpeg
then
brew install ffmpeg
again.
Here is homebrew by the way.
https://brew.sh/index_se.html
*Update. Just tried your file and indeed, it doesn´t process, go figure!
Will check further. Stay tuned.
This isn´t an original canon H.264 MOV file but prores file right? Think you want to run original H264 files for this workflow. I also noted that your black sequence is pretty long at the end. Probably more than three seconds, is that right?
@Lars Steenhoff
Tested some more and I managed to clean your prores file as well. I had to change threshold settings and also black length estimation prolonged to 5 seconds from earlier 3. Now it needs to be tested on darker footage so it doesn´t break elsewhere. Seemed to work on my test files over here. Corrected an error as well.
New script.
for i in *.MOV; do
duration=$(ffprobe -loglevel quiet -of compact=p=0:nk=1 -show_entries format=duration -i "$i")
if (( $(echo "$duration < 5" |bc -l) )); then
snippet=$(echo 2)
else
snippet=$(echo 5)
fi
first_black=$(ffmpeg -i "$i" -to $snippet -vf "blackdetect=d=0.1:pix_th=0.09" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
last_black=$(ffmpeg -i "$i" -to $snippet -vf "reverse,""blackdetect=d=0.1:pix_th=0.09" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
if ! [ x"$last_black" = x ]; then
last_black=$(echo $duration - $last_black | bc -l)
fi
trimmed=$(echo $last_black - $first_black - 0.01 | bc -l)
if ! [ x"$first_black" = x ] && ! [ x"$last_black" = x ]; then
ffmpeg -ss $first_black -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"
elif ! [ x"$first_black" = x ] && [ x"$last_black" = x ]; then
ffmpeg -ss $first_black -i "$i" -t $duration -vcodec copy -acodec copy new_"$i"
elif [ x"$first_black" = x ] && ! [ x"$last_black" = x ]; then
ffmpeg -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"
fi
done
Thanks, I installed the homebrew and ffprobe.
the reason why this is a prores file, is that I recorded it with and external recorder triggered by the hdmi record trigger from h264. I did not remember when I upload it was the pores version.
Actually I aslo have the internal camera file, it has the same length of black file.
https://www.dropbox.com/s/s8xzqghjjaddnod/953A9793.MOV?dl=0
The new script cleaned up both of them well :)
EDIT: it seems the h264 still has some black part at the end.
All-I vs IPB
I had the camera on IPB I don't know if thats affects anything for the black frames?
I mean how the file can be cut of, or this does not matter?
thunderbooksmbp:black thunderbook$ for i in *.MOV; do duration=$(ffprobe -loglevel quiet -of compact=p=0:nk=1 -show_entries format=duration -i "$i"); if (( $(echo "$duration > 3" |bc -l) )); then snippet=$(echo 2); else snippet=$(echo 3); fi; first_black=$(ffmpeg -i "$i" -to $snippet -vf "blackdetect=d=0.1:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2); last_black=$(ffmpeg -i "$i" -to $snippet -vf "reverse,""blackdetect=d=0.1:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2); if ! [ x"$last_black" = x ]; then last_black=$(echo $duration - $last_black | bc -l); fi; trimmed=$(echo $last_black - $first_black - 0.01 | bc -l); if ! [ x"$first_black" = x ] && ! [ x"$last_black" = x ]; then ffmpeg -ss $first_black -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"; elif ! [ x"$first_black" = x ] && [ x"$last_black" = x ]; then ffmpeg -ss $first_black -i "$i" -t $duration -vcodec copy -acodec copy new_"$i"; elif [ x"$first_black" = x ] && ! [ x"$last_black" = x ]; then ffmpeg -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"; fi; done
thunderbooksmbp:black thunderbook$ for i in *.MOV; do
> duration=$(ffprobe -loglevel quiet -of compact=p=0:nk=1 -show_entries format=duration -i "$i")
> if (( $(echo "$duration < 5" |bc -l) )); then
> snippet=$(echo 2)
> else
> snippet=$(echo 5)
> fi
> first_black=$(ffmpeg -i "$i" -to $snippet -vf "blackdetect=d=0.1:pix_th=0.09" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
> last_black=$(ffmpeg -i "$i" -to $snippet -vf "reverse,""blackdetect=d=0.1:pix_th=0.09" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
> if ! [ x"$last_black" = x ]; then
> last_black=$(echo $duration - $last_black | bc -l)
> fi
> trimmed=$(echo $last_black - $first_black - 0.01 | bc -l)
> if ! [ x"$first_black" = x ] && ! [ x"$last_black" = x ]; then
> ffmpeg -ss $first_black -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"
> elif ! [ x"$first_black" = x ] && [ x"$last_black" = x ]; then
> ffmpeg -ss $first_black -i "$i" -t $duration -vcodec copy -acodec copy new_"$i"
> elif [ x"$first_black" = x ] && ! [ x"$last_black" = x ]; then
> ffmpeg -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"
> fi
> done
ffmpeg version 3.3.3 Copyright (c) 2000-2017 the FFmpeg developers
built with Apple LLVM version 8.1.0 (clang-802.0.41)
configuration: --prefix=/usr/local/Cellar/ffmpeg/3.3.3 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-frei0r --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid --enable-opencl --enable-videotoolbox --disable-lzma --enable-libopenjpeg --disable-decoder=jpeg2000 --extra-cflags=-I/usr/local/Cellar/openjpeg/2.2.0/include/openjpeg-2.2 --enable-nonfree --enable-vda
libavutil 55. 58.100 / 55. 58.100
libavcodec 57. 89.100 / 57. 89.100
libavformat 57. 71.100 / 57. 71.100
libavdevice 57. 6.100 / 57. 6.100
libavfilter 6. 82.100 / 6. 82.100
libavresample 3. 5. 0 / 3. 5. 0
libswscale 4. 6.100 / 4. 6.100
libswresample 2. 7.100 / 2. 7.100
libpostproc 54. 5.100 / 54. 5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '953A9793.MOV':
Metadata:
major_brand : qt
minor_version : 537331968
compatible_brands: qt CAEP
com.apple.quicktime.make: Canon
com.apple.quicktime.model: Canon EOS 5D Mark III
com.apple.quicktime.rating.user: 0.000000
creation_time : 2017-08-27T16:44:02.000000Z
Duration: 00:00:17.48, start: 0.000000, bitrate: 27129 kb/s
Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 25531 kb/s, 25 fps, 25 tbr, 25k tbn, 50k tbc (default)
Metadata:
creation_time : 2017-08-27T16:44:02.000000Z
timecode : 17:54:07:22
Stream #0:1(eng): Audio: pcm_s16le (sowt / 0x74776F73), 48000 Hz, stereo, s16, 1536 kb/s (default)
Metadata:
creation_time : 2017-08-27T16:44:02.000000Z
timecode : 17:54:07:22
Stream #0:2(eng): Data: none (tmcd / 0x64636D74), 0 kb/s (default)
Metadata:
creation_time : 2017-08-27T16:44:02.000000Z
timecode : 17:54:07:22
Output #0, mov, to 'new_953A9793.MOV':
Metadata:
major_brand : qt
minor_version : 537331968
compatible_brands: qt CAEP
com.apple.quicktime.make: Canon
com.apple.quicktime.model: Canon EOS 5D Mark III
com.apple.quicktime.rating.user: 0.000000
encoder : Lavf57.71.100
Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, bt709), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 25531 kb/s, 25 fps, 25 tbr, 25k tbn, 25k tbc (default)
Metadata:
creation_time : 2017-08-27T16:44:02.000000Z
timecode : 17:54:07:22
Stream #0:1(eng): Audio: pcm_s16le (sowt / 0x74776F73), 48000 Hz, stereo, s16, 1536 kb/s (default)
Metadata:
creation_time : 2017-08-27T16:44:02.000000Z
timecode : 17:54:07:22
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame= 330 fps=0.0 q=-1.0 Lsize= 48488kB time=00:00:12.77 bitrate=31090.9kbits/s speed= 163x
video:46004kB audio:2476kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.017650%
ffmpeg version 3.3.3 Copyright (c) 2000-2017 the FFmpeg developers
built with Apple LLVM version 8.1.0 (clang-802.0.41)
configuration: --prefix=/usr/local/Cellar/ffmpeg/3.3.3 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-frei0r --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid --enable-opencl --enable-videotoolbox --disable-lzma --enable-libopenjpeg --disable-decoder=jpeg2000 --extra-cflags=-I/usr/local/Cellar/openjpeg/2.2.0/include/openjpeg-2.2 --enable-nonfree --enable-vda
libavutil 55. 58.100 / 55. 58.100
libavcodec 57. 89.100 / 57. 89.100
libavformat 57. 71.100 / 57. 71.100
libavdevice 57. 6.100 / 57. 6.100
libavfilter 6. 82.100 / 6. 82.100
libavresample 3. 5. 0 / 3. 5. 0
libswscale 4. 6.100 / 4. 6.100
libswresample 2. 7.100 / 2. 7.100
libpostproc 54. 5.100 / 54. 5.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f8936003400] overread end of atom '?mak' by 2 bytes
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f8936003400] overread end of atom '?swr' by 3 bytes
Guessed Channel Layout for Input Stream #0.0 : 4.0
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'YORAN800_S001_S001_T009.MOV':
Metadata:
major_brand : qt
minor_version : 537199360
compatible_brands: qt
creation_time : 2017-08-27T17:30:00.000000Z
make : Atomos
make-eng : Atomos
encoder : NinjaBld - 5.2
encoder-eng : NinjaBld - 5.2
timecode : 17:54:07:24
Duration: 00:00:17.20, start: 0.000000, bitrate: 156029 kb/s
Stream #0:0(eng): Audio: pcm_s16le (sowt / 0x74776F73), 48000 Hz, 4.0, s16, 3072 kb/s (default)
Metadata:
creation_time : 2017-08-27T17:30:00.000000Z
Stream #0:1(eng): Video: prores (apch / 0x68637061), yuv422p10le(bt709, top coded first (swapped)), 1920x1080, 152908 kb/s, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 2500 tbn, 2500 tbc (default)
Metadata:
creation_time : 2017-08-27T17:30:00.000000Z
encoder : Apple ProRes 422 (HQ)
Stream #0:2(eng): Data: none (tmcd / 0x64636D74), 0 kb/s (default)
Metadata:
creation_time : 2017-08-27T17:30:00.000000Z
timecode : 17:54:07:24
Output #0, mov, to 'new_YORAN800_S001_S001_T009.MOV':
Metadata:
major_brand : qt
minor_version : 537199360
compatible_brands: qt
timecode : 17:54:07:24
make : Atomos
make-eng : Atomos
encoder : Lavf57.71.100
Stream #0:0(eng): Video: prores (apch / 0x68637061), yuv422p10le(bt709, top coded first (swapped)), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 152908 kb/s, 25 fps, 25 tbr, 10k tbn, 2500 tbc (default)
Metadata:
creation_time : 2017-08-27T17:30:00.000000Z
encoder : Apple ProRes 422 (HQ)
Stream #0:1(eng): Audio: pcm_s16le (sowt / 0x74776F73), 48000 Hz, 4.0, s16, 3072 kb/s (default)
Metadata:
creation_time : 2017-08-27T17:30:00.000000Z
Stream mapping:
Stream #0:1 -> #0:0 (copy)
Stream #0:0 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame= 318 fps=0.0 q=-1.0 Lsize= 236864kB time=00:00:12.69 bitrate=152834.9kbits/s speed=52.2x
video:232082kB audio:4776kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.002831%
ffmpeg -i INPUT.MOV -c copy -map 0:a OUTPUT.WAV
Next I try to extract the wav file, and want to change the script to process all the .mov files in the folder and make the wav file with the corresponding names.
to allow Davinci Resolve to import the audio and raw file sync up.
Here is the corresponding MLV for testing
https://www.dropbox.com/s/hhnn68a24mer6q6/953A9793.MLV?dl=0
And all the files together
https://www.dropbox.com/sh/a3a5wtyppgddnt4/AABh0Snrmb3fJC4T4CqrvTVra?dl=0
I will put davinci resolve compliance and automation workflows in Switch eventually. For dng folder embedding you´ll need some additional wav metadata. You can inject the data with bwfmetaedit.
You can check row 190 and onwards in here:
https://bitbucket.org/Dannephoto/switch/src/fa38c2375f43b051bf54268e3912dd6c946321a9/source_code/mlv_dump_01.txt?at=default&fileviewer=file-view-default
If you want to get the wav files from the script you can run this right away next to your MOV files:
for i in *.MOV; do
duration=$(ffprobe -loglevel quiet -of compact=p=0:nk=1 -show_entries format=duration -i "$i")
if (( $(echo "$duration < 5" |bc -l) )); then
snippet=$(echo 2)
else
snippet=$(echo 5)
fi
first_black=$(ffmpeg -i "$i" -to $snippet -vf "blackdetect=d=0.1:pix_th=0.09" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
last_black=$(ffmpeg -i "$i" -to $snippet -vf "reverse,""blackdetect=d=0.1:pix_th=0.09" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
if ! [ x"$last_black" = x ]; then
last_black=$(echo $duration - $last_black | bc -l)
fi
trimmed=$(echo $last_black - $first_black - 0.01 | bc -l)
if ! [ x"$first_black" = x ] && ! [ x"$last_black" = x ]; then
ffmpeg -ss $first_black -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"
ffmpeg -i new_"$i" -c copy -map 0:a new_$(echo "$i" | cut -d "." -f1).WAV
elif ! [ x"$first_black" = x ] && [ x"$last_black" = x ]; then
ffmpeg -ss $first_black -i "$i" -t $duration -vcodec copy -acodec copy new_"$i"
ffmpeg -i new_"$i" -c copy -map 0:a new_$(echo "$i" | cut -d "." -f1).WAV
elif [ x"$first_black" = x ] && ! [ x"$last_black" = x ]; then
ffmpeg -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"
ffmpeg -i new_"$i" -c copy -map 0:a new_$(echo "$i" | cut -d "." -f1).WAV
fi
doneQuoteEDIT: it seems the h264 still has some black part at the end.
will check.
Seems there is a pure white frame at the end of the newly created file. Not sure what´s going on.
I'm not sure if its something todo with All-I vs IPB
I had the camera on IPB I don't know if thats affects anything for the black frames.
Because the prores does it well with the same settings. ( prores is All-I )
and because we don't recompress the .MOV file maybe there are only certain frames that are allowed to be cut.
Purely speculating.
Actually, I think it´s the floating point numbers. FFmpeg can´t cut the files with that precision on the frames so it does some crude cutting instead(Now this is speculation) However. Here´s a version which simply shaves off the the floating point leaving us with integer values. Didn´t test very much but it cuts you example H.264 file. Probably would be better if we could round numbers.
By the way. I actually managed to trick a dark shot for blackframe with the latest changes that worked on your prores file.
-vf "blackdetect=d=0.1:pix_th=0.09"
If you´re working with in cam MOV files I suggest you set the script to:
-vf "blackdetect=d=0.1:pix_th=0.01" in both places in the script.
*erased* not working.
This new scripts cuts off a bit too much, it also cuts the footage in the end.
I put them all in top of each other in resolve and manually cut the original black frames to see the difference
Thanks for pm with files. I guess it will be pretty hard to match exactly all the time, especially the ending part. I see to it that wav will match and work in Switch. Easier to cut wav when you extracted the amount of dng files. Thanks for testing and feedback.
If anybody wants to run MOV proxy/wav automation in Switch I just released a version which supports it.
http://www.magiclantern.fm/forum/index.php?topic=15108.msg189314#msg189314
cool, I will check it soon !
Found a way to allocate the entire SRM memory without locking the exposure controls... sort of:
- When allocating the entire SRM memory (normally used for still photos, for holding uncompressed 14-bit raw data), Canon code locks the shutter button and shows the BUSY screen (to prevent you from taking any more pictures until some buffer becomes free again).
- Unfortunately, they also lock the exposure controls after you press the shutter halfway.
- I've tried a bunch of tricks, such as calling their unlock routines used in this allocator (ClearBusy (http://www.magiclantern.fm/forum/index.php?topic=19300.msg189251#msg189251), RealClearBusy); they did unlock the GUI, but pressing the shutter fully (by mistake or for whatever reason) gave ERR80. Unlocking the GUI for a short time and re-locking it again did not re-activate the exposure controls.
- Low-level note: RealSetBusy/RealClearBusy only changes a GPIO; the MPU does react to this change.
- There is another way to lock the GUI - using PROP_ICU_UILOCK (our wrapper is gui_uilock). That one lets you lock only the shutter button (UILOCK_SHUTTER), but does not lock the exposure controls.
- Guess what: having both locks enabled (UILOCK_SHUTTER and Canon's SRM lock) works and DOES NOT LOCK THE EXPOSURE CONTROLS!!!
Therefore, one possibility would be to avoid the UILOCK_NONE state as long as the entire SRM memory is allocated.
However, this has another issue: half-shutter events are no longer present. Not sure how to solve it...
The MPU does send a message that tells the main CPU to show the BUSY screen. Maybe I should just interpret that as half-shutter press. I believe the message is some part of:
45B7B> **INT-36h*:000afe84:00:00: *** mpu_recv(26 24 0a 08 6f 00 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00 01 01 00 00 00 00 00 00), from ff2e87f8
CDA4B> **INT-36h*:000afe84:00:00: *** mpu_recv(26 24 0a 08 6f 00 01 00 01 01 a0 10 00 68 01 01 50 20 30 01 01 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00), from ff2e87f8
Good luck decoding that (http://www.magiclantern.fm/forum/index.php?topic=17596.msg171988#msg171988), though...
Fixing the busy screen. How cool is that. Seems a deep mystery...
If anyone is still interested there is a way to get rid of that last black frame by doing multiple runs on stubborn files. By putting in a while loop we can keep recopy the file until it chops the very last blackframe out.
Some talk about drop frames and black frames over here.
https://video.stackexchange.com/questions/10383/remove-black-frames-from-vob-files-with-ffmpeg
for i in *.MOV; do
duration=$(ffprobe -loglevel quiet -of compact=p=0:nk=1 -show_entries format=duration -i "$i")
if (( $(echo "$duration < 5" |bc -l) )); then
snippet=$(echo 2)
else
snippet=$(echo 5)
fi
first_black=$(ffmpeg -i "$i" -to $snippet -vf "blackdetect=d=0.1:pix_th=0.09" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
last_black=$(ffmpeg -i "$i" -to $snippet -vf "reverse,""blackdetect=d=0.1:pix_th=0.09" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
if ! [ x"$last_black" = x ]; then
last_black=$(echo $duration - $last_black | bc -l)
fi
trimmed=$(echo $last_black - $first_black - 0.01 | bc -l)
if ! [ x"$first_black" = x ] && ! [ x"$last_black" = x ]; then
ffmpeg -ss $first_black -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"
ffmpeg -i new_"$i" -c copy -map 0:a new_$(echo "$i" | cut -d "." -f1).WAV
elif ! [ x"$first_black" = x ] && [ x"$last_black" = x ]; then
ffmpeg -ss $first_black -i "$i" -t $duration -vcodec copy -acodec copy new_"$i"
ffmpeg -i new_"$i" -c copy -map 0:a new_$(echo "$i" | cut -d "." -f1).WAV
elif [ x"$first_black" = x ] && ! [ x"$last_black" = x ]; then
ffmpeg -ss 0 -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"
ffmpeg -i new_"$i" -c copy -map 0:a new_$(echo "$i" | cut -d "." -f1).WAV
fi
#rerun ending part until we´re satisfied
while ! [ x"$last_black" = x ]; do
duration=$(ffprobe -loglevel quiet -of compact=p=0:nk=1 -show_entries format=duration -i new_"$i")
last_black=$(ffmpeg -i new_"$i" -to $snippet -vf "reverse,""blackdetect=d=0.01:pix_th=0.09" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
if ! [ x"$last_black" = x ]; then
trimmed=$(echo $duration - $last_black | bc -l)
ffmpeg -ss 0 -i new_"$i" -t $trimmed -vcodec copy -acodec copy new_new_"$i"
rm new_"$i"
mv new_new_"$i" new_"$i"
fi
done
done
Better: count the frames in the "sidecar" MLV and trim the MOV to the same number of frames.
Hello,
When I use the proxy mode, H264 files get hot pixels. Is there a tool to remove it on H264 files ? By the way, when using 10/12 compressed mode, I get pink zone in the highlight. Is it removable ? the goal is not to correct it for master file but to have a clean proxy for offline editing.
Hi All
I have been using Full-resolution LiveView: 5796x3870 at 7.4 fps (128ms rolling shutter) - continuous*) at 5 FPS! with success to capture still macro.
Given the low frame rate (5fps) has anyone used this , if so what is the recommend shutter speed for such slow rate under ML for a stationary subject/object. I see that "they" state to double the shutter speed of the FPS. which means if its 5fps then my speed should be 1/10. Given the subject is stationery and there is no movement is there any advantage of increasing the shutter speed. I have done some tests and cannot tell the difference other then loss of light.
cheers
Denis
QuoteBetter: count the frames in the "sidecar" MLV and trim the MOV to the same number of frames.
This is what I do with the wav file. Seems the way to go for proxy files as well. Just tested and verified a perfect cut from the problematic file from Lars Steenhoff. And it´s fast. Since first cut always seems correctly detected, ending part can be calculated via mlv_dump and the MLV file.
Needs later version of mlv_dump to work and the corresponding MLV next to the proxy file.
for i in *.MOV; do
MLV=$(echo "$i" | cut -d "." -f1 | tail -c 5).MLV
duration=$(ffprobe -loglevel quiet -of compact=p=0:nk=1 -show_entries format=duration -i "$i")
if (( $(echo "$duration < 5" |bc -l) )); then
snippet=$(echo 2)
else
snippet=$(echo 5)
fi
first_black=$(ffmpeg -i "$i" -to $snippet -vf "blackdetect=d=0.1:pix_th=0.09" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
#grab amount of dng frames from MLV metadata
frct=$(mlv_dump *"$MLV" | awk '/Processed/ { print $2; }')
FPS=$(mlv_dump *"$MLV" | awk '/Processed/ { print $6; }')
trimmed=$(echo $frct/$FPS | bc -l | awk 'FNR == 1 {print}')
if ! [ x"$first_black" = x ]; then
ffmpeg -ss $first_black -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"
elif [ x"$first_black" = x ]; then
ffmpeg -ss 0 -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"
fi
doneThe trustworthy number is to grab:
frct=$(mlv_dump *"$MLV" | awk '/Processed/ { print $2; }')Grabbing the Processed number is a bit slow on bigger MLV files since it will process the whole file before giving you the information. There is a
Frames Video: metadata tag which could be used but it´s never exact so not useful here.
I will rewrite the processing in Switch maybe tonight...
You can specify -vframes to avoid doing the (possibly inexact) FPS math.
-vframes, never heard of. Is there any difference running -vframes from only -v? Speed increase?
My understanding is that this:
frct=$(mlv_dump *"$MLV" | awk '/Processed/ { print $2; }')
FPS=$(mlv_dump *"$MLV" | awk '/Processed/ { print $6; }')
this:
frct=$(mlv_dump -vframes *"$MLV" | awk '/Processed/ { print $2; }')
FPS=$(mlv_dump -vframes *"$MLV" | awk '/Processed/ { print $6; }')
and this:
frct=$(mlv_dump -v *"$MLV" | awk '/Processed/ { print $2; }')
FPS=$(mlv_dump -v *"$MLV" | awk '/Processed/ { print $6; }')
Will do exactly the same thing?
My bad, it's -frames:v. Things have changed since I've last read the manual :D
hehe.
Let´s see.
I get this:
dans-MacBook-Pro:~ dan$ mlv_dump -frames:v /Users/dan/Desktop/MLV_files/h264proxy/953A9793.MLV
MLV Dumper
-----------------
Mode of operation:
- Input MLV file: '/Users/dan/Desktop/MLV_files/h264proxy/953A9793.MLV'
- Verify file structure
File /Users/dan/Desktop/MLV_files/h264proxy/953A9793.MLV opened
File /Users/dan/Desktop/MLV_files/h264proxy/953A9793.M00 not existing.
Processing...
Reached end of chunk 1/1 after 328 blocks
Processed 315 video frames at 25.00 FPS (12.60 s)
Done
And exactly the same without:
dans-MacBook-Pro:~ dan$ mlv_dump /Users/dan/Desktop/MLV_files/h264proxy/953A9793.MLV
MLV Dumper
-----------------
Mode of operation:
- Input MLV file: '/Users/dan/Desktop/MLV_files/h264proxy/953A9793.MLV'
- Verify file structure
File /Users/dan/Desktop/MLV_files/h264proxy/953A9793.MLV opened
File /Users/dan/Desktop/MLV_files/h264proxy/953A9793.M00 not existing.
Processing...
Reached end of chunk 1/1 after 328 blocks
Processed 315 video frames at 25.00 FPS (12.60 s)
Done
Default -frames:v setting when nothing selected? How to verify?
It's not an undocumented option (hint (https://www.google.com/search?q=%22frames%3Av%22)).
FFmpeg domain, not mlv_dump. That changes stuff. Let me have a look.
-frames:v
Not really working with -vcodec it seems and then we´re back to reencoding delays. Anyway. I trust these lines below. Been running a long time with wav files in Switch and before that :).
frct=$(mlv_dump *"$MLV" | awk '/Processed/ { print $2; }')
FPS=$(mlv_dump *"$MLV" | awk '/Processed/ { print $6; }')
trimmed=$(echo $frct/$FPS | bc -l | awk 'FNR == 1 {print}')
$ time ffmpeg -i BLACK_4GB.MOV -frames:v 500 -vcodec copy -acodec copy x.mov -y -loglevel error -hide_banner
real 0m2.530s
user 0m0.420s
sys 0m1.296s
$ time ffmpeg -i BLACK_4GB.MOV -vframes 500 -vcodec copy -acodec copy y.mov -y -loglevel error -hide_banner
real 0m2.628s
user 0m0.448s
sys 0m1.296s
$ time ffmpeg -i BLACK_4GB.MOV -to 10 -vcodec copy -acodec copy z.mov -y -loglevel error -hide_banner
real 0m2.575s
user 0m0.480s
sys 0m1.252s
$ stat x.mov y.mov z.mov | grep Size
Size: 483990897 Blocks: 945304 IO Block: 4096 regular file
Size: 483990897 Blocks: 945296 IO Block: 4096 regular file
Size: 484135017 Blocks: 945584 IO Block: 4096 regular file
$ ffprobe -i BLACK_4GB.MOV |& grep fps
Stream #0:1(eng): Video: prores (apch / 0x68637061), yuv422p10le(bt709), 1920x1080, 384385 kb/s, SAR 1:1 DAR 16:9, 50 fps, 50 tbr, 5k tbn, 5k tbc (default)
$ vbindiff y.mov z.mov
-0000 0010: 71 74 20 20 00 00 00 08 77 69 64 65 1C D8 FD 8C qt .... wide....
+0000 0010: 71 74 20 20 00 00 00 08 77 69 64 65 1C DB 30 0C qt .... wide..0.
-1B78 3FA0: 85 09 FF B7 30 AE 5B FC 00 0E C4 00 69 63 70 66 ....0.[. ....icpf
+1B78 3FA0: 85 09 FF B7 30 AE 5B FC 00 00 00 00 00 00 00 00 ....0.[. ........
...
Minor differences at the end of the file and in the header, but neither of these variants is re-encoding the file.
Interesting. Tried again and working. At least -vframes and -frames:v options, and with vcodec. Must have missed something before.
8)
-0000 0010: 71 74 20 20 00 00 00 08 77 69 64 65 1C D8 FD 8C qt .... wide....
+0000 0010: 71 74 20 20 00 00 00 08 77 69 64 65 1C DB 30 0C qt .... wide..0.
-1B78 3FA0: 85 09 FF B7 30 AE 5B FC 00 0E C4 00 69 63 70 66 ....0.[. ....icpf
+1B78 3FA0: 85 09 FF B7 30 AE 5B FC 00 00 00 00 00 00 00 00 ....0.[. ........
I did some real life testing between my frame_count/fps solution and the -frames:v option. One anomaly is that last frame is prolonged(probably by ffmpeg due to audio sync/iframe etc). The prolonged last image is much longer with the -frames:v option.
Check files:
-frames:v
https://bitbucket.org/Dannephoto/magic-lantern/downloads/vframes.MOV
frame_count/fps
https://bitbucket.org/Dannephoto/magic-lantern/downloads/divisioned.MOV
Conclusion based on this very file only. -frames:v should be closer to perfect in theory but it isn´t? Or maybe I´m just lucky with a more crude number.
Build August 26 - 5d3 1.1.3
Crop 3,5K 23,98 fps 14 bits Losless
Message when move joystick
Plenty of death pixel when zoom x5
(http://thumb.ibb.co/fhyK1k/20170831_172921.jpg) (http://ibb.co/fhyK1k)
(http://thumb.ibb.co/cw9EZ5/20170831_173243.jpg) (http://ibb.co/cw9EZ5)
Man you guy's are working.. Loving the progress.
I think there is a bug in the builds since 19th August with 50/60p 3x3.
I am getting some "Frame Jumps" I don't know how to explain it, but its like a frame will repeat it self 5-10 frames later. If that makes sense? At first I thought it was slow external drive causing it but I see it also after transcoding. It might happens only once in an MLV and sometimes 2-3 times. This bug might have been there all along, I think I have seen it "here and there" before, but very random, I will have to have a look at older takes from this summer. But since 19th almost every MLV has this problem.
I have not tried yet to cut the frames out, but not sure if this repeat frame overrides the one that "should have been in its place".
But then again it could also be the Converter, Danne's Batch script, I don't know. I cant see the footage until I have converted the MLV's to DNG.
At the moment, I am on crappy satellite internet again.. This Sat is extra bad, there will be no uploading, but might have some proper internet tomorrow and I will upload a MLV.
But I thought I atleast mention it now in case someone else experienced this and could chime in.
5D3 123. Aug 20th build. The shots I noticed this most on were shot a week ago. I have since then updated to 26th, but have not looked at anything I shot with 26th build. (no time)
Hey a1ex or somone knows the answer
why we can't use 12or 10bit lossless for 720p mode
Can you explain?
My camera canon 700d
I have no idea, I don't even have a 700D...
https://www.chiark.greenend.org.uk/~sgtatham/bugs.html
Please provide feedback on what works, what not, what happens when it doesn't work. Same for EOS M.
If it's a regression (that is, if it used to work in some older build), regardless of what camera model or feature, please run "hg bisect" to identify the change that introduced the issue.
I get the "busy" message while the 5x crop and raw rec is on an (can't change apperture and shutter speed). In my memory it didn't happen on previous builds. I'm on the latest available Aug 26 5DIII123.
Read last 10-20 posts.
You can press menu button twice to get rid of the signal.
Quote from: Danne on September 02, 2017, 11:33:57 AM
Read last 10-20 posts.
You can press menu button twice to get rid of the signal.
My bad, found it. Thank you.
When will the experimental build become nightly build?
and why is it still experimental?
Answered on... the download page.
what is the difference between 1:1 crop and full frame?
More about the BUSY screen:
* Tried applying the SRM_ClearBusy hack and manually block the full shutter event. Result: the camera ignores the event and stays in LiveView for about 2 seconds, then the MPU throws ERR80.
* Tried decoding the BUSY message (the one from UILOCK_SHUTTER) - it's complicated, they are different in each video mode. After getting it somewhat working, I've noticed it also blocks autofocus. Go figure...
* In the end, I've noticed that freeing one SRM buffer (and keeping the others allocated) appears to be a good compromise (even though it's still a hackish solution):
- no BUSY screen
- with N-1 buffers allocated from SRM, and nothing from Shoot memory, Canon firmware:
- will still record H.264 (it will use the last buffer in this case)
- will take a picture, then ERR70 (because it expects the SRM buffers to be freed in the same order as allocated)
- will go outside LiveView and back, but will show BUSY on the screen
- therefore, in this case, using the last SRM buffer is NOT safe
- with the entire Shoot memory is allocated, and then N-1 buffers from SRM, Canon firmware is very unlikely to use the last buffer:
- will not start recording H.264 (if the allocation was done in standby) -> ERR70 (timeout allocating memory).
- will not stop recording H.264 (if the allocation was done while recording) -> lockup, ERR70 after 10-20 seconds (similar)
- will not finish taking a picture (BUSY screen after capturing the exposure; will overwrite the additional SRM buffer)
- will not enter H.264 playback mode from LiveView (black screen, Cannot playback movie after 10-20 seconds)
- will enter CR2 playback mode from LiveView
- will exit LiveView (to regular photo mode)
- will enter Canon menu
- all the failed actions from above will succeed as soon as we will deallocate our Shoot and SRM memory buffers (if done before the timeout)
- with the exception of still photo capture, none of the above actions resulted in overwriting the additional SRM buffer
- therefore, in this case, I believe using the last SRM buffer during LiveView is *probably* safe.
In any case, this method should be a lot safer than using the entire SRM memory after free (as in previous builds without the BUSY screen).
I also believe mlv_rec (in its current state) should now be usable on the current codebase (except maybe for full-res LiveView), as the memory management trickery has been moved into the memory backend.
BTW - if any of you has any clue on how to reproduce this bug (http://www.magiclantern.fm/forum/index.php?topic=19300.msg189513#msg189513), it will be very appreciated.
Forgot to mention - 700D and EOS M will need an extra constant (easy to find).
New build posted.
Quote from: a1ex on September 03, 2017, 03:30:09 PM
700D and EOS M will need an extra constant (easy to find).
SRM_BUFFER_SIZE ?
Sorry. for the big file:
https://mega.nz/#!pQwT0BQT!r2P1GdlBTgXBcQrx3g4Cqao_pVUO6TLv_cOxZt-zKmQ 751 MB
here is one example of the repeat frame.
And I can basically reproduce this all the time.
Frame 336 and 338 are the same.
Justy downloaded your file. It´s file 335 and 337 that are the same over here at least. Strange.
That is weird, I went to check if I saw it wrong. But here it is 336 and 338.
I will reconvert the file and see if it changes again. How did you convert it ?
Maybe you could preview the MLV file itself with the MLVFS MLRawViewer thing?
Both in Switch and mlv_dump alone. I don´t think that is the problem.
If viewed in MlRawViewever it´s 336 and 338.
Yes I was checking the DNG's in MLRawViewer.
Just tested latest crop_rec_4k build. The busy signal is gone! What an achievement. The camera is set free again :).
So, what did I test. Decided to run a few MLV takes in 14bit-lossless 24fps and (5D mark III 1.1.3) with proxy recording on. This combo works great and continuously. My findings:
Short files are ok regarding metadata. Then I recorded a 3 minute MLV and suddenly metadata for processed frames weren´t to be found. I got this instead. Check the end. Anything obvious here?:
Reached end of chunk 1/3 after 2305 blocks
File Header (MLVI)
Size : 0x00000034
Ver : v2.0
GUID : 11411502559624145806
FPS : 23.976000
File : 1 / 0
Frames Video: 2265
Frames Audio: 0
Class Video : 0x00000021
Class Audio : 0x00000000
Block: RAWI
Offset: 0x00000034
Number: 1
Size: 180
Time: 0.756000 ms
Res: 1920x1080
raw_info:
api_version 0x00000001
height 1318
width 2080
pitch 3640
frame_size 0x00493450
bits_per_pixel 14
black_level 2047
white_level 16200
active_area.y1 28
active_area.x1 146
active_area.y2 1318
active_area.x2 2078
exposure_bias 0, 0
cfa_pattern 0x02010100
calibration_ill 1
Got a new RAWI, throwing away average buffers etc.
Got a new RAWI, throwing away previous frame buffers etc.
mlv_dump(6212,0xa533e1c0) malloc: *** error for object 0x848000: pointer being realloc'd was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6
Checking in mlv_dump.c code:
int frame_size = MAX(bit_depth, block_hdr.raw_info.bits_per_pixel) * block_hdr.raw_info.height * block_hdr.raw_info.width / 8;
/* resolution change, old data will be thrown away */
if(frame_arith_buffer)
{
print_msg(MSG_INFO, "Got a new RAWI, throwing away average buffers etc.\n");
free(frame_arith_buffer);
}
if(prev_frame_buffer)
{
print_msg(MSG_INFO, "Got a new RAWI, throwing away previous frame buffers etc.\n");
free(prev_frame_buffer);
}
Does it help if you disable SRM memory?
If yes, try this change:
--- a/src/exmem.c
+++ b/src/exmem.c
@@ -444,1 +444,1 @@
- if (shoot_full_suite)
+ if (0)
*UPDATE
Most of this post was due to a buggy version of mlv_dump I was using from my own ml-dng branch.
Disabling SRM memory doesn´t help. I see though that end chunk is shown so could grep for those for now if needed.
With srm ON:
Reached end of chunk 1/3 after 2309 blocks
Got a new RAWI, throwing away average buffers etc.
Got a new RAWI, throwing away previous frame buffers etc.
mlv_dump(12500,0xa533e1c0) malloc: *** error for object 0x887000: pointer being realloc'd was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6
With srm OFF:
Reached end of chunk 1/2 after 2308 blocks
Got a new RAWI, throwing away average buffers etc.
Got a new RAWI, throwing away previous frame buffers etc.
mlv_dump(11467,0xa533e1c0) malloc: *** error for object 0x895000: pointer being realloc'd was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6
And maybe I should count the actual dng files processed from the MLV file when syncing MOV proxies. Not sure that all files are always extracted. Maybe:
$(find *.dng | wc -l)
* Checked some more. The MOV is way longer than the MLV file. It´s like the MLV file stops in the middle of the recording. MOV is maybe twice the length in a three minute clip.
*august 26th build is giving away the same error
* Error also occurs when not recording proxy with 14bit-lossless mode.
* testing mlv_rec. Notice mlv_play.mo isn´t compaitble with mlv_rec (tc_symbols)
*the *** set a breakpoint in malloc_error_break doesn´t happen with regular mlv_rec, only mlv_lite
*UPDATE
Tested with mlv_dump compiled in crop_rec_4k branch(ran my own ml-dng version before, sorry:
Reached end of chunk 1/3 after 2305 blocks
Got a new RAWI, throwing away average buffers etc.
Got a new RAWI, throwing away previous frame buffers etc.
Reached end of chunk 2/3 after 2278 blocks
Got a new RAWI, throwing away average buffers etc.
Got a new RAWI, throwing away previous frame buffers etc.
Reached end of chunk 3/3 after 296 blocks
Processed 4839 video frames at 23.98 FPS (201.83 s)
*mlv_dump_on steroids processes all dng files! Go figure...
I ran:
mlv_dump_on_steroids --relaxed --dng Input.MLV
Same command didn´t work with crop_rec_4k mlv_dump
*Started looking at my own version of mlv_dump and it seemed I had some old version included. Not sure how that was possible. I updated to this:
https://bitbucket.org/Dannephoto/magic-lantern/branch/ml-dng-unified_11c
And all was working again.
*Conclusion
Messy testing but mlv_rec recording seems fine. Crop_rec_4k mlv_dump version seems broken since it can´t convert any MLV file I tested. Goes into segmentation fault 11.
Recorded a test clip, about 6.5 minutes, 5 chunks, 14-bit lossless 1920x1080, mlv_lite; it plays fine in mlv_play.
mlv_dump from crop_rec_4k:
mlv_dump M04-0015.MLV
MLV Dumper
-----------------
Mode of operation:
- Input MLV file: 'M04-0015.MLV'
- Verify file structure
File M04-0015.MLV opened
File M04-0015.M00 opened
File M04-0015.M01 opened
File M04-0015.M02 opened
File M04-0015.M03 opened
File M04-0015.M04 not existing.
Processing...
Reached end of chunk 1/5 after 2261 blocks
Got a new RAWI, throwing away average buffers etc.
Got a new RAWI, throwing away previous frame buffers etc.
Reached end of chunk 2/5 after 2267 blocks
Got a new RAWI, throwing away average buffers etc.
Got a new RAWI, throwing away previous frame buffers etc.
Reached end of chunk 3/5 after 2303 blocks
Got a new RAWI, throwing away average buffers etc.
Got a new RAWI, throwing away previous frame buffers etc.
Reached end of chunk 4/5 after 2307 blocks
Got a new RAWI, throwing away average buffers etc.
Got a new RAWI, throwing away previous frame buffers etc.
Reached end of chunk 5/5 after 765 blocks
Processed 9847 video frames at 25.00 FPS (393.88 s)
Done
No errors in valgrind. No obvious issues near frame 336 either.
Can you upload the buggy file somewhere?
Been a bit messy with testing. Conclusion:
mlv_dump from crop_rec_4k branch is getting segmentation fault 11 on all files tested today.
My version of ml-dng mlv_dump was causing the error before which is separate from any of the other issues. Tested a newer build of mine and it works processing the correct amount of dng files. Correct frame processing metadata is showing as well. mlv_rec seems all fine. Sorry for the detour.
Tested with a 3:21min MLV with accompanying proxy and it worked perfectly. And I mean perfectly matched from first to last frame. Over 4000 dng files.
About the 336 frame issue I don´t have a clue how to create it. I think Kharak is using 5D mark III 1.2.3 version.
QuoteRecorded a test clip, about 6.5 minutes, 5 chunks, 14-bit lossless 1920x1080, mlv_lite; it plays fine in mlv_play.
I think mlv_rec together with mlv_play is broken.
This change in makefile causes segmentation fault 11 in mlv_dump:
https://bitbucket.org/hudson/magic-lantern/commits/9ce033951922df7a721bfe48a665d9d03656a7d8?at=crop_rec_4k
Hi
cr2hdr app is unable to extract few of my MLV files that exceeds 4GB limit (MLV and M00, M01....). it only creates 1 pink frame. any help with that?
Did you check posts above yours?
Anyway. I renamed the app to Switch and hopefully fixed that issue of yours. Test and report back.
http://www.magiclantern.fm/forum/index.php?topic=15108.0
@ Danne
worked like charm. Thanks a lot!
Quote from: Danne on September 04, 2017, 12:34:25 AM
About the 336 frame issue I don´t have a clue how to create it. I think Kharak is using 5D mark III 1.2.3 version.
I run 123 yes.
Quote from: Kharak on September 03, 2017, 06:48:23 PM
Sorry. for the big file:
https://mega.nz/#!pQwT0BQT!r2P1GdlBTgXBcQrx3g4Cqao_pVUO6TLv_cOxZt-zKmQ 751 MB
here is one example of the repeat frame.
And I can basically reproduce this all the time.
Frame 336 and 338 are the same.
Just to be clear, I think what I wrote about reproducing the issue can be misinterpreted. I don't mean that it is always frame 336 and 338 on every MLV. I mean that almost every MLV has repeated frames, but at random. I read my post and see it could be interpreted as 336 and 338 are always the repeated frames.
But it is always 2 frames apart. I thought it was more random, like some 1-5 frames apart, but so far I see it is always 2 frames apart for the repeat frames. 181-183, 144-146 etc.
I tried cutting the repeat frame out to see if it goes 1-2-(1)-3-4-5-6..etc. But I believe it does overrides the 3rd frame so it is 1-2-(1)-4-5-6.. Cutting the frame out makes it pretty much unnoticeable, but it is there.
I've looked at it last night, but don't have the conclusions ready - meanwhile, can you try to reproduce on 1.1.3?
Yes, I will try 113
@a1ex
On the experimental page you write: "Continuous: 3072x1308 @ 24p, 1920x1080 @ 45p, 1920x960 @ 50p, 1920x800 @ 60p, 4096x2560 @ 8p, 5796x3870 @ 5p."
1920x800 60p 3x3 is not continuous. Atleast not on 1.2.3. I will try 1.1.3 tomorrow, is it supposed to be continuous on 113?
Just checked - it is (still) continuous on 1.2.3.
Did the first test with all the speed hacks enabled (global draw off, frozen LV, card warm up 1GB). Result: it filled the 32GB card (Komputerbay 1000x) on the first try. Details:
- started from ML defaults
- formatted CF card as FAT32 from camera (ML was running from SD)
- loaded crop_rec, mlv_lite, mlv_play, file_man
- set crop mode 50/60p 3x3
- set 60 fps in Canon menu
- enabled raw video (1920x800, 14-bit lossless - it's the default)
- noted compression rate for the test scene (52%)
- disabled global draw, enabled frozen LV and card warm up (1GB)
- rebooted
- pressed REC after the LED activity settled
- when recording stopped, rebooted again and Canon's free space warning appeared.
It was a bit on the edge though - the recording icon was orange all the time, showing ~ 70-80 seconds. The meaning of this number is: if the average write speed would drop to 95% of the measured value (this percentage is hardcoded - 100/105 - see predicted_frames_left), recording would stop after 80 seconds (or whatever that number is).
After that, ran a second test with global draw on (all others unchanged from above) and got similar results (card nearly filled with a single clip, 7 chunks, but the estimation dropped to ~ 16-50 seconds and there were 2GB of free space left). The icon was still orange throughout the test and the preview was in grayscale. Note the writing speed usually drops when the card becomes full (at least with my card).
A third test (with ML defaults, crop_rec to 50/60p 3x3) stopped after less than 10 seconds (so, Frozen LV is important).
A fourth test (back to first settings, test scene with 57% ratio - ISO 6400) stopped after ~ 30 seconds (red icon).
Didn't try on 1.1.3, but I expect it to be a little faster.
BTW - did any of you play with this speed hack (http://www.magiclantern.fm/forum/index.php?topic=19300.msg184290#msg184290) ?
Okey, Frozen LV explains a lot. I never use Frozen LV, its too hard to work with for me.
Yes, I think all cards slow down when 5% is left, maybe more. All my Lexar Cards do the same. For instance when I have 5 GB left, 60P is almost useless.
You are on 123. Are you unable to reproduce the Repeat frames? I still have not looked at any shots I did with Aug-26th Build (just random shots) and I see there is a newer one out now. I will try the newest Experimental before downgrading to 113. I assume you were already on the Aug 26th Build when you tried to reproduce? Maybe even the newest one ?
I just checked footage shot over a period from 14th to 19th of July and they also have plenty of repeat frames, some MLVs have several. And some even more severe a problem where about 3 frames are missing, so 1 2 1 X X X 7 8 9... etc. I cant remember what build that was shot on though, but I believe it was the one from 4th of July or thereabouts.. :/ Atleast it shows its not a recent bug introduced.
my settings:
1920x800 60p Crop mode 3x3
14 Bit lossless
Global Draw ON (But I always hold Half-Shutter right after hitting record to make it go away which increases recording times)
RAW Zebras ON
Histogram RAW Based RGB - Scaling: Log, ETTR Hint, Clip Warning ON.
Real Time Preview
Shutter Fine Tuning: +1.25
Digic Peaking - Slightly Sharper
Exposure Override ON
Movie Simulation.
if you could try with these settings again to reproduce. Perhaps the Half Shutter - Clear Overlays thing could be of interest?
And no I have not tried the Speed Hack, it seems a bit too much over my head.
I tested the latest proxy mode by loading the firmware: magiclantern-crop_rec_4k.2017Sep03.5D3113
Module loaded :
MLV Lite
MLV Rec
MLV Sound
MLV Play
I had a sdhc card and a CF card inside the 5D mkIII.
The proxy file has a Timecode starting at 00:07:58:00
The proxy has sound embeded
The Raw file has a timecode starting at 00:00:00:00
The frames are only aligned at the beginning if I trim 32 black frames on the H.264 file. The raw file is also shorter of 14 frames at the end.
When I recorded different clips, they all seem to have different in and out. And it made it worst to add the pre-recording to two seconds.
My point is that it's propably useless to have proxys with MLV_Lite right now since there is no time code or a consistent way to get them in sync for post production. The best improvement would be to have a consistent timecode.
Useless? Certainly not. TC can easily be changed in ffmpeg.
http://www.magiclantern.fm/forum/index.php?topic=15108.msg189532#msg189532
Also read around the last 30 posts here on the subject.
Quote from: The_bald_guy on September 05, 2017, 06:45:47 PM
My point is that it's propably useless to have proxys with MLV_Lite right now since there is no time code or a consistent way to get them in sync for post production.
Have you tried using Danne's switch (http://www.magiclantern.fm/forum/index.php?topic=15108.0)? It trims the H.264 proxy to the same length as the DNG image sequence along with matching timecode. (Danne also replied while I was still writing this post but he didn't mention his awesome program.)
Quote from: a1ex on September 03, 2017, 03:30:09 PM
Forgot to mention - 700D and EOS M will need an extra constant (easy to find).
The SRM_BUFFER_SIZE for the EOSM and 700D (https://bitbucket.org/hudson/magic-lantern/pull-requests/852/srm_buffer_size-for-eosm-and-700d/diff#comment-None) were pretty easy to find. An interesting difference between these cameras was that on the EOSM the SRM_BUFFER_SIZE wasn't visible until I pressed record or took a silent still while on the 700D it was available as soon as the camera booted up. Maybe this is related to the EOSM not getting into mv1080 mode until it is recording H.264?
QuoteMaybe this is related to the EOSM not getting into mv1080 mode until it is recording H.264?
Cross my fingers. Lossless working and just bought a sigma 10-20mm f3.5 for it filming crop_rec lossless.
Tried proxy filming as well but files came out corrupted and early stop etc...
Quote from: Danne on September 05, 2017, 07:24:14 PM
Tried proxy filming as well but files came out corrupted and early stop etc...
Flashback to when mv1080 (via H.264 proxy) was almost working on the EOSM:
http://www.magiclantern.fm/forum/index.php?topic=16608.msg178438#msg178438
(https://s24.postimg.org/mvvjqkkbp/mv1080eosm_000000.png)
Quote from: dfort on September 05, 2017, 07:13:34 PM
An interesting difference between these cameras was that on the EOSM the SRM_BUFFER_SIZE wasn't visible until I pressed record or took a silent still while on the 700D it was available as soon as the camera booted up.
It should be visible as soon as the SRM memory is allocated by something in ML. The latest mlv_lite, if enabled and the camera is in movie mode, will allocate this memory during standby (as soon as the camera starts); this change is recent - the memory allocation used to be performed when pressing REC. The silent picture module will allocate the memory when taking a picture. Best guess: different settings (maybe raw video wasn't enabled at startup) or maybe the EOS M had an older mlv_lite.
There's a bunch of activity displayed if you enable the console - there you can tell when the memory is (re)allocated.
SRM_BUFFER_SIZE for the 550D
https://www.dropbox.com/s/04yxfu5e98llgz8/srm_malloc_cbr_550D.jpg
srm_malloc_cbr(156510, 48000064, 1f68000)
A1ex or David, could one of you please add the following line to the consts.h header file within the platform/550D.109 folder of the crop_rec_4k branch? (My old ML fork does not yet contain the crop_rec_4k branch and I didn't found a way to sync newer branches to my repository. I guess I need to fork it again.)
#define SRM_BUFFER_SIZE 0x1F68000
Next time I try to find the SRM_BUFFER_SIZE for the 50D and 5D2 too.
May be I missed something.
I tested the latest proxy mode by loading the firmware: magiclantern-crop_rec_4k.2017Sep03.5D3113
Module loaded :
MLV_Lite
crop_rec
ettr
file_man
pic_view
1920X1080
There is a scale difference in mov and dng images. I used scale - 99,1%, Position - 967;542 in AE for mov image to fit to dng one.
mlv_dump.exe on 2017Sep03
I believe the H.264 is interpolated from 1904 x about 1072. Does this give a better match?
Recording at this resolution might require only a small offset (a few pixels, if any). Didn't test - it's just what I'd expect to happen.
Thank you! No problem for an exact match, only interesting.
Quote from: Kharak on September 05, 2017, 02:01:16 AM
You are on 123. Are you unable to reproduce the Repeat frames?
Still unable to reproduce. Added a check in mlv_play, to spot the issue without swapping the cards - it triggers on your sample footage, but could not get the warnings on my own footage, even with your settings and usage pattern.
Instead, this check uncovered another bug: when recording was about to stop, there was a speed optimization to save fewer frames at a time, so their buffers could be reused for other frames. The bug was: rather than writing say 7 frames instead of 10, and group the remaining 3 with whatever else will get recorded meanwhile, it was writing all the 10 frames, and then it was writing those 3 frames again (possibly grouped with others). The bug was introduced with variable frame size (required for lossless compression, but also affected uncompressed recordings). It's not present in unified. Didn't benchmark the change, but I expect it to squeeze a few more frames when recording is not continuous.
sorry if this has been covered but i read almost all the posts and i don't remember seeing it. when i use compressed raw recording at 14bit my liveview is normal perfectly fine perfectly usable. when i drop to 12bit it gets darker. then when i switch to 10 bit its so dark that its unusable for framing. i know if i push the half shutter it brightens up like its normal but thats too clumsy for filming. i don't have a 5d mark iii to test it with and see if that camera is doing the same thing. you guys probably already know about this but just putting it out there in case I'm missing something... I reread the first page and i get it just wondering if anything has happened since april 1st on it. also don't think I'm trying to complain or ask for features I'm not just simply trying to talk.
@Teamsleepkid - you probably have Preview set to Real-time. Try the default, Auto. Yeah, it probably looks crappy but it isn't dark!
Sorry for late reply. I am immensely busy these days. I will get back to you on the 113 and september build aswell. Asap!
Hi there,
I tested this on April and 4K half fps + gray preview worked fine for me. Now the the September builds when I enabled Crop Mode it doesnt change anything in the resolution (still 1080 :( ) + no preview at all. Is there some options I don't know about I should enable ?
Thanks !!
Generic troubleshooting steps:
- check for warnings in the menu (grayed out items - these will print the reason why they are disabled)
- check the Modified settings tab
Loving the quick start times of MLV Lite on newest build!!!!
I'm thinking I should I update the video on the main page to explain the "BUSY" bug from half touch or DOF button? I can explain the Q menu work around or the trash button work around. Or I can wait. I just don't want my video to cause confusion.
Quoteto explain the "BUSY" bug from half touch or DOF button?
How do you mean. This is actually fixed by some deep workaround from a1ex in latest build.
Quote from: Danne on September 10, 2017, 05:39:33 PM
How do you mean. This is actually fixed by some deep workaround from a1ex in latest build.
In September 6 2017 build?
Hmmm I downloaded yesterday and on 1.1.3 it does the busy thing unless you hit the Q button twice or hit the trash button twice. I probably missed something but I am on latest build 1.1.3 5d3 sep 6 2017 crop rec 4k build
Shot some 3.5K 12bit Lossless stuff today using September 6th build & running on 1.1.3. So far everything is running smoothly 8)
Quote from: goldenchild9to5 on September 11, 2017, 01:51:37 AM
Shot some 3.5K 12bit Lossless stuff today using September 6th build & running on 1.1.3. So far everything is running smoothly 8)
Still wonder how you folks manage to record 12-bit lossless? My cam says its only for photo mode.. Running 1.1.3. Thanks!Please ignore this post, for first time it now is choosable! Great work! It's has been a problem when choosing in mlv lite menu.
Hi Magic Lantern users,
I'm using a 2 year old ml build.
Everything works perfect, but with the old build I have, it's a little bit annoying that you can't see the right framing when shooting in crop mode - you always see the 5x magnification, but the camera records in 3x magnification...
That's why I'm interested in ths crop_rec experiment build.
I want to shoot in 1080p 25fps 14bit in 3x magnification and see the correct frame (not 5x magnification).
Is this the right build for me?
How do I install over the old magic lantern build?
Do I just need to download the magiclantern-crop_rec_4k.2017Sep06.5D3123.zip and replace it with my old build on the sd card? Then install in the camera?
Sorry for asking, but I'd rather ask before I make something wrong and worst case break my camera :)
hope you can understand!
Thanks so much for your help
First you make a back up of the current build you use in case you like it better.
Then you just overwrite the files on the card with the new one.
All set
ok, it works great so far :)
the only thing is that I'm not able to get higher resolution than 1920x1080, see here:
the
(http://thumb.ibb.co/i6oTUQ/12.jpg) (http://ibb.co/i6oTUQ)
I have enabled only the crop_rec module and the mlv_lite module.
Any clue why I can't enable a higher res than 1080p?
Thanks so much!
Yes you need to first enable the crop mode, and after press the zoom button on the camera to go to 5x zoom.
Then you go back to the menu and choose the resolution.
Mac OS. How do you turn the MLV into DNG's? The referenced "mlv.dump.osx" download does not open on my Mac with a "there is no app. set to open the document..." I tried MLRawviewer 1.4.3 but it crashes.
Quote from: mannfilm on September 14, 2017, 11:41:27 PM
Mac OS. How do you turn the MLV into DNG's? The referenced "mlv.dump.osx" download does not open on my Mac with a "there is no app. set to open the document..." I tried MLRawviewer 1.4.3 but it crashes.
MSVFS? http://www.magiclantern.fm/forum/index.php?topic=13152.0
There are a couple of workflows
you can try
MLVFS
http://www.magiclantern.fm/forum/index.php?topic=13152.0
Switch
https://www.magiclantern.fm/forum/index.php?topic=15108.0
MLV App
https://www.magiclantern.fm/forum/index.php?topic=20025.0
MLVToMovie
https://www.magiclantern.fm/forum/index.php?topic=17893.0
Footage
https://www.magiclantern.fm/forum/index.php?topic=18949.0
hey guys, I know this has been discussed, but I've been through all 46 pages of this thread and tired to figure this out on my own without bothering anyone. I also tried to reach out on IRC about it with no luck. I'm having trouble with the april 4th null pointer bug with the new build. I get the error message every time I install a newer 4k build after the april 4th (which I know is now known to have these bugs). I need someone to walk me through the process to remove the error. I've dumped my rom, I've formatted cards, I've cleared canon settings, I've done the battery pull... I still get the error message without fail. If I install the normal nightly build it works fine, just not the 4k. I've read through the forum several times but it is still unclear to me how to resolve it. I've tried contacting you on IRC like it says to do but have not got a response from anyone on the ML side. I'd love to start testing this new build but my camera won't allow me to do so... If anyone knows what process I have to go through in order to get it up and running again I would be very grateful. Thank you in advance, and sorry, I know plenty of you will think my issue is redundant. Running a 5D3 1.2.3. with latest 4k build.
1 frame half press trouble
Thank you ML devs and testers for the awesome features in the crop_rec exp builds! I have been testing the many recent versions on a telecine machine I have been building and will exhibit next weekend at the NYC Maker Faire. The project uses the crop_rec with half press / 1 frame only externally triggered by a movie projector synced with a reed switch to capture each frame of the film into an mlv file.
First, to get the 1 frame per half press working was confusing to me since the menu suggested that pre-recording had to be active. When I tested with pre-recording active I was capturing many frames per half press. By turning off pre-recording I was capturing only 1 frame per half press. I thought it was all good.
With more testing it seems that the frame that is being recorded has inconsistent timing, sometimes right on time and sometimes later than the half press, but before the next half press, which shows a vertical blur because the projector is advancing the film. I have linked some test captures from a 16mm film shot at the Watkins Glen F1 race NY in 1969 and a couple of projector project photos.
The projector is advanced by a stepper motor and programmed to turn a specific numbers of frames at a fairly precise speed. By changing the stepper controller settings I see the same issue running it at 10FPS (200 frame files), 5FPS (100 frame file), and 2.5FPS (50 frame files) so running slower shows the same results. I have rechecked the reed sw trigger with an LED and it blinks correctly 1 time per each turn. I have also tried many different crop resolutions, bit depths, fps override with very fps as low as 1 second, and with and without compression, the issue seems to be about the same. That said the total number of captures frames is correct to the number of frames the steeper runs before stopping (sometimes the 10FPS captures are 1 or 2 frames short).
Is there a way to update pre-recording to add a buffer setting that only saves the newest single frame rather than a group of frames by seconds? I have been trying to keep up with this thread since April so if I have missed something please let me know.
Thanks for your help!
https://drive.google.com/open?id=0BxP1ckq84DqobzlqMFN6b3JhcjA (https://drive.google.com/open?id=0BxP1ckq84DqobzlqMFN6b3JhcjA)
(http://thumb.ibb.co/nkN5eQ/2017_09_16_DSC6043.jpg) (http://ibb.co/nkN5eQ)
Quote from: Rcorrell1 on September 15, 2017, 01:58:06 AM
I've tried contacting you on IRC like it says to do but have not got a response from anyone on the ML side.
I see, sorry I wasn't online at 1 A.M...
Quote
https://community.linuxmint.com/tutorial/view/12
Disappearing: If you ask a question, wait for an answer. Don't just ask a question and go a few minutes later. Some people are connected all day, but that doesn't mean they're in front of their screen or that they're looking at the channel right now. Leave the IRC open in the background and do something else in the meantime. The more you wait the more likely you'll get an answer.
Quote from: miket on September 17, 2017, 02:44:31 AM
First, to get the 1 frame per half press working was confusing to me since the menu suggested that pre-recording had to be active.
Yeah, not sure how to re-word it. Internally, the 1-frame trigger is implemented as pre-recording with only one frame.
QuoteWith more testing it seems that the frame that is being recorded has inconsistent timing, sometimes right on time and sometimes later than the half press, but before the next half press [...] I see the same issue running it at 10FPS (200 frame files), 5FPS (100 frame file), and 2.5FPS (50 frame files)
I didn't do any timing tests, but this suggests the half-shutter event may be detected too late in the camera (in particular, at low FPS, where other settings are also updating slower than usual). Can I assume the issue is not present at high frame rates?
The half-shutter event is detected right away, but it's processed at next frame (where the pre-recorded frame from memory is queued for saving). Is the delay larger than one frame? (will look at the files tonight)
Quote
Is there a way to update pre-recording to add a buffer setting that only saves the newest single frame rather than a group of frames by seconds?
Unless I'm missing something, the 1-frame trigger already works that way.
A1ex, thanks for taking a look. I neglected to mention that I'm using a 5d3 with 1.1.3.
QuoteCan I assume the issue is not present at high frame rates?
I have not tried to capture faster than 10 frames per second, I'll give it a go and see what happens. The projector would normally run somewhere between 15 to 24 fps when projecting a film. Slower FPS captures may be useful to others who try this project with SD card cameras though.
BTW, what's the reason for setting FPS override to a low value? Do you need long exposures?
If the camera frame rate is a few times higher than the projector frame rate, I don't expect any sync issues (there will be some jitter from the mismatched clocks, but as long as the projector is not advancing, it should be fine). Otherwise, if the two frame rates are close, but not identical, I expect trouble because the clocks will drift; the half-shutter trigger is not genlock (it does not affect LiveView frame timing - it simply saves whatever it finds in buffer).
A closer look at the footage shows the motion blur is, in many cases, at the top of the frame or on the entire frame, but never just at the bottom. The frame is captured from top to bottom, with rolling shutter - that suggests the defective frames were captured too early. The issue is clock drift, from what I can tell - try a higher frame rate setting for LiveView. The output frame rate will be driven by your external clock (so the data rate won't change), but the jitter will be a lot lower when LiveView is sampled at some high frame rate.
As a starting point, try running the projector at 1 FPS, LiveView at 30 FPS and give the sync signal in the middle of the interval where the film is not advancing. Once that is working fine, increase the speed and see how far you can push it, fine-tuning the settings and the timings. If you need more speed, try 1080p45 or 1920x800 60p.
For high speed scanning (with both projector and LiveView running at similar frame rates, doesn't matter if 1, 5, 10, 20, 30, 60 fps or whatever), you need genlock. Or mechanical shutter while the film is advancing.
A1ex, thanks for taking the time to review my files and for these suggestions. I was using the FPS override to get longer exposures. That's with ISO 100 and f16 on the EL Nikkor so I have plenty of light to test with a shorter exposure at higher ISO.
I took a guess on the positioning of the reed switch and cut slots to give adjustment through about 90 degrees of shaft rotation starting after the film stopped moving. Another thing I can try is to move the reed sw 180 degrees so that it triggers just before the film advance. The film advance completes in less than 90 degrees of shaft rotation leaving about 3/4 of a turn with the film stationary.
The original mechanical shutter has been removed for a couple of reasons, first to make room to rework the lamp as the original was 1000 watts. The second reason is the mechanical shutter rotates faster than the frame advance to reduce flicker and exposes each frame 3 times and that would leave less time for exposure with this project.
If you are interested there is a great video by the Engineer Guy - How a Film Projector Works https://www.youtube.com/watch?v=En__V0oEJsU (https://www.youtube.com/watch?v=En__V0oEJsU)
@miket
I am wondering why you are stopping down your lens so much. I assume you are shooting directly at a 90 Degree angle on to the surface of the Film? Do you need the deep focus ?
I am not familiar with Nikkor lenses, though apparently they are amazing. But stopping down to f16, are you not degrading the quality? I think they call it introducing "Diffraction". Something related to the physics of lens elements and the Aperture being to Tiny at f16. From my experience, most lenses reach their peak sharpness between f8 - f11. (unless if its fastest is f8)
Just suggesting and wondering at the same time, I find your project very interesting.
And remember reading another post here from some years ago. Some guy was doing exactly what you are doing. You should try and search for it on the forum, cause it sounds like you both have the same issues with the Sync drifting of, but I can not recall if he managed to resolve his problem or not.
Good luck!
Quote from: a1ex on September 17, 2017, 07:36:20 PM
As a starting point, try running the projector at 1 FPS
A1ex, I tested today with your recommended settings and retiming the reed switch, the 1.25fps and 2.5fps worked with no blurred frames which is great. At 5fps the blur returned. I'm going to rework the trigger magnet, maybe it's holding the half press too long. Based on the record LED the frame gets captured on half press (not release) but the delay on the release may be holding up other functions. I'll report back after this change.
Quote from: Kharak on September 18, 2017, 01:45:37 AM
@miket
I am wondering why you are stopping down your lens so much. ...But stopping down to f16, are you not degrading the quality? I think they call it introducing "Diffraction".
Hello Kharak,
Thanks for your suggestion. That's my understanding too about the diffraction at higher f stops. I'm stopping down and prioritizing the increase to a depth of field since this is macro and the focal depth is only a few of MM. It looks so far like the 5d3 with the 6 element 50mm EL Nikkor is out resolving the detail in the 16mm film.
I plan on posting details about the build here on the forum in the Tutorials and Creative Uses section once it all works for others who may want to build a system. There are a lot of similar projects on the web that I have learned from. The goal of the project is to minimise the hardware modifications needed so more makers will attempt build systems to digitize historical films with very high quality.
Hi Alex
I think this may be a bug. I am using crop rec 4k - Full-resolution LiveView: 5796x3870 at 5fps with some success howeever have encountered an issue which I was able to reproduce twice.
For a video session where I take several short MLVs, the 1st MLV is always coming out at 1920 x 1290 and subsequent MLVs are all 5784x3864, I did 2 tests as shown below ie , my folders WorkArea 26e and WorkArea 27 ( 2 seperate sessions) - see images below. THE ML settings are not changed
TEST 1 - WorkArea 26
(http://thumb.ibb.co/igcSb5/9_18_2017_5_30_37_PM.jpg) (http://ibb.co/igcSb5)
(http://thumb.ibb.co/gbPjOk/9_18_2017_5_23_17_PM.jpg) (http://ibb.co/gbPjOk)
TEST 2 - WorkArea 27
(http://thumb.ibb.co/e1UJik/9_18_2017_5_31_31_PM.jpg) (http://ibb.co/e1UJik)
(http://thumb.ibb.co/gpjUpQ/9_18_2017_4_56_54_PM.jpg) (http://ibb.co/gpjUpQ)
SAMPLE extract of the MLV FILE that is from the 1st folder ie 1920x1290
I used the mlv_dump -f 5 -o to reduce the file size
https://www.dropbox.com/s/107iuqok07zzspy/test.MLV?dl=0
SAMPLE extract of the MLV FILE that is from the subsequent folder ie 5784x3864
I used the mlv_dump -f 5 -o to reduce the file size
https://www.dropbox.com/s/xrt4bx4ywtkfe5e/working137.MLV?dl=0
cheers
Denis
Quote from: denishogan on September 18, 2017, 12:00:23 PM
For a video session where I take several short MLVs, the 1st MLV is always coming out at 1920 x 1290 and subsequent MLVs are all 5784x3864
Take a look at the crop_rec indicator on the bottom left, on the info bar. If there's a warning sign right after enabling crop_rec, you may have to give it a few seconds to refresh the display. The video mode parameters can only be changed before entering LiveView; ML does that (exits and re-enters LiveView) shortly after you close ML menu. If you start recording right away, it will not be able to refresh the display and will record with the current settings (most likely 1080p).
Thanks Alex, I will test tomorrow and ensure I wait a few seconds
cheers
Denis
Quote from: miket on September 18, 2017, 02:28:19 AM
Based on the record LED the frame gets captured on half press (not release) but the delay on the release may be holding up other functions.
That's correct - the buffered is queued for saving to card shortly after the half-shutter press (at the next LiveView frame). If you try this at a very low frame rate, you will see the delay (uniform distribution between 0 and 1/FPS). What actually gets saved, is the frame that was already captured (in memory) at the moment of queuing (so you can usually expect some part of the frame to be captured before the trigger signal). Didn't actually test this behavior; it's just what I'd expect to happen. Such a test can be easily done with some ambient light activated at the same time as the half-shutter trigger.
I don't expect any side effects from the half-shutter release delay - this event is not even handled by ML in this trigger mode. Canon firmware will handle it in background (e.g. for exposure metering or whatever else they do on half-shutter press).
So, whatever the "duty cycle" of the trigger signal may be, as long as it's detected, I expect the results to be identical - the only important factor being when half-shutter state changes from "not pressed" to "pressed" (and the phase difference between that event and the LiveView image capture clock).
On my 5d3 with 1.1.3 the 3K crop mode only goes to 3072x1504. I am trying to get to 1920 verticle. I can get to 3075x1920 in 3K crop mode once but after making a few recordings and shutting down to remove the CF card the verticle res is lower and won't go any higher in raw video menu. What am I forgetting to check? I have live view at 1920 30p all I frames (or IPB). Leaving menus, leaving live view and returning or shutting down and powering back up don't help. Thanks!
I've been doing some lossless compression full resolution lossless compression testing on the 700D and EOSM. Looks like this has been a problem for a long time and I didn't notice it until recently. This might also have something to do with video recording stopping prematurely with lossless compression on these cameras. I was able to revive one of my closed branches (https://bitbucket.org/daniel_fort/magic-lantern/commits/1df43b139bc1e78b9f5adab8d9cc90e0e46fa47d) where this was working but am having a tough time merging it with the current crop_rec_4k branch.
Another issue that came up was that the EOSM raw buffer seems to have shifted 8 pixels on the X axis in this branch so the focus pixels are showing up again in the various MLV processing apps. Need to do more testing to see what settings are affected and what caused the shift.
Lossless 14/12bit movie shooting seems to work here both crop_rec and regular mv720(no premature stops). Did you set FPS override to 24fps?
Didn´t manage to save into a lossless fullres DNG but saves to DNG if not lossless.
I tested on an eosm.
@dfort: if the linked changeset is working fine, try this to compare it to current crop_rec_4k:
hg diff -r 1df43b139bc -r crop_rec_4k > changes.patch
but there are a lot of changes. It may be helpful to test the current crop_rec_4k with lossless.c from 1df43b139bc:
hg up crop_rec_4k -C
hg revert -r 1df43b139bc modules/silent/lossless.c
If that works, the change we are looking for is in lossless.c; otherwise, it's somewhere else.
It may be helpful to try "hg bisect", considering the changes made on top of crop_rec_4k (1df43b139bc vs 6591b1c69642, or even a31a03714911 vs b98026c9978f) - at least a31a03714911 should be working (http://www.magiclantern.fm/forum/index.php?topic=18443.msg188650#msg188650):
hg diff -r 6591b1c69642 -r 1df43b139bc > 700d-eosm.patch
hg diff -r b98026c9978f -r a31a03714911 > 700d.patch
However, these patches won't apply cleanly to future revisions of crop_rec_4k, so it's best to see whether FRSP lossless compression was working in vanilla crop_rec_4k, immediately after merging (ebf206a3a2d1).
hg diff -r ebf206a3a2d1 -r 1df43b139bc
This shows minor differences in lossless.c (except for a missing decompress_init, which shouldn't matter), so I expect it to work. If so, narrowing down will be as easy as running "hg bisect" on crop_rec_4k:
hg bisect --reset
hg up ebf206a3a2d1 -C
hg bisect --good
hg up crop_rec_4k -C
hg bisect --bad
Quote from: a1ex on September 20, 2017, 07:19:56 PM
hg diff -r ebf206a3a2d1 -r 1df43b139bc
This shows minor differences in lossless.c (except for a missing decompress_init, which shouldn't matter), so I expect it to work.
Nope--it doesn't work. That's one of the early revisions that had the scary null pointer warning but it doesn't save a lossless compressed FRSP DNG on either the EOSM or 700D.
On the EOSM it does flash a warning. I had to shoot a video with my phone to read it:
(https://farm5.staticflickr.com/4406/37164900416_e77b9c3d81_z.jpg) (https://flic.kr/p/YC8Q67)
Maybe that's a clue?
Quote from: a1ex on September 20, 2017, 07:19:56 PM
If so, narrowing down will be as easy as running "hg bisect" on crop_rec_4k:
I had to read up on "hg bisect" and it looked like something I was thinking about doing manually--start with a working version and step through the revisions until it breaks. Unfortunately the only revision that works with both the 700D and EOSM is 1df43b139bc so if I try this:
hg bisect --reset
hg up 1df43b139bc -C
hg bisect --good
hg up crop_rec_4k -C
hg bisect --bad
abort: starting revisions are not directly related
I'm pretty sure lossless compression for EOSM/700D was done by cherry picking bits from ErwinH and me. At least I can't find a formal pull request. Back when this was going on I was looking at only video files on the crop_rec_4k branch. I didn't even know it was supposed to be able to do lossless compression with the silent module.
So the next thing I'm going to try is to make small changes on the working version until it breaks.
In this case, I'd try something like this:
hg bisect --reset
# take Erwin's compression changes only (less chances for conflicts)
hg diff -r b98026c9978f -r 5b6936ed7aab > 700d-compress.patch
# crop_rec_4k revision Erwin started from
hg up b98026c9978f -C
# make sure the patch applies cleanly
patch -p1 < 700d-compress.patch
# compile and test again, just to be sure it works
# the result should be identical to 5b6936ed7aab -
# Erwin's version before decompression and 5D3 patch
hg bisect --good
# crop_rec_4k after merging (not working)
hg up ebf206a3a2d1 -C
hg bisect --bad
then, at every step, before compiling, apply the changes from 700d-compress.patch:
patch -p1 -F 100 < 700d-compress.patch
then compile and test as usual,
then, undo the patch before answering whether the build was working or not:
hg revert --all; hg bisect --good # or hg bisect --bad
and re-apply it again at the next iteration.
Thanks a1ex, that's a lot more sophisticated than my kindergarten method of adding one file at a time to the branch that works. Though I have managed to isolate it to something, I don't know what yet, in silent.c.
[EDIT] Getting closer but not worth bringing too much attention to it because I didn't follow instructions--yet. I started with the working version of silent.c and copied pieces of the current silent.c until I narrowed it down to the save_lossless_dng function. Yeah, that makes sense. Now to figure out which commit broke FRSP lossless compression on the EOSM/700D. Probably best if I go back and re-read the previous post.
Continuing on the quest to find what's going on with FRSP lossless compression on the 700D/EOSM. Knowing that the problem is with the save_lossless_dng function in modules/silent/silent.c, there's what I did:
# Made a copy of my repository using the Mac Finder
# Yeah--I know, bad form. Got a space in the path name but hey.
# Why did I make a copy? You'll see in a minute.
cd magic-lantern\ copy/
hg bisect --reset
hg diff -r b98026c9978f -r 5b6936ed7aab > 700d-compress.patch
hg up b98026c9978f -C
217 files updated, 0 files merged, 381 files removed, 0 files unresolved
# Apply patch
patch -p1 < 700d-compress.patch
patching file modules/silent/lossless.c
patching file src/boot-hack.c
patching file src/debug.c
patching file src/raw.c
patching file src/tasks.c
# Looks like Erwin didn't touch modules/silent/silent.c so I'll be copying
# silent.c to my "real" repository -- now it should be clear Why
# I made a copy. I wanted to isolate just the changes in silent.c
# This also allowed me to stay on 700D.115 for these tests.
hg bisect --good
hg up ebf206a3a2d1 -C
130 files updated, 0 files merged, 7 files removed, 0 files unresolved
hg bisect --bad
Testing changeset 16504:8040bdc22e2f (24 changesets remaining, ~4 tests)
89 files updated, 0 files merged, 12 files removed, 0 files unresolved
# At first I was applying the patch and reverting but then I realized that
# since there were no changes to silent.c in the patch this had no effect
# so I stopped doing that. Cutting to the chase:
...
The first bad revision is:
changeset: 16504:8040bdc22e2f
branch: crop_rec_4k
user: alex@thinkpad
date: Sat Aug 19 01:18:35 2017 +0300
summary: exmem: do not keep track of full shoot memory allocations
Looking up that commit, it didn't touch silent.c. However, what I did discover is:
This works--lossless compressed DNG on FRSP:
static int save_lossless_dng(char * filename, struct raw_info * raw_info)
{
struct raw_info out_raw_info = *raw_info;
ASSERT(out_raw_info.bits_per_pixel == 14);
/* compress the image in-place */
/* skip the top bar (that way, we'll be able to avoid race conditions) */
int dy = out_raw_info.active_area.y1;
int dm = dy * out_raw_info.pitch;
void * output_buffer = out_raw_info.buffer; /* output buffer = real buffer */
out_raw_info.buffer += dm; /* input buffer = real buffer + top bar size */
out_raw_info.frame_size -= dm;
out_raw_info.height -= dy;
out_raw_info.active_area.y1 -= dy;
out_raw_info.active_area.y2 -= dy;
struct memSuite * out_suite = CreateMemorySuite(output_buffer, raw_info->frame_size & ~0xFFF, 0);
out_raw_info.frame_size = lossless_compress_raw(&out_raw_info, out_suite);
ASSERT(out_raw_info.frame_size < raw_info->frame_size);
out_raw_info.buffer = output_buffer;
DeleteMemorySuite(out_suite);
int ok = save_dng(filename, &out_raw_info);
if (!ok) bmp_printf( FONT_MED, 0, 83, "DNG save error (card full?)");
return ok;
}
But what is in the latest version of silent.c doesn't work:
static int save_lossless_dng(char * filename, struct raw_info * raw_info)
{
struct raw_info out_raw_info = *raw_info;
ASSERT(out_raw_info.bits_per_pixel == 14);
/* fixme: not all models are able to allocate such a large contiguous chunk */
int max_compressed_size = ((uint64_t) raw_info->frame_size * 80 / 100) & ~0xFFF;
struct memSuite * out_suite = shoot_malloc_suite_contig(max_compressed_size);
if (!out_suite)
{
bmp_printf( FONT_MED, 0, 83, "Out of memory");
return 0;
}
ASSERT(out_suite->size == max_compressed_size);
out_raw_info.frame_size = lossless_compress_raw(&out_raw_info, out_suite);
if (out_raw_info.frame_size > out_suite->size)
{
bmp_printf( FONT_MED, 0, 83, "Warning: output truncated (%s)", format_memory_size(out_suite->size));
out_raw_info.frame_size = out_suite->size;
}
out_raw_info.buffer = GetMemoryAddressOfMemoryChunk(GetFirstChunkFromSuite(out_suite));
int ok = save_dng(filename, &out_raw_info);
if (!ok) bmp_printf( FONT_MED, 0, 83, "DNG save error (card full?)");
shoot_free_suite(out_suite);
return ok;
}
Still trying to narrow this down even further.
Just to double-check: does it execute this line?
bmp_printf( FONT_MED, 0, 83, "Out of memory");
(if in doubt, add a beep or some LED blinks)
I don't get the "Out of memory" message but do get the "DNG save error (card full?)" message.
Note that with the working version of the save_lossless_dng function I get a "RAW error, falling back to YUV overlays" but the compressed DNG is saved.
I know, I was asking whether that line of code gets executed, not how the visible feedback looks like.
if (!out_suite)
{
beep();
bmp_printf( FONT_MED, 0, 83, "Out of memory");
return 0;
}
No beep no message no lossless compressed DNG saved.
Found out that if I back up all the way to the silent.c in a9c8ad4 (https://bitbucket.org/hudson/magic-lantern/commits/a9c8ad412af5c282bbad6889ce5f65d974ca3ada) and keeping everything else at the lastest in crop_rec_4k the FRSP lossless compression DNG issue with EOSM/700D is not present. Updating to commit f6d95f2 (https://bitbucket.org/hudson/magic-lantern/commits/f6d95f2d7b5ab59fb4b003ae128a5293921a25ee) the issue shows up. The commit message is:
Quotesilent: do not compress DNGs in place
(fixes various issues, but memory management is more difficult)
Not sure if I've got my terms correct, changeset, commit, revision, whatever but these are the changes in the save_lossless_dng function. Maybe there's just one little thing in there that should be tweaked?
static int save_lossless_dng(char * filename, struct raw_info * raw_info)
{
struct raw_info out_raw_info = *raw_info;
ASSERT(out_raw_info.bits_per_pixel == 14);
- /* compress the image in-place */
+ /* fixme: not all models are able to allocate such a large contiguous chunk */
+ int max_compressed_size = ((uint64_t) raw_info->frame_size * 80 / 100) & ~0xFFF;
+ struct memSuite * out_suite = shoot_malloc_suite_contig(max_compressed_size);
+
+ if (!out_suite)
+ {
+ bmp_printf( FONT_MED, 0, 83, "Out of memory");
+ return 0;
+ }
- /* skip the top bar (that way, we'll be able to avoid race conditions) */
- int dy = out_raw_info.active_area.y1;
- int dm = dy * out_raw_info.pitch;
- void * output_buffer = out_raw_info.buffer; /* output buffer = real buffer */
- out_raw_info.buffer += dm; /* input buffer = real buffer + top bar size */
- out_raw_info.frame_size -= dm;
- out_raw_info.height -= dy;
- out_raw_info.active_area.y1 -= dy;
- out_raw_info.active_area.y2 -= dy;
+ ASSERT(out_suite->size == max_compressed_size);
+
+ out_raw_info.frame_size = lossless_compress_raw(&out_raw_info, out_suite);
- struct memSuite * out_suite = CreateMemorySuite(output_buffer, raw_info->frame_size & ~0xFFF, 0);
- out_raw_info.frame_size = lossless_compress_raw(&out_raw_info, out_suite);
- ASSERT(out_raw_info.frame_size < raw_info->frame_size);
- out_raw_info.buffer = output_buffer;
- DeleteMemorySuite(out_suite);
+ if (out_raw_info.frame_size > out_suite->size)
+ {
+ bmp_printf( FONT_MED, 0, 83, "Warning: output truncated (%s)", format_memory_size(out_suite->size));
+ out_raw_info.frame_size = out_suite->size;
+ }
+
+ out_raw_info.buffer = GetMemoryAddressOfMemoryChunk(GetFirstChunkFromSuite(out_suite));
int ok = save_dng(filename, &out_raw_info);
if (!ok) bmp_printf( FONT_MED, 0, 83, "DNG save error (card full?)");
+ shoot_free_suite(out_suite);
+
return ok;
}
Quote from: miket on September 17, 2017, 02:44:31 AM
(http://thumb.ibb.co/nkN5eQ/2017_09_16_DSC6043.jpg) (http://ibb.co/nkN5eQ)
holy moly.... the first thing i can think of - steampunk :)
cool!
Quote from: miket on September 17, 2017, 02:44:31 AM
1 frame half press trouble
Thank you ML devs and testers for the awesome features in the crop_rec exp builds! I have been testing the many recent versions on a telecine machine I have been building and will exhibit next weekend at the NYC Maker Faire. The project uses the crop_rec with half press / 1 frame only externally triggered by a movie projector synced with a reed switch to capture each frame of the film into an mlv file.
could you possibly genlock your steampunk device's motor to the camera?
so the camera's card LED blinks a msec whenever a MLV frame was captured, signalling the stepper should advance one frame?
then the projector runs as fast as the camera is recording...
Found a fix for EOSM/700D to get lossless compression working on full resolution silent pictures. Posted it in the Full-resoution silent pictures topic (http://www.magiclantern.fm/forum/index.php?topic=12523.0) so instead of double posting, I'll just quote myself:
Quote from: dfort on September 23, 2017, 08:50:25 PM
Did a very hackish fix to get lossless compressed Full Resoution Silent Pictures DNG files working with the EOSM and 700D without breaking it on the 5D3. Put up test builds on my bitbucket download page (https://bitbucket.org/daniel_fort/magic-lantern/downloads/) and submitted a pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/858/fix-lossless-compression-on-frsp-for-eosm/diff).
Not the ideal solution, basically it just uses the newer version of save_lossless_dng for the 5D3 and the older version of that function for all other cameras. The "RAW error, falling back to YUV overlays" warning has returned to the EOSM and 700D but the current version of silent.mo doesn't save the DNGs and gives no warning on these cameras.
The movie projector scanner that @miket posted (http://www.magiclantern.fm/forum/index.php?topic=19300.msg190023#msg190023) reminds me of a similar project from a user who was trying to adapt a EOSM for his film scanner (https://www.magiclantern.fm/forum/index.php?topic=19005.0) to do full resolution (>4K) scans. It was challenging on that camera because after shooting a full resolution silent picture (FRSP) the LiveView would blank out and the EOSM does nearly everything through LiveView including triggering the shutter via an audio signal (Audio Remote Shot). The solution was to make a special build that put the camera into LiveView after every FRSP exposure. I preferred returning to LiveView rather than leaving LiveView paused but there's probably a good reason why a1ex paused LiveView so I also made it a menu option. It still needs some more work but maybe it is worth resurrecting?
A new menu item can change the way FRSP behaves, the default is to pause LiveView as it currently does:
(https://farm5.staticflickr.com/4332/37244299382_4146bf561c.jpg) (https://flic.kr/p/YK9LCN)
or you can choose the option to return to LiveView after each exposure:
(https://farm5.staticflickr.com/4417/37273528381_e21a2818ba.jpg) (https://flic.kr/p/YMJzp4)
If you're in a mode other than Full-res, the menu item is greyed out and the tool tip shows that the option is only for full-res pictures.
(https://farm5.staticflickr.com/4365/37244297402_a15b5bba4e.jpg) (https://flic.kr/p/YK9L3E)
I took another look at it and it is working in the crop_rec_4k branch. I believe where we left off was to hide instead of grey out the menu item, like the Slit-Scan Mode options and there is an annoying beep every time a shot is saved that is probably telling me that I'm doing something that I shouldn't be doing by forcing the camera into LiveView.
Quote from: a1ex on September 17, 2017, 07:36:20 PM
If the camera frame rate is a few times higher than the projector frame rate, I don't expect any sync issues
@A1ex
After reworking the magnet glued to the projector gear and many changes to the reed switch orientation and timing, along to changing crop mode to 3k, it finally all works! I ran 200 frame demos all weekend at the NYC World Maker Faire at the New York Hall of Science this weekend and I didn't have a single blurred frame. That was running the projector at 10 FPS (about 600 shaft RPM), 3072 x 1728 with 14 bit lossless.
My cousin and I were presenting 2 projects as "DIY Vintage Film Restoration with a Pi or DSLR" which was a hit with both the attendees and the staff. Joe and I are now sharing 3 editors choice blue ribbons. Thanks for Magic Lantern! I will be writing up the details to post in the forum soon.
(http://thumb.ibb.co/ga8yb5/8ygwc1_Wutt4_Y_S8_Jdcio_CPC4_S6_Eu_Ius5e_Qsr_KRg_Ce_Fgc_MZ7y_ITu_Bl_A_w1200_h630_p.jpg) (http://ibb.co/ga8yb5)
(http://thumb.ibb.co/dcy9ik/qqefyt_RG2oo_BQLOf_Inb2_JQm_Zf_Vy7d_Z_WBi9_KKCb_si_VAg_DFNll_I9_Q_w1200_h630_p.jpg) (http://ibb.co/dcy9ik)
@miket:
if you get some problems with speed or reliability - as i wrote before - we can modify mlv_lite to use the card LED to genlock your projector to the MLV frame rate.
Congratulations for your achievements! Learned about projector Ampro Stylist (built around 1952?) using bakelite for some parts. Just out of curiosity: Are all those bakelite parts original or do you had to replace some parts using 3D-printing (or else)?
@g3gg0: Yes, it resembles steampunk but it is doing something and actually is producing something. A projector about 65 years old + Arduino + Magic Lantern. Worlds collide and making an impression!
@g3gg0 and @Walter Schulz, thanks for the kind words! I don't think I could easily genlock the projector with a signal from the camera due to the mechanics of it - for example startup and stopping needing motor acceleration and decel, the Arduino is running the AccelStepper library to handle this currently. But the overall goal is simplicity as a first priority.
Primarily I'm building this to digitize my family films, but the secondary goal is to make the build easy enough so that others will adapt the ideas to fit other projectors and massively digitize the worlds family and historical films. The barrier to having film collection digitized/scanned is often the high price. If a DIY machine based on Magic Lantern can be(relatively)easily made and have a high-quality output then I think more people will give it a try.
To that end, the recording this weekend were working at 10FPS which is more than normal projection half speed. If it can get to 16 to 18FPS or so then the stepper motor would not be needed, eliminating some more of the complexity. Testing higher speeds is on the to-do list.
I don't know the age of my projector, I picked it up used from craigslist but early 50's sounds right. The operation manual first edition date was 1949. All of my bakelite is in great shape and most of the main bakelite and metal parts look identical to the older Ampro Imperial used in the pi project. My Super Stylist has Sound on Film built in. I never turned on the vacuum tube amp though, none of our films have sound. The machine looks cool and sounds great when it's running too.
Quote from: miket on September 26, 2017, 08:32:34 PM
...Primarily I'm building this to digitize my family films, but the secondary goal is to make the build easy enough so that others will adapt the ideas to fit other projectors and massively digitize the worlds family and historical films. The barrier to having film collection digitized/scanned is often the high price. If a DIY machine based on Magic Lantern can be(relatively)easily made and have a high-quality output then I think more people will give it a try.
Oh I'm with you on this one @miket!
Which of the builds is the most stable to use on 5d3 firmware 1.1.3?
I am mostly interested in shooting at 24p in 16:9 and 2.20:1 with sound with the highest resolution possible, therefore mostly continuous.
Can I use MLV producer for converting these footages to cDNGs?
@Ibrahim
For me (and this is a personal opinion based on my own testing) is crop_rec_4k.2017Sep06.5D3113
I tested all versions, included the last one (Sep27).
I'm using it in producction, but it has a risk that I asume... Only using it in raw HD with proxy recording for the sound (never 3k) and in 3x3_50p for slowmotion.
pc-bel...how you set exactly for have a 50P ? with every release , all my test on 1920 50p (raw) was failed....
5d3 + 123
problem was always..few seconds and stop recorders...no possibile to have stable/continued recording...
@jimiz
You need one of the latest builds from crop rec 4k. In Canon menu (not ML menu) choose 1280 50p All-I. Then in ML menu choose the movie submenu and activate Crop mode, choosing 1920 50/60 3x3. Exit menu and that's all. You can record in 1920x960 50p (2:1 aspect ratio or less). You will need a fast CF card (1066x). The live view preview is not ok. You can't see the inferior part of the frame, but it's easy to handle...
"mlv_lite has lossless compression and H.264 proxy, mlv_rec has uncompressed 14/12/10-bit video and native sound recording"
Activating "mlv_lite" on My SL1 running "crop_rec_4k.2017Sep27" causes all Modules to "err" ~
Using "mlv_rec / Raw video (MLV)" seems to work fine & allows recording ~ Have not Processed yet ~
Trying to run "crop_rec_4k.2017Sep28" makes My SL1 Very Unhappy ~
ORR ~ DeanB
hi guys,
need some help, please,
i am using april experimental build
1.
can not record in 48p 50p or 60[...recording stops after less than second with error, asking me to rebot camera and decrease vertical resiolution
I am not sure what i am doing wrong
-in canon manu is 720 all
-12 bit lossless
-fps override 48, or 50, or 60 fps
-low light preview
2. How to get 3072x1728 or more vertical resolution?....or 3072x1920....or 3840x1536? I can get max 3584x1320
and
3. what is most stable of latest experimental builds, that you can recommended? I know that, there is no more"stable" builds, but some of latest builds are more stable then the others....thats what i meant.
Thanks a lot,
and thank to magic lantern team for everything they done
Quote from: banertop on September 30, 2017, 12:37:14 AM
3. what is most stable of latest experimental builds, that you can recommended?
The newest experimental build is the one you should use. Did you notice that older experimental builds are not available for download? If you want stability, go with a nightly but then of course you won't be able to do the other tricks. Sorry, I can't help you with the other questions because I haven't tried them myself.
Part of the reason for posting experimental builds is to get feedback from users so here's some feedback.
Up until a month ago mlv_rec wasn't working (http://www.magiclantern.fm/forum/index.php?topic=19300.msg189162#msg189162) on the crop_rec_4k branch. All the cool 4k, lossless compression features only work in mlv_lite so there probably hasn't been much feedback on how well mlv_rec is doing here. There are problems. One worth noting is that in Movie crop mode (the 600D crop-mode not the crop_rec module) it isn't starting, at least not on the EOSM and 700D.
(https://farm5.staticflickr.com/4440/36750994123_bf5e9cbdab.jpg) (https://flic.kr/p/XZysaH)
There is a message in the debug menu.
700D
(https://farm5.staticflickr.com/4451/37421390401_776ae8cdce.jpg) (https://flic.kr/p/Z1NpCe)
EOSM
(https://farm5.staticflickr.com/4421/36750993943_3f8f2aaf4a.jpg) (https://flic.kr/p/XZys7B)
Can't back test this because it has been this way since mlv_rec has been working in this branch--and it does seem to work fine in other modes, just Movie crop mode is a problem.
This probably affects other cameras with the Move crop mode feature. I've heard reports that this isn't affecting the 100D, maybe a 100D user could verify?
Wasn't working on 100D last I tried.
to dfort
thank you for your answer
I use april 29 build right now, so i will try with newest build.
But still dont understand why 48p, 50p and 60p dont work for me and why i can not find the way to increase vertical resolution to 3072x1728 or 3072x1920?
i am shooting with 5d iii, 1.1.3....using lexar 128gb 1066x
thanks
Quote from: banertop on October 01, 2017, 04:39:34 PM
But still dont understand why 48p, 50p and 60p dont work for me and why i can not find the way to increase vertical resolution to 3072x1728 or 3072x1920?
@banertop
The crop rec resolutions are tied to the Canon menu Movie rec. size settings. Once you have the latest exp build installed activate 3 modules, crop_rev, mlv_lite, and mlv_play. and restart the camera. Turn on live view and open the canon menu (menu button), in menu shoot 4 - select Movie rec. size and press set - select 1920 24 (either ALL-I or IPB). Press set again and exit the menu, might be a good idea to exit and return to live view.
Then from the Magic Lantern menu movie menu select crop mode and press set, select 3K 1:1 and press set, select RAW video and press Q, you should be at 3072x1728. If you don't see it then try another exit and return to from live view. I think this mode is 24 FPS only
I'm also running 5d3 on 1.1.3 with the 128GB 1066 Lexar and I don't know how to get to 3072x1920. I'm still on magiclantern-crop_rec_4k.2017Sep06.5D3113 but will be upgrading this week to try out the latest build.
to miket
i will try all that, with latest september build......
Thank you very much for your time and detailed answer
What did you find more usable, and best for your type of work, 3.5k with 1320 vertical resolution, or 3k with 1728 vertical res, witch is more close to 16:9?
Quote from: banertop on October 01, 2017, 08:01:35 PM
What did you find more usable, and best for your type of work, 3.5k with 1320 vertical resolution, or 3k with 1728 vertical res, which is more close to 16:9?
@banertop
Update on 3072x1920, in 3K 1:1 crop mode then RAW video and press Q, if I change the aspect ratio to 1.2:1 or 3:2 (maybe other taller ratios) I can then use the joystick RH to increase resolution all the way to 3072x1920 and that holds through exit and reopen live view and camera off and on. Thats a lot of data so you may only get a few seconds of great looking recording before the buffers are full and recording auto stops, there is an estimation on the bottom of the ML movie menu.
I'm using the crop mode with 1 frame half press trigger for a home movie digitization project so I haven't really used the regular video mode very much. If you loaded the MLV play module you can play the files in camera for a quick check. After you have stopped the recording open the ML menu go to Movie then RAW video and press Q. Scroll down to playback where your MLV filename should show up, and press set to start playback.
very informative....
i have been using regular magic lantern raw 1080p for a while, but experimental build i s a bit tricky....now it is more clear
thank you very much, again
May be it can be useful for the developement.
Using now the latest build (october02) on my 5D3 1.1.3.
Today I got an error (don't remember if it was error 77, I was in the middle of a shooting) and needed to pull the battery.
This is the crash log:
ASSERT: FALSE
at RscMgr.c:894, MovieRecorder:ff0efaf4
lv:1 mode:3
MovieRecorder stack: 1716f0 [171850-16f850]
0xUNKNOWN @ de48:171848
0xUNKNOWN @ 17bbc:171820
0x000178B4 @ ff0ebd58:171808
0xUNKNOWN @ 178e4:1717f8
0xUNKNOWN @ 1796c:1717d8
0xFF1ED5B4 @ ff1d8444:171770
0xFF13C210 @ ff1ed600:171760
0x00001900 @ ff0efaf0:171728
0x00069868 @ 69c1c:1716f0
Magic Lantern version : crop_rec_4k.2017Oct02.5D3113
Mercurial changeset : 96b44eb365d4 (crop_rec_4k) tip
Built on 2017-10-02 13:23:12 UTC by jenkins@nightly.
Free Memory : 137K + 3800K
Small update: LiveView no longer darkens (http://www.magiclantern.fm/forum/index.php?topic=10111.msg191218#msg191218) when using 8..12-bit lossless compression.
Does this trick work on camera models other than 5D3?
5D Mr3 - The flight is normal!
Magic Lantern Crop Rec 4k.2017Oct07.5D3113
Thx ;D
Small update??? THIS IS A BIG DEAL!!!!! Amazing work ALEXXXX. :)
How cool is this 8)
very cool:) maybe this has been covered to death but any way to push the resolutions farther? I'm guessing that 1792 is the max width for eosm? anyway really cool live view working now.
Quote from: a1ex on October 07, 2017, 01:03:13 AM
Does this trick work on camera models other than 5D3?
Working on 700D and EOSM.
Just wondering--why is it that when putting these cameras in crop_rec 3x3 mode and raw the LiveView refresh rate slows way down in Auto Preview mode yet it seems to work fine when in Real-time Preview mode. I'd rather see an image that is stretched vertically (like it does when recording H.264) rather the slow refresh rate (when recording MLV).
What resolution?
The heuristic (raw_rec_should_preview) picks the slow preview for video modes with resolutions higher than default (where Canon preview doesn't work well), and for aspect ratios taller than 16:9 (to avoid hiding the picture under the info bars).
Works on 100D :).
Been testing crop_rec on my 100D lately to test for focus pixel maps from Bouncyball. I notice a black border at the top lately confusing pixel maps.
(https://s1.postimg.org/89ed11rh9b/Screen_Shot_2017-10-07_at_10.00.26.png)
Was it there from the beginning? (I didn't change any offsets lately)
Take a look at raw_info_update_cbr -> calc_skip_offsets. These values are hardcoded for 5D3 - you may need to change them.
To verify them, use raw_diag's OB zones analysis - should look like this (the blue contour should mark the active area):
(https://farm2.staticflickr.com/1634/23859109724_ae1c9a9fc7_o.png)
From the very start no but I only filmed one file back then. Will check further when I get back home. Thanks for pointers.
Indeed, what you did with this "small update", (Oct. 07 experimental build) is really a BIG thing, A1ex! Screen is no longer dark in the 10/12-bit MLV and RAW recording modes and the pink highlights are gone! This makes the 100D a very usable for practical video shooting camera since now we can see what we record and frame properly during camera moves.
I found one bug though. The RAW_tweak module causes trouble when I try recording with the MLV_rec module. I get the message "Threads failed to start" and that's it. In the RAW_rec module recording works (RAW video is recorded with several 512 byte files) but then, when I try to play it back, I get a black screen and camera crashes. I have to pull the battery out. So, I disabled the raw_tweak module and can now record normally. Sound also works with the MLV_module. There are a few more things that need to be fixed though:
1) Playback of already recorded 10/12-bit MLV and RAW videos activates the counter indicating palyback but produces a black screen, i.e. video does not appear on the screen;
2) I get "hick-up" jumps when recording in 1920x1080, 2496x1080 and 2520x1080 resolutions @5x magnification that produce excellent video quality.
3) In the MLV_rec mode, when I activate the "Moovie crop mode" and press the record button, the BUSY sign appears and recording does not start. I need to turn camera off and on again to be able to record in the Moovie crop mode.
I hope, this bug report is helpful.
Hi alex, tested the new tweaks to liveview, works well, i did notice the image still darkens ever so slightly in lossless 8bit, but still greatly improved, 12, 11, 10bit lossless are great, just thought i'd share my findings, thanks again.
Thanks Alex for this great update.to all @100D users : can you record in 1728*972 continuous ?
@Danne : is this build can support h264 proxy in parallel ? Even if it's very small bitrate like 5mbit/s ?
This is awesome news!
Quote from: 12georgiadis on October 07, 2017, 01:32:28 PM
can you record in 1728*972 continuous ?
Yes, you can - in the crop_rec mode at 24 fps and 8...11 bit lossless compression. I am not sure if you can record continuous at 12-bit loss less too but I did about 1500 frames in that mode and stoped it. For me, the most important thing is that you can record 10-bit MLV with sound for 15 s. in that resolution and in my opinion, this makes the 100D a real game changer.
Thanks for the infos IDA_ML. Of course it's incredible to have 10bits mlv with sound ! But if we have H264 proxy mode (for preview and offline edit) + continuous, making a short indie film with 100D becomes possible !
Proxy recording at the moment seems to crash the camera. It doesn´t seem to work with eosm as well. Havn´t had the time to really check more into it.
I've only been able to do proxy recording on the 5D3. It is wonderful and would be great to have it working on all cameras.
with 12 bit lossless in 1080x1920 5D3 most the time need less then 40MB/sec. can the 100D, 700D, and Eos M, do 1080p continuous? at least in 2:35 or 8-11 bit lossless...
PS
I just saw this
EOSM, 700D, 100D:
1736x688 @ 50/60p with 3x3 pixel binning (actually column binning / line skipping)
Lossless compression: 1736x976 @ 24p continuous (this is the maximum 16:9 resolution with 3x3 sampling)
:-[
Checked in crop_rec but it´s not obvious to me why 100D has out of bounds black bar issue. Well I realise that height goes all the way up to 724(1648x724) and that this is not within the capabilities f the 100D(Seems it should be 1648x1660 instead). Eosm(1648x696) and 700D seems to have different limits set. I tried the raw_diag.mo and although it says that it saves a ppm file in a specific folder I can´t find it anywhere. Couldn´t get a screenshot either so here is a cell phone upload :P. Blue line out of the buffer at the top? Seemes buffer portion to the left, outside blue not used?
(https://s1.postimg.org/9oxtsajd0f/Screen_Shot_2017-10-08_at_20.04.32.png)
Wow, I wonder how it even works...
(with skip offsets so far away from the truth, I would expect either raw backend errors, or at least wrong black level)
Remember DeafEyeJedi's test with white level? You may have to do something similar for raw offsets: run raw_diag's OB zones test for each crop_rec camera (100D, 700D, EOSM, 5D3 for double-checking) in all supported video modes (Canon modes x crop_rec modes). Ideally also with crop_rec turned off, just in case.
Or imagine some automated test (maybe a Lua script) to do this job for you (ideally also validating the borders).
The screenshots should be in the directory mentioned at the bottom (on your card, of course). Tip: convert them to png if you plan to upload them.
Let me start by checking my 100D and eosm. Checked canon mode offsets(non crop_rec) and they seemed fine at first glance. Will go through them more thoroughly.
Sidenote. Only time I get a file in the mentioned folder is when selecting Dump RAW Buffer and i get a scrambled, corrupted DNG file. The Auto Screenshot ppm thingy seems broken.
(https://s1.postimg.org/1e86r04d5r/Screen_Shot_2017-10-08_at_20.34.34.png)
100D test:
All modes without crop_rec seleted seems ok. The blue lines are fitting the buffer perfectly when selecting mv1920, mv720, mv480
Crop_rec on:
Only able to test mv720 mode with the result two posts up. in mv1080 I get raw_detect error since there is a big white bar at the bottom so not even working to test.
Quote from: anto on October 08, 2017, 04:35:38 PM
:-[
Why the sad face?
On the EOSM/700D (and probably 100D now) it is possible to record continuously in 1648x724 lossless with crop_rec. Do you really want 1920x1080 raw in these cameras? As far as I know it can't be done because of the hardware but you can get really, really close. I just tried this on the EOSM and it works:
1920x1072, 24FPS (override), 8-11 bit lossless -- continuous!
How? This is the zoom mode which goes all the way up to 2512x1072 on these cameras. Works the same as on the 5D3, press the magnifying glass button until you get into 5x then check the mlv_lite settings and you'll see the increased resolution.
@Danne: just checked raw_diag on 5D3 (auto screenshot and dump raw buffer enabled): in movie mode it saved valid PPM and scrambled DNG (with valid header) on the CF card, exactly where it said. Fix in progress solved.
In photo mode, the DNG was clean.
If your screenshot saving issue persists in photo mode, I might be able to reproduce it in QEMU. A few cameras (those passing the FA_CaptureTestImage test (https://builds.magiclantern.fm/jenkins/view/QEMU/job/QEMU-tests/lastSuccessfulBuild/console)) are able to the silent picture module in full-res mode, show the captured image, trigger the raw overlays and even save a regular (uncompressed) DNG on the virtual card. The code from raw_diag is even less demanding (emulation-wise).
Test on eosm(ppm works):
(https://s1.postimg.org/15fpscrs4f/Screen_Shot_2017-10-08_at_21.05.17.png)
(https://s1.postimg.org/6ke8as3prj/Screen_Shot_2017-10-08_at_21.05.25.png)
The issue seems to be with mv720. All other modes looks ok. (that is if mv1080p above is ok) 100D can´t work with mv1080p with crop_rec.
OB-zones ppm are being saved on the 100D when in photo mode. The DNG is still scrambled even when saved in photo mode.
Quote from: dfort on October 08, 2017, 08:57:03 PM
...I just tried this on the EOSM and it works:
1920x1072, 24FPS (override), 8-11 bit lossless -- continuous!
1920x1078 on the 700D--oh so close to full HD.
Danne reported that zoom mode on the 100D goes all the way up to 2520x1080 so on that camera it should be possible to do 1920x1080 continuous with 8-11 bit lossless 24fps. Of course it crops the sensor, no audio, yadda yadda but hey--seems like ML has reached the Holy Grail of
1920x1080 HD raw video on the 100D. :D
We now return you to the ongoing discussion of OB zones.
enjoy a 9-bit 14bit DNG from a 100D 2520x1080p 8)
https://bitbucket.org/Dannephoto/magic-lantern/downloads/M08-2236_1_2017-10-08_0001_C0000_000001.dng
Sample MLV:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/M08-2236.MLV
Hm, seems it 14bit lossless. Is the 9-11 bits working on zoom modes?
White level on this one should be 2461 (2462 is too much). Fix pushed.
(your file is 9-bit lossless)
9bit :D thanks for claryfing. (guess white/black level says this)
Than you very much for the great build!
When mlv_rec.mo is loaded, it's impossible to load mlv_play.mo - error.
Is it normal? 5D3.
Would be nice if mlvrec worked with mlv play, but it does not yet. so yea it nothing wrong on your side. I also have the same.
by the way if you unload mlvrec and restart the cam with only mlvplay you can play files recorded in mlvrec. not so convenient but at least you can check the footage
Workaround: also load the silent picture module.
Fix: move lossless.c to a library module (auto-loaded by the backend if other modules require it). Precondition: add support for library modules in ML.
Tested latest build. Over the top?:
Black Level : 2047
White Level : 2459
https://bitbucket.org/Dannephoto/magic-lantern/downloads/M08-2336_1_2017-10-08_0001_C0000_000000.dng
Another thing. When previewing the 9bit file in mlv_play you get the the lowered light with pinks while previewing on the 100D. The files are looking fine when checked on the computer. Previewing 9-12 bit were correctly looking in mlv_play in earlier builds before the static 14bit live view fix.
A safety margin of 0.01 EV shouldn't hurt... I guess.
Didn't check mlv_play - you had correct highlights before this change and they turned pink after lowering the white level?
Thanks for the workaround for mlv_play +mlv_rec !
@A1ex
I had issues before as well but, as I was testing before I think the issues dissapeared. Not entirely sure, was very tired when checking the first time. I think the issues was there from the start on the 100D. Is it acting weird with other cameras?
I have one issue, When I use the config preset startup key (MENU) and I save the config will the crop module selected for load at startup, it does not load the module.
Alex, thanks for the workaround for mlv_rec+silent picture module+ mlv_play. But preview works only for 14bpp. There is black screen for 12,10bpp. Canon 5D3.
Was fooling around recording 2520x1080p 8 seconds takes with my 100D in 24fps. The following issue appears in every single file:
http://www.magiclantern.fm/forum/index.php?topic=19300.msg189633#msg189633
Doubled frames causing a slip here and there in the files. What´s also fishy is that the first frame in the following recorded clip will contain one frame from the preceeding file. Not all files but most when filming 2520x1080p. It´s like it´s still in memory in the camera and wants out. Said and done. What to try next. Disabling srm memory maybe helps but no, the problem persists. I then tried lowering aspect ratio to 3xzoom 1920x1080p and the problem is still present. I also tried non compresses 12/10bit but still the double frame issue. Tried regular 14bit and files comes out very short but without doubled frame but still too short to be reliable for this test.
Here is an example file with a doubled frame:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/3xzoom_100D_Frame_57-59.MLV
The issue can happen anywhere but often after a couple of seconds. Tried changing timers in FPS but didn´t work. Seemed to worsen things at one point but can´t really be sure of that. One theory I have is that maybe the issue appears due to 3xzoom code now is centred? Just guessing here. Can´t really see what´s up here.
I have seen that If I use mlv_lite with h264 the resulting DNG files have a 1 pixel line on the left side. 1080p 25fps Pal ( canon 5d3 1.2.3 )
Is this a known issue?
There is no 1 pixel line on any side, mlv_lite with h264, Canon 5D3 1.1.3. Tested 14-bit lossless and 12-bit lossless, 1920x1080, 24fps
https://www.dropbox.com/s/ineacrodd1x0lb7/M10-1506_000000.dng?dl=0
you can see it in this DNG file, black Line of pixels on the left side
This is two pixels line.
8). true, by the way 14 bit lossles was used. did you not get the same?
I have black lines at left with my 5D MKIII 1.2.3 images too.
Is this related to mlv_dump settings?
Quote from: Lars Steenhoff on October 10, 2017, 06:53:50 PM
8). true, by the way 14 bit lossles was used. did you not get the same?
There are no black pixel lines, mlv_lite with h264, Canon 5D3 1.1.3. Tested 14-bit lossless too.
Not sure what this has to do with H.264 or bit depth, but it's a valid bug - fix pushed. There were two black pixels on the left edge, not just one.
mlv_rec and/or the unified branch may also need this fix (not checked).
Bugs like this are subtle enough to go unnoticed and/or unreported for a long time; as I'm not a regular video user, and not very good at pixel peeping, I wasn't aware of it. After searching for similar stuff, noticed this report from 2013 (http://www.magiclantern.fm/forum/index.php?topic=5247.msg38848#msg38848)...
Thanks for the fix!
Im sure it has nothing to do with h264, I was just trying to report as accurate as possible my settings when I noticed the pixels.
I just double checked and mlv_rec does NOT have this issue.
Hi all- Long-time forum reader, first time commenter.
Wanted to chime in that I also get the two dark pixel column bug. It's been present for me since I started experimenting with crop_rec on steroids back in 2017 June. I was about to summarize my observations and post when I saw it come up again in the thread. Thus far, I've only tested on 5D3 1.2.3.
I went through the thread and found a few other mentions, with some variations:
http://www.magiclantern.fm/forum/index.php?topic=19300.msg184143#msg184143
http://www.magiclantern.fm/forum/index.php?topic=19300.msg184217#msg184217
http://www.magiclantern.fm/forum/index.php?topic=19300.msg185903#msg185903
http://www.magiclantern.fm/forum/index.php?topic=19300.msg186013#msg186013
In my case, I've isolated it to a combination of using one of the lossless recording modes and having a horizontal resolution of exactly 1920 in some, but not all shooting modes. Shooting 14-bit lossless with 1920 pixels wide in 3x3 binned, regular old 1to1 cropped, and 50/60p gets me the two dark (but not black) pixel columns at the left edge. There is also a one pixel column of lighter pixels immediately adjacent. They change a little moving to 12 and 8-11 lossless; an additional slightly darkened area a few pixels wide also appears to the right of the dark bars. If I make the horizontal resolution slightly smaller, or in a crop mode slightly larger, the bug goes away. It does not appear to come up in 3.5K centered shooting mode. Changing vertical resolution or using speed hacks like Global Draw off do not have any effect.
Hope this helps. Let me know if I can provide more information.
someone please...
4096×2160 24p 10bit raw, Continuous please!!!!
always great thanks to ML team.
Quote from: mr.smith on October 12, 2017, 07:01:42 PM
4096×2160 24p 10bit raw, Continuous please!!!!
I want more -- 5184x3456 14bit raw on the EOSM. Come on guys, you can do it! Oh yeah, FRSP already does. Now I want 120p continuous!!!!
Ok--back to reality. I'd like to resurrect the discussion on the OB zones issue with the EOSM, 100D, 700D and probably 650D. This happens when using crop_rec 3x3_1X at maximum possible image size.
EOSM example (black bar on top of frame)
(https://farm5.staticflickr.com/4463/37641138091_1108e496bb_z.jpg) (https://flic.kr/p/ZmdEXv)
I'm at work without my development computer or cameras right now but this looks like something interesting:
/* max resolution for each video mode (trial and error) */
/* it's usually possible to push the numbers a few pixels further,
* at the risk of corrupted frames */
static int max_resolutions[NUM_CROP_PRESETS][5] = {
/* 24p 25p 30p 50p 60p */
[CROP_PRESET_3X_TALL] = { 1920, 1728, 1536, 960, 800 },
[CROP_PRESET_3x3_1X] = { 1290, 1290, 1290, 960, 800 },
[CROP_PRESET_3x3_1X_48p] = { 1290, 1290, 1290, 1080, 1040 }, /* 1080p45/48 */
[CROP_PRESET_3K] = { 1920, 1728, 1504, 760, 680 },
[CROP_PRESET_UHD] = { 1536, 1472, 1120, 640, 540 },
[CROP_PRESET_4K_HFPS] = { 3072, 3072, 2500, 1440, 1200 },
[CROP_PRESET_FULLRES_LV] = { 3870, 3870, 3870, 3870, 3870 },
};
Secifically, the CROP_PRESET_3x3_1X on the EOSM, 100D, 700D, 650D is not 800 vertically in mv720 mode. The full raw buffer isn't even that big. As far as I can tell there's no other place in the code that sets max_resolutions.
On the EOSM there are 28 black pixels at the top of the frame and the image is 724 pixels high. I couldn't find an EOSM mv720 full raw buffer in my flickr library but I have one for the 100D that Danne shared with me.
(https://farm5.staticflickr.com/4487/37257348780_bcb3d65171_z.jpg) (https://flic.kr/p/YLiDLq)
It is 727 pixels high and the out of bounds area takes up 28 pixels off the top (sound familiar?) so the maximum possible vertical resolution would be 699 pixels, maybe slightly less, so that 800 pixel max resolution may be fine for the 5D3 but it doesn't apply to these other cameras.
To change height and width for the 100D this is the place in crop_rec.c:
/* optical black area sizes */
/* not sure how to adjust them from registers, so... hardcode them here */
static inline void FAST calc_skip_offsets(int * p_skip_left, int * p_skip_right, int * p_skip_top, int * p_skip_bottom)
{
/* start from LiveView values */
int skip_left = 146;
int skip_right = 2;
int skip_top = 28;
int skip_bottom = 0;
switch (crop_preset)
{
case CROP_PRESET_FULLRES_LV:
/* photo mode values */
skip_left = 138;
skip_right = 2;
skip_top = 60; /* fixme: this is different, why? */
break;
case CROP_PRESET_3K:
case CROP_PRESET_UHD:
case CROP_PRESET_4K_HFPS:
skip_right = 0; /* required for 3840 - tight fit */
/* fall-through */
case CROP_PRESET_3X_TALL:
skip_top = 30;
break;
case CROP_PRESET_3X:
case CROP_PRESET_1x3:
skip_top = 60;
break;
case CROP_PRESET_3x3_1X:
case CROP_PRESET_3x3_1X_48p:
if (is_720p()) skip_top = 0;
break;
}
if (p_skip_left) *p_skip_left = skip_left;
if (p_skip_right) *p_skip_right = skip_right;
if (p_skip_top) *p_skip_top = skip_top;
if (p_skip_bottom) *p_skip_bottom = skip_bottom;
}
This:
/* start from LiveView values */
int skip_left = 146;
to:
/* start from LiveView values */
int skip_left = 66;
and this:
if (is_720p()) skip_top = 0;
to:
if (is_720p()) skip_top = 28;
Will result in 1728x698 crop_rec recordings.
Strange.
case CROP_PRESET_3x3_1X:
case CROP_PRESET_3x3_1X_48p:
if (is_720p()) skip_top = 0;
EOSM/100D/700D/650D are using only 3x3_1X so changing that line to if (is_720p()) skip_top = 28; should have no effect.
So did that get rid of the black bar?
Yes.
Well if it's x724 and you put in -28 when otherwise 0 when set to mv720 something should happen...
Could you try your 700D?
The line should apply for both?
CROP_PRESET_3x3_1X
And
CROP_PRESET_3x3_1X_48p
?
Tried your suggestion on the 700D and also a simpler suggestion a1ex made on the 650D pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/872/crop_rec_4k-for-650d104/diff#comment-47074957) and got the same results. Basically, don't run this:
/* update skip offsets */
int skip_left, skip_right, skip_top, skip_bottom;
calc_skip_offsets(&skip_left, &skip_right, &skip_top, &skip_bottom);
raw_set_geometry(raw_info.width, raw_info.height, skip_left, skip_right, skip_top, skip_bottom);
I put the EOSM/100D/650D/700D into a group called "is_basic" because they only do the basic 3x3_1X thing. This will make it easy to test for this group and skip that section.
So here is the before:
700D crop_rec 1648x724
(https://farm5.staticflickr.com/4472/36996183353_a358b77e91_z.jpg) (https://flic.kr/p/Yne7oe)
And after -- not running that section of code:
700D crop_rec 1736x688
(https://farm5.staticflickr.com/4462/36996183753_c8437b108a_z.jpg) (https://flic.kr/p/Yne7v8)
Something interesting happened to the LiveView with this change. Before Auto Preview defaulted to Framing but not calling calc_skip_offsets/raw_set_geometry changed Auto Preview to Real-time which for crop_rec is stretched vertically. Either way works fine.
By the way, I've been using Danne's Switch app to process the MLVs and at one point I tried mlv_dump on the command line and Adobe Camera Raw stretched out the image--guess it was adjusting it as if it were a regular mv720 clip. However, this is also how it looks like on the LiveView.
(https://farm5.staticflickr.com/4512/36996520453_10236a18aa_z.jpg) (https://flic.kr/p/YnfQAi)
Quote from: dfort on October 13, 2017, 08:37:24 AM
By the way, I've been using Danne's Switch app to process the MLVs and at one point I tried mlv_dump on the command line and Adobe Camera Raw stretched out the image--guess it was adjusting it as if it were a regular mv720 clip. However, this is also how it looks like on the LiveView.
Pretty sure it does that. Suggestion: if the MLV has RAWC metadata, this can tell whether the image requires any stretching (and if so, how much). The aspect ratio would be (binning_x + skipping_x) / (binning_y + skipping_y).
Yes, should follow metadata. It's not atm. Mlv_dump version in Switch adds default scale tag corrections first automatically then corrects it if told it's processing crop_rec.
Regular(unified) mlv_dump doesn't touch the default scale tag.
Another ML mystery solved. Here is the RAWC block from the shot that I posted.
Block: RAWC
Offset: 0x000000e8
Number: 2
Size: 32
Time: 0.815000 ms
raw_capture_info:
sensor res 5184x3456
sensor crop 1.62 (APS-C)
sampling 3x3 (read 1 line, skip 2, bin 3 columns)
I really want to resolve that black bar at the top issue and even though skipping over calc_skip_offsets/raw_set_geometry does seem to do it, there is an issue with the way ACR is displaying the processed DNG files depending on the app used to process the MLV files.
Let's call the code as it currently stands as "Before" and skipping over calc_skip_offsets/raw_set_geometry as "After":
Before
Mode of operation:
- Input MLV file: 'Before.MLV'
...
Block: RAWI
...
Res: 1648x724
raw_info:
api_version 0x00000001
height 726
width 1808
pitch 3164
frame_size 0x00230CE8
bits_per_pixel 14
black_level 2047
white_level 16200
...
Block: RAWC
Offset: 0x000000e8
Number: 2
Size: 32
Time: 0.786000 ms
raw_capture_info:
sensor res 5184x3456
sensor crop 1.62 (APS-C)
sampling 3x3 (read 1 line, skip 2, bin 3 columns)
After
Mode of operation:
- Input MLV file: 'After.MLV'
...
Block: RAWI
...
Res: 1736x688
raw_info:
api_version 0x00000001
height 726
width 1808
pitch 3164
frame_size 0x00230CE8
bits_per_pixel 14
black_level 2047
white_level 16200
...
Block: RAWC
Offset: 0x000000e8
Number: 2
Size: 32
Time: 0.815000 ms
raw_capture_info:
sensor res 5184x3456
sensor crop 1.62 (APS-C)
sampling 3x3 (read 1 line, skip 2, bin 3 columns)
Processed in MLVFS and opening the DNG files in Adobe Camera Raw:
Before
(https://farm5.staticflickr.com/4495/37413781960_54faf75b60_z.jpg) (https://flic.kr/p/Z18pUd)
After
(https://farm5.staticflickr.com/4458/37413784640_8eda97b873_z.jpg) (https://flic.kr/p/Z18qGq)
The mlv_dump version that is in the crop_rec_4k shows both the Before and After processed DNG files without vertical stretching but maybe that's wrong? My understanding is the mlv_dump code that is in the main repository doesn't read the RAWC metadata. However, Danne's Switch app has two modified versions of mlv_dump (the other is mlv_dump_on_steroids) and they both produce DNG files that display in ACR exactly the same as MLVFS -- when these mlv_dump builds are run from the command line. Switch adjusts the DNGs so crop_rec footage is not stretched provided you choose option 01 from the startup menu:
(https://farm5.staticflickr.com/4491/37623170576_75a79370d8.jpg) (https://flic.kr/p/ZjCzQL)
So I'm confused now. I'd like to resolve the black bar issue in the crop_rec module and skipping over calc_skip_offsets/raw_set_geometry does seem to do it but it is also causing issues with some of the MLV processing apps. I can't see anything in the metadata that would cause this so I'm a bit hesitant to commit that change.
Been discussed several times before. Crop_rec gets treated as mv720 files. Detected the same since rawc isn't integrated. Default scale tag is set into the dng file.
Therefore the manual extra labour in Switch...
Ok, I get it that the default scale tag is different in the dng file but something in the MLV file must have changed when skipping over calc_skip_offsets/raw_set_geometry in crop_rec.c.
No issues when using Switch--you figured this out long ago Danne but it seems that you're doing some black magic on the DNG files after processing through mlv_dump to cover up the underlying issue.
These were processed in MLVFS - both the Before.MLV and After.MLV were placed in the same directory so they were processed with the same settings:
Before
ExifTool Version Number : 10.13
File Name : Before_000000.dng
...
Camera Model Name : Canon EOS REBEL T5i
...
Black Level : 2047
White Level : 16200
Default Scale : 1 1
Default Crop Origin : 0 0
Default Crop Size : 1648 724
After
ExifTool Version Number : 10.13
File Name : After_000000.dng
...
Camera Model Name : Canon EOS REBEL T5i
...
Black Level : 2047
White Level : 16200
Default Scale : 1 1.666666667
Default Crop Origin : 0 0
Default Crop Size : 1736 688
I want to get rid of the black bar at the top of the processed DNG files but I don't want them stretched in ACR. I also want to resolve this issue in crop_rec.c without using any black magic in post production.
It's all in dng.c. Detection derived from aspect ratio. Gets fooled with crop_rec since you corrected it. Just implement rawc metadata and it can be done straight in code instead like I do, using exiv2.
It seems to me that the MLV processing apps should be able to produce DNGs that scale properly in ACR without having to ask the user if this is crop_rec or mv720 and without having to fix the DNG files with exiv2. MLVFS seemed to be doing a good job before the change I tried in crop_rec.c.
For the record here is some information extracted with mlv_dump -v on three different MLV files from the same camera.
mv720 - plain vanilla, MLVFS adjusts the DNG Default Scale so it displays in ACR properly.
Block: RAWI
Res: 1736x688
raw_info:
height 726
width 1808
Block: RAWC
sampling 5x3 (read 1 line, skip 4, bin 3 columns)
Before - this is crop_rec in its current state which produces a black bar on the top of EOSM/100D/650D/700D crop_rec files but MLVFS does handle Default Scale properly in the DNG files.
Block: RAWI
Res: 1648x724
raw_info:
height 726
width 1808
Block: RAWC
sampling 3x3 (read 1 line, skip 2, bin 3 columns)
After - Skipping over calc_skip_offsets/raw_set_geometry in crop_rec.c, no black bar but MLVFS doesn't handle this properly and is stretched vertically in ACR.
Block: RAWI
Res: 1736x688
raw_info:
height 726
width 1808
Block: RAWC
sampling 3x3 (read 1 line, skip 2, bin 3 columns)
I'm coming to the conclusion that the problem is in the MLV processing apps. The developers should be looking at the RAWC sampling information. That includes you @g3gg0 - it looks like mlv_dump does nothing with the DNG Default Scale metadata. Though other developers seemed to have added feature in mlv_dump to adjust Default Scale but apparently they are not using the information in RAWC sampling.
So--I'll go ahead and make that change in crop_rec.c on the pull request and let the MLV processing app developers figure this out.
Nothing wrong with crop_rec.c. Check this in dng.c(dmilligan):
int32_t par[4] = {1,1,1,1};
double rawW = dng_info->raw_info->active_area.x2 - dng_info->raw_info->active_area.x1;
double rawH = dng_info->raw_info->active_area.y2 - dng_info->raw_info->active_area.y1;
double aspect_ratio = rawW / rawH;
//check the aspect ratio of the original raw buffer, if it's > 2 and we're not in crop mode, then this is probably squeezed footage
//TODO: can we be more precise about detecting this?
if(aspect_ratio > 2.0 && rawH <= 720)
{
// 5x3 line skpping
par[2] = 5; par[3] = 3;
focal_resolution_x[1] = focal_resolution_x[1] * 3;
focal_resolution_y[1] = focal_resolution_y[1] * 5;
}
//if the width is larger than 2000, we're probably not in crop mode
//TODO: this may not be the safest assumption, esp. if adtg control of sensor resolution/crop is implemented, currently it is true for all ML cameras
else if(rawW < 2000)
{
focal_resolution_x[1] = focal_resolution_x[1] * 3;
focal_resolution_y[1] = focal_resolution_y[1] * 3;
}
Thought I broke the 5D3 with these changes but it turned out that it wasn't me!
Steps to reproduce, if you dare. Just make sure you're close to your computer and card reader.
- Download latest 5D3.113 experimental (didn't test this on 123)
- Turn on only the crop_rec and mlv_lite modules
- Set crop_rec to 1920 50/60 3x3 which I take it is the 3x3_1X mode, right?
- Turn off camera.
- Turn on camera
- Panic because the LV screen is blank!
- Recite Walter's mantra -- remove card, remove battery, reinsert battery, start camera without card.
- Delete the ML/SETTINGS off the card and you're back in business.
This particular sequence works fine here. However...
Quote from: a1ex on August 28, 2017, 08:38:49 PM
sometimes the screen remains black when switching modes around (not sure how to reproduce); in this case, you don't need to go to photo mode; just press MENU twice (see first post).
On 5D3, the 50/60 3x3 preset is indeed 1x, but also with higher vertical resolution (unlike the smaller cameras).
Yep, hitting menu twice resurrects LiveView. Haven't used crop_rec on the 5D3 much but glad to know I didn't break it!
I see a lot of "is_basic"(dfort) workarounds in crop_rec.c to make crop_rec working for 100D/700D/eosm/650D. i tried like this and works. Seems easier to just categorize the cameras. Thoughts?
https://bitbucket.org/Dannephoto/magic-lantern/commits/a01a8f4420cfe37a7b52ca071ae3525906c02296
The correct skip offsets are already in raw.c - your approach requires duplicating them, does not reduce the number of branches and... you have a typo in one of the offsets ;)
The current solution does not override the offsets for these models (these offsets happen to be the same as without crop_rec). Compare these two screenshots (https://bitbucket.org/hudson/magic-lantern/pull-requests/872/crop_rec_4k-for-650d104/diff#comment-47066658) side by side.
Yes, probably. I updated dforts pull request with two screenshots of raw_diag images and a question.
Can someone please help me with removing the april 4th null pointer bug??? I'm not getting any help from anyone, just being made fun of and being told to contact people I already have several times. I've gone on IRC several times, made posts in this forum, and DM'ed with no responses. Please, if anyone can point me in the right direction to remove this bug in any way that would be great!
Thank you!!
If you don't want to use an outdated and buggy build you may want to consider using a newer one.
Ceterum censeo experimental build page esse delendam.
FRSP in photo mode with Full-res LV mode and Lossless DNG is now really fast! Preview for 2s, just a little stripped pink flash (before it was a long flash). I don't know if it's this commit (842e25f) (https://bitbucket.org/hudson/magic-lantern/commits/842e25f0a782c6e8cbf4f88bdf70cc25b427162e?at=crop_rec_4k) but really appreciate!! (last build 17oct16 used)
"Ceterum censeo experimental build page esse delendam." Walther you had me howling from laughter. When I get my breath back, I'm going toagree with you. I install every new official build, and it certainly pays off - and when I try out an experimental, I reinstall the official build immediately after testing. I find it exciting like Christmas every time I install a new build - and many times I have been totally awed by the improvements.
Quote from: kichetof on October 19, 2017, 08:20:06 PM
FRSP in photo mode with Full-res LV mode and Lossless DNG is now really fast! Preview for 2s, just a little stripped pink flash (before it was a long flash). I don't know if it's this commit (842e25f) (https://bitbucket.org/hudson/magic-lantern/commits/842e25f0a782c6e8cbf4f88bdf70cc25b427162e?at=crop_rec_4k) but really appreciate!! (last build 17oct16 used)
I've tested the first build with this feature (Aug 20 (https://builds.magiclantern.fm/jenkins/view/Experiments/job/crop_rec_4k/39/)), but didn't notice any speed difference...
Found a bug: to use crop_rec in photo mode (any preset), movie mode *must* be set to 1080p30. Nobody else noticed?!
Quote from: a1ex on October 19, 2017, 08:52:28 PM
I've tested the first build with this feature (Aug 20 (https://builds.magiclantern.fm/jenkins/view/Experiments/job/crop_rec_4k/39/)), but didn't notice any speed difference...
Some tests and you're right! I'll try to test some builds from September, I noticed a difference, that's why I've been so enthusiastic :)
Issue? Whenever I attempt "MLV_Lite" on My SL1 I get a "Busy...Please Wait" Flag
across the top of the View Screen. Recording can be started & does seem to occur.
I have tried numerous diff' settings but It's always there.
Have tried in Latest Nightly & "crop_rec_4k.17~10~10.100D101"
Also > Unable to take Screen Shots while "MLV_Lite" is active.
ORR ~ DeanB
Workaround: disable SRM memory. However, that means less memory available for raw recording. For 5D3 I've found a way to avoid the BUSY screen (http://www.magiclantern.fm/forum/index.php?topic=19300.msg189561;topicseen#msg189561), but other cameras may show different behavior.
The screenshot doesn't work because mlv_lite preallocates all the shooting memory during standby (because this is a slow operation that would delay recording). That's not an issue with 5D3, since it has enough memory in the general-purpose pools (such as AllocateMemory), but smaller cameras are not that lucky.
Can you do a quick test?
- enable the console from the Debug menu
- enable raw video (mlv_lite)
- take a picture of the screen with a second camera
From console:
(https://s1.postimg.org/64yyc35yjj/Screen_Shot_2017-10-20_at_13.20.08.png)
Is the srm memory fix for 5D mark III active for other cameras too or only 5D mark III? Trying to follow exmem.c from september 3rd to see if anything could be unchecked to test busy screen fix also for the 100D.
*update(mv1080)
(https://s1.postimg.org/9njvt6whjz/Screen_Shot_2017-10-20_at_13.27.14.png)
It's active on all models, but probably the 100D handles the BUSY screen differently.
You could try to allocate manually 1, 2, 3 SRM buffers (and so on), e.g. with srm_malloc_suite, and document what happens with the BUSY screen. If that one comes up as soon as one SRM buffer is allocated... I'm not sure what we can do. If it comes up after allocating all SRM buffers, but does not disappear when freeing one of them (in 5D3 it disappears), we can probably find a different workaround (such as reallocating everything from scratch after finding out how many SRM buffers are available).
Ok, now I can´t even reproduce the "busy" screen at the top or at the bottom. Tested latest crop_rec_4k. I had it several times in the past but now stressing shooting crop_rec, mv1080 etc both mlv_lite and mlv_rec for about ten files each and all worked fine. Could alter iso, shutter etc while recording, nothing stuck. So probably the fix is active and working but I know i had the issue described with busy screen at the top several times before. I will try my best to reproduce.
Question. How to allocate manually? snippet in exmem.c:
printf("[SRM] free all buffers\n");
/* free all SRM buffers */
/* note: first buffer is use-after-free */
for (int i = 0; i < COUNT(srm_buffers); i++)
{
if (srm_buffers[i].buffer)
{
ASSERT(!srm_buffers[i].used);
if (!srm_buffers[i].use_after_free)
{
SRM_FreeMemoryResourceFor1stJob(srm_buffers[i].buffer, 0, 0);
}
srm_buffers[i].buffer = 0;
srm_buffers[i].use_after_free = 0;
}
}
srm_shutter_unlock();
srm_allocated = 0;
}
REQUIRES(mem_sem)
struct memSuite * _srm_malloc_suite(int num_requested_buffers)
{
printf("srm_malloc_suite(%d)...\n", num_requested_buffers);
/* alloc from operating system, if needed */
srm_alloc_internal();
if (num_requested_buffers == 0)
{
/* if you request 0, this means allocate as much as you can */
num_requested_buffers = COUNT(srm_buffers);
}
adding number here?
if (num_requested_buffers == 0)
or here?:
srm_allocated = 0;
Don't edit the library, just call its functions.
Quote from: a1ex on October 20, 2017, 01:28:41 PM
with srm_malloc_suite
e.g. srm_malloc_suite(1) in don't click me, then 2, then 3...
I have tried 16oct crop_rec build on 100D. No problem with RAW recording (12bit lossless most the time) at ISO100.
At ISO>100 most time "Raw detect error" (usually first ISO>100 clip correct and then further tries with this error) - was the same situation with previous builds.
Hi guys,
hate to ask, ....because i am pretty sure that someone asked this before....but i did not manage to find it......
Is it possible..., or why it is NOT possible, to hack somehow higher resolutions recording in h264 in crop mode....?
I am very happy with 5d iii raw...more than happy, actually i sold my gh4, and did not buy gh5 , just to buy 5d iii again, after all new achievement of ML team.....but, sometimes it could be nice to have decent h26h, for some projects....
Again, thanks for all your work, and all that magic you give to us
Quote from: banertop on October 22, 2017, 10:11:41 PM
Hi guys,
hate to ask, ....because i am pretty sure that someone asked this before....but i did not manage to find it......
Is it possible..., or why it is NOT possible, to hack somehow higher resolutions recording in h264 in crop mode....?
@Banertop I'm not sure this is the correct answer but it takes a lot of processing power to take the raw data and compress it to the very complex algorithms of h.264. The 5diii can do it at 1080p but I doubt high resolution is possible. RAW data is just RAW data taken from the sensor and sent straight to the card with maybe some metadata and other bits of info attached. It takes less battery power then h.264 too. The 10,12,14 bit compression that ML is using is not nearly as complex because it doesn't need to be.
Thank you very much...
i thought that it is something about processing power.....
but, the good thing is the fact that i really now rarely need h264....raw is raw!
thanks again
Shameless cross post: https://www.magiclantern.fm/forum/index.php?topic=16650.msg192161#msg192161 sound recording with mlv_lite based on the crop_rec_4k branch.
Quote from: jacekp on October 21, 2017, 06:47:39 PM
I have tried 16oct crop_rec build on 100D. No problem with RAW recording (12bit lossless most the time) at ISO100.
At ISO>100 most time "Raw detect error" (usually first ISO>100 clip correct and then further tries with this error) - was the same situation with previous builds.
Open the console (Debug menu) to see the reason you've got this error.
I've been trying to track down this issue and it is getting pretty complicated for me. First of all, I reported it here (http://www.magiclantern.fm/forum/index.php?topic=5951.msg191830#msg191830) and here (http://www.magiclantern.fm/forum/index.php?topic=19300.msg190836#msg190836). I was trying to figure out when it first started happening and it looks like since mlv_rec has been useable on the crop_rec_4k branch--the unified branch doesn't exhibit this bug. Only the cameras that can do Movie crop mode are affected -- a.k.a. the 600D crop hack, so that should include EOSM/700D/100D/650D. I've been able to reproduce it on both the EOSM and 700D.
- Start with a fresh download or delete the ML/SETTINGS directory from the sd card.
- Turn on only mlv_rec
- Activate Movie crop mode and RAW video (MLV)
- Try a test recordings, any image size or bit option
Yes, it works but now is when it starts to get weird. Reboot the camera and try it again. It won't record with Movie crop mode turned on. Other modes work fine though.
Something is going on with the configuration files? These are the EOSM configuration files:
MAGIC.CFG
# Magic Lantern crop_rec_4k.2017Oct16.EOSM202 (8ee7858f0d7e (crop_rec_4k) tip)
# Built on 2017-10-16 20:24:23 UTC by jenkins@nightly
# Configuration saved on 2017/10/24 14:21:13
beta.warn = 24
menu.first = -4
disp.mode.x = 199
MLV_REC.CFG
# Config file for module mlv_rec (MLV_REC.MO)
mlv.video.enabled = 1
mlv.write_speed = 3911
MENUS.CFG
0 3FFFBE8F 3FE9C8FD Movie\Movie crop mode
0 3F7FBE87 3F68BA2E Movie\RAW video (MLV)
I also posted the saved settings from EOSM and 700D on my Bitbucket downloads page (https://bitbucket.org/daniel_fort/magic-lantern/downloads/), file name Movie_crop_mode_bug.zip.
Yeah, not high priority because it looks like mlv_lite might soon be getting sound capabilities and mlv_rec could be phased out but I thought I'd report this anyway.
No idea what's going on; a video showing the issue might help (including the steps required to bring it back to a "working" state).
If it's really one of these config entries, it should be easy to narrow down (by deleting only some of these files or entries).
Quote from: a1ex on October 23, 2017, 11:58:03 PM
Open the console (Debug menu) to see the reason you've got this error.
Most of the time after switching to ISO>100 and half press shutter button (or after 1 succesfull record) whole console in non-stop messages (with different numbers in brackets)":
"Black 3/5: stdev too large (9745/100, ref 1900/100)
"Black 3/5: stdev too large (8944/100, ref 1790/100) ..."
Could be either some offsets being still wrong, or some hot pixels in the OB area.
To diagnose, download raw_diag.mo and get a screenshot with the "OB zones" analysis. If that one looks good, try taking a few simple silent pictures (with silent.mo) in the affected video mode. You may have to turn off raw recording, as it preallocates the entire memory.
Quote from: a1ex on October 25, 2017, 05:57:37 PM
Could be either some offsets being still wrong, or some hot pixels in the OB area.
To diagnose, download raw_diag.mo and get a screenshot with the "OB zones" analysis. If that one looks good, try taking a few simple silent pictures (with silent.mo) in the affected video mode. You may have to turn off raw recording, as it preallocates the entire memory.
Did OB Zones in ISO100 and ISO1600.
(http://thumb.ibb.co/inoCi6/OB_ZONES.png) (http://ibb.co/inoCi6) (http://thumb.ibb.co/ghZUVm/OB_ZON_1.png) (http://ibb.co/ghZUVm) (http://thumb.ibb.co/gOTCi6/OB_ZON_4.png) (http://ibb.co/gOTCi6)
Could not take silent picture both in ISO100 (which should work) and ISO1600.
Tried silent mode: "Simple, DNG" and "Simple, Lossless DNG". Tried in photo mode 'Manual' and movie mode 'Manual'. Tried without mlv_lite and with mlv_lite but disabled "RAW recording".
Some console outputs in attachment: after camera start, after silent picture in photo mode and twice in Movie mode.
(http://thumb.ibb.co/mLm1Am/onload.jpg) (http://ibb.co/mLm1Am) (http://thumb.ibb.co/eD4A36/in_photo_mode.jpg) (http://ibb.co/eD4A36) (http://thumb.ibb.co/gj78qm/movie_mode_try2.jpg) (http://ibb.co/gj78qm) (http://thumb.ibb.co/gHOyqm/movie_mode_try1.jpg) (http://ibb.co/gHOyqm)
When trying again silent mode picture i get all the time only (even in ISO 100):
"[SRM] alloc all buffers
srm_malloc_suite =>0
srm_malloc_suite(2)..."
When switch to ISO > 100 i usually get additionally my usual messages "Black 3/5: stdev too large......"
no pictures.
The zones look good.
If the silent module doesn't work, try enabling Dump RAW Buffer in raw_diag. That one should work with less memory.
edit: trying to reproduce the memory issue - does the 100D really have only 1 SRM buffer? (check with Debug - Free Memory).
Quote from: a1ex on October 26, 2017, 07:56:32 PM
The zones look good.
If the silent module doesn't work, try enabling Dump RAW Buffer in raw_diag. That one should work with less memory.
edit: trying to reproduce the memory issue - does the 100D really have only 1 SRM buffer? (check with Debug - Free Memory).
Free Memory:
(http://thumb.ibb.co/hW8KwR/memory.jpg) (http://ibb.co/hW8KwR)
Dump RAW Buffer (only raw_diag module enabled with no analysis). ISO 100 image and ISO 800 image (with ISO > 100 got my usual "Black 3/5: stdev too large......" and no image, but after restart succesfully got one):
(http://thumb.ibb.co/mG0zwR/RAWiso100.png) (http://ibb.co/mG0zwR) (http://thumb.ibb.co/j3H1Am/RAWiso800.png) (http://ibb.co/j3H1Am)
hosting jpg (http://pl.imgbb.com/)
Workaround Note: Now I know when I can do succesfull ISO>100 image. When the Console is not throwing "Black 3/5: stdev too large......" messages I can succesfully record ISO>100. After record ISO>100 it usually throws "Black 3/5: stdev too large......" and when I swich to ISO 100 it stops throwing errors. I can then go back to ISO>100 and If I don't do many other clicks it remains without "Black 3/5: stdev too large......" and I can do second ISO>100 record...
With 2 SRM buffers I'm unable to reproduce this screenshot: http://ibb.co/eD4A36
Was the free memory screenshot taken in LiveView, in the same video mode?
Can you also upload the DNG? (I'm interested in looking at the invisible areas).
Quote from: a1ex on October 27, 2017, 01:41:43 PM
With 2 SRM buffers I'm unable to reproduce this screenshot: http://ibb.co/eD4A36
Was the free memory screenshot taken in LiveView, in the same video mode?
Can you also upload the DNG? (I'm interested in looking at the invisible areas).
That screenshot http://ibb.co/eD4A36 was in Photo mode. In "LiveView [Manual] Movie mode" was http://ibb.co/gj78qm . Free memory screenshot was also in "LiveView [Manual] Movie mode".
Today I made a ML reinstall and enabled only raw_diag and silent. This time I succefully made silent pictures (as usual: LiveView, [M]-Manual, MovieMode), but after silent picture in ISO>100 (1600), I got my usual "Black 3/5: stdev too large......" and couldn't make next picture, console:
(http://thumb.ibb.co/fJRL36/silent_try_after_silent_ISO800.jpg) (http://ibb.co/fJRL36)
DNG-s:
- previous Dump RAW Buffer in ISO800 (I have lost ISO100 image, sorry)
https://magicznyfilm.com/img/DumpRAWBuffer.DNG
- current Silent picture in ISO100
https://magicznyfilm.com/img/silent_ISO100.DNG
- current silent pictre in ISO1600
https://magicznyfilm.com/img/silent_ISO800.DNG
Nothing wrong with the samples...
Are you able to compile ML? There is an "if (0)" in raw.c - turn that into "if (1)" to save all frames that caused the error.
I've been prepping for a 3k shoot this Monday. I've been doing some exposure tests. Depending on the mode I'm in, the spot meter changes the reading of a gray card. It also changes in real terms when I measure the card in Lightroom. I'm not particularly needing these inconstancies to go away. Mainly I just want to nail my exposure so I have good highlight details but don't have to pull up shadows until they become noisy. I tested the gray card in four different settings and got sometimes some very different results.
Click on image to see the whole spreadsheet:
(http://thumb.ibb.co/cHXTQm/Screen_Shot_2017_10_28_at_9_31_03_AM.png) (http://ibb.co/cHXTQm)
Now here's the strangest part: I had exposure on the 5diii set to Manual. FPS Override was on and set to 23.976. ISO was 320. AND ... AND shutter speed was set to 1/60. But when I looked at the meta-data in Lightroom, the 3k crop DNGs said they was exposed at 1/50. Is the meta-data wrong or is the shutter speed operating at a different speed than indicated on the camera? Anyone know?
Lastly, I noticed in Lightroom that one of the process options for the video DNGs was "embedded". Anyone have info on how this choice effects the image?
Generally, you should look at the shutter speed printed by ML. That one is computed from low-level registers and is not rounded too much. Exact value in Expo menu, with 3 decimal places. Same from the frame rate - actual value is on the top bar, also with 3 decimal places.
If you can get the desired frame rate without FPS override, it's best to keep it off. This feature doesn't play nice with crop_rec, and it's not easy to fix (they both touch the FPS registers).
QuoteIf you can get the desired frame rate without FPS override, it's best to keep it off. This feature doesn't play nice with crop_rec, and it's not easy to fix (they both touch the FPS registers).
When I don't use FPS override, the ML settings say 29.776 when I punch in to 5x (whether the Canon settings are 1080/24 or 1080/30). And it actually records at 29.776. So, I don't know how to record 3k at 23.976 without FPS Override.
(http://thumb.ibb.co/cArGT6/IMG_7529.jpg) (http://ibb.co/cArGT6)
(http://thumb.ibb.co/b6G0o6/IMG_7530.jpg) (http://ibb.co/b6G0o6)
Hi guys, great work on these new exciting features -as always.
I started playing with the 2.5k compressed raws the other day and tried to put it to the test: harsh exposure contrasts and low light... it held up pretty well!! Went up to iso 1600 on this and manage to bring back +0.75 usable(ish - it's a bit mushy) exposure with resolve. here's the test:
https://vimeo.com/240408593
I had a question on the other hand: I happen to shoot interviews sometimes and wanted to see if smaller bitrates would help reduce file size (my interviews are usually in contrôles environments and don't require much color correction). From what I gathered, both 10bits compressions don't have liveview -so far?- ... what I don't understand is why the normal 10bits reduces files size by 70% (according to the menu) and the compressed 10bits only reduces it by 43% (I can't remember the exact figures but it's in those lines). How's that so?
Cheers!
Hi everyone.
I've been recording in 1080p 48fps, but the liveview is too slow. Is there a way to get it playing realtime or just faster in general? I've tried disabling global draw and the only modules enabled are the raw video ones.
shooting tomorrow. been using the build with sound and compressed raw but i noticed that something is wrong with it. some clips are fine others are split up and the have there own folders in resolve. the clips that are good don't look like folders they look like film strips. using danes switch to make the dngs. looking for a gold master for my shoot. probably doesn't exist but i thought id ask. thanks.
Quote from: D_Odell on June 22, 2017, 09:30:27 AM
(http://thumb.ibb.co/k5wbOQ/IMG_0571.jpg) (http://ibb.co/k5wbOQ)
(http://thumb.ibb.co/gLOO3Q/IMG_0572.jpg) (http://ibb.co/gLOO3Q)
Or is it nothing to worry about?
Still having this problem, or is it a problem? It remains locked down when using crop rec. Do anyone know if it affect performance?
When i set to Crop_rec 3k .. my live view is with latency. With this situation i can not focus any moving objects. I s there any way to get real time liveview in crop_rec?
Quote from: ali farahani on November 10, 2017, 05:24:55 PM
When i set to Crop_rec 3k .. my live view is with latency. With this situation i can not focus any moving objects. I s there any way to get real time liveview in crop_rec?
Live view only work in crop rec if you shoot 3x 1080 1:1 then you have propper framing and live view. Otherwise (higher resolutions without crop) you use Frozen LV so you at least can frame :) CPU power and memory isn't enough I guess..
Hi Ali, REAL TIME LIVEVIEW would indeed be the Magic Lantern holy grail, unfortunately it's not currently possible, as for focusing you have one of two options,
You can use an external monitor and you'll get the x10 zoom continuous feed displayed (no latency) perfect for focusing (only available in the 3.5K CROP MODE), or you can half press the shutter, and liveview will display the 10x zoom until you take your finger off the shutter button. So go into crop mode 3.5K CROP MODE, set your resolution you wish to shoot at, 3K or whatever, then compose your shot, half press shutter and then focus, finger off the shutter and hit record.
You who uses crop_rec a lot, which option do you use most? I often tend to use center crop. All because of lens characteristics, better performance in the glas etc. But I would have loved higher vertical resolution in this option. Since I only shot 1.85:1 or 1.78:1 and never 2.35:1 there is not so high resolution, since it ebbs out at 1340 or such. But time will tell.
How about you?
How are you shooting 1920x1080 48p ?
What exact canon settings etc.
I keep having Corrupted frames, when I shoot 1080 48p. 45p is close to continues and all frames are whole, but 48p gives corrupted frames no matter what.
I am on fir 123.
Settings from first post, starting from Canon and ML defaults.
There's no more 1920x1080 48p in recent builds, guess why (http://www.magiclantern.fm/forum/index.php?topic=19300.msg189157#msg189157). If you still have problems at 1920x1040 48p, try decreasing the vertical resolution in the crop_rec submenu. How far do you have to decrease it?
Last time I've checked, all crop_rec modes were centered (maybe off by a few pixels, give or take). Is this no longer true?
Cache lockdown issue already answered (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182619#msg182619).
Quote from: a1ex on November 11, 2017, 12:29:59 AM
Settings from first post, starting from Canon and ML defaults.
There's no more 1920x1080 48p in recent builds, guess why (http://www.magiclantern.fm/forum/index.php?topic=19300.msg189157#msg189157)Last time I've checked, all crop_rec modes were centered (maybe off by a few pixels, give or take). Is this no longer true?
Last time i tried there were substantial more than a few pixels when compared center crop to 3K crop rec. But it's not a complaint only wonder what everyone uses :)
Quote from: a1ex on November 11, 2017, 12:29:59 AMCache lockdown issue already answered (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182619#msg182619).
Thanks for that!
I can't find the option to decrease vertical resolution on the crop rec submenu and have the most recent build I believe. How do I opt between the two?
(https://i.imgur.com/uDy9gBK.jpg?1)
Also, I have a doubt that I looked around and still couldn't figure out maybe because it is too basic. Here it goes: does crop rec mean zooming x5 in liveview while actually capturing the equivalent of x3? I struggle with this question because last week I'm pretty sure I recorded in 1080p48 continuous without zooming in.
Quote from: Kanakori on November 11, 2017, 09:29:53 PM
I can't find the option to decrease vertical resolution on the crop rec submenu and have the most recent build I believe. How do I opt between the two? (https://i.imgur.com/uDy9gBK.jpg?1)
Also, I have a doubt that I looked around to find the answer but couldn't because I think it is too basic. Here it goes: does crop rec only work zooming in x5? I initially thought it would capture the whole frame, but then I read it is 5x zoom in liveview and actually capturing the equivalent of x3.
Crop rec is determined by resolution, do you want less vertical you change aspect ratio, for example 2.60:1. Higher vertical 1.78:1 etc..
Ok thanks, very kind of you! So the closest i can get to 1040 is 1:85:1 (gives me 1038). Is that it?
Correct! It's dependent of aspect ratio..
Otherwise you need crop lines and make the rest in post.
Ok, ok but now when I play around with the camera the image moves like jelly on liveview. Is that normal?
In crop_rec mode slow shutter and low frame rate is normal if you want proper framing.
@Kanakori & Whomever
Especially on Non Camera Specific Threads it would be very helpful to include info on
what Camera & Firmware You are discussing.
ORR ~ DeanB
I've found that 10bit lossless mlv_lite converted to .dng is giving me different colors than 14bit, 10bit and 14bit lossless, all of which are looking the same - ISO 3200 recording.
I tought this black level issue was solved earlier but looks like I was wrong.
Is there anything that I have to add to mlv_dump code to compensate this?
Right - the black level adjustment is not implemented for 8..12-bit lossless in mlv_dump (only for uncompressed).
Have some sample files I could look at, where the black level issue is obvious? (no camera with me right now)
Thansk a1ex!
Here you can download my .mlv files: https://we.tl/sFVu3jnWSM (https://we.tl/sFVu3jnWSM)
(the clips are very short so you can download faster - dozen frames only each)
I forgot to add that I'm using 5D MKIII with 1.2.3 firmware.
Changing black level to 2048 ("exiftool *.dng -BlackLevel=2048" or "mlv_dump --black-fix=2048") appears to do the trick.
Oddly enough, the 10-bit DNG with black level set to 2048 looks cleaner than the 14-bit one, to me. Can you confirm?
14-bit, 10-bit lossless black=2048, 10-bit lossless black=2047; rendered with dcraw -a -b 8:
(http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/14bit_000001-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/10bit-lossless_000001-2048-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/10bit-lossless_000001-2047-crop.jpg)
Do you mind capturing the same scene at 12-bit and 8-bit?
I got to record everything again because it is dawn here and there is day light coming from the window. Looks like there is an issue with 12bit lossless too:
https://we.tl/OUs5E7lZeq (https://we.tl/OUs5E7lZeq)
(14/12/10bit + 14/12/8bit lossless)
EDIT: I can say to you that the 10bit lossless .dng file with black level at 2048 in ACR (Photoshop) looks different - it may appear cleaner because it has less banding and for sure looks brighter to me than the 14bit equivalent! This may be very good because my main use for 10bit lossless will be recording documentary style videos at night with some interviews.
Take a look at those attached files - exposure levels raised a bit:
10bit
(http://thumb.ibb.co/iOsuf6/10bit_lossless_noise.jpg) (http://ibb.co/iOsuf6)
14bit
(http://thumb.ibb.co/hiLfL6/14bit_noise.jpg) (http://ibb.co/hiLfL6)
I hope this 'find' is good news for all of us!
Confirmed - the 10-bit DNG looks a little brighter as well. Actually, this applies to any lossless preset from 8...12-bit - these use a different raw type (http://www.magiclantern.fm/forum/index.php?topic=18393.0) to reduce the bit depth using digital gain.
Interesting - the 8-bit DNG looks best with black level set to 2049; however, the 10-bit DNG from previous set looks bad at 2049. Why?!
The scaling factor between the two is about 1.2 (0.26 stops). Linear regression on x14bit vs x8bit-lossless (one frame) gives [1.1971 1.2035 1.1934 1.2467] (corrections for R,G1,G2,B). The same method, on 14bit vs 10bit-lossless, gives [1.1864 1.1828 1.1842 1.1966]. After averaging all frames in the MLVs, linear regression gives [1.1878 1.1952 1.1854 1.2060] and [1.1799 1.1764 1.1793 1.1718].
Uncompressed: 14, 12, 10-bit
(http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x14bit-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x14bit.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x12bit-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x12bit.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x10bit-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x10bit.jpg)
Lossless: 14/2047, 12/2049, 8-bit/2049
(http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x14bit-lossless-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x14bit-lossless.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x12bit-lossless-2049-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x12bit-lossless-2049.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x8bit-lossless-2049-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x8bit-lossless-2049.jpg)
Lossless: 14/2047, 12/2049 adjusted by -0.26 EV, 8-bit/2049 adjusted by -0.26 EV:
(http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x14bit-lossless-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x14bit-lossless.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x12bit-lossless-2049-adj-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x12bit-lossless-2049-adj.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x8bit-lossless-2049-adj-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x8bit-lossless-2049-adj.jpg)
Lossless 8-bit: 2047, 2048, 2049, 2050
(http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x8bit-lossless-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x8bit-lossless.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x8bit-lossless-2048-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x8bit-lossless-2048.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x8bit-lossless-2049-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x8bit-lossless-2049.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x8bit-lossless-2050-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x8bit-lossless-2050.jpg)
Lossless 10-bit: 2047, 2048, 2049, 2050 (from previous set):
(http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/10bit-lossless-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/10bit-lossless.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/10bit-lossless-2048-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/10bit-lossless-2048.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/10bit-lossless-2049-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/10bit-lossless-2049.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/10bit-lossless-2050-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/10bit-lossless-2050.jpg)
Lossless 12-bit: 2047, 2048, 2049, 2050:
(http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x12bit-lossless-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x12bit-lossless.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x12bit-lossless-2048-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x12bit-lossless-2048.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x12bit-lossless-2049-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x12bit-lossless-2049.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x12bit-lossless-2050-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x12bit-lossless-2050.jpg)
The above are 100% crops; click for full images.
Makefile (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/Makefile) used for rendering.
Averaged MLVs (first set, 14-bit vs 10-bit 2048 -0.26 EV):
(http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/14bit-avg-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/14bit-avg.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/10bit-lossless-avg-2048-adj-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/10bit-lossless-avg-2048-adj.jpg)
Averaged MLVs (second set, 14-bit vs 8-bit 2048/2049 -0.26 EV):
(http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x14bit-avg-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x14bit-avg.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x8bit-lossless-avg-2048-adj-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x8bit-lossless-avg-2048-adj.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x8bit-lossless-avg-2049-adj-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x8bit-lossless-avg-2049-adj.jpg)
edit: mlv_dump averages at the same bit depth as the input (not ideal); here's the same 14-bit vs 8-bit test, but averaged after 14->16 conversion:
(http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x14bit-avg16-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x14bit-avg16.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x8bit-lossless-avg16-2048-adj-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x8bit-lossless-avg16-2048-adj.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x8bit-lossless-avg16-2049-adj-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/10bit/x8bit-lossless-avg16-2049-adj.jpg)
=> there's a 0.5 LSB roundoff error with mlv_dump -a.
hi, sorry if this has been asked in the past- I find the forum very hard to search for answers.
Can I crop a 4:3 ratio (for anamorphic) out of the 5D's full frame and get 1080p resolution at 50/60fps? If not, what would be the highest frame rate with the fastest cards?
cheers paul
Hi,
I've just bumped into a problem I don't remember anyone reporting on the crop_rec or lossless branch - I don't know exactly where this issue belongs, so could be posting to the wrong place.
Since it came out in April, I've been using the 4k losless branch, and have never come across this before: coloured vertical pinstripes across the right-hand side of the frame.
(http://thumb.ibb.co/bYQFL6/M24_0844_1_2017_11_24_0001_C0000_000400.jpg) (http://ibb.co/bYQFL6)
jpg to img (http://imgbb.com/)
I'm using the crop_rec_4k.2017Oct28.5D3123 release, and was shoting in 3.5k crop_rec mode, with a resolution of 3584x1320 at 1 fps. This is shot on a pinhole lens, with a raised exposure of 1/30, ISO 800. It's shot in 14bpp lossless.
The MLV processing was done in the latest version of Switch, using its mlv_dump option, which I imagine is the latest.
The vertical stripes persist for nearly the first 700 frames, towards the end of this range getting more sparse and faint. Around frame 700, they are no longer visible, and the footage appears normal and artefact-free.
The MLV file is 8.4 Gb, and even if I cut it down to only contain the first 800 frames, it will still be very large to upload, though probably small enough to send by WeTransfer, if looking at it helps?
Thanks,
Sacha
This artifact is visible on the main screen when zooming to 10x; does it help if you only zoom to 5x? You can disable the 10x zoom in the Prefs menu.
Unfortunately I don't have an easy fix for this one, though I have some ideas where to start looking. The image must be centered before Canon code captures the first LiveView image (which, to my understanding, is a dark frame); currently, the centering takes place a bit later.
I don't need the MLV, as I've seen this defect before (though it was only on x10 LiveView, not on x5 footage).
I've made new tests comparing files at 14/10bit and 14/10bit lossless with different ISO values. In daylight it doesn't look like the 10bit lossless files are better at banding than 14bit ones as in the previous set I've uploaded with tungsten light at ISO 3200 (post #1315). Another interesting thing is it looks to me (I may be wrong here) that 14bit lossless looks a little bit better than 14/10bit files.
All 4 pictures for each ISO have the same adjustments in ACR (Photoshop) - WB values were used clicking on the second gray patch in the color target:
Always 14bit > 14bit lossless > 10bit > 10bitlossless
ISO 1600
(http://thumb.ibb.co/mz6aL6/ISO_1600_14bit.jpg) (http://ibb.co/mz6aL6)
(http://thumb.ibb.co/dSEUf6/ISO_1600_14bit_lossless.jpg) (http://ibb.co/dSEUf6)
(http://thumb.ibb.co/c2ggtR/ISO_1600_10bit.jpg) (http://ibb.co/c2ggtR)
(http://thumb.ibb.co/fPqKf6/ISO_1600_10bit_lossless.jpg) (http://ibb.co/fPqKf6)
ISO 3200 (last one 10bit lossless Black Level at 2048)
(http://thumb.ibb.co/cgP1tR/ISO_3200_14bit.jpg) (http://ibb.co/cgP1tR)
(http://thumb.ibb.co/dNnAnm/ISO_3200_14bit_lossless.jpg) (http://ibb.co/dNnAnm)
(http://thumb.ibb.co/h8nMtR/ISO_3200_10bit.jpg) (http://ibb.co/h8nMtR)
(http://thumb.ibb.co/m0yuDR/ISO_3200_10bit_lossless.jpg) (http://ibb.co/m0yuDR)
ISO 6400 (last one 9bit lossless Black Level at 2048)
(http://thumb.ibb.co/itmQL6/ISO_6400_14bit.jpg) (http://ibb.co/itmQL6)
(http://thumb.ibb.co/cNXbSm/ISO_6400_14bit_lossless.jpg) (http://ibb.co/cNXbSm)
(http://thumb.ibb.co/jujqnm/ISO_6400_10bit.jpg) (http://ibb.co/jujqnm)
(http://thumb.ibb.co/j1fVnm/ISO_6400_9bit_lossless.jpg) (http://ibb.co/j1fVnm)
ISO 12800 (last one 8bit lossless Black Level at 2049)
(http://thumb.ibb.co/mDQrtR/ISO_12800_14bit.jpg) (http://ibb.co/mDQrtR)
(http://thumb.ibb.co/hGmaL6/ISO_12800_14bit_lossless.jpg) (http://ibb.co/hGmaL6)
(http://thumb.ibb.co/jcnh06/ISO_12800_10bit.jpg) (http://ibb.co/jcnh06)
(http://thumb.ibb.co/caUvL6/ISO_12800_8bit_lossless.jpg) (http://ibb.co/caUvL6)
As you can see, every 10/9/8bit lossless file does have a colour shift in my real world use, and I can see some small pink and green dots in the conversion too. Unfortunately while I appreciate the smaller size and the potential better dynamic range and better banding handling in the images (as seen in my first comparison #1315), I think for now I'll still going to use 14bit lossless for my low light work. In my workflow it is impossible to get different ISO takes to match each other while at 10bit lossless. Until ISO 1600 this is somewhat a small problem, but above it (my need), it turns to a big one.
I think I don't know what 'average' mean in the .dng conversion yet, hope my test is not flawed because of this.
Hi a1ex,
I already had a conversation with you about vertical stripes - through the full width - in the liveview, and you explained that you know about it, that it's harmless, and that you'd need to work hard at removing it.
This report isn't about that, it's actually in the MLV file recorded in the 3.5k mode (with 5x magnification); the vertical stripes are recorded in the first 700 frames, only appear in the right half of the image's width, each of different colour and with random spacing. And, the only other similarity to the live view is that they fade towards the end of the 700 frames, visually disappearing from all further frames.
The reason I reported it is that: I've never experienced it before; I haven't read about it anywhere in the forums since you introduced the 4k_compressed branch; and as you're still treating the 4k_compressed branch as experimental and not stable, I thought you may want to know about edge pathological cases. Though, to be fair, since I don't know what would reproduce the behaviour, that does limit the usefulness of the report. :-[
@Sganzerla: nice test; may I have the MLVs? (same as before, a few frames at each setting).
BTW - 14-bit and 14-bit lossless are identical (https://www.magiclantern.fm/forum/index.php?topic=19300.msg183231#msg183231). You are seeing two different noise samples - that's all.
@smasry: yes, I know how this defect looks in x10 zoom. I didn't know it can also end up in the recorded MLV, so your report is definitely useful. I'm not able to reproduce it though... (I only get the stripes in x10 zoom, but they go away in x5).
@a1ex
Here it is: https://we.tl/IdWVFsg91b (https://we.tl/IdWVFsg91b)
(I didn't use 12bit options because it is of no interest to me)
Hi @a1ex,
Yeah, reproducibility is the problem here. If it helps to 'read' the MLV file, all you need to do is ask.
But, that's not the end of troubles, sorry. I've just had time to process another MLV from the same batch, this time a full-resolution crop_rec mode MLV, at 5784x3248, 1 fps. The problem here is quite a bit worse; the usable image is only in the lower half of each DNG, with travelling colourful aberrations and horizontal lines in the top half (maybe top 60% of the image).
(http://thumb.ibb.co/hi2WdR/M24_0945_000013.jpg) (http://ibb.co/hi2WdR)
Another MLV from the entire batch fails to process completely, in Switch, mlv_dump and MLV App, but let's not even go there.
The only thing that I did between these shots was to change resolution and/or bit depth including from 14-bit lossless to 14-bit. Nothing 'seemed' out of the ordinary, apart from the [usual and expected] slow-down in Magic Lantern responsiveness, which is correlated to the selected fps.
One thing was annoying and unexpected, but not insurmountable: trying to get back into the Magic Lantern menu often failed, and pressing the 'bin' button would (two thirds of the time) instead of drawing the ML menu, bring up Canon's own LiveView HUD picture style selector. Waiting longer and trying again would sometimes work properly, other times it would leave the LiveView on, drawing only the bottom two-three menu items on top, without any of the rest of the ML menu.
Again, I've only experienced this now, and only through playing with different fps values. It never happens in non crop_rec mode or at the expected 23.976 or higher, (or lower, down to 4) fps.
So, this shoot - while only a test - went wrong in three different ways.
If you'd like to see any MLVs, just ask.
Thanks
Hi everyone, I am sorry for being little intrusive, but I have posted a question at the Canon T2i/550D forum here (http://www.magiclantern.fm/forum/index.php?topic=13111.msg193653#msg193653), but no one is answering.
As this question is most likely related to crop module (and this version is built on top of it with RAW support), can someone from here answer me the simple question: can we use this module on EOS 550D and achieve ~ same 3x crop mode as it is possible with stock firmware on EOS 600D ? Is the compiled module version 'stable' enough on 550D model?
Many thanks!
Regards
edit: a1ex answered my question here:
http://www.magiclantern.fm/forum/index.php?topic=13111.msg193772#msg193772
@Smasry
Is it possible that this is an FPS Override Issue & Not a corp_rec Issue ?
ORR ~ DeanB
Quite likely - finding a way to reproduce should clear the mystery.
@OlRivrRat it's possible; without bumping into it again (and again), and working out the common factor triggering it, I just can't say.
As you can see, the damage to the frame is huge in the full-res LiveView, and extensive on the 3.5k one. I'll leave this be until I have more time to play with combinations of settings, with 'exotic' fps choices! If something similar comes up again, I'll report back.
Thanks all.
I have to report a bug.
On the 5D Mk III 1.1.3 with the latest 4K raw video recording build from October 28, if crop mode 1920 50/60 is set together with 1920 25 ALL-I video mode, FPS override menu works.
But if you set that crop mode together with 1280 50 ALL-I video mode, as soon as you enter the FPS override menu, the camera freezes and you have to remove the battey.
I can replicate this issue as many times as I want.
Quote from: anDyIII on December 04, 2017, 12:23:04 AM
But if you set that crop mode together with 1280 50 ALL-I video mode, as soon as you enter the FPS override menu, the camera freezes and you have to remove the battey.
I can replicate this issue as many times as I want.
Can confirm that too, but only if you have "exact FPS" selected.
And another issue, at least for me, can't shoot 48p - I get errors.
I'm finally trying out the new crop_rec advancemens introduced earlier this year, but I can't seem to set the 1920x960 @50fps mode.
I'm running firmware 1.2.3 with the 12.19.2017 build. I've set the video mode to 1280/60p in the Canon menu, and enabled the crop_rec and mlv_lite modules. Crop mode in ML is set at 1920 50/60, but I can't set the resolution above 1920x800. Changing the aspect ratio and FPS override have no effect. I must be missing something simple and would appreciate any help...
Try changing "Video system" from NTSC to PAL. Setting is located in Canon menu.
Quote from: Walter Schulz on December 23, 2017, 11:20:25 AM
Try changing "Video system" from NTSC to PAL. Setting is located in Canon menu.
That worked perfectly. Thanks!
Hi guys,
sorry if it was asked before,.....could not find it.....
Using Experimental build from september,... 5d iii,... firm 1.1.3,... cf Lexar 128gb 1066x
My two question are about using 3k mode
1. In 3k (1:1) mode, i can get 4 sec at 3072x1728. Is that the maximum right now, or i am doing something wrong? (my fps override is set at low light)
2.Is it possible to enable canons live view during shooting? When i try it (pressing info button) it becomes glitchy ) Only i can get is preview in global draw, that becomes unusable once i hit record button. Any way to get usable preview while recording in 3k mode, even black and white....?
Thank you for your time, guys
@banertop The reason why you probably getting on 4 seconds of recording it might be that your using 14bit, and not 14bit Lossless, or 12bit Lossless ect.. When recording in 3K mode LiveView is in greyscale, and laggy that's normal, you just have to get use to it, or use and external monitor but you have to upgrade to 1.2.3. A little trick that I've been using is while recording or before recording I press the info button on the camera which gives me a zoomed in image but it's clean and Live. I can focus properly that way.
Try underexposing your scene by one stop or so. This should increase your record times substantially.
Thank you guys....
But, I am shooting 12 bit or 14 bit lossless...no 14 bit, and still get only 4 sec....
When i am shooting in 3.5k mode i get 3.5k continuously....
In all other modes i get continuously resolutions that are the maximum resolutions for continuously recording......but only in 3k mode i have the problem......4 sec....
About preview....i know about info button trick.....but (in 3k mode) if i press info button i get black screen with some color dots or stripes....like glitch effect.....so, i can not use info while recording.......In other modes, like 3.5k, i can use info button for zoomed preview while recording.....
Quote from: banertop on December 25, 2017, 12:33:45 PM
(my fps override is set at low light)
Why? I never used lossless combined with low light. What framerate are you using? Have you tried Exact or even turn it off?
Well,....when i set exact, i dont have any preview......everything is black, with color dots and stripes or stratches, but, when i set fps to low light, i got preview in global draw.......
If i press info, it still stays black screen with that dots (like glitch).....Thats why....just to get any kind of preview in 3k mode....
Again, it only happens in 3k (1:1) mode
O, sorry >))
23.976 fps
And that was set in fps override?
Yes, set in fps override.....
In general, that 3k mode is like mystery to me>)....no preview....strange glitch screen....
When i turn to low light (fps override) i get global draw preview.....before that, if i try to record, i get "raw error" message
After i set it to low light, i could record.....but only 4sec in 3072x1728
Or maybe, if someone want to share his settings for shutting in 3k mode....
Because, there are many "turn this on", or "turn this off" things.....so it is going to be easier to understand what i am doing wrong.....
Of course, if someone have time...
Thanks for help, guys
Quote from: banertop on December 29, 2017, 04:15:55 PM
Or maybe, if someone want to share his settings for shutting in 3k mode...
This video needs small updates but it should still get you 3k continuous. It's also on page one of this topic.
PM me if you need more 3k tips
Quote from: hjfilmspeed on December 29, 2017, 07:16:59 PM
This video needs small updates but it should still get you 3k continuous. It's also on page one of this topic.
The quide is very helpful, but I think the question was about the 3k 1:1 mode from the crop rec module. The video describes 3.5K where you have to punch in.
Yes RTI, thank you, you are right....
This video is great, and helps a lot.......but it is all about 3.5kmode, witch i do not have problem with.....
Problem is with 3k mode....
1. How to get more then 4 sec in 3072x1728
2. how to get preview usable during shooting,in 3.5k mode with pressing info i get preview while recording.....how to do the same in 3k mode....because, when press info button, i get black screen with color dots, stripes....
thanks for your time
Quote from: banertop on December 29, 2017, 09:08:55 PM
Yes RTI, thank you, you are right....
This video is great, and helps a lot.......but it is all about 3.5kmode, witch i do not have problem with.....
Problem is with 3k mode....
1. How to get more then 4 sec in 3072x1728
2. how to get preview usable during shooting,in 3.5k mode with pressing info i get preview while recording.....how to do the same in 3k mode....because, when press info button, i get black screen with color dots, stripes....
thanks for your time
You won't have preview in 3K 1:1 (it's broken, it's even written when you select it in the menu). As for 3.5k 1:1 - it's quite usable. Select "framing" from the menu - this will give you a preview of the real framing, but there's a trick when you half press the shutter button - it will show you the canon real time image, though 5x punched in but it's very useful to check your focus.
"but there's a trick when you half press the shutter button - it will show you the canon real time image, though 5x punched in but it's very useful to check your focus."
Do you think about 3.5 or 3k mode?
It was confusing, because one of previews guys said that he press info button to get canon zoomed preview in 3k mode, but i can not get it...
So, half press shutter button is trick for 3k or 3.5k mode (preview with info button works for me in 3,5 mode, to)
Quote from: banertop on December 29, 2017, 09:52:08 PM
"but there's a trick when you half press the shutter button - it will show you the canon real time image, though 5x punched in but it's very useful to check your focus."
Do you think about 3.5 or 3k mode?
It was confusing, because one of previews guys said that he press info button to get canon zoomed preview in 3k mode, but i can not get it...
So, half press shutter button is trick for 3k or 3.5k mode (preview with info button works for me in 3,5 mode, to)
half shutter for 3.5K crop mode. I'm not sure about the info and 3k 1:1 - I didn't manage to get it working...
I will try it with 3k mode, too.
Thank you for your help,
Cheers
Maybe I missed it, but it doesn't seem like anyone has answered banertop's questions.
1) How to get more than 4 seconds at 3072x1728 in 3K 1:1 crop mode? The answer is indeed you cannot. The 3K 1:1 mode can only record at 14-bit lossless, so even if you set 12-bit lossless or 8-11, it's still going to be 14 lossless. That's a data rate of around 120MB/s! Staying with a 16:9 aspect ratio, the largest continuous resolution you can get in this mode is probably 2560x1440 (84.1MB/s).
2) In 3K 1:1 mode you don't need to use FPS Override if you want to shoot 23.976. That's only needed to come down from 30 in the 3.5K centered mode. Using FPS Override with Exact FPS is the cause of your black screen/dots non-functioning preview. Turn it off completely and enjoy the framing preview.
Quote from: MichaelVito on January 04, 2018, 04:14:37 AM
Maybe I missed it, but it doesn't seem like anyone has answered banertop's questions.
1) How to get more than 4 seconds at 3072x1728 in 3K 1:1 crop mode? The answer is indeed you cannot. The 3K 1:1 mode can only record at 14-bit lossless, so even if you set 12-bit lossless or 8-11, it's still going to be 14 lossless. That's a data rate of around 120MB/s! Staying with a 16:9 aspect ratio, the largest continuous resolution you can get in this mode is probably 2560x1440 (84.1MB/s).
2) In 3K 1:1 mode you don't need to use FPS Override if you want to shoot 23.976. That's only needed to come down from 30 in the 3.5K centered mode. Using FPS Override with Exact FPS is the cause of your black screen/dots non-functioning preview. Turn it off completely and enjoy the framing preview.
What about recording on sd+cf to get 120MB/s ? Or it is not possible ?
Envoyé de mon iPhone en utilisant Tapatalk
to MichaelVito
Thanks a lot>))
I figured it out in the meanwhile....
Confusion starts when someone said that the problem could be something else....
Thank again
@banertop did you get continuous rec in 3k 1:1? Thats why I suggested 3.5k mode because you can get more reliable 3k.
No, i did not....
Just as MichaelVito wrote...it is not possible
I get some nice 1 min or more (app.) at 2.88k (1:95:1 aspect ratio), witch is so close to 16:9
In general, the lowest the iso is, the more seconds you will get
I shoot 2880x1620 (about 7 seconds) or 2880x1206 (continuous) to get some really great looking HD final output. So 3 pixels become 2, like an Alexa.
With 3.5K centered 5x mode, because vertical resolution is capped at 1320, you hit this wall quickly if you need taller than 2.39:1 aspect ratio. Plus, you need FPS Override to bring the 30fps down to 23.976 or 24, which introduces more instability.
It's fun to use crop_rec on steroids to push the camera to its limits, both 3K and 3.5K open up fun experiments with some Ben-Hur super wide aspect ratios, but I think the 2x crop in 3K 1:1 mode is the most useful when you're doing (or at least trying to do) real work. :)
When using the losslessly compressed modes (8...11, 12 and 14-bit lossless), the compression ratio depends very much on the scene. Bright scenes with a lot of fine detail will compress much less than darker scenes with less fine detail. This of course requires different bandwidth and recording speeds. Therefore, you will experience a substantial variation in recording times dependent on scene.
I sometimes borrow a 5D3 from a friend of mine and experiment with it. Due to the excellent video quality, the 3072x1728 resolution at 24 fps is also my favorite since it fills up the entire area of my 16:9 monitor. Typically, I use it for landscape videography, timelapses and scenes with a lot of fine detail. With the fastest card available (1066x) and a normally lit and well exposed (0.0 mark on the histogram) scene, I typically get between 140 and 170 frames before buffer gets full. When I underexpose the scene by 1 stop (1.0 mark on the histogram), I get about twice as much frames recorded (8-12 seconds @24fps). Two stops underexposure provides between 420 and 480 frames. I use the 3K 1:1 mode with the Preview set to Frozen LV for this type of shooting. For very contrasty and underexposed scenes, I also use Dual ISO and am quite satisfied with the results at this resolution. It greatly reduces noise in the dark areas which compensates for 1 to 2 stops underexposure for reasonable record lengths.
I also would like to see card spanning for this mode which would allow continuous recording at this resolution but obviously this is not possible with MLV-Lite. Otherwise, the developers would have implemented it.
Quote from: MichaelVito on January 05, 2018, 05:23:33 AM
I shoot 2880x1620 (about 7 seconds) or 2880x1206 (continuous) to get some really great looking HD final output. So 3 pixels become 2, like an Alexa.
With 3.5K centered 5x mode, because vertical resolution is capped at 1320, you hit this wall quickly if you need taller than 2.39:1 aspect ratio. Plus, you need FPS Override to bring the 30fps down to 23.976 or 24, which introduces more instability.
It's fun to use crop_rec on steroids to push the camera to its limits, both 3K and 3.5K open up fun experiments with some Ben-Hur super wide aspect ratios, but I think the 2x crop in 3K 1:1 mode is the most useful when you're doing (or at least trying to do) real work. :)
MichaelVito - I would LOVE to shoot in 2880x1206. But today, its still a bucket dream. Which build are you using, please?
I downloaded magiclantern-crop_rec_4k.2017Dec19.5D3123.zip and followed instructions on Jonathan DeNicholas excellent Vimeo feed: https://vimeo.com/217313287.
Loading from X2000 Lexar SD (rec. to Komputabay CF 1000).
Vimeo instruction contradicts suggestion to access 3.5k 1:1 in crop mode (yes x5 is enabled and x10 disabled). Its greyed out. No access.
FYI, all modes above 1920 give me nothing but interference (on Liveview it looks like an old tv interference graded in pinkish).
So I bit my tongue and resorted to a (simpler menu) nightly build.
Any ideas, anyone, for me to see the light and to carry me into this great, 4k in the sky kind of place where you all live happily ever after?
1 Would you suggest I download any modules?
2 Uninstall and reload?
3 Another option that I missed?
Forgive me for growing an issue that may have been dealt with but I looked for a day on the forums to see if there was anything that sounded close to my issue with the pink screen.
Thank you
Quote from: Kokoe on January 08, 2018, 05:06:18 PM
Vimeo instruction contradicts suggestion to access 3.5k 1:1 in crop mode (yes x5 is enabled and x10 disabled). Its greyed out. No access.
Every single time a ML menu option is grayed out, there is a message at the bottom of the screen, telling you why it's grayed out. Please read it out loud and do what it says. If that doesn't solve your issue, read on.
Quote from: Kokoe on January 06, 2018, 05:18:37 PM
I tried everything. Firmware 1.2.3. Firmware 1.1.3 (and I ain't stupid - I can write in C) but sadly, still no 4k for me. Max setting 1920 that is stable. Even the excellent Topic: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView post video that told me exactly what to did not work. Why? If the instructions were complete then why didn't it work?
I suggest it is because the build used on the Vimeo film is the ONLY one it works on.
The video looks fine to me, and it's not outdated (other than the build number printed in the Help menu). If you are seeing anything different:
1) make sure you are really following
all the steps from the video
2) if in doubt, watch the video with sound enabled
3) document the difference between what you are seeing and what's in the video properly (https://www.chiark.greenend.org.uk/~sgtatham/bugs.html).
I'd even go as far as asking you to find
one (1) single difference on the user interface side between the latest build and what's shown in the video. Without doing that, I somehow doubt you'll be able to tell whether this tutorial is outdated or not.
P.S. Sorry for not giving the answer directly - I want you to find the solution on your own.
Quote from: dfort on December 10, 2017, 11:16:24 PM
Instead of giving you the answer I'm trying to show you how to find the answer. Like the old proverb, "Give a man a fish and he eats for a day, teach a man to fish and he'll sit in a boat all day drinking beer."
Quote from: a1ex on January 08, 2018, 07:26:06 PM
Every single time a ML menu option is grayed out, there is a message at the bottom of the screen, telling you why it's grayed out. Please read it out loud and do what it says. If that doesn't solve your issue, read on.
The video looks fine to me, and it's not outdated (other than the build number printed in the Help menu). If you are seeing anything different:
1) make sure you are really following all the steps from the video
2) if in doubt, watch the video with sound enabled
3) document the difference between what you are seeing and what's in the video properly (https://www.chiark.greenend.org.uk/~sgtatham/bugs.html).
I'd even go as far as asking you to find one (1) single difference on the user interface side between the latest build and what's shown in the video. Without doing that, I somehow doubt you'll be able to tell whether this tutorial is outdated or not.
P.S. Sorry for not giving the answer directly - I want you to find the solution on your own.
Thanks a1ex. Don't be sorry. I don't expect to be handed any 'fish' on a plate at all or I become less valuable to the team in future testing (and which I intend to do live on client's work). I also don't want to treat any work in a way that disheartens the creation of something that others might take away and never comes back to contribute toward. So please forgive me for asking questions to get to a point of knowing the genius in the work. I spent 2 hours experimenting today and 2 days prior learning as much as I could before I got to this point and became loop ignorant as well as stuck.
Anyway, I tried a search using the Forum Search button for '3.5K 1:1 centred x5' but it brought up a '0' result. A search for 'crop mode' successfully brought me here.
Contrary to your assumption, I have followed everything again in Jonathan's video, word for word and listened carefully, and three times now (varied with reformats in FAT32, reboots etc. on three versions of crop-red inc 2 firmware changes) but without knowing where crop mode lives off other 'needed' menu items, little wonder I am stuck.
'Crop Mode' is greyed out when selecting 3.5k 1:1
Same message at base of LV saying 'To use this mode, please enable the x5 zoom in Liveview'
PREFS main menu, select 'Liveview zoom tweaks'
Enable x5 (already done)
Disable x10 (already done)
Head back to the 'Crop Mode under MOVIE main menu.
It still says 'To use this mode, please enable the x5 zoom in Liveview'
Little wonder that head scratching goes on with the thought as to whether I have something turned on (did not touch anything since installing ML clean) that Jonathan had on that I don't.
But without knowing what, I thought I would ask if someone with more experience than I, an ML student might learn? By being taught to ask questions. if you want to head me off in a direction then please give me a clue which one. I am happy to learn.
I really do appreciate the work you guys have put in and bravo on this major advance. (Not to mention a huge bravo for coming on here and supporting it directly. I feel honoured and embarrassed, but thank you all the same).
Quote from: Kokoe on January 08, 2018, 08:19:21 PM
Anyway, I tried a search using the Forum Search button for '3.5K 1:1 centred x5' but it brought up a '0' result. A search for 'crop mode' successfully brought me here.
Forum search does not use strings with less than 3 letters. Go Google:
http://bfy.tw/Fw9I
The issue is not string length, but proper spelling. Google can find things even with some misspellings, while forum search cannot.
3.5K 1:1 centered x5
gives the expected result in the search box (try it).
Quote from: Walter Schulz on January 08, 2018, 08:25:27 PM
Forum search does not use strings with less than 3 letters. Go Google:
http://bfy.tw/Fw9I
Wow! That link was nothing short of amazing !!! Almost felt surreal watching it.
Quote from: a1ex on January 08, 2018, 08:52:53 PM
The issue is not string length, but proper spelling. Google can find things even with some misspellings, while forum search cannot.
3.5K 1:1 centered x5
gives the expected result in the search box (try it).
Doing that now ... Where I come from the word is 'centred' ... We would need to be sharing at least one beer on a porch to broach the subject (safely and with beings intact) as to who was responsible for something called 'American English' (a contradiction in itself).
Quote from: a1ex on January 08, 2018, 07:26:06 PM
P.S. Sorry for not giving the answer directly - I want you to find the solution on your own.
I did find the solution. The answer is 'the zoom (magnify button) must be depressed to x5 because it affects ML MENU's ability to select greater resolutions than 1920' ...
Now who on earth would have figured that one out as easy (or stupid, depending on your assessment of me as a noob). OK, so I am an honorary member of the 'DUH' club but head is well scratched and for good reason. Allow me to explain.
Among the Canon's I have had (and relevant to this post, am using today and which in total number: 5), the one thing that one does tend to do on all of them (dumb though we DOP's are) is to think of a <mag> button as a control for a zoom function for anything other than tighter focussing. In this case, a button that 'zooms' picture content less for focussing but more important, so that a zoom calculation enables a mode that affects the ability to access something in a non-linked menu as a prime function (the path to 4k recording).
Yes, how wrong I could be but a given misunderstanding comes from a lack of knowledge, not a proof of idiocy. I will listen to that video again though.
But, in this case, lest we wander on, although zoom was set to x5 in the PREFS main menu selection, the message 'To use this mode, please enable the x5 zoom in Liveview' does not provide enough information to lead one to that realisation. Not immediately. But given that, most messages have to be shortened to fit them in. it was easy for one to think it meant selecting 'Liveview movie tweaks' in the PREFS menu because, surprise, surprise, there it is sitting in there.
So adding 'To use this mode press the third button up on your camera'... is not your fault at all. Or mine, for that matter. Because dammit Canon, you should have known ML needed a bigger screen to explain it in much greater detail.
Thank you, a1ex.
Since you are a native English speaker, can you suggest a better help text that would fit there? (up to about 60-70 characters)
BTW, not all cameras have the zoom button in the same position, but all of them will label this mode on screen as x5.
Trying to be as succinct as possible without losing an obviousness in the meaning but respecting your limit of space either/or mixes of these could work:
1 To use this mode, exit ML menu & press index/magnify button (59)
2 Mode only selectable after exiting menu & pressing magnify button (65)
3 To use 3.5k 1:1 crop mode exit ML menu & press 3rd button up on left (68)
4 3.5k 1:1 crop mode only selectable via button:index/magnify set to x5 (69)
(I think 2 to be the most truistic). Line 3 knocks out the 'button position rationale' that you gave - e.g.: Canon 70D button is 'top right' so possibly harder to position with text. (Less explaining on screen may be necessary however, following Walter's epoch of a manual).
FYI: The reason for the two lines above looking the same in length (as in at first glance) is that although one boasts a longer character count the actual <real> length is down to the set of the font vs. the number of characters.
Not sure which fonts work best at the size for readability for users and may be a limit in coding (having not worked out pixel height vs. x height comparison on screen) but generally, some book font sans are tighter set (Futura vs Helvetica, for example).
Hope his helps and is possibly of use. Thank you for letting me provide some input.
Hello everybody,
I'm sorry if it's not the right place to ask, but are there any reasons to keep 5DIII with Magic Lantern and not to buy Sony a7SII (or rather upcoming a7SIII) for filming/video 4K, avoiding all the hassle with ML? Obviously a7s doesn't have enough resolution for stills, so i mean video-wise.
Thank you a lot!
Well there is a big difference between 12 bit 4:2:2 compressed raw 4k and the highly compressed 8 bit 4:2:0 XAVCS 4k slog footage from the A7SII.
The 12 bit raw is infinitely more gradable and depending on where you plan to deliver, you may require at least a 10 bit 4:2:2 source. 8 bit 4:2:0, despite how nice it CAN look, simply isn't a professional delivery standard.
I would personally choose the new GH5S over an A7SII.
4K is overrated, higher bitdepth raw, dynamic range and lowlight performance is worth so much more. 2k Raw = sweetspot res vs filesize compromise.
A lot of people I know don't even shoot 4k with the A7s ii. They shoot HD. But they like the compact size and the low light capabilities. I used one for the first time over the weekend. Lots of pros and cons vs. 5diii, but the reason I used it was because I was shooting in extreme darkness. Shot the rehearsal with the 5diii and got no exposure, but the Sony got some great images.
The A7s ii doesn't beat the quality of RAW. Really hoping to combine the Lossless RAW with audio recording so I can use it for interviews without needing slates for syncing and get a few more years out of the camera. :)
https://vimeo.com/251956004 (https://vimeo.com/251956004)
Two bugs squashed and another one found.
We've been having an interesting discussion and testing sessions on the 100D topic. Turns out that there was a bug reported a while back that the 100D will "hiccup" in zoom mode no matter what resolution you use. In addition, while testing the latest focus pixel maps we found out that the 100D in mv1080crop mode (a.k.a. Movie Crop Mode or crop hack) will not work with the 14, 12 and 10 bit uncompressed settings. I made some adjustments to the raw buffer height and skip offsets and was able to not only resolve the 100D issues but also add the full 1080 height to the zoom mode on the EOSM and 700D. I'm assuming that the 650D is also in this group of similar cameras so I made adjustments on that platform too.
Here is the pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/897/raw-buffer-adjustments-for-eosm-100d-650d/diff) with the changes and I posted test builds on my Bitbucket downloads page (https://bitbucket.org/daniel_fort/magic-lantern/downloads/).
Here's what I did and why.
First of all the EOSM doesn't report the full buffer height but there is no code to add 1 to the height like there is to subtract 1 for the 5D3. The 700D and 650D also under report height by 1 pixel so I added 1 pixel to the height of the EOSM/700D/650D cameras. The 100D reports the height accurately. We found that subtracting 1-pixel from the height resolves the "hiccup" bug but it sacrifices the maximum image height in zoom mode so I reverted that change.
Next, I looked at the skip offsets. Turns out that there was a little space at the top of the zoom mode buffer so by adjusting that I was able to get rid of a similar "hiccup" issue on the EOSM so I'm hoping this will also work on the 100D. I was able to reproduce the Movie Crop Mode issue on the 700D and resolved it by adjusting the skip_bottom setting. The skip offsets were carefully determined using the raw_diag module from the iso-research branch but it appears that they needed this slight adjustment, probably related to the raw buffer height underreporting issue.
#if defined(CONFIG_EOSM) || defined(CONFIG_700D) || defined(CONFIG_650D) || defined(CONFIG_100D)
skip_top = zoom ? 26 : 28;
skip_left = 72;
skip_right = 0;
skip_bottom = zoom ? 0 : mv1080crop ? 2 : 4;
#endif
I did find a bug on the EOSM that I couldn't resolve. When in zoom mode at maximum 2520x1080 resolution if you set the FPS override to "Optimize for Exact FPS" or "Low Jello, 180d" the right side of the image gets chopped off and glitches appear every few frames. Basically, a black bar on the left right side of the frame that can be seen on the LiveView screen if the Preview is set to Auto or Framing. The good news is that it doesn't affect 1920x1080 resolution and the EOSM can record continuously at that image size in zoom mode with 8..11bit lossless compression.
(https://farm5.staticflickr.com/4668/38985928825_d70f300232.jpg) (https://flic.kr/p/22p45oD)
There is some bad news in all this, at least for me--some of these fixes don't work with the new focus pixel maps I just finished working on. Guess there's no such thing as "finished" in Magic Lantern development.
Quote from: dfort on January 25, 2018, 04:54:19 AM
We found that subtracting 1-pixel from the height resolves the "hiccup" bug but it sacrifices the maximum image height in zoom mode so I reverted that change.
Long answer on bitbucket; I have a feeling most users will prefer clean footage (without hiccups) over "perfect" 1080p resolution :D
Quotea black bar on the left of the frame that can be seen on the LiveView screen if the Preview is set to Auto or Framing.
I see the bar on the right.
This kind of black bars with FPS override indicate the FPS_TIMER_A_MIN for EOSM is way too low (and may need fine-tuning for each video mode, as it's done on other models). I did question that value back then, when it was backported from TL, but it takes a dedicated tester like IDA_ML, with lots of attention to detail, to narrow down such "subtle" issues that make the camera unusable in certain video modes.
These need double-checking on all models, in all video modes, with raw_diag (OB zones screenshot) and FPS override set to maximum value; that way, the timer A will be minimized, revealing this issue.
QuoteI did find a bug on the EOSM that I couldn't resolve. When in zoom mode at maximum 2520x1080 resolution if you set the FPS override to "Optimize for Exact FPS" or "Low Jello, 180d" the right side of the image gets chopped off and glitches appear every few frames. Basically, a black bar on the left of the frame that can be seen on the LiveView screen if the Preview is set to Auto or Framing.
This bug, altough in my case it's on the right side (as a1ex), is there for at least 9 months, i have it in my test from April ( https://youtu.be/FyfRVbYHYh8 (https://youtu.be/FyfRVbYHYh8) ). Strangely enough, it disappeared when fps was set to 15 fps (https://youtu.be/Wvs2KDwk0M8 (https://youtu.be/Wvs2KDwk0M8)). With build from December the 19th it's not true any more, and stripe is there to stay (only in exact fps mode though).
I will test your new build tomorrow.
Quote from: a1ex on January 25, 2018, 09:33:56 AM
I have a feeling most users will prefer clean footage (without hiccups) over "perfect" 1080p resolution :D
Dfort,
A1ex is right. We all would prefer perfectly clean footage without hiccups and corrupt frames and WITH SOUND in all video modes rather than the 2 missing pixels. Upscaling from 1078 to 1080 which Resolve does automatically anyway, will hardly affect quality.
My suggestion to you is: Please prepare a build that in your opinion works well in the above way, making sure that sound is working properly in all modes and I will test it for you. Leave the focus pixels for the very final stage after we all make sure that everything else works fine. Then I will perform the Pixel Scanner test again and you can make corrections to the focus pixel maps.
Keeping in mind the latest remarkable developments on the 100D, I personally would like to see a reliable and stable build, with all functions working properly, that all 100D users can take and use with confidence on their trips and vacations. I think that the 100D is very close to this stage of its development. So, here is my request to all developers involved with the 100D in one way or another. Please help Dfort complete his mission! Thank you all in advance.
+1 for sound in mlv_lite.
Quote from: loknar on January 25, 2018, 11:08:59 AM
...in my case it's on the right side...
Quote from: a1ex on January 25, 2018, 09:33:56 AM
I see the bar on the right.
Ok, don't know my left from my right -- edited my post (https://www.magiclantern.fm/forum/index.php?topic=19300.msg196350#msg196350). So yes, there is a black bar on the right like in the video @loknar posted. If you stretch the image you'll see that it isn't just a black area:
(https://farm5.staticflickr.com/4706/39182208694_97853a5e39.jpg) (https://flic.kr/p/22Gp4yj)
Quote from: a1ex on January 25, 2018, 09:33:56 AM
This kind of black bars with FPS override indicate the FPS_TIMER_A_MIN for EOSM is way too low ...
Thanks! Good to know what to tweak.
Quote from: a1ex on January 25, 2018, 09:33:56 AM
...it takes a dedicated tester like IDA_ML, with lots of attention to detail, to narrow down such "subtle" issues that make the camera unusable in certain video modes.
+1 for IDA_ML
Quote from: a1ex on January 25, 2018, 09:33:56 AM
This kind of black bars with FPS override indicate the FPS_TIMER_A_MIN for EOSM is way too low (and may need fine-tuning for each video mode, as it's done on other models).
Looks like there has been some fine-tuning on the EOSM but the zoom setting probably wasn't tested at 2520x1080. The current settings work fine at 1920x1080.
#elif defined(CONFIG_EOSM)
#define TG_FREQ_BASE 32000000
#define FPS_TIMER_A_MIN (ZOOM ? 666 : MV1080CROP ? 532 : 520)
#undef FPS_TIMER_B_MIN
#define FPS_TIMER_B_MIN ( \
RECORDING_H264 ? (MV1080CROP ? 1750 : MV720 ? 990 : 1970) \
: (ZOOM || MV1080CROP ? 1336 : 1970))
Fiddling with the FPS timer A in the FPS override Advanced settings, the black bar on the right completely disappears at 716 (FT +50) with the Exact FPS and Low Jello, 180d options. Should I add this to the current pull request?
Sure. That's quite a difference...
Cross-checking: the black zone starts at 2320 out of 2520 pixels, so it's 200 pixels wide. In raw.c, column_factor for EOSM is 4 (meaning: one unit in 0xC0F06800/4 covers 4 pixels horizontally). Since you had to adjust FPS timer A by 50 units, that probably means the FPS timer A has the same units as the register that controls the image capture resolution.
This column_factor is probably the number of columns read out in parallel. On 5D3, this value is 8, and the well-known vertical stripe pattern repeats every 8 pixels.
Ok, done. New build looks good on the EOSM now. Posted a new test build.
Helloo
PLS!!!!!!!!!!!!!HELP ME,
I have a 5d3 fimrware 1.2.3
after instalation magiclantern-lua_fix.2018Jan20.5D3123 i dont see crop_rec , i don't see nothig 4k.
My full resolution is 1920 :(. Why??????
You downloaded the lua_fix build, not the crop_rec_4k build. Look for this one on the experiments downloads page (https://builds.magiclantern.fm/experiments.html):
magiclantern-crop_rec_4k.2017Dec19.5D3123
You'll find it under: "4K raw video recording; lossless compression"
Any suggestion is greatly appreciated:
Using 5dIII 1.2.3, the storage card is: SanDisk Extreme Pro 128GB, 160MB/s.
I followed the steps to use crop_rec. Downloaded and installed the Dec 2017 version. Shot video using the steps as per the video in this forum a few pages back, and the MLV files were created. But when I import the files into Resolve 14, all I get is noise. I have successfully did the same procedure with the regular 5dIII 1.2.3 Dec 2017 release (not crop_rec), and everything works fine. (Just shooting 1920x1080 24p). Resolve imports the files and they can be edited.
My overall goal is to shoot 1920 x 1080 48p, but I have never been able to do that either using crop_rec. Same thing, the files seem to be corrupted. I purchased the EosHD Shooter's Guide (from EOSHD.com), which explains a lot and is helpful. I followed the steps there. Same thing, the MLV files seem to be corrupt when imported into Resolve 14.
So basically, to sum up: I have not had any success creating editable MLV files using the crop_rec release.
Any suggestions are very helpful and appreciated as to the corrupted files problem. Must be some setting I am missing. But no idea at this point. Thanks in advance.
Quote from: [email protected] on February 01, 2018, 01:25:18 AM
...the MLV files seem to be corrupt when imported into Resolve 14.
You need to convert the MLV files to DNG image sequences before importing into Resolve.
Now that I had such good luck with hg bisect on another crop_rec_4k issue, I'm going to resurrect the discussion we started back in Reply #1157 (https://www.magiclantern.fm/forum/index.php?topic=19300.msg190202#msg190202) Specifically:
Quote from: a1ex on September 21, 2017, 02:25:14 AM
# compile and test again, just to be sure it works
# the result should be identical to 5b6936ed7aab -
# Erwin's version before decompression and 5D3 patch
hg bisect --good
Turns out that when I followed the instructions the changeset wasn't working with FRSP lossless on the EOSM. I tracked down the problem to commit f6d95f2 (https://bitbucket.org/hudson/magic-lantern/commits/f6d95f2d7b5ab59fb4b003ae128a5293921a25ee). Déjà vu:
Quote from: dfort on September 22, 2017, 10:39:14 PM
Found out that if I back up all the way to the silent.c in a9c8ad4 (https://bitbucket.org/hudson/magic-lantern/commits/a9c8ad412af5c282bbad6889ce5f65d974ca3ada) and keeping everything else at the lastest in crop_rec_4k the FRSP lossless compression DNG issue with EOSM/700D is not present. Updating to commit f6d95f2 (https://bitbucket.org/hudson/magic-lantern/commits/f6d95f2d7b5ab59fb4b003ae128a5293921a25ee) the issue shows up.
It
should be printing out an "Out of memory" error message if there is a problem but everything seems to work fine when you are shooting pictures though no lossless DNG files are saved to the card.
I hacked together a silent.c that works with all cameras (https://bitbucket.org/hudson/magic-lantern/pull-requests/858/fix-lossless-compression-on-frsp-for-eosm/diff) but it isn't a satisfactory solution. There's a note on the pull request to add some printf's to see where it fails. Not sure where to start with that but I'd really like to get this issue resolved properly.
Regarding printf's, you need to find out why it fails (why it takes some particular code path, what are the return values etc). Some example in pseudocode:
int x = check_this();
if (x > y)
{
do_this();
return 5;
}
else
{
return do_that();
}
After adding printf's, it could look like this:
int x = check_this();
printf("check_this returned %d, expecting > %d\n", x, y);
if (x > y)
{
printf("doing this\n");
do_this();
return 5;
}
else
{
printf("doing that\n");
int ret = do_that();
printf("do_that returned %d\n", ret);
return ret;
}
In particular, we need to find out why the DNGs are not saved - likely bad input data to save_dng. Was save_lossless_dng able to allocate memory? (print the input and output values: max_compressed_size and out_suite). Did the lossless compression work? What return value did you get? (it returns compressed size or negative error code). Does the raw data get recognized by save_dng as a lossless stream? (return value of is_lossless_jpeg). What's the size of the data it's trying to write to card? And so on.
If it's able to allocate memory, but for some reason, lossless_compress_raw fails, compare with your version and see what's different. Do you try to compress an image with different size? (maybe smaller? maybe some dimension is modulo some power of 2? no idea here, there may be additional hardware restrictions on certain cameras).
Hi, my English isn't good so I apologize for my mistakes.
I try to figure out how to use Crop mode 1920 50/60 3x3 in proper way.
First I switch fps in Canon menu to 50.
Second I turn on RAW video (1920x818) and Crop mode (1920 50/60 3x3).
I able to record 1920x818, but:
in my live view I can see:
(https://s18.postimg.cc/mky156fwp/ml2.jpg)
but my footage looks:
(https://s18.postimg.org/yyavcm0ll/image.jpg)
It's a diffrent position.
Any advice to solve my problem?
magiclantern-crop_rec_4k.2017Dec19.5D3123
Pushed some updates, mostly integrating previous work in one build:
- experimental mlv_snd support from ErwinH (thanks IDA_ML, Danne and dfort for identifying and squashing the bugs)
- latest Lua stuff (http://www.magiclantern.fm/forum/index.php?topic=14828.msg194706#msg194706) (please try the linked tests on this build, too)
- latest memory updates for raw backend (http://www.magiclantern.fm/forum/index.php?topic=5601.msg196569#msg196569) (please report back on 70D and 100D, will require compiling from source)
- various small fixes
- an experiment that lets you move the preview window around in x5 mode using the focus box
Caveat:
you may lose frames if you move the focus while recording!(nothing new, this happens in all ML builds, but now you might be tempted to move it)
@Tom_LS: from the download page:
Quote
You will generally need to use the (non-realtime) preview from the raw recording module.
Modes with higher vertical resolution will have real-time preview only for the top of the frame.
Great updates!
I like to test latest manual lens updates with the 4k branch,
would the way to test them just to import manual lens branch into 4k crop, now that Lua fixes are in place in the 4k branch?
That's next - I already have some local changes to fix the issues there. I just wanted to integrate the refactors first - there were some changes on the lua_fix branch that touched many parts of the code (refactors to get consistent behavior among all camera models).
Quote from: a1ex on February 01, 2018, 10:02:41 AM
...you need to find out why it fails (why it takes some particular code path, what are the return values etc).
Looks like you added another check, Silent pictures: check for lossless compression errors (https://bitbucket.org/hudson/magic-lantern/commits/3b202398eee93a2a7b5ae5efdf1003d3bcae6362?at=crop_rec_4k). The message flashes by pretty quickly so I shot a video of the screen to make sure I'm reporting the error message properly:
Lossless compression error: -2(https://farm5.staticflickr.com/4757/40043397661_71753c1b07.jpg) (https://flic.kr/p/241uT2X)
Semaphore timeout (in other words, the encoder did not run).
Now compare with your version and see what parameters you have different: maybe image size? The version with in-place compression did have slightly lower vertical resolution; maybe that was making a difference?
Quote from: a1ex on February 02, 2018, 05:51:34 PM
Now compare with your version and see what parameters you have different: maybe image size?
My version is just a hack of commit 3b20239 (https://bitbucket.org/hudson/magic-lantern/commits/3b202398eee93a2a7b5ae5efdf1003d3bcae6362?at=crop_rec_4k), running the new code on the 5D3 and the old code on all other cameras. In fact simply replacing only silent.c in the latest crop_rec_4k with the version from commit a9c8ad4 (https://bitbucket.org/hudson/magic-lantern/commits/a9c8ad412af5c282bbad6889ce5f65d974ca3ada) it will work on the EOSM/700D.
What seems to be different is that the "skip top bar" code was removed in the newer "do not compress DNGs in place" version. Is that where the slightly lower vertical resolution happens? We recently reduced the vertical resolution in the raw buffer for the 100D (https://bitbucket.org/hudson/magic-lantern/commits/4b11e709fcdaddab0d39a047e65511c80e9bf263?at=crop_rec_4k) like the 5D3. Maybe that's needed here? Thing is, we also added some vertical resolution to the EOSM raw buffer (https://bitbucket.org/hudson/magic-lantern/commits/b75c50941fc6d746db80509a0e865388ebe038aa?at=crop_rec_4k).
The changes done for 100D and EOSM happen in a different place in the pipeline - before lossless compression. That issue is unrelated - there, the problem was at bringing the captured image into RAM. Now we've got the image into RAM, can save clean uncompressed DNGs, but the lossless encoder has some trouble compressing it.
Just print out_raw_info.width and height with your version, and with mine, then try modifying it (right before the call to lossless_compress_raw).
Wow guys! RAWLite (compressed raw) with sound included in the MLV in the latest 5Dmk3 update! Congrats to all of you who made it possible. I will battle test the beast and send results.
Cheers!
BTW the proxy dosent work with that but I think it's better to make proxys straight from Resolve.
Thanks again coding team!
Quote from: a1ex on February 02, 2018, 07:42:03 PM
Just print out_raw_info.width and height with your version, and with mine, then try modifying it (right before the call to lossless_compress_raw).
Ok, this is what I did:
printf("out_raw_info.width %d\n", out_raw_info.width);
printf("out_raw_info.height %d\n", out_raw_info.height);
out_raw_info.frame_size = lossless_compress_raw(&out_raw_info, out_suite);
On the EOSM with the old version of silent.c, commit a9c8ad4 (https://bitbucket.org/hudson/magic-lantern/commits/a9c8ad412af5c282bbad6889ce5f65d974ca3ada), it reports on the console:
out_raw_info.width 5280
out_raw_info.heigh 3477However, your current version reports:
out_raw_info.width 5280
out_raw_info.heigh 3529Next I tried changing the height with:
out_raw_info.height = 3477;and it saved a compressed DNG but this is obviously not the way to do it because the file wouldn't open.
[EDIT] I take it that this test doesn't apply to FRSP?
In [1]: 5280*14//8*3477 % 16
Out[1]: 8
In [2]: 5280*14//8*3529 % 16
Out[2]: 8
Try adjusting out_raw_info.active_area.y2 too (decrease it by the same amount). If that works, can you try intermediate sizes to see where it breaks?
It works! Turns out that height -1 is all that's needed.
out_raw_info.height = 3528;
out_raw_info.active_area.y2 = 3528;
So taking what we learned recently with the 100D (https://bitbucket.org/hudson/magic-lantern/commits/4b11e709fcdaddab0d39a047e65511c80e9bf263), I tried this and it works too:
(out_raw_info.height)--;
(out_raw_info.active_area.y2)--;
Would there be a problem if this is applied to all of the cameras?
Probably not. In photo mode, I take the size directly from EDMAC configuration; double-checked with the transfer model (http://www.magiclantern.fm/forum/index.php?topic=18315.0) and appears to be right the way it is now:
height = shamem_read(RAW_PHOTO_EDMAC + 4) + 1; /* size N */
On 5D3, this gives an even number (also visible in edmac.mo, channel #0). On 700D/M/100D, I assume that's an odd number. Maybe force it to be always even?
height = shamem_read(RAW_PHOTO_EDMAC + 4) + 1; /* size N */
height &= ~1;
OMG I've been waiting this for looong time! mlv_lite + mlv_snd = great! You are magicians! But some issues:
1. mlv_dump (latest from crop_rec_4k) seems to miss numbers in frames' names, looks like no missing frames, all parts with no gaps stay synced with the sound, I've got 373-375, 753-755 and 1133-1135 (like even numbers are missing)
(plz ignore my fancy pants :D)
(https://thumb.ibb.co/k4bhZR/framez.jpg) (https://ibb.co/k4bhZR)
2. Sound starts recording before several frames and stops after.
(https://thumb.ibb.co/dgNvom/framez2.jpg) (https://ibb.co/dgNvom)
*file deleted*
Same with crop 3K 1:1 2560x1440. I think it is still usable with clapper for syncing.
5D3 1.2.3, external stereo mic, ML build magiclantern-crop_rec_4k.2018Feb02.5D3123, SanDisk Extreme Pro 128GB
The same issue-missed numbers 375 and 755 (total 854), for 23,976 FPS override -on ExactFPS.. 1920x1080, mlv_lite + mlv_snd 5D3 1.1.3, mlv_dump (latest from crop_rec_4k)
Missed numbers 392,787,1183,1579 (Total 1658) for 25FPS FPS override -off.,1920x1080
I had this issue before on both mlv dump & mlv dump on steroids (Canon 700d).
Yeah mlv dump jump (1) every many frames and caused missed frame but actually the problem only affect the numbering of frames.
The missing frame is not missed at all, only jumped (1) forward causing blink between frames.
Fast solution I used: Select all of frames and rename to any character like (A) now Windows will create the frame numbers correctly. :D
Yet another boring test: https://www.youtube.com/watch?v=MMGbWQLAArs (https://www.youtube.com/watch?v=MMGbWQLAArs)
This time, odd nums missed too, 2560x1440 gives longer pieces without num jumping but longer delay between sound and video rec starting. Did not notice any frame drops so just a bit more time in post needed, maybe simple renaming script would be a temporary workaround.
Fix committed; if you can compile from source, please try. Still got a couple of things to fix for 100D etc.
hg up crop_rec_4k -C
hg merge crop_rec_4k_mlv_lite_snd
make 5D3.113_zip
(if you need help compiling, watch the vid (https://twitter.com/autoexec_bin/status/913530810686418944))
Quote from: a1ex on February 03, 2018, 12:36:54 AM
...On 700D/M/100D, I assume that's an odd number. Maybe force it to be always even?
height = shamem_read(RAW_PHOTO_EDMAC + 4) + 1; /* size N */
height &= ~1;
I tried that on the EOSM which has an odd number height and it is working but I've noticed some issues. First of all to make sure I'm doing this properly, that change needs to be done in raw.c, #ifdef CONFIG_RAW_PHOTO:
/* autodetect image size from EDMAC */
width = shamem_read(RAW_PHOTO_EDMAC + 8) * 8 / 14; /* size B */
height = shamem_read(RAW_PHOTO_EDMAC + 4) + 1; /* size N */
height &= ~1; /* force it to be always even https://www.magiclantern.fm/forum/index.php?topic=19300.1375 */
The GoodOne thing that I didn't expect was that a long press is no longer needed to bring Live View back. This has been one of my pet peeves with FRSP. I tired various Image review settings on the Canon menu and they all work as expected, going back to Live View when the review time is reached. That means that the "Long half-shutter will take another picture." message would no longer be needed.
The Bad and The UglyHowever, when shooting several FRSP and switching between compressed and uncompressed sometimes I'm getting corrupted DNG's. This is only happening with the uncompressed ones, the compressed DNG's are always fine. Not sure what is causing it but it seems that switching from lossless to regular triggers the issue. Starting in uncompressed mode works fine.
Indeed, that's ugly. Does it happen with your approach?
Unfortunately, yes.
Fortunately this can be reproduced with the latest crop_rec_4k from the Experimental downloads page (https://builds.magiclantern.fm/experiments.html) on every camera I tried -- EOSM, 700D and 5D3. (Noticed that the latest experimental was built off the crop_rec_4k_mlv_lite_snd branch.)
Shoot a DNG FRSP, switch to Lossless DNG and shoot then switch back to regular DNG and shoot once again. An uncompressed DNG shot after a lossless DNG will be no good.
Regressed to my pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/858/fix-lossless-compression-on-frsp-for-eosm/diff) version and it was there too--before and after merging in the latest changes and fixing the conflicts. Also present in commit a9c8ad4 (https://bitbucket.org/hudson/magic-lantern/commits/a9c8ad412af5c282bbad6889ce5f65d974ca3ada) which I based my pull request off of and commit 600b490 (https://bitbucket.org/hudson/magic-lantern/commits/600b490fd7ccdc987583009b83c49cb585dc0a2e) which was the merge that introduced lossless compression into the crop_rec_4k branch has it. I would have gone further back but it was getting late.
Found the issue.
(http://a1ex.magiclantern.fm/bleeding-edge/drink.png)
s/drive/code :)
Ha ha!
Thanks for the fix. My EOSM, 700D and 5D3 are happy campers.
Quote from: a1ex on February 04, 2018, 08:58:51 AM
Found the issue.
(http://a1ex.magiclantern.fm/bleeding-edge/drink.png)
s/drive/code :)
Too much wine lately :P
... more than 1 year ago (hg blame src/chdk-dng.c -r crop_rec_4k | grep dng_compression)
That was a subtle bug; you don't see these reported often ;)
For magiclantern-crop_rec_4k.2018Feb04.5D3123:
-straight to resolve the sound is still a bit unsynced like 0.5 sec.
-more serious bug - sound ignores prerec and records since first press.
(https://thumb.ibb.co/cBamJx/prerec.jpg) (https://ibb.co/cBamJx)
Second audio track is the same from clip but inserted separately and synced manually.
Good news - mlv_dump does not jump frame nums anymore.
1920x1080 25FPS
For magiclantern-crop_rec_4k.2018Feb04.5D3113:
-mlv_dump to AE - the sound is still a bit unsynced, about 2 frames (2/25 sec.) for FPS override -on, ExactFPS and Lowlight. The sound is unsynced, about 3 frames for FPS override - off
- mlv_dump does not jump frame nums anymore.
Sorry, I don't edit video, and I have no plans to learn this overnight; will leave it to the community for now. By design, all video and audio frames are timestamped at microsecond level. However, whether your workflows use this metadata or not, I have no idea.
Adding pre-recording support will require implementing the same logic in mlv_snd; with the vanilla version from g3gg0, the easiest thing to do was to simply record all the sound (including the paused sections from the video clip) and let the postprocessing tools figure it out later (from timestamps). The only side effect is a bit of storage space wasted, and a bit of extra burden on the postprocessing tools.
What I can try would be to add an option to mlv_dump to discard the extra audio data (not matched by any video frames), but I'd be surprised if there aren't any post workflows that can already handle this.
having a preview in MlRawViewer with the right sound is also nice,
and being able to do a quick export with sync sound can be very nice for the workflow.
Not all post tools have this option to sync or offset audio. Like the MlRawViewer for example.
I have a couple questions....
Is it true that live preview works when using an external monitor?
Also are there any risks using this? Has anyone actually bricked a camera and not revived it?
Switch, MLVFS, Mlv app, mlvp match the audio file(shortens too long audio) already.
Switch is by far perhaps the safest route you can pretty much take without getting a DUI.
Now pour me down some fine wine and let's get the party started, guys...
kyrobb,
Markus,
lostfeliz,
Thank you very much for your replies. Somehow i missed the fact that AS7sII is 8 bit... My favorite time to shoot is twilight, so A7sII is very tempting. But bit depth is more important for me. There are option to use some external Ninja device to be able to shoot in 10 bit, i haven't figured out it yet.
You are obviously more or less pros in comparison with me, but i honestly don't understand why you think 4k is overrated. I have Macbook 13.3-inch (2560 x 1600), and it's not the biggest. I easily see the tremendous difference in resolution when i watch youtube in 1080 or 1440 or vimeo 2k and 4k. More details is better, it's just more natural for eyes to see more details like we do in life. How can you argue with this? Even Iphone videos are 4k, we just got use to them. I shoot all videos in 4k on iphone because they look so much better and more real than 1080p. Worse scenario - when i shoot something from a moving car, i can't even see what i'm shooting in 1080p, iphone has wide lens. 4k is much better.
Besides, downscale from 4k to 2k will be better than originally 2k (they say, i didn't try myself, but quite obvious to me).
It's quite impossible for me to choose between bit depth and resolution, both equally important for me.
If today we still don't mind watching 1080p videos, then in a couple of years or say in 10-20 years it will be the same as with 480p today - unbearable to watch. It's only my amateur opinion. I'm curious why you think otherwise.
BTW, i love to extract frames from videos and use it as photos. Higher resolution obviously makes a huge difference. (there is a video in youtube where a photographer compares Red camera (75 fps 8k video) to Hasselblad (photo) and gets impressed with the variety of moments (75 photos/sec) that he misses on Hasselblad. So i'm not the only one freak who does it:)
So i use ML 12 bit, 3k crop with framing live view and suffer)) I second to the mentioned question - will external monitor via HDMI give normal live view?
Thank you and have a great day!
Hello again everybody,
I'm sorry for these stupid questions, i hate to interrupt your development and bug fixing process with this, but i've been wanting to know for a while:
In 4k experimental version:
1) When i choose data format between
14-bit
12-bit
10-bit
14-bit lossless
12-bit lossless
11...8-bit lossless
Is "14-bit lossless" worse quality (less data in video) than "10-bit"?
Is "12-bit lossless" worse quality than "12-bit"?
If in 12-bit lossless format my camera is already pushing the limits (close to be automatically stopped), then i should not try anything above this in this list, such as 10-bit, correct?
2) Is 1080p48 a 14 bit or less bit? Should i use this 4k experimental version to shoot 48 fps or 48 fps is available in stable version as well?
3) Is 1080p24/30 14 bit available in stable version?
4) Will buying a faster card than my Komputer bay 64Gb 1000x CF UDMA7 (advertised record speed 130MB/sec) let me achieve better quality? I know there's a limitation in bus.
5) Are there any external devices exist for 5dM3, like those for other cameras, that may help achieve better quality? I don't mind any extra devices on camera, ugliness of camera or extra expenses.
6) Will external monitor via HDMI give normal live view (not framing)?
7 ) I use settings 3k crop x5, 3072x1308, 2.35:1, 12 bit lossless with framing live view from this tutorial https://vimeo.com/217313287
Is this the best what can be squeezed out of 5dM3 and ML, providing that i wouldn't like to go less than 2:35:1 and 3k or about that?
Or i would appreciate any links to some info...
Thank you a lot for your time.
To answer "whysodifficult"'s question about 10bit,12bit,14bit.
14-bit - No edit to the original Canon 14bit Raw File
12-bit - Is 4 LSB Trimmed (nearly lossless - Almost identical to 14bit visually)
10-bit - Is 8 LSB Trimmed (Small loss of detail in shadows - Still Visually similar to 12bit Overall)
14-bit / 12 / 10 lossless - Bitdepth changes with each type but is Losslessly compressed. This Means there is almost no difference between compressed and not compressed, this is because the compression is Lossless. Lossless compression tries to eliminate data that is not visually useful, therefore visually it is the same.
But because Higher ISO's generate more noise this makes it more difficult for the lossless compression method to work.
14bit Losssless Compression - Only works with 100iso or lower. Works well with LiveView but the only limitation is its pretty much locked at ISO100 but you get 14bit RAW :)
12bit lossless Compression - 100iso to 1600iso - This is my preferred Shooting mode as i almost never go above 1600iso, and at lower ISO's 100-800 Compression is usually always more than 50% which is great for longer shoots.
11bit / 10bit / 8bit - This is variable based on the ISO you are using. 11bit up to 1600iso, 10bit up to 6400iso (i think?).
So all in all this latest branch of Compressed RAW is great for reducing file sizes (in camera) without any visual loss.
Or if your keen to try, (14-12bit-10bit lossless) enables higher resolutions at 24p-30p such as 2.5k 3k 4k.
I think pretty soon a close to 3k 16:9 option should be included as a easy setting for continous shooting :)
Quote from: Kyle Kearns on February 05, 2018, 04:53:52 PM
I have a couple questions....
Is it true that live preview works when using an external monitor?
Also are there any risks using this? Has anyone actually bricked a camera and not revived it?
time ago (few release ago) I try the ninja 2 , on hdmi output, like a 2' monitor on 5d3 , and you have same preview of 5d3 not better !
I think you have only a little vantage, with monitor outside 5d3, and lcd in that closed....more long life battery, and 50% low temperature...
Quote from: kizza1234 on February 08, 2018, 03:10:13 AM
To answer "whysodifficult"'s question about 10bit,12bit,14bit.
14bit Losssless Compression - Only works with 100iso or lower. Works well with LiveView but the only limitation is its pretty much locked at ISO100 but you get 14bit RAW :)
This is completely false.
You can shoot from 100-6400, but the compression struggles more the higher your ISO just like 12 bit lossless will struggle more the higher the ISO.
And there is no ISO lower than 100, unless you go ISO_reg.mo and manually lower the Analog gain or ISO Registries or however it is done.
There might be some confusion over how to use "Full-res LV" on cameras other than the 5D3 and @OlRivrRat pointed out a bug in the 100D topic (https://www.magiclantern.fm/forum/index.php?topic=16040.msg196947#msg196947) so I thought I'd run it on the 700D and came up with yet another issue.
Here are the steps, first of all just activate the crop_rec and silent modules to keep the variables down to a minimum.
(https://farm5.staticflickr.com/4609/25287433967_50bfab956f.jpg) (https://flic.kr/p/EwyH7P)
Make sure you have the Canon movie settings on 1280/60 (or 50 for PAL) -- Keep the camera in movie mode don't switch to photo mode or it won't work at all.
(https://farm5.staticflickr.com/4662/39448054334_feaa491540.jpg) (https://flic.kr/p/236TAcQ)
Turn on crop_rec -- there is only one option on cameras other than the 5D3 and that's the 3x3 720p setting.
(https://farm5.staticflickr.com/4743/39448053384_b05e3097f1.jpg) (https://flic.kr/p/236TzVs)
Set the silent options to "Full-res LV" and either DNG or DNG Lossless. Because the issue was found with the intervalometer I'm turning it on though that isn't really the issue.
(https://farm5.staticflickr.com/4653/39448053794_5b934dbdc4.jpg) (https://flic.kr/p/236TA3w)
The first DNG will look like crop_rec 3x3 720p should look like:
(https://farm5.staticflickr.com/4650/39448057134_0ce8c942fc.jpg) (https://flic.kr/p/236TB37)
But after the first shot the crop_rec module will be deactivated and the shots will come out with the regular 720p 3x5 sampling:
(https://farm5.staticflickr.com/4760/25287437947_ef25e10819.jpg) (https://flic.kr/p/EwyJir)
The bug reported on the 100D topic is that DNG's aren't being saved at all when the silent options are set to "Full-res LV" but I'm finding that the 700D and EOSM are saving DNG, DNG Lossless and MLV files fine. The crop_rec module being deactivated by the silent module is something I just now discovered. Not sure where to look for a fix for that, much less the issue that OlRivrRat reported on the 100D.
@dfort:
Full-res LV is just an automation (http://www.magiclantern.fm/forum/index.php?topic=19300.msg188883#msg188883) around crop_rec's preset with the same name, which is only available on 5D3. Check the commit (https://bitbucket.org/hudson/magic-lantern/commits/812b4e5d5413f97cb5f5323ed6dab9876e8c1594) to see how it works:
- enables Full-res LV in crop_rec [ this part fails if that option is not available ]
- refreshes the LiveView
- takes a regular silent picture
- disables crop_rec [ this part works, as all cameras running crop_rec have an "OFF" option ]
There's no easy way to check whether a particular choice is available in the crop_rec menu, other than manually walking the list of choices.
@kizza1234, @whysodifficult:
Lossless = without any loss (14-bit lossless = bitwise
identical to 14-bit raw after decompression, e.g. same MD5 if you want). Removing data not visually useful =
lossy (that's what H.264 does). Reducing the bit depth is also a lossy operation.
Quote from: a1ex on April 15, 2017, 01:12:36 PM
14-bit lossless will give raw data identical to uncompressed 14-bit (100% identical, not one single bit different, contrary to some earlier report (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182741#msg182741)), while "12-bit lossless" should be interpreted as "14-bit to 12-bit conversion - lossy by definition - followed by lossless compression", so it should have the same number of useful levels as uncompressed 12-bit. Please note the 12-bit lossless is not identical to 12-bit uncompressed - they differ by a constant value and possibly by some round-off error, and the same is true for lower bit depths.
Quote from: dmilligan on June 15, 2017, 02:40:04 AM
"Lossless" compression algorithms compress data in such a way that the original data can be reconstructed exactly, bit for bit. "Lossy" compression algorithms compress data in such a way that the original data cannot be reconstructed exactly, but you can come close and the differences are optimized to be things that aren't very noticeable based on human perception. They are two very different beasts.
14 - 12 = 2, not 4.
Read the help text at the bottom of screen in ML menu - it should cover these things (display the current data rate, explain how each bit depth preset is created and so on).
Hi all, sorry for the misinformation A1ex is correct and was wrong about the compression :/
Gets so confusing lossy and lossless haha
12bit lossless is beast!
@Alex
Thanks Much for the Full-res LV explanation. Might be good to remove the Option from Cam's
on which it will not work.
In the 5D3 World is there an advantage to FRSP Full-res LV over FRSP Full-res?
Full-res LV works at fast shutter speeds (up to to 1/15000, but with 128ms rolling shutter) and has trouble at slow shutter speeds (didn't try to push it too far), while the "classic" version works only at slow shutter speeds (with gradient (https://www.magiclantern.fm/forum/index.php?topic=12523.msg121962#msg121962) - fastest shutter speed being between about 1/33" at the top of the frame and 1/6" at the bottom).
They use very different backends (one works in LiveView by modifying the resolution; the other exits LiveView temporarily and uses the still photo capture functions), so their UI behavior is also noticeably different (and it's not trivial to switch between them).
Quote from: a1ex on February 09, 2018, 06:01:56 PM
Full-res LV works at fast shutter speeds (up to to 1/15000, but with 128ms rolling shutter) and has trouble at slow shutter speeds (didn't try to push it too far) ...
With regard to this I have a few questions to you, A1ex:
-----------------------------------------------------------
1) As explained on the Experimental page, this feature works very well on the 5DMkIII which allows continuous recording at 5796x3870 resolution and a speed of 5 fps. I was wondering if this same Full-res LV feature could be implemented also in the 100D, 700D and other similar cameras, at a lower frame rate of course, due to the lower write speed of about 40 MB/s. I remember you saying that 2 fps should be doable. This feature would be great for FRSP time lapses since there is no dead time between the frames recorded and this would result in very smooth time lapses. Combined with Dual ISO, this would be another dream come true for those using these cameras.
2) If the answer to the above question is yes, would it be possible to add a menu option applying also lossless compression, (selectable between 14, 12 and 8 ... 11 bits) to the frames to save card space and pack whole image sequences in MLV containers to greatly enhance sorting? If also a change in the file name is made, say a letter "T" for time lapse, this will made file sorting even easier. The same question applies also to shooting single FRSP DNGs using IS lenses that you recently added to the above cameras - I love this feature! If also such picture sets, shot at different locations, could be packed in MLV containers with a different letter in the file name, say "S" for picture sets consisting of various single LL DNGs, this will save the photographer a lot of time sorting his pictures according to the location at which the picture set was shot.
3) Right now, the thumbnails of the single FRSP-DNGs show up as black rechtangles in Explorer. Is it possible that they show up as small picture thumb nails to make finding a specific picture easier? This works with Cannon's CR2 files. Is it possible to make it work also with ML DNGs, keeping in mind that they use the same type of lossless compression?
Thank you in advance for your answers.
1) Yes, but it has to be done by somebody who has the camera in his hands, as it's model-specific and mostly trial and error with the adtg_gui module. For 700D, mk11174 got very close (https://www.magiclantern.fm/forum/index.php?topic=19300.msg182818#msg182818). Some (https://www.magiclantern.fm/forum/index.php?topic=19336.msg196400#msg196400) more (https://www.magiclantern.fm/forum/index.php?topic=21427.msg195448#msg195448) hints (https://www.magiclantern.fm/forum/index.php?topic=12523.msg120635#msg120635) around (https://www.magiclantern.fm/forum/index.php?topic=16608) the (https://www.magiclantern.fm/forum/index.php?topic=10111.msg191218#msg191218) forum (https://www.magiclantern.fm/forum/index.php?topic=16040.msg191039#msg191039) (and first post from this thread, of course).
2) Yes, but these are very easy coding tasks (anyone can give it a try); I prefer to focus on the more difficult stuff.
BTW - now that lossless compression works well on the cameras that support it, I was thinking to remove the uncompressed options, to keep the code simpler.
3) The DNGs do have previews, but they are uncompressed (uncommon); Explorer may expect them to be plain JPEGs? It would require additional reverse engineering to understand how to create JPEGs. The same routines, if figured out, would also provide MJPEG recording.
(the previews do show up in Geeqie, which is what I use for sorting pictures)
Quote
BTW - now that lossless compression works well on the cameras that support it, I was thinking to remove the uncompressed options, to keep the code simpler.
I'm all for simplifying. Question though. Stabilitywise, are the similar now? Not filming a whole lot these days I remember early stops with lossless due to specific conditions, too bright scenery, hdr recording. Did not encounter this with uncompressed modes. However, maybe separate compressed and uncompressed in two branches meaning crop_rec_4k solely lossless and 10bit_12bit branch compressed? My two cents.
Please keep the uncompressed as its the one format that is suppored by all the post software since the beginning of magic lantern, And I have had more problems with compressed than uncompressed.
Also uncompressed is probably faster to display as there is no need to decompress.
And for higher isos there is no worry to reach the compression limit.
please keep it 8)
I fully agree - MLV_REC is now redundant. I still would leave the 10, 12 and 14-bit UNcompressed options in the RAW video mode just in case we experience some instability as Danne pointed out.
Geeqie seems to be a working option on Mac and Linux but not for Windows, unfortunately? Are you aware of a similar software for Win?
We were talking about silent pictures, but probably it's not a bad idea for video either. Still, for video we've got a problem with older models - I was unable to get lossless compression in LiveView on 60D, for example, so that would break the compatibility.
For silent pictures (not video), Greg was able to get lossless compression working outside LiveView on 500D, to some extent, though we did not try too hard to decode the pictures. So, at least outside LiveView, I'm confident lossless compression can be ported on all models.
BTW, updated ISO research modules (https://builds.magiclantern.fm/modules.html#iso-research) to load on the latest crop_rec_4k builds (including 100D) without having to compile from source. Will test later. The raw_diag module should also load on lua_fix, as it has no special dependencies, but adtg_gui requires the patch manager.
Quote from: a1ex on February 10, 2018, 02:01:40 PM
BTW, updated ISO research modules (https://builds.magiclantern.fm/modules.html#iso-research) to load on the latest crop_rec_4k builds (including 100D) without having to compile from source. Will test later. The raw_diag module should also load on lua_fix, as it has no special dependencies, but adtg_gui requires the patch manager.
Yes, I always wanted to test these miracle crooked ISO numbers that provide lower noise and improved dynamic range but never knew how to do that on the 100D. I will start reading on that as soon as I can and hope to be able to test soon. I will also be looking forward to your findings on this.
Quote from: a1ex on February 10, 2018, 12:59:23 PM
.... For 700D, mk11174 got very close (https://www.magiclantern.fm/forum/index.php?topic=19300.msg182818#msg182818). ...
@a1ex I can't see C0FO[****] Registers, the others shows very well.
What should I do to see these C0FO Registers?
I use latest build from 4k branch on 700d.
For 700D, define ENGIO_WRITE_FUNC and ENG_DRV_OUT_FUNC in adtg_gui.c (copy from stubs.S; they used to require fiddling, but I hope it's no longer the case), then select Advanced -> DIGIC registers.
Currently trying to sketch a walkthrough; here's the first draft (still editing).
Here's another adtg_gui walkthrough (https://www.magiclantern.fm/forum/index.php?topic=10111.msg191218#msg191218) for a different purpose; the basics are still valid.
Background:
- What are these registers? (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/HACKING.rst#rst-header-basic-concepts) (TLDR: that's how the ARM CPU interacts with other peripherals - by writing to / reading from registers)
- What are ADTG and CMOS registers? (https://www.magiclantern.fm/forum/index.php?topic=6751) (TLDR: CMOS = sensor controller, ADTG (http://magiclantern.wikia.com/wiki/ADTG) = ADC + timing generator, ENGIO = Canon's image processing engine)
For changing the LiveView resolution, we are interested in:
- ENGIO registers 0xC0F06800/4 on D5, 0xC0F06084/8 on D4 (raw.c: raw_lv_get_resolution); these adjust the resolution of the captured raw buffer (it's the first set of registers we have to tweak, but it's not enough)
- Video timer (http://magiclantern.wikia.com/wiki/Register_Map#Video_timer) aka FPS override (http://magiclantern.wikia.com/wiki/VideoTimer) (these are directly related to resolution; we did not know that at first, but found out later)
- CMOS registers (to select what image area to scan, and also the column binning factors)
- ADTG registers (line skipping factor and some others)
The number of registers is huge (overwhelming). Hundreds? No. Thousands. Many thousands. Address space covers millions of possible registers. We only understand a tiny part of them (http://magiclantern.wikia.com/wiki/Register_Map). Don't be discouraged.
The logging code has noticeable overhead and may push LiveView beyond its limits. Be prepared to encounter crashes or corrupted images, and be prepared to search for workarounds. For example, on 5D3, I can switch from 50p to 25p with logging enabled, but not the other way around. Enabling logging in x5 mode gives corrupted frames, unless I reduce the FPS. There is an option named Force low FPS, just for that.
Let's start with some basics, for example FPS override.
There are two timers: A and B. FPS is MainClock / timerA / timerB. MainClock is TG_FREQ_BASE in fps-engio.c (24MHz on 5D3, 32MHz on 100D). Browse the FPS override submenu in a few video modes, compute the FPS from timer values and make sure you understand the math before proceeding.
Now try changing FPS from adtg_gui. Timer A is 0xC0F06008, B is 0xC0F06014. Start from a low-FPS video mode (24/25p), turn off FPS override, enable ADTG registers, enable DIGIC*) registers, select to show Image size regs only. Go to PLAY mode and back - the registers are identified only when Canon code changes them. Locate the two registers in the menu.
*) actually ENGIO - it wasn't clear to me when I wrote the module. I should rename it.
On 5D3, in 1080p25, you will see: C0F06008 = 0x1df01df, C0F06014 = 0x7cf. Do the math to get 25 FPS. Do not skip this step.
Now increase the value of timer B (e.g. 0x7df). Go to PLAY mode and back to apply the changes (they are only applied when Canon code changes these registers again). Notice the FPS value printed by ML on the top bar changed.
Do the same with timer A (for example, change it to 0x1ef01ef). Notice there are some more registers with the same value (0x1df01df or 0x1df). Update them too.
Level 1 complete - back to theory.
Timer A is directly related (https://www.magiclantern.fm/forum/index.php?topic=19300.msg196374#msg196374) to horizontal resolution. Timer B is directly related to vertical resolution. They are not the same, but if you increase one, you may also have to increase the FPS timers.
Now that we know how to adjust stuff, let's find out what registers are relevant to image resolution. The first step is to look at the registers configured by Canon code, save them to a log file (https://www.magiclantern.fm/forum/index.php?topic=16608.msg175392#msg175392) and compare them (https://www.magiclantern.fm/forum/index.php?topic=16608.msg176768#msg176768).
For example, let's compare 1080p25 with 720p50. On 5D3, the raw resolutions in these two modes (usable in mlv_lite) are 1920x1290 and 1920x672 (same resolution horizontally, different vertically). Full buffers (including black bars) are 2080x1318 and 2080x692 (edmac.mo -> Show EDMAC channels -> raw_write_chan in edmac-memcpy.c -> channel 4 on 5D3 and 18 on other models, or Debug -> Show console).
Unfortunately, on 5D3 I can only compare the ADTG/CMOS registers (without the DIGIC/ENGIO ones), as it locks up when trying to log the latter in 50p. It might work on other models, or it might not. We'll still get useful info out of this.
Step by step:
- select 720p50 in Canon menu
- enable ADTG Registers
- go to PLAY mode and back to get all the registers
- select Show -> Modified from now on (also displayed as "since HH:MM:SS")
- go to PLAY mode and back, make sure it shows nothing (if it does, repeat a few times)
- select 1080p25 in Canon menu
- go to ADTG registers menu and notice some registers appeared
- to double-check: go back to 720p50 and make sure it shows nothing
- back to 1080p25 and the registers should appear again
- select Advanced -> Log Registers Now (to save the modified registers to a log file)
That's the general procedure for "diff-ing" two video modes.
There are already a few dozens of registers, but you can ignore some of them:
- shutter blanking: that actually gives the shutter speed
- ADTG 8/9/a/b and 8882/4/6/8: ISO gains
- likely most other registers with "small" values
The interesting stuff:
- ADTG2[800c]: line skipping factor (2 in 1080p and 4 in 720p)
- ADTG2[8178, 8179, 82F8, 82F9, 8196, 8197]: likely related to vertical resolution (look them up in crop_rec source)
You may also diff the 1080p and x5 zoom modes - this should work with DIGIC registers as well, without locking up. There you'll notice registers 0xC0F06800/4 for adjusting the capture resolution, and many many others.
Finally, you can compare the x5 zoom with the regular photo mode. This may require some fiddling - I've got it working by using plain FPS override set to 10 fps to avoid image distortion, rather than adtg_gui's "Force low FPS" (which appears to interfere with photo mode).
Level 2 complete. You may now examine the differences between Canon video modes, look up the registers in adtg_gui.c (https://bitbucket.org/hudson/magic-lantern/src/iso-research/modules/adtg_gui/adtg_gui.c) and make some guesses about their possible meaning.
TODO: logs.
Let's now try to adjust the horizontal resolution. We've already got some 20MB (https://www.magiclantern.fm/forum/index.php?topic=5601.msg196632#msg196632) reserved for the raw buffer from Canon code, so we don't need to worry about memory yet. Let's start from some 1:1 crop mode (x5 zoom or something else).
Horizontal resolution is adjusted from the low halves of 0xC0F06800/4. First do the math for the current video mode (raw_lv_get_resolution). Enable ADTG Registers and DIGIC Registers to see them, then select "Show: Image size regs only" to find them faster, then go to PLAY mode and back to refresh all the registers. Select "Force low FPS" if the image is broken.
Examples:
5D3 1080p: [0xC0F06800] = 0x10017, [0xC0F06804] = 0x528011b => 2080x1319 (column_factor (https://www.magiclantern.fm/forum/index.php?topic=19300.msg196374#msg196374) = 8 for 5D3, 4 for 100D).
5D3 x5: [0xC0F06800] = 0x10017, [0xC0F06804] = 0x56601eb => 3744x1381.
Increase horizontal resolution in x5 by 1 step (8 pixels on 5D3) => change 0xC0F06804 to 0x56601ec. Go to PLAY mode and back to x5 to apply the change. Notice the image is "slightly" distorted (some sort of diagonal look). This is good!
Enable raw video (mlv_lite), set preview to Framing, set image resolution and aspect ratio to maximum.
Before the change: 3584x1320.
After: 3592x1320.
Level 3 complete. You have successfully increased the resolution in x5 mode by a whopping 8 pixels!
(the real-time preview is broken, don't know how to fix it yet, but the preview looks good with the Framing option, so the raw data is valid)
Still with me?
Impressive findings, maybe scary for ordinary users.
April 1st 2018 is the Benchmark ;)
Quote from: a1ex on February 10, 2018, 05:36:34 PM
Still with me?
Finally I have defined ENGIO_WRITE_FUNC and ENG_DRV_OUT_FUNC, I was having some issues with compiling and understanding somethings.
I will reply as soon as I understand other things :D
Thanks for this amazing explaining!
Will it be possible for resolutions above 2k to have working full live view (other than 5x)? Or is it a hardware limitation?
Thanks heaps.
Asked and answered (https://www.magiclantern.fm/forum/index.php?topic=19300.msg183706#msg183706) (also 2 posts above yours).
TLDR: it's not hardware limitation, but our own lack of knowledge (http://wiki.magiclantern.fm/faq#why_is_it_so_difficult_to_change_the_liveview_resolution_when_elements_like_bit_rate_control_and_hdr_video_seem_to_be_relatively_easy_to_implement).
Okay a1ex for Level 1
BaseClk = 32 MHz = 32000000 Hz (Canon 700D)
Timer A = 639 | Timer B = 1999
clk1 = 32000000 / (639+1) = 50000
fps = 50000 / (1999+1) = 25fps
I get the values from my camera C0F06008 = 0x27f027f & C0F06014 = 0x7cf (Same as 5D3).
But how I can calculate these values without FPS override (I get timer A & B values from FPS override) How is 0x7cf = 1999 what the math ?
EDIT: I understood that is 0x7cf a Hexadecimal value.
After tweaking C0F06008 & C0F06014 I got 34.04 from 25fps which is nice experiment, I tweaked even more, I end with around 42 to 45 fps with working liveview but the image was more darker comparing to 25fps (liveview freezing & corrupted frames at 45fps while recording, at 42fps was fine) .
kizza1234
jimiz
Kharak
a1ex
Thanks for your answers!
It's pity we can't extend hi-res possibilities with Ninja-like device. (I'm watching 60 fps videos in youtube, such a smoothness, but i'm dreaming))
Was interesting to know A1ex's answer about normal LV on hi-res.
Well then i'll try uncompressed modes as well, maybe there will be difference for me as well.
The last improvements are fantastic!!!! MLV_lite with audio recording!!!!
I follow this topic from message number 1 and the amount of work seems to be giant. Thanks to everybody!!!
Even although I read all the messages related to the new audio implementation, I have a doubt. My audio is still out of sync. I'm using MLVFS to get the DNGs and using the last ML version (5feb) in a 5Dmk3. Is this a bug to solve? or is a problem with my particular workflow? (MLVFS to get DNGs and direct import to PremierePro CC2018).
Thanks.
Sorry, one more question.
I can't make to work H264 proxy recording when I use mlv_lite+audio. It stops working in few seconds. Is it normal, or its me?
Thanks.
Quote from: pc_bel on February 14, 2018, 10:13:49 AM
...Even although I read all the messages related to the new audio implementation, I have a doubt. My audio is still out of sync. I'm using MLVFS to get the DNGs and using the last ML version (5feb) in a 5Dmk3. Is this a bug to solve? or is a problem with my particular workflow? (MLVFS to get DNGs and direct import to PremierePro CC2018).
Thanks.
Audio records a bit extra before first and after last frame (which is may be helpful), I use mlv_dump, import video and audio into Resolve separately then sync manually. A bit longer but fine by me.
Quote from: SanchYESS on February 14, 2018, 04:26:18 PM
Audio records a bit extra before first and after last frame (which is may be helpful), I use mlv_dump, import video and audio into Resolve separately then sync manually. A bit longer but fine by me.
SanchYESS,
When you sync manually the clips in resolve are they all synced at the same point for any clip length? For example do you just slide the audio back 10 frames for each clip? Or is it a different point for each clip?
Hopefully its the same offset so we can use one of the mov converters that can offset the audio...
Thanks!
Need opinion please.
I know they are the same performance with ML, but wich is the best choose EOS M, 100D or 700D ?
Thanks SanchYESS!!! :)
Hope it has solution, because syncing every clip is time consuming if you have thousands of them :-[
Maybe I can edit and then sync all clips together... If all have the same offset...
Quote from: a1ex on February 04, 2018, 09:27:45 PM
Sorry, I don't edit video, and I have no plans to learn this overnight; will leave it to the community for now. By design, all video and audio frames are timestamped at microsecond level. However, whether your workflows use this metadata or not, I have no idea.
Adding pre-recording support will require implementing the same logic in mlv_snd; with the vanilla version from g3gg0, the easiest thing to do was to simply record all the sound (including the paused sections from the video clip) and let the postprocessing tools figure it out later (from timestamps). The only side effect is a bit of storage space wasted, and a bit of extra burden on the postprocessing tools.
What I can try would be to add an option to mlv_dump to discard the extra audio data (not matched by any video frames), but I'd be surprised if there aren't any post workflows that can already handle this.
Offset is ufortunately slightly different.
I currently have max 133 clips but barely the third of them needs to be synced so I can do it manually (and I have separately recorded audio so anyway I have to) but it would be great to add option to sync audio while converting with mlv_dump. Anyone knows if some other tool can use timestamps to sync?
Is audio off in the beginning? Could you upload a sample MLV with unsynched audio? Let's check time stamp.
By the way. What converter are you using to process dng files? Mac, Win? You mention mlv_dump. Tried Switch, Mlv app, mlvfs, mlvp, raw2cdng to mention a few...
Hi Danne,
I can send you a out of sync mlv file.
I send you a PM.
Thanks.
Quote from: anto on February 15, 2018, 03:56:34 PM
Need opinion please.
I know they are the same performance with ML, but wich is the best choose EOS M, 100D or 700D ?
If the size does not matter to you:
For photography EOS M doesn't have built-in flash, both EOS M & 100D can shoot continuous 4fps, the 700D can shoot up to 5fps + it's have fully articulated screen which is very useful feature to see what's happening on the screen without external monitor in different types of shooting like low/high angle and shooting yourself ..etc.
In 1080p (actual resolution will be 1736x976) RAW video mode for EOS M the camera shoots with 5x3 bining pattern which means you will have more aliasing, moire and losing in vertical resolution, the solution was to enable crop_rec module, and re-enabling 3x3 bining pattern but you will get very wide screen (1736x688) unlike Canon 100D & 700D you can shoot normally in 3x3 bining without this things.
This the only differences, I went for 700D only for the screen :D
You can ask more questions in the right place at General Chat thread (https://www.magiclantern.fm/forum/index.php?board=26.0)
Got a MLV from pc_bel so I thought we´d look at metadata. Are we looking at timestamp of AUDF and VIDF? If so.
First audio block:
Block: AUDF
Offset: 0x12b5ab00
Number: 187
Size: 38656
Time: 9.071000 ms
Frame: #0000
Space: 232
First video block:
Block: VIDF
Offset: 0x00000600
Number: 13
Size: 1815040
Time: 89.100000 ms
Frame: #0000
Crop: 152x132
Pan: 152x133
Space: 32
Audio starts:
Time: 9.071000 ms
Video starts:
Time: 89.100000 ms
89.100000-9.071000=80.029
Audio starts 80.029 ms too early, is this correct information? Is audio always early or could video block start before audio under any circumstance?
@Danne
I'm guessing that, since You Wizards of ML Are Way More Savvy than I on
the Inner Workings of Our EOSs, the following has already been considered >
"Audio starts 80.029 ms too early, is this correct information?"
I would guess that the Audio Starts Being Recorded "On Time" & that the Video
Starts Being Recorded "Late", due maybe to the complexity of both & the time it
takes for the Processor to Deal With the Extra Complexity of the ML Vid'.
Could it also be that the Audio Recording is being handled "By Canon" while
Video is being handled "By ML".
Probably obvious that I possess No Savvy about this stuff > Just some ideas
I thought I'd throw out there.
ORR ~ DeanB
QuoteIs audio always early or could video block start before audio under any circumstance?
I'm searching in my mlv files and at this time, all the clips have de audio late. Keep on searching.
Well,
After viewing 30-40 mlv files, few have audio in sync and the rest are out of sync. Always audio late. All clips are recorded same day with same adjustments in camera, so I can't explain why the differences in audio sync. ::) :-[
By now only solution is sync them in post. Hope some dev find a solution.
Thanks, specially to Danne for his interest!!!
Took a closer look and hopefully managed to get a slightly better audio sync (https://bitbucket.org/hudson/magic-lantern/commits/7fcd17733b77543fa771e0dde40f7187f471fc55). Caveat: I only know this from debug messages; I'm afraid I don't have the equivalent of pixel peeping skills on the audio side (to tell whether audio is in sync or off by some split-second), sorry.
If my understanding of the video and audio pipelines is correct, then:
- video frames are timestamped as soon as the image is read out (evfReadOutDoneInterrupt); the capture process probably happens a little earlier, unsure about the exact timing
- audio frames are timestamped right after StartASIFDMAADC (when the audio recording starts)
- first audio frame is now started shortly after first video frame was captured (~ 2ms in my tests, but I had the trace module active)
It would have been probably better to timestamp the beginning of the exposure, but finding that requires either polling the EDMAC channel that brings the raw image into RAM, or a better understanding on how LiveView works, or - the easiest way - estimating it from FPS and rolling shutter info. Are these ~20ms or so important when it comes to audio sync?
Quote from: a1ex on February 17, 2018, 11:16:36 PM
Are these ~20ms or so important when it comes to audio sync?
Finally, a question I might be able to answer with some degree of confidence. When syncing film dailies we did our best to get sync to the nearest perf. There are 4-perforations per frame of 35mm film so at 24fps that's the equivalent of about 10.5ms, right? Well nobody can actually tell if sync is off unless they examine the waveform (audio pixel peeping) up to about 1-frame so that would be 42ms. When we went digital we were willing to accept sync within 2-frames which the average person can't detect so that's 84ms. I've worked on TV movies where the dailies came in with the sync off 3-frames and that didn't seem to bother anyone except the assistant editor--that would be me.
So no, those ~20ms or so variations shouldn't matter. That's not even a frame out of sync.
Flicks (https://github.com/OculusVR/Flicks)?
Anything within 29429400 flicks is golden. ;D
@A1ex:
2 ms is more than enough for synch. Thanks a lot for fixing.
I added millisecond cutting support in Switch thanks to ffmpeg so audio beginning should be spot on now when processing MLV files into dng files. Should work for all MLV files but needs to be tested.
Commit:
https://bitbucket.org/Dannephoto/switch/commits/a253b813e80b7c4048799847f9e58681952b343e
Download section:
https://bitbucket.org/Dannephoto/switch/downloads/
*Update
Not working yet! Will fix the bug soon...
FIXED!
The module adtg_gui.mo needs to be fixed with latest updates.
(https://image.ibb.co/hfrjy7/VRAM0.jpg) (https://imgbb.com/)
Thanks A1ex for fixing the audio delay!!!!
Still not tested but I'm sure is ok.
Thank you!!!
Question to A1ex,
===========
This has probably been discussed already but I couldn't find where. So, I apologize for asking.
The 3K/4K_crop_recording mode is fantastic since it provides high-resolution RAW video with superb quality and detail, however, it has one serious drawback. The crop factor imposes some serious restrictions on the lenses used and the videographer needs high-quality fast wide angle lenses to be able to compensate for it. These are expensive. On the other hand, if one films in-door at low light, excellent low-cost lenses like the 50/1,4 or 85/1,8 may become too long and unusable due to the crop factor.
I have been going through the first post of this thread as well as the explanation concerning 4K RAW recording on the Experimental page and there are two statements about the 5DMkIII that I got very interested in:
1) Continuous: 3072x1308 @ 24p, 1920x1080 @ 45p, 1920x960 @ 50p, 1920x800 @ 60p, 4096x2560 @ 8p, 5796x3870 @ 5p
2) Full-width LiveView - decrease vertical resolution in the crop_rec submenu, all the way to 5796x400 @ 48 fps :)
My question is:
=========
If the 5DMkIII can record in 4096x2560 @ 8 fps and 5796x400 @ 48 fps wouldn't it be possible to make it record at a more usable and higher than FHD resolution and 24 fps WITHOUT the crop factor or with a much smaller crop factor? This is possible already at 1920x1080 (FHD) resolution without aliasing. What about 3072x1728@24 fps or 4096x1744@24 fps for a limited time of course? Or maybe 2900x1812@24 fps for continuous recording?
Additional question:
---------------------
The maximum vertical resolution of the 7D in the 5x-magnification mode is 1200 pixels while it is only 1080 in the 100D. What restricts this number to 1080 in the 100D if both sensors are APS-C and have about the same pixel count?
Thank you in advance for your reply.
1) From first post:
* 3072x1920 @ 24p (1:1 crop)
* 4096x1440 @ 25p (1:1 crop)
You can further crop any of these from the raw video options.
Column binning factor is limited to either 1 or 3; line skipping/binning factor is a bit more flexible, but... (https://www.magiclantern.fm/forum/index.php?topic=17021.msg165204#msg165204)
2) Without crop_rec, all video modes are configured by Canon code (whatever resolution they provide, is the maximum we can record).
Wanted to try out the slightly better audio sync changes on the 6d, but got some errors.
(Compiled the build myself, so not sure if I did everything right, but I guess that's not the problem here.)
On screen it says, 'Data corruption at slot 36' for every frame.
This error only happens with lossless recording options.
Everything is fine with normal 14 to 10 bit recording.
Unfortunately I can get the 'screenshot after 10seconds' option to work.
Here's the 'Data corruption at slot 36' for every frame error log
Error logs:
ML ASSERT:
0
at mlv_lite.c:2630 (compress_task), task compress_task
lv:1 mode:3
compress_task stack: 1e4340 [1e43d0-1e33d0]
0x0044C93C @ b7f0a4:1e4370
0x0044C478 @ 44c998:1e4340
Magic Lantern version : Nightly.2018Feb20.6D116
Mercurial changeset : 02e5918a6ed5 (crop_rec_4k_mlv_lite_snd)
Built on 2018-02-20 19:00:42 UTC by magic_lantern@Vasses-iMac.
Free Memory : 343K + 2335K
And the camera crashes when I look into the 'Free memory dialog', here's the crash log.
ASSERT: hSuite != 0
at RscMgr.c:2228, guess_mem:ff0f41ec
lv:1 mode:3
guess_mem stack: 1e32f8 [1e33c8-1df3c8]
0xUNKNOWN @ ea80:1e33c0
0x0045C15C @ 44db3c:1e3378
0x0045C0E8 @ 45c1f4:1e3368
0x0045BFC8 @ 45c13c:1e3358
0x004A3150 @ 45bfe8:1e3350
0x00001900 @ ff0f41e8:1e3330
0x0044C478 @ 44c4f4:1e32f8
Magic Lantern version : Nightly.2018Feb20.6D116
Mercurial changeset : 02e5918a6ed5 (crop_rec_4k_mlv_lite_snd)
Built on 2018-02-20 19:00:42 UTC by magic_lantern@Vasses-iMac.
Free Memory : 336K + 2267K
Thanks for the updates on sounds syncing A1ex and everyone that helped! I have done a few tests. It looks like it is much closer to sync now, but it seems to me that the audio track is 2 frames early. I need to manually adjust each audio track 2 frames forward to get it right on. I am using 113 firmware with crop record 3.5k enabled with sound. I am also using MLVFS for processing. Anyone else getting the same result?
I hope this helps. Let me know if there is something else I can test to help narrow down the offset.
Thanks!
Rob
Quote from: Levas on February 20, 2018, 08:43:57 PM
(Compiled the build myself, so not sure if I did everything right, but I guess that's not the problem here.)
Actually it looks like it is. The 6D hasn't been merged into the crop_rec_4k branch yet. I've got an open pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/875) but haven't merged in the mlv_lite_snd changes to it yet.
Quote from: rob_6 on February 20, 2018, 10:43:43 PM
Thanks for the updates on sounds syncing A1ex and everyone that helped! I have done a few tests. It looks like it is much closer to sync now, but it seems to me that the audio track is 2 frames early. I need to manually adjust each audio track 2 frames forward to get it right on. I am using 113 firmware with crop record 3.5k enabled with sound. I am also using MLVFS for processing.
I have tested 1920x1080@25fps, 1/60, 14bit,12bit,10bit, 14lossless,12lossless,10lossless, FPS-off, 5DMkIII 113 firmware. Sinc is better than 1 frame. magiclantern-crop_rec_4k.2018Feb17.5D3113, mlv_dump.exe
Tested!
Audio for me is now in sync!!!
5Dmk3 1.1.3 Crop_recFeb17th
Thanks.
Looking closer, audio is 2 frames before images as Rob_6 said.
Much much better than before. Thanks.
Quote from: a1ex on February 20, 2018, 08:16:36 PM
Column binning factor is limited to either 1 or 3; line skipping/binning factor is a bit more flexible, but... (https://www.magiclantern.fm/forum/index.php?topic=17021.msg165204#msg165204)
Thank you for your reply, А1еx.
What about a combination of the two binning/line skiping factors 1 and 3? Wouldn't it be possible to implement an algorithm that randomly alternates between 1 and 3 during sensor sampling? This technique is commonly used for weighting interdigital transducers and reflector gratings in surface acoustic wave (SAW) devices to achieve certain filter or reflection characteristics and/or to suppress unwanted interference. It is also used for precise adjustment of the center frequency in narrowband SAW devices such as resonators which is hard to do due to discrete step size of the pattern generator for photo masc fabrication. If for example, a resonant frequency of 433 MHz is needed, the pattern generator can provide only 430 or 440 MHz due to its discrete step size. The solution is a random combination of these two consecutive steps to achieve precisely the desired 433 MHz. Some of the transducer/reflector vertical strips are sampled at 440 MHz and the rest - slightly more often at 430 MHz. In my opinion, if implemented in sensor sampling, this technique will not only solve the crop factor problem but also reduce moire and aliasing when line skipping. Just a thought.
Not sure if this parameter can be altered during image capture, and even if it does, it would have very tight timing requirements. Normally, the main CPU configures the exposure parameters, starts it, and waits until it's finished (actually it has a lot of other tasks to run during this wait).
We have no datasheet for the ADTG chip (the timing generator that handles line skipping/binning); all we know about it is what we have found by trial and error (adtg_gui), so I have no idea where to start in order to configure the chip for variable line skipping (or whether it can do that).
Compressed Raw + Audio works on 5D3 1.2.3. I have not nitpicked at synchronicity, just did some initial tests to see if sound worked, I could not see any issues there.
Have not tried with 3.5k, I suppose the audio will decrease the recording time of Crop Modes?
And for anyone else that might come across this issue. I couldn't get the Audio meters to appear at first, meaning my sound recording was not on, even though Sound recording was on in Raw menu. It turned out that Audio Recording was disabled in Canon Menu. I don't know how, I never touch Audio option in Canon Menu. Is this something the previous version of MLV_Lite.mo did?
This new release is really nice, being able to shoot with sound recording with Lossless Raw, will save so much space. I kept sticking to mlv_rec because I needed the sound. Thank you to all developers/testers.
From help of this (https://www.magiclantern.fm/forum/index.php?topic=19300.msg182818#msg182818) and this (https://www.magiclantern.fm/forum/index.php?topic=19300.msg182819#msg182819) and a1ex (https://www.magiclantern.fm/forum/index.php?topic=19300.msg197098#msg197098)I was able to got 5208x2624 2:1 aspect ratio at 6.527fps on 700D!
(https://preview.ibb.co/fN2vdH/IMG.jpg) (https://ibb.co/gWUadH)
(https://preview.ibb.co/bOXiXc/IMG.jpg) (https://ibb.co/jr4adH)
Here is the settings:
(https://preview.ibb.co/hHuSQx/Sett.jpg) (https://ibb.co/jDeu5x)
The actual ADTG2[8172/8] N values is 6fd or 67c and you have to set it 8fd (I don't know why it's looks different from the outside)
And the new register that fixed the distorting in the vertical resolution (https://www.magiclantern.fm/forum/index.php?topic=19300.msg182819#msg182819) is ADTG2[82b6]
I couldn't go further than that because of the memory? When I did I got "too small srm memory" on the screen and this:
ML ASSERT:
0
at ../../src/raw.c:664 (raw_lv_realloc_buffer), task shoot_task
lv:1 mode:3
shoot_task stack: 1c5c40 [1c5e20-1c3e20]
0x000CD8E8 @ ace74:1c5db0
0xUNKNOWN @ cd93c:1c5d98
0x00AC399C @ ac5ec0:1c5d28
0x000C3CBC @ ac39ec:1c5d18
0x000C3C7C @ c3cc0:1c5d00
0x000C32D0 @ c3c94:1c5cf0
0x000C107C @ c3370:1c5c90
0x0007F60C @ c121c:1c5c70
0x0007EF78 @ 7f668:1c5c40
Magic Lantern version : Nightly.2018Feb11.700D115
Mercurial changeset : d6369a63e04b+ (crop_rec_4k_mlv_lite_snd)
Built on 2018-02-11 11:04:05 UTC by Bilal4Fakhouri@WORKS.
Free Memory : 124K + 1895K
So is that the hardware limitation?
Also even when I changed it to 2.5fps (write speed of 33mb/s) I have not got continues recording.
And 3072x1306 aspect ratio 235:1 at 20fps (I was able to reach 23.976 but with some corrupted frames and I still working to get it right)
(https://thumb.ibb.co/dHvWJH/res.jpg)
(https://thumb.ibb.co/e8Te5x/li.jpg)
(https://preview.ibb.co/idF95x/3k_sett.jpg) (https://ibb.co/cNufCc)
The actual ADTG2[8172/8] N values is 67c or 8fd and you have to set it 77c (I don't know why it's look different from the outside)
@a1ex Are the FPS Timers only the way to control or decrease /increase the fps?
Quote from: theBilalFakhouri on February 22, 2018, 05:21:36 PM
I couldn't go further than that because of the memory? When I did I got "too small srm memory" on the screen and this:
This is great news - it means the safeguards regarding raw buffer size are working!
When the image size exceeds the amount (what we believe) Canon code has reserved for the raw buffer, the raw backend should allocate a full-size image. A possibly cross-platform solution (edit: committed (https://bitbucket.org/hudson/magic-lantern/commits/7ad3754768b26a106d057550ca359089ba837fcf)) is something like this (https://www.magiclantern.fm/forum/index.php?topic=19336.msg194165#msg194165):
#define RAW_LV_BUFFER_ALLOC_SIZE (SRM_BUFFER_SIZE - 0x1000)
That's because SRM_BUFFER_SIZE (the largest contiguous block we can get from Canon's memory manager) matches the full-res image size (as 14-bit uncompressed; actually it's a tiny bit larger than that). You need to use a slightly smaller amount because our memory backend also has some overhead.
Yes, the two FPS timers are the only ways to control the FPS. The nearby registers might be useful to break some limits, but their exact meaning is unknown.
Another helpful change is in raw.c - by default, the optical black areas are checked for consistency (e.g. to avoid wrong black level for corrupted frames, which are usually transient). In autodetect_black_level, just set *black_mean to 2048 and *black_stdev_x100 to some nonzero value, without doing all the math - this will help previewing raw buffers where only half of the image is valid, for example.
Interesting..
Sorry to interrupt you guys, is it just me or the digic peaking isn't working for 5D 1.2.3?
indeed great news. Keep up your work.
Crossing fingers. Maybe a PR with some useful "presets" would make 700D owners happy - though it's not 1st April yet.
Good Job ;)
Quote from: RTI on February 22, 2018, 09:01:19 PM
Sorry to interrupt you guys, is it just me or the digic peaking isn't working for 5D 1.2.3?
It works for me.
@theBilalFakhouri
Good work 8)
I'm trying to get higher resolutions on the 6d.
But either way my camera freezes or I get super scrambled live view previews :P
How did you go to work, just trial and error, did you use logical order of changing registers ?
I tried with calculating and changing the stuff all the way...but that always results in horrible scrambled live view feed.
And changing step by step, ended most of the times in a freezed camera
If other 6d users are willing to try, here are some interesting register values:
For full sensor readout at about 4/5 fps use:
C0F0[6008] = 5970597
C0F0[6014] = e8d
Excuse me, big trouble to understand well how use correct this exp. version of ML.
canon 5d3 fw123
well I installed the last release of Feb. but I not understand and IF ML work fine in my Canon or there any errors meaby in installation files...
Setting dir is delete...all the another files (ML) are overwrite and all correct.
In setting I have many MOD "raw" to activate and also id is a very little info, I not understand what are the right to acrivate , IF CAN be all activated or just what need....
CROP_REC
MLV_LITE
MLV_PLAY
MLV_SND
MLV_REC
RAW_REC
RAW_TWK
IF I activate all that (and nothing another)...I turn off/on...enter in ML menu I have a big error on display
so I turn off " RAW_REC " and all seem to work fine (no errors)...
BUT there are 2 MLV to use....MLV_RAW and MLV_LITE
and CROP_REC
what I not understand is.. CROP_REC work togheter with MLV_RAW / LITE or just alone ?! if work alone , must be activated with zoom x 5 taste ?
i try but not luck... I set canon 1920 I-All or 720..but not understand the right steps to setting before begin rec.
MLV_RAW , always with "zoom x 5" (canon x 720 25p) how setting right for have a working 1920x960 50p ? i try nothing work....but also 3072x1308 24 ?!?
and 3072x1920 24 1;1 crop ?!? with crop_rec activated ?!
very big big confusion and no steps/guide about here....
not possible to have a little guide , with this LATEST ML 4k (no much old) what MOD acitvated and how set the reflex for every risolutions possible exactly , so when must be Zoom x 5 and when not need or must be active CROP option ?
I Know , the expert smile about that, but I'm sure we are many to not undestand and no helps around....
thanks for the support and patience!!!
Some minor updates:
- explicit option to set the shutter speed range (ae5667f (https://bitbucket.org/hudson/magic-lantern/commits/ae5667f4)):
- original (1/33 ... 1/4000 in 1080p or 1/64 ... 1/4000 in 720p)
- full-range (1/FPS ... fastest exposure time allowed by hardware: 1/15000 in full-res LV, 1/30000 in 50/60p 3x3 etc)
- previously, the "original" method was used for presets that did not change FPS, and "full-range" otherwise; now it's YAMLMO
- this should be very helpful for "Full-res LV" snapshots from silent.mo (how I've tested it) and 45/48p (the default choice was inadequate)
- in x5 mode, the FPS from Canon menu (24, 25 or 30) is used by default, without requiring FPS override (34d25aa (https://bitbucket.org/hudson/magic-lantern/commits/34d25aa))
- x10 mode is centered too - no more vertical bars (04e8221 (https://bitbucket.org/hudson/magic-lantern/commits/04e8221))
- raw memory management should be ready for full-res LiveView on all models (7ad3754 (https://bitbucket.org/hudson/magic-lantern/commits/7ad3754), cc @theBilalFakhouri)
- major updates to memory backend (https://bitbucket.org/hudson/magic-lantern/pull-requests/906/memory-backend-improvements/diff) (for general-purpose memory, not the one usable for raw recording)
@leozan: there used to be a video tutorial (http://vimeo.com/217313287) on the first page, but the link is not working here... [edit: solved, thanks hjfilmspeed]
Quote from: a1ex on February 24, 2018, 09:18:18 PM
Some minor updates:
- raw memory management should be ready for full-res LiveView on all models (7ad3754 (https://bitbucket.org/hudson/magic-lantern/commits/7ad3754), cc @theBilalFakhouri)
Unless I miss understood
You mean just
digic 5 , I see nothing there for
digic 4 .
does this mean d4 is dead ?
Memory management, not anything else. DIGIC 4 too, on models with a working CONFIG_EDMAC_RAW_SLURP (current or future).
The green lines from the linked changeset are not tied to any particular model; they are now applied to all models. They were previously applied to 5D3 only, but now @theBilalFakhouri reached a stage where this change could make a difference in his experiments. Furthermore, this very change was confirmed during DIGIC 4 experiments - click on the links (https://www.magiclantern.fm/forum/index.php?topic=19300.msg197702#msg197702) ;)
Builds from this branch are posted only for models where things are known to work (crop_rec and lossless compression). At the time of writing, that happens to be just a few DIGIC 5 models, but the backend is ready to accept the others, too. For example, it compiles for 60D, but none of the new features will work yet. However, it does run adtg_gui and lets one tinker with it.
Pure Awesomeness @a1ex 8)
I was just testing the last build 24 few on 5d mk3 1.2.3
Here's some things I noticed.
h264 proxy has no sound when I enable sound recoding module + mlv lite.
sound menu is not yet in normal mlv recoding, so if I load sound and mlv rec, no way to enable sound for mlv rec.
1) expected (the audio DMA is not multi-user)
2) cannot confirm (menu is there (https://www.magiclantern.fm/forum/index.php?topic=16040.msg196849#msg196849) with mlv_rec + mlv_snd loaded; furthermore, sound is enabled by default (https://www.magiclantern.fm/forum/index.php?topic=16040.msg197157#msg197157))
Ok I will double check for the sound on mlv rec.
for 1. I understand that it is choice between sound in h264 or raw + sound.
for switching between:
A ("h264 proxy with canon sound" + "mlvlite & no mlv snd") the old proxy workflow. where I use the h264 for confidence check with sound. ( and can use h264 with sound for quick send by mail etc )
and
B ("h264 without sound" + "mlvsnd & mlv lite"). the new mlvlite mlv snd workflow. where I use the h264 for confidence check image only.
Could there be some preset setting in the menu?
@A1ex,
The ability to use original canon FPS without fps override in 3.5K CROP is great.
I have noticed one small thing :
If you activate the FPS override in crop mode and then deactivate it, you fall back to 30.125fps instead of the original canon fps value.
- exiting and reentering the crop mode with the loop button (x10 >No zoom > x5)
- entering Canon menu and reselecting the fps mode
Will get you back to the original canon fps setting so really no big issue there just a mention.
Quote from: a1ex on February 24, 2018, 09:18:18 PM
@leozan: there used to be a video tutorial (http://vimeo.com/217313287) on the first page, but the link is not working here...
thanks A1EX I found this video , but was made with a old april release, and there are many new different options now in the last one ...
for begin is super (video) but ofcourse I have many anotehr questions, I think very easy to answer for you or another expert.
like..
in this video, for recording in "high resolution" , are activated CROP_REC.mo and MLV_LITE.mo (raw 1.1 lite)
Canon menu ofcourse in 1920 P 24 or 25 fps I-ALL
so later start again the 5D3 in video ML menu is set , raw video (example 2:35:1 , 3072 , 12 bit loss, preview "framing" )
so, CROP MODE on > 3,5 1:1 x5zoom
last , out of menu, and pressed ZOOM (x5)
ready to rec.
SO first question...but "crop option" MUT be always select for recording in high resolution up 1920 ? so..must always mork togheter with RAW (MLV_Lite) ? or there are anothers modes without "crop options" ?!?
I not undesrtand why are presents also:
MLV_REC
RAW_REC
RAW_TWK
IF must be enable (mod) all , or not and for what exactly need that ?!
second question (not showed in any video) HOW recording in H264 High resolution up 1920 , what mod must be enabled and how correct set menu /canon and ML before recording....
THANKS very much !!!
;)
(https://thumb.ibb.co/mG7SDH/IMG_0666.jpg) (https://ibb.co/mG7SDH)
Why this errors when I enable the 4 mod ???
I just put a clean last ML exp. of yeserday on my SD like ever..(delete before the old files...)
never see before
:o
Cause crop_rec only works with Mlv_lite module. Not mlv_rec.
:o ?!
so crop_mod can be enabled only with mlv_lite and not with mlv_rec ?
sorry but what need Mlv_rec ?
confusion....
:(
MLV_Rec records sounds, MVL_Lite does not.
Quote from: Ottoga on February 26, 2018, 08:43:31 AM
MLV_Rec records sounds, MVL_Lite does not.
Sure?
https://builds.magiclantern.fm/jenkins/view/Experiments/job/crop_rec_4k/65/changes -> Item 3.:
"Merged crop_rec_4k_mlv_lite_snd into crop_rec_4k (experimental mlv_snd support for mlv_lite from ErwinH)"
so mlv_rec.mod normaly not needit ?
Only for recording audio ...
but without CROP.mod , right? look error picture ....
?
Do the following configurations work for you?
- mlv_lite, mlv_snd, crop_rec (do *not* load mlv_rec!)
- mlv_rec, mlv_snd (do *not* load mlv_lite! Do *not* load crop_rec!)
yes....I can't enable togheter CROP_REC.mod and MLV_REC.mod or I have error when I turn ON 5d3 and I enter in ML menu'
but if I enable CROP & MLV_LITE...work fine , and there is audio rec too !
what I not undestand, is ..what you can NEED MLV_REC.mod ?
Fiddling with adtg_gui on the 6d, but not much luck.
Does anybody know how I change these N type registers in ADTG_gui ?
82f3N
8172N
8178N
8179N
82f8N
82f9N
When I select them to override, the value changes, it's not hexadecimal anymore once overriden, how do I set these values ?
I can get all type of resolutions, full sensor width, 5472 x960, also, 3072 x 960 and 2880 x 960, but I can't get normal raw data image preview, so the resulting image is a mess :P
Quote from: jimiz on February 26, 2018, 12:03:14 PM
what I not undestand, is ..what you can NEED MLV_REC.mod ?
I believe I have seen some conversations that now that mlv_lite has sound, mlv_rec is no longer necessary. But that only happened a couple weeks ago, so give it time.
Quote from: Levas on February 26, 2018, 12:05:54 PM
Does anybody know how I change these N type registers in ADTG_gui ?
The N registers are NRZI (https://en.wikipedia.org/wiki/Non-return-to-zero)-encoded (at least on 5D3; on other models YMMV).
This should help (same as these (https://bitbucket.org/hudson/magic-lantern/src/ba55e574dd6d898005bb72c5a6a947f119dfe051/modules/adtg_gui/adtg_gui.c?fileviewer=file-view-default#adtg_gui.c-204), but in Python):
def nrzi_decode(in_val):
val = 0
if in_val & 0x8000:
val |= 0x8000;
for num in range(31):
old_bit = (val & 1<<(30-num+1)) >> 1;
val |= old_bit ^ (in_val & 1<<(30-num));
return val;
def nrzi_encode(in_val):
out_val = 0;
old_bit = 0;
for num in range(31):
bit = 1 if in_val & 1<<(30-num) else 0;
if (bit != old_bit):
out_val |= (1 << (30-num));
old_bit = bit;
return out_val;
Registers 0x805E / 0x8060 (shutter blanking = frame duration - exposure time) are NRZI on all models, and a good candidate for understanding how they work.
Quote
I can get all type of resolutions, full sensor width, 5472 x960, also, 3072 x 960 and 2880 x 960, but I can't get normal raw data image preview, so the resulting image is a mess :P
Full width is actually great news (assuming ML "grayscale" preview and/or recorded MLVs are OK).
Yes, the real-time preview is going to be broken for anything with modified horizontal resolution. The registers to fix it are somewhere in these logs (https://www.magiclantern.fm/forum/index.php?topic=2388.msg197313#msg197313) - good luck figuring out which ones :D
That's massive progress btw - two weeks ago I was completely clueless where to start looking; now I believe I've narrowed it down to a few thousand registers :D
BTW - next steps for understanding how to fix the preview (just in case anyone might want to help):
- annotating known registers in these logs (easy, see g3gg0's signature)
- plugging the register values from the above logs in QEMU (definitely not an easy coding task)
- trying to understand what they do (lots of registers, plenty of low-hanging fruit, plenty of difficult ones, very time consuming)
Quote from: jimiz on February 26, 2018, 12:03:14 PM
what I not undestand, is ..what you can NEED MLV_REC.mod ?
Keep in mind we are working within an experimental branch derived from "main" (unified) nightly build. Nightly build's mlv_lite does not support audio (yet). Killing mlv_rec in experimental has no priority and might complicate - I suppose - merging experimental build results back into "unified".
Quote from: a1ex on February 26, 2018, 12:20:33 PM
Full width is actually great news (assuming ML "grayscale" preview and/or recorded MLVs are OK).
Unfortunately, the ML preview is also a mess, as are the recorded MLV's :P
Quote from: a1ex on February 26, 2018, 12:20:33 PM
That's massive progress btw - two weeks ago I was completely clueless where to start looking; now I believe I've narrowed it down to a few thousand registers :D
O jus a few thousand registers to try, I'm almost there ;D
Yeah, just have to convince a computer (https://www.youtube.com/watch?v=PyBNR8ThrNQ) to figure it out :D
Sounds ideal, a computer that does the work...at the moment I'm doing the 'boring' trail and error work :P
The build I'm using to test this gives memory patches, once I enable adtg_gui module:
Just wondering, is this normal ?
(https://farm5.staticflickr.com/4766/40501060971_7b25bbe5f9_o.jpg)
Right, the display could be a little nicer. Maybe it could use a built-in disassembler?
But is it normal ?
I'm using kind of Frankenstein build.
Didn't compile this myself, using the 6d 4 februari build from Dfort and a adtg_gui.mo from another build from Dfort, probably a few builds before that.
Haven't find out how to compile a normal adtg_gui module myself.
When I compile a adtg_gui module from the normal Hudson magic-lantern build - updated for Crop_rec_4k, I get a very different adtg_gui than I have from Dfort ?
The one in the Hudson-Crop_rec_4K is very basic and doesn't have the advanced options ???
What do I have to clone, pull&update to be able to compile a very clean build with the latest normal adtg_gui ?
Try this one: http://builds.magiclantern.fm/modules.html#iso-research
If you really want to modify the source, try the iso-research branch. Most modules should be interchangeable with crop_rec_4k.
@a1ex
Thanks for the commit, now I can get
5208x3240 and this really close of 5208x3490, I couldn't take a picture with silent.mo (I could do it in 5208x2624) I was able to get one frame with mlv_lite but sometimes I got blank MLV file with size of 24mb and other one was 49mb (maybe this happen with loading other modules?) .
But still I wasn't able to get the full resolution when I adjust the register for that I got frozen live view, I think this related to the buffer? I can get the full height (with corrupted frames need to adjust something to fix it but not frozen) by decreasing horizontal resolution at same other registers settings.
Quote from: a1ex on February 22, 2018, 06:52:35 PM
You need to use a slightly smaller amount because our memory backend also has some overhead.
So should I lower the resolution to the native .cr2 5184x3456 and maybe more?
-- The camera freezes when using adtg_gui with 720p 50/60, any solution ?
These two registers have same value for shutter speed on 700D to getting high shutter speed on low fps:
ADTG2[805f]
ADTG2[8061]
Maybe it's known, but for documenting.
@goldenchild9to5
Thanks!
Quote from: nikfreak on February 22, 2018, 09:05:04 PM
indeed great news. Keep up your work.
Crossing fingers. Maybe a PR with some useful "presets" would make 700D owners happy - though it's not 1st April yet.
I hope it works fine especially for 3K at 23.976, but it seems like these small cameras are limited to get that fps in higher resolutions.
But it's still cool 8)
Quote from: reddeercity on February 22, 2018, 09:43:02 PM
Good Job ;)
@reddeercity
Thank you, we are waiting for 5D2 :D
@Levas
Thanks!
This good news to 6d!
Maybe the 6d will be more similar to 5D3 (look into crop_rec.c) , but I wrote a small guide for my little experiment on this for anyone need to try:
I am talking about 700D for getting custom resolutions, the other cameras have DIGIC 5 will be similar too (not in everything you should discover somethings):
You have 30fps in the box, you will pay fps for more resolution (vice versa) and the higher resolution you already have in x5 mode (2520x1080 on 700D \ 3584x1320 on 5D3 without crop_rec) the more resolution you will get at 23.976 fps without corrupted frames for example.
Some informations:Basic informations (https://www.magiclantern.fm/forum/index.php?topic=19300.msg197098#msg197098) must read.
Any register has 3 parts (and more), 0x this is not a part.
e.g. In 1080p at 25fps Timer B C0F06014 = 0x7cf
If you change:
The left part you will get Large step changing in the register Ex: 7 to 6 the fps will be 28.669
The middle part you will get Medium step changing in the register Ex: c to d the fps will be 25.201
The right part you will get Small step changing in the register Ex: f to e the fps will be 25.012
Some registers have two Main parts and every part is for controlling specific thing EX:
C0F06804 = 0x4540298 454 = Vertical resolution 298 = Horizontal resolution (There are a math for calculate the actual frame size I didn't know it).
Increasing Timer A (Lowering the fps via timer A) is very important for increasing Horizontal resolution.
Increasing Timer B (Lowering the fps via timer B) is very important for increasing Vertical resolution.
I usually start with x5 mode.
For increasing
Horizontal resolution:
First of all you should increase FPS Timer A (lowering the fps) and increase the registers related to it (C0F0600c - C0F06010):
On x5 mode C0F06008 = 0x2cb02cb
Now increase one Large step :
C0F06008 = 0x3cb03cb You will notice the lowering in fps in live view (Press magnification button multiple time to see the update).
In C0F06804 :
Change C0F06804 = 0x4540298 to C0F06804 = 0x4540328 and make update (Magnification button) you will see the broken live view and frames, and the resolution will not increase because of the registers related to it didn't change yet which are:
C0F06824 = 0x2ca
C0F06828 = 0x2ca
C0F0682c = 0x2ca
C0F06830 = 0x2ca
All of these registers has the same value and if you missed one of them you probably will have frozen live view. Now increase them one large step
to 0x3ca
Make update and see the increase in resolution, safe frames with broken live view!
Okay if you had black bar in the left of the image you must now correct the offset via CMOS 5 for vertical offset.
Try decreasing/increasing it to get the correct value (trial and error).
Now for increasing
Vertical resolution:
1-Lower the fps via Timer B.
2-Increase the registers related to it which are:
ADTG[8172] N = 0x67c One large step 0x77c
ADTG[8178] N = 0x67c One large step 0x77c
C0F0713c = 0x455 One large step 0x555 (If you increase/decrease this too much you will have frozen live view and corupted frames, try to increase all of the registers as needed only maybe 0x500 rather than 0x555 will be enough, this will help for getting safe frames and good fps.
C0F06804 = 0x4540298 five Meduim steps 0x4a40298 make update (it's better to make update from PLAY mode in some registers) and check the resolution.
If you see a white-black bar in the bottom you should adjust CMOS 7 (trial and error).
Now if you have distorting in vertical you should adjust ADTG2[82b6] = 0x67f One Large step 0x77f .
Somethings I have noticed:
Vertical resolution needs more fps than horizontal resolution to increase it?
Disappearing Timer B in 1080p 23.976 (@a1ex this is a issue?).
Vertical stripes when adjusting small steps for both timers (you need to be careful).
CMOS registers updating directly when tweaking them you don't need to press PLAY mode or magnification button.
Sometimes you feel the camera froze, switch to photo mode and press trash button twice and you will get back.
Helpful tips:
Document (https://docs.google.com/spreadsheets/d/12rN6HFjVpDYWCyWe7vBs2O7DqgnF7vt0EB8a2lKhMEY) the important registers for each mode and try to study them.
Write the resolutions you want to get and write each register for it. (https://docs.google.com/spreadsheets/d/1TzPxh7ai5JAoEI8zllaiiF1IDF5tAQam2BKRujyIYQU/)
Use "Modified from now on" it will help you a lot.
Turning off adtg_gui when adjusting settings is a lot faster.
Keep learning and trying every day about these registers so your mind will analysis everything for you and you will feel the problems, it's about training.
(I still editing this maybe it will take more time but this what you will need).
Now I got the following resolutions on 700D:
- Very close to Full-resolution LiveView 5208x3240 at 6.5 fps
- 3K 2.35:1 3072x1306 at 20 fps
- 3.5K 3540x1080 Very Wide at Edit: 23.85 fps
- 4K 4096x2560 at 9.5 fps
- UHD 3840x2160 at 12 fps
- 2520x1386 16:9 at 23.976 fps
- 1920x1920 at 23.976 fps (corrupted frames not working yet)
This the latest updates, yeah it's needs more working to get most of them at 23.976, this is all the registers for each mode Here (https://docs.google.com/spreadsheets/d/1TzPxh7ai5JAoEI8zllaiiF1IDF5tAQam2BKRujyIYQU)
Maybe I will not to able to continue now for exams, feel free to play with them!
You've got 2 limits here:
- total captured size (including optical black bars) - always larger than the one displayed in the Raw Video menu - must be less than SRM_BUFFER_SIZE - 0x1000
- lossless encoder limit (apparently H <= 3528 on 700D (https://www.magiclantern.fm/forum/index.php?topic=19300.msg196749#msg196749)) - only applies when saving a lossless DNG (does not apply to uncompressed DNG)
In your case:
5208x3490 W
6.527fps
C0F06800 0x30010
C0F06804 0xccc0538
the total resolution would be: ((0x538 - 0x10) * 4) x (0xccc - 3) = 5280 x 3273 (was that a typo when copying the register values?)
Assuming this value was for 5208x3240, that would give a left bar size of 72 (matches raw.c) and a top bar size of 33 (raw.c uses 28 for LiveView and 52 for photo mode).
On a CR2 file from 650D, "dcraw -i -v" shows "Full size: 5280 x 3528". This image would take 5280 * 3528 * 14/8 = 0x1f16ac0 bytes uncompressed (SRM_BUFFER_SIZE 0x1F24000) and matches the encoder limit exactly. (BTW, that limit might be actually a register we have to change in order to solve the puzzle for 650D / 70D / possibly also 6D).
Anyway - if the resolution given by C0F06800 and C0F06804 does not exceed SRM_BUFFER_SIZE - 0x1000, it should be fine (although I doubt you will be able to capture more pixels than in a CR2). What could happen is that the top optical black bar would end up larger than usual (so the active area stays the same, but the black area ends up larger). In this case, increase the upper half of C0F06800 to trim the black bar.
Photo mode: 0x30010, 0xdcc0538 -> 5280 x 3529 -> 0x1f18ed8.
Sorry guys, but wasting my time trying to get this (* 3072x1920 @ 24p (1:1 crop)) to work. Is not working for me. Any advice. Using 5D mark III!
Sorry forgot to say using build (magiclantern-crop_rec_4k.2018Feb24.5D3123)
Prokopios, I don't think it will shoot that aspect ratio. I think you can shoot 3072 x 1320.
@Prokopios make sure you enable MLV_Lite module, Crop_rec module & if you want sound mlv_snd from the 9th tab in the magic lantern than restart your camera. Enter magic lantern menu after restart switch liveview on than go to the 4th tab and enable crop mode / raw video options & lastly hit the zoom button to go into 5X mode. That"s it you should be good to go after that. Let me know if that works. Also here is a video breaking it down https://vimeo.com/217313287
Guys, he's asking about 3072x1920 @ 24p (see first post), not about 3072x1320 in x5 zoom mode...
http://rawcalculator.bitballoon.com/calculator_desktop -> record time about 5 seconds.
Quote from: a1ex on March 09, 2018, 06:57:39 AM
http://rawcalculator.bitballoon.com/calculator_desktop -> record time about 5 seconds.
Why is 5D4 in this calculator?
(https://thumb.ibb.co/fYjHXn/calculator.png) (https://ibb.co/fYjHXn)
This calculator is giving different results from these
* 3072x1920 @ 24p (1:1 crop)
* 3840x1536 @ 24p (1:1 crop) (corrupted frames at 1600)
* 4096x2560 @ 12.5p (1:1 crop) - continuous*) at 8 FPS
* 4096x1440 @ 25p (1:1 crop)
The 4K half frames is just an example 4096 X 1440, but the calculator says 3584x1320 when you choose 4096 resolution. Maybe i am missing something??
Hi folks!
Anyone willing to share the limits of 600d and which version they're using? And also which version they think is the "most stable". I am mostly interested in 24fps, 16:9 and even 2.35:1.
@ibrahim - Off topic, the 600D isn't supported in the crop_rec_4k branch. You will have better luck asking on the Canon 600D / T3i (https://www.magiclantern.fm/forum/index.php?topic=15360.0) topic.
Please tell me.
Do you have plan to add Full-resolution LiveView: 5796x3870 at 24 fps(8bit or 10bit) in the near future?
Or, add normal 4k with 16:9 aspect ratio?
I want to use Magic Lantern normally.
If 16:9 aspect ratio feature has in ML, more people use ML, I think.
4k or higher resolutions in 24/25fps
in 16:9 ratio probably never become reality.
Hardware limitations, the sensor can't be readout that fast.
With current crop_rec options it's already reading 150Megapixel per second.
See also this post from Greg:
Quote from: Greg on April 02, 2017, 09:41:03 PM
These are just approximate calculations. You can initially estimate the speed of the sensor.
5D3 22MPx x 6fps = 132MPx/s
4096px x 2560px x 12.5fps = 131MPx/s
7D 18MPx x 8fps = 144MPx/s
70D 20MPx x 7fps = 140MPx/s
6D 20MPx x 4.5fps = 90MPx/s
500D 15MPx x 3.4fps = 51MPx/s :-[
Btw, my 4 year old smartphone 8MPx x 30fps = 240MPx/s :P
More accurate calculations - https://www.magiclantern.fm/forum/index.php?topic=12656.0
Of course we must remember about the speed of writing.
Quote from: Levas on March 14, 2018, 07:40:50 PM
4k or higher resolutions in 24/25fps in 16:9 ratio probably never become reality.
Never say "never" :)
Actually, it's a question of porting ML to Digic 6+.
You're right, I forgot to add "On the Canon 5d3" ;D
Other cams, that's a different story.
I think , time come a working raw 14 bit 4K without problems and limits also in a 5d4....the price of the 5d4 used will be 50$ ... :D
Something weird happened with the latest March 10th build on my 700D. MLVProducer (3261) couldn't properly remove the focus dots in my footage and was completely incapable of removing false colour artifacts. I switched back to the February 24th build and everything worked correctly with it. I have no idea where the problem could be exactly, but I thought I'd report.
Quote from: mvrck on March 15, 2018, 08:00:49 PM
Something weird happened with the latest March 10th build on my 700D. MLVProducer (3261) couldn't properly remove the focus dots in my footage and was completely incapable of removing false colour artifacts. I switched back to the February 24th build and everything worked correctly with it. I have no idea where the problem could be exactly, but I thought I'd report.
If you read the very first commit change of the build
57614b3:
Raw backend: use (*height)-- adjustment on all DIGIC 5 models
700D, 650D, 100D, M: fine-tuned skip offsets
Allow odd values for skip_bottom
650D/M x5 zoom: skip_top is the same as with regular 1080p or 720p or mv1080crop
700D/650D/M 720p
before: H=727, skip_bottom = 4 => active area y2=723 (real 724)
after: H=725, skip_bottom = 1 => active area y2=724
100D 720p:
before: H=726, skip_bottom = 4 => active area y2=722 (real 724)
after: H=726, skip_bottom = 2 => active area y2=724
650D/M 1080p:
before: H=1190, skip_bottom = 4 => active area y2=1186 (2 white lines at bottom)
after: H=1188, skip_bottom = 0 => active area y2=1188 (no bad lines at bottom)
EOSM movie crop: H=1060, top=28, active area y2=1060, no bad lines at bottom
(unsure about skip_bottom on 100D - to be tested with a silent DNG in movie crop mode)
See also:
https://www.magiclantern.fm/forum/index.php?topic=18443.msg198140#msg198140
https://www.magiclantern.fm/forum/index.php?topic=16054.msg195213#msg195213
In english, it means that the recorded image has been moved a few pixels, so your FPM (Focus Pixel Map) is now invalid. AWPstar has to update his FPM in MLVProducer.
The MLV metadata contains the offset for this change (so, unless the maps are hardcoded for some particular video mode, things should work). Only the x5 zoom was affected - the top offset in that mode was incorrect on these models (there were two black pixels at the top).
Example metadata from 5D3, with a similar change (just to make sure the focus pixel tools have the metadata for aligning the maps). Before:
Block: RAWI
Res: 1920x1080
raw_info:
height 1380
width 3744
active_area.y1 60
active_area.x1 146
active_area.y2 1380
active_area.x2 3742
Block: VIDF
Crop: 1824x300
After:
Block: RAWI
Res: 1920x1080
raw_info:
height 1380
width 3744
active_area.y1 62
active_area.x1 146
active_area.y2 1380
active_area.x2 3742
Block: VIDF
Crop: 1824x300
In both cases, it's the crop field that shows what area was recorded from the full-size raw buffer, relative to the first pixel in a silent DNG, which contains the complete raw buffer (caveat: the offsets are not relative to the first active pixel, but to the first - usually black - pixel). That's the metadata to be used for aligning the focus pixel maps.
In this particular case, the recorded area was not changed at all, but the metadata does show some difference. If the focus maps were aligned on the first active pixel (which changed in the metadata), then we have a problem (of not following the spec).
I'm unable to predict the exact metadata from the 700D, sorry. The crop offset might have been changed, or might have not - I have no idea. Please post the relevant lines from mlv_dump -v, as in the above example.
700D
Before:
Block: RAWI
Res: 1920x1080
raw_info:
height 1108
width 2592
active_area.y1 26
active_area.x1 72
active_area.y2 1108
active_area.x2 2592
Block: VIDF
Crop: 280x28
After:
Block: RAWI
Res: 1920x1078
raw_info:
height 1106
width 2592
active_area.y1 28
active_area.x1 72
active_area.y2 1106
active_area.x2 2592
Block: VIDF
Crop: 280x28
Now for the strange part -- I'm seeing focus pixels too but only on lossless compressed files. Looked at the metadata on the uncompressed footage and it was the same as the lossless compressed. Seems to be a recurring problem (https://www.magiclantern.fm/forum/index.php?topic=16054.msg195771#msg195771). Stranger still is that my map files were updated in January and should work fine with these files but aren't being accepted by the app developers (https://bitbucket.org/dmilligan/mlvfs/pull-requests/13/update-for-1-1-sampling-modes/diff) so that's why we're seeing focus pixels.
That "could have pixels shifted 8-pixels to the right" sounds fishy. No metadata difference?
One strange notice:
QuoteIn fact now that I reviewed them I'm not sure why I included Crop in my script, only Pan is needed to line up the focus pixel maps.
Correction: only crop is needed to line up the focus pixel maps (at least in theory). These offsets will match the edmac_copy_rectangle call.
Wait a minute, found the bug. The lossless compression routine uses a different rounding - that explains the 8-pixel offset. Whoops...
Hotfix:
diff -r fe4e5f79138f modules/mlv_lite/mlv_lite.c
--- a/modules/mlv_lite/mlv_lite.c
+++ b/modules/mlv_lite/mlv_lite.c
@@ -2494,7 +2494,7 @@
int compressed_size = lossless_compress_raw_rectangle(
outSuite, fullSizeBuffer,
- raw_info.width, skip_x, skip_y,
+ raw_info.width, (skip_x + 7) & ~7, skip_y & ~1,
res_x, res_y
);
Workaround for existing files: panPosX & ~7 will give the correct X offset for existing lossless files, while (panPosX + 7) & ~7 should match cropPosX and should be correct for uncompressed files. Focus pixel maps covering both "original" and "offset by 8" are also a good workaround.
For lossless files where panPosX is multiple of 8, the metadata is correct (both rounding methods will give the same result).
In any case, this not a difference between March 10th and February 24th build. The only difference that could affect focus pixel maps, between these two builds, is the 2px difference in the top optical black bar. Recorded area was not changed in the particular case tested by dfort (which may or may not match the initial report).
Dots be gone 8)
My map files still work though they are now overkill.
Quote from: a1ex on March 16, 2018, 04:38:39 AM
Correction: only crop is needed to line up the focus pixel maps (at least in theory).
Right, corrected that on the post. Though my script is converting Pan to Crop because of a bug that was fixed a while back:
##
# Allow for legacy MLV files with cropX, cropY bug
#
cropX=$((panPosX + 7 & ~7))
cropY=$((panPosY & ~1))
Déjà vu.
magiclantern-crop_rec_4k.2018Mar10.5D3113 Thank you very much!
One Q: how to set 5796x3870 @ 5p. Only 5784x3870 is available. And what about FPS 5p? Low light or Exact FPS?
Low light should be fine; however, the two settings (FPS override and crop_rec) are conflicting, as they override the same registers. Use with care - not all settings are working. In particular, the newest shutter speed option doesn't seem to work well with FPS override, regardless of the preset (just tested).
I've started to rewrite the backend to handle both features (and upcoming ones) without conflicts, but didn't get very far.
Easiest workaround: I could add a basic FPS option in the crop_rec submenu (already having that locally, as timer overrides).
Thank you Alex for reply.
But in case of 5796x3870 @ 5p, FPS override must be on. And what about 5796?
That works with the silent picture module (which saves the entire image, including black borders); mlv_lite needs to start and stop recording at 8 pixel boundaries and skips the black borders; therefore, it crops 6 good pixels from the left and 6 from the right (in this particular case, with skip_left = 138 and skip_right = 2). Declare these two as 136 and 0 in raw.c and you should get 5800 in raw video menu, then crop 2 columns from the left and 2 from the right in post to get 5796.
Thank you for clarifying.
Any way to lock the half-shutter zoom view on without pressing the button? I find that view very useful.
Try Prefs tab -> Misc key settings -> Sticky HalfShutter.
I have a feeling the question was about the preview, where half-shutter toggles between real-time and framing-accurate. In this case, the option is in the raw video submenu (press Q).
FYI, sticky half-shutter does not come without side effects (locks up the GUI while it's sticky).
Sticky-half-shutter was indeed what I was after, thank you.
So I switched preview to real time and x5 jumps straight to that view, focus dots, histo and all without touching the shutter button and sticky-half-shutter off but I have a feeling that's how it worked anyway and I got confused but all good now. Framing not really essential as I have markers on lcd screen.
Just what I needed, thanks.
Hello Guys,
I would like to share with you my fresh done test with the last build of the crop module for my 5D Mark III V1.2.3 (2018Mar10).
My purpose was to try to use an anamorphic adapter on an already 2,35:1 crop. My curiosity was about to see if after the squeeze of the images at this resolution is still useable without crop it in post.
It's really ultra anamorphic but I love it, what do you think?
- Resolution: 1920 X 818 @50p
- Format: Mlv Lite 14bit
- Sound: On
- Recording Media: Lexar 1066x 64GB
No color correction was made, just a bit of adjustment thru Camera Raw but very little, I love the color of my anamorphic converter together with the Magic Lantern Raw.
As usual the recording time is unpredictable, sometimes the camera let me shoot long shot, sometimes very short.
A super THANK YOU to all the people involved in the making of this awesome crop module!
Enjoy :)
https://youtu.be/R5vtuiRQ6CM
A few days ago I grabbed the SL1 to get some QuickShots of some Nicely Lit Clouds &
the 1st 2 ended up with Odd Patterns in them. Has anyone else had the happen?
https://photos.app.goo.gl/1TSGDdHFbZEwhM5F2
SL1 was running on > magiclantern-crop_rec_4k.2018Mar10.100D101
Hi everyone, a quick question/clarification about mlv_lite 60p
Canon 5d MIII firmware 113, ML4k firmware March 10 2018, lexar 64gb 1066x.
canon menu : video 1280x720 60p ALL-I
modules loaded: mlv_lite, crop_rec
RAW video ON, 1920x800
crop mode, enable, 1920 50/60 3x3 selected
I cannot get continues recording...only 20, or 25 30 seconds.
If I press the magnify button one time, 5x, the video is no longer 59.940 (60p) it changes to 29.776 (30p) is that normal? or is anything else I have to change to get the 60p when I am in 5x?
Quote from: tigerbengal2009 on March 31, 2018, 10:33:57 AM
If I press the magnify button one time, 5x, the video is no longer 59.940 (60p) it changes to 29.776 (30p) is that normal? or is anything else I have to change to get the 60p when I am in 5x?
The x5 mode is only affected by the x5 preset in crop_rec; all the others operate on regular (non-zoomed) modes. By default (even with vanilla Canon firmware), the x5 zoom mode operates at 29.776 FPS on 5D3 (not to be confused with 29.970 aka 30p) and captures a 3.5K wide image (YMMV on other models). Only a small part gets displayed on the screen, but the entire frame can be recorded.
The x5 preset will override the FPS in x5 (https://www.magiclantern.fm/forum/index.php?topic=19300.msg197777#msg197777) to 24/25/30p, as selected in Canon menu, for convenience, but that won't give you 3.5K at 60 FPS.
The "1920 1:1 tall" preset also operates on 50/60p - you may be looking for that.
Hm, i set shutter speed at 60, but on footage it's always 51. When set 30, it's also lower. Is this a familiar problem? 5DM3, 3-3,5k crop
(https://thumb.ibb.co/cg9V5S/Screen_Shot_2018_03_31_at_4_12_47_PM.png) (https://ibb.co/cg9V5S)
Its 1st of April. The Front Image of ML website shows a broken camera :(
I guess we will have news soon
HAPPY BIRTHDAY "3,5 K" ML !!!!! ;) :D
So, is there something to talk about today? ???
In any case, happy birthday! :-D
Hi A1ex, thank you for your response, at first I didn't understand what you mean about using the 1920 1:1 tall, but after trying it , the look is close to the x5, thanks for the advice.
My main goal is to record 1920x800 at 60p, and for some reason I can only have 14 seconds of recording. Using all the settings recommended
-5d Mark III firmware 1.1.3
-Canon Menu: 1280x720 60fps ALL-I
-RAW Video: On, 1920x800, 16:9, 14 bit loss less, preview: Real-time
-crop mode 1920 50/60 3x3
-fps override : OFF
-iso: 100
-vutowb
-Shutter 1/130
-USING a fast card lexar 1066x 64gb
ML Main menu said you can record continuously.
I only want to be able to record 1 minute but what am I doing wrong???? Hope you can point me in the right direction, thanks A1ex
Quote from: a1ex on March 31, 2018, 11:01:10 AM
The x5 mode is only affected by the x5 preset in crop_rec; all the others operate on regular (non-zoomed) modes. By default (even with vanilla Canon firmware), the x5 zoom mode operates at 29.776 FPS on 5D3 (not to be confused with 29.970 aka 30p) and captures a 3.5K wide image (YMMV on other models). Only a small part gets displayed on the screen, but the entire frame can be recorded.
The x5 preset will override the FPS in x5 (https://www.magiclantern.fm/forum/index.php?topic=19300.msg197777#msg197777) to 24/25/30p, as selected in Canon menu, for convenience, but that won't give you 3.5K at 60 FPS.
The "1920 1:1 tall" preset also operates on 50/60p - you may be looking for that.
Quote from: jimiz on April 01, 2018, 08:59:56 AM
HAPPY BIRTHDAY "3,5 K" ML !!!!! ;) :D
Thanks :)
Quote from: tigerbengal2009 on April 01, 2018, 09:53:14 PM
preview: Real-time
First post recommends something else (real-time uses too much CPU), though I might have something that will help (check twitter).
yes I changed preview to Frozen LV, same thing a few seconds as before.
OK I'll wait for twitter post.
By the way Happy Birthday ML 4K...Thank you very much A1ex and all the great people who made possible this awesome feature for the 5d Mark III !!!
Quote from: a1ex on April 01, 2018, 10:02:51 PM
Thanks :)
First post recommends something else (real-time uses too much CPU), though I might have something that will help (check twitter).
So I absolutely love MLV Lite and the Crop rec 4k module. So many awesome things in the newest build. I had a couple of thoughts.
It would be awesome to be able to get into our favorite crop rec 4k mlv lite settings faster like custom presets. Presests specific to MLV lite and crop rec 4K.
Ex: on 5D3 I use 3k x5 mode so if I went into my hypothetical module I would pick 3k and my settings would come up
3072
2.35:1
23.976
Canon 1920x1080 23.976
x5 zoom
canon zoom enabled
1/50 shutter
12bit lossless
framing
sound on
all gets chosen.
Or my 60fps option:
1920
2.35:1 (1920x800)
60fps
Canon 1280 60p mode
1/125 shutter
10bit uncompressed
Real time live view
No sound
All gets chosen.
I threw the shutter option in there because it's just one more step we don't have to take if we are trying to stick with a certain shutter degree ex 180° shutter.
That way we can get in and out of our favorite crop rec 4k MLV lite settings faster when doing some run and gun.
Also it's real tricky monitoring in the framing modes. I see how they help with write speed and its incredible that we even get framing period but I would love to be able to contribute to better live view. Especially for the less intensive settings.
I realize NONE OF THIS IS AND EASY TASK. And I am sure most of this has been mentioned. I am sorry for my brain storming but I thought I would throw it out there. I'm just a humble tester and user so take this all as a grain of salt. I am overjoyed with 1920x1080 24fps RAW with sound and fast start by itself. Just contributing my thoughts.
Should be doable with Lua scripts. Save these under ML/SCRIPTS:
5x.lua
-- 3K 24p x5 with sound
-- close ML menu to get into LiveView
menu.close()
-- change settings from ML menu
menu.set("Movie", "Crop mode", "3.5K 1:1 centered x5")
menu.set("Movie", "RAW video", "ON")
menu.set("Movie", "FPS override", "OFF")
menu.set("RAW video", "Aspect ratio", "2.35:1")
menu.set("RAW video", "Data format", "12-bit lossless")
menu.set("RAW video", "Preview", "Framing")
menu.set("Sound recording", "Enable sound", "ON")
menu.set("Overlay", "Global Draw", "ON, all modes")
menu.set("Display", "Clear overlays", "OFF")
-- etc
-- FIXME: user needs to switch to 1080p24 from Canon menu
lv.zoom = 1
while menu.get("FPS override", "Actual FPS", "") ~= "23.976" or not camera.gui.idle do
display.notify_box("Please switch to 1080p24 from Canon menu.")
msleep(500)
end
-- ready to enter x5 crop mode
lv.zoom = 5
-- finally set the resolution
-- fixme: doesn't seem to work if you have fine-tuned it
menu.set("RAW video", "Resolution", 3072)
-- finished!
display.notify_box(string.format(
"Ready!\n%s %s",
menu.get("RAW video", "Resolution", ""),
menu.get("FPS override", "Actual FPS", "")
),
2000
)
60p.lua
-- 1920x800 60p, no sound
-- close ML menu to get into LiveView
menu.close()
-- change settings from ML menu
menu.set("Movie", "Crop mode", "1920 50/60 3x3")
menu.set("Movie", "RAW video", "ON")
menu.set("Movie", "FPS override", "OFF")
menu.set("RAW video", "Aspect ratio", "2.35:1")
menu.set("RAW video", "Data format", "10-bit")
menu.set("RAW video", "Preview", "Real-time")
menu.set("Sound recording", "Enable sound", "OFF")
menu.set("Overlay", "Global Draw", "ON, all modes")
menu.set("Display", "Clear overlays", "Recording")
-- etc
-- close x5 zoom, should it happen to be enabled
lv.zoom = 1
-- FIXME: user needs to switch to 720p60 from Canon menu
while menu.get("FPS override", "Actual FPS", "") ~= "59.940" or not camera.gui.idle do
display.notify_box("Please switch to 720p60 from Canon menu.")
msleep(500)
end
-- finally set the resolution
-- fixme: doesn't seem to work if you have fine-tuned it
menu.set("RAW video", "Resolution", 1920)
-- finished!
display.notify_box(string.format(
"Ready!\n%s %s",
menu.get("RAW video", "Resolution", ""),
menu.get("FPS override", "Actual FPS", "")
),
2000
)
Was it hard? You can customize pretty much any ML settings. You can assign these presets to hotkeys or create custom menus, too. See the Lua API (https://builds.magiclantern.fm/lua_api/modules/menu.html) for more.
The only rough edges were:
- When switching the video setting in Canon menu, which is not available to Lua. A long time ago, ML had the option to switch video modes from code... until I've got a corrupted H.264 (fixable by clearing Canon settings). Maybe worth a second look, after understanding how these settings are changed.
- With the dynamic raw video resolution menu (doesn't seem to work well if you have fine-tuned it to some value not in the pickbox)
Tip: use these often enough and they will appear in the Recent (star) menu.
BTW, why did you choose 10-bit uncompressed (70%) over 14-bit lossless (50-65%) for 60p? Are you shooting at ISO 6400+ ?
So So So Awesome! I need to try scripting. I have been using ML for to long not to try scripts
Quote from: a1ex on April 04, 2018, 03:02:29 PM
BTW, why did you choose 10-bit uncompressed (70%) over 14-bit lossless (50-65%) for 60p? Are you shooting at ISO 6400+ ?
I had some issues when shooting in 14 bit lossless at ISO 100 when shooting up at the trees on overcast day. Using March 10 2018 build 5d3 113.
Can shutter speed be changed with Lua too? 1/125 with 60 or 1/50 for 23.976?
That might be a bit too verbose, but works:
camera.shutter = 1/50
See also Bracketing with flash (https://www.magiclantern.fm/forum/index.php?topic=21741.0) (another simple example).
Thanks for easy to follow lua examples. A piece here and there helps moving forward.
Modifying a lua script from a1ex... this is a proud day for me especially since it worked. Thank you thank you thank you for scripting this a1ex!!!
These have shutter speed changes depending on frame rate. The shutter change has to be put after the frame rate change otherwise you will crash the script since the canon wont allow you to use a shutter lower then 1/60 in a 60p mode.
Check this before you use because I KNOW nothing about ML code but I was able to quickly toggle between 1920 2.35:1 24fps and 1920x800 60fps. Much faster. I assume this will only work in manual mode since shutter speed is being modified.
-- 1920x800 60p, no sound
-- close ML menu to get into LiveView
menu.close()
-- change settings from ML menu
menu.set("Movie", "Crop mode", "1920 50/60 3x3")
menu.set("Movie", "RAW video", "ON")
menu.set("Movie", "FPS override", "OFF")
menu.set("RAW video", "Aspect ratio", "2.35:1")
menu.set("RAW video", "Data format", "10-bit")
menu.set("RAW video", "Preview", "Real-time")
menu.set("Sound recording", "Enable sound", "OFF")
menu.set("Overlay", "Global Draw", "ON, all modes")
menu.set("Display", "Clear overlays", "Recording")
-- etc
-- close x5 zoom, should it happen to be enabled
lv.zoom = 1
-- FIXME: user needs to switch to 720p60 from Canon menu
while menu.get("FPS override", "Actual FPS", "") ~= "59.940" or not camera.gui.idle do
display.notify_box("Please switch to 720p60 from Canon menu.")
msleep(500)
end
-- finally set the resolution
-- fixme: doesn't seem to work if you have fine-tuned it
menu.set("RAW video", "Resolution", 1920)
camera.shutter = 1/125
-- finished!
display.notify_box(string.format(
"Ready!\n%s %s",
menu.get("RAW video", "Resolution", ""),
menu.get("FPS override", "Actual FPS", "")
),
2000
)
-- 1920x818 24p with sound
-- close ML menu to get into LiveView
menu.close()
-- change settings from ML menu
menu.set("Movie", "Crop mode", "OFF")
menu.set("Movie", "RAW video", "ON")
menu.set("Movie", "FPS override", "OFF")
menu.set("RAW video", "Aspect ratio", "2.35:1")
menu.set("RAW video", "Data format", "12-bit")
menu.set("RAW video", "Preview", "Real-time")
menu.set("Sound recording", "Enable sound", "ON")
menu.set("Overlay", "Global Draw", "ON, all modes")
menu.set("Display", "Clear overlays", "OFF")
-- etc
-- FIXME: user needs to switch to 1080p24 from Canon menu
lv.zoom = 1
-- close x5 zoom, should it happen to be enabled
lv.zoom = 1
while menu.get("FPS override", "Actual FPS", "") ~= "23.976" or not camera.gui.idle do
display.notify_box("Please switch to 1080p24 from Canon menu.")
msleep(500)
end
-- finally set the resolution
-- fixme: doesn't seem to work if you have fine-tuned it
menu.set("RAW video", "Resolution", 1920)
camera.shutter = 1/50
-- finished!
display.notify_box(string.format(
"Ready!\n%s %s",
menu.get("RAW video", "Resolution", ""),
menu.get("FPS override", "Actual FPS", "")
),
2000
)
Definitely triple check these before using ha ha warning I stink at this it might break your cam ha ha.
I'm running into an error when I try to load multiple modules on crop_rec_4k builds.
The error initially showed up on the magiclantern-crop_rec_4k.2017Sep06.5D3123 build, which I had installed on the SD card and was working previously.
Here is an image of the error I'm getting when trying to load multiple modules:
(https://thumb.ibb.co/nQhVp7/linkerror.png) (https://ibb.co/nQhVp7)
I've tried:
-Deleting all ML folders/files and replacing them on the SD card
-Deleting all ML folders/files, formatting card, copying latest build (magiclantern-crop_rec_4k.2018Mar10.5D3123) to SD card
I still get the same error when loading modules.
I can install and run the following builds with no errors:
magiclantern-Nightly.2018Feb04.5D3123
magiclantern-crop_rec_4k.2017Jun19.5D3123 (does not contain the MLV raw 2.0 module)
I've tried searching for a solution and haven't found anything that worked. Any help on this would be appreciated.
something is change with mods on crop rek 4k versions...
look and read circa here...
https://www.magiclantern.fm/forum/index.php?topic=19300.msg197822#msg197822
meaby is that your problem...like me and anothers guy , time ago...
Quote from: jimiz on April 26, 2018, 01:44:30 PM
something is change with mods on crop rek 4k versions...
look and read circa here...
https://www.magiclantern.fm/forum/index.php?topic=19300.msg197822#msg197822
meaby is that your problem...like me and anothers guy , time ago...
Thanks! Hard to keep up with the changes. What's strange to me is that the build I was using worked previously and then stopped working.
I don't know if this discussion should be here but I've found that I have some issues making 4:3 recordings.
Tried with 1440x1080px and 1712x1284px - I'm not interested in the anamorphic use, more a creative style
With 1440x1080px it is possible to record with a 4:3 aspect ratio and get realtime preview but you need to press half the shutter and mantain it pressed (the crop window goes away when recording, you can only see a full 16:9 image). Not perfect but it is possible to use tape to cover the sides in the worst case scenario.
With higher resolution (1728x1290px for example) we can press half the shutter and get a realtime preview of the image but as soon as you hit record and press the shutter, it changes the real aspect preview to a croped one, wide aspect, not very useful at all. And this extra resolution (above 1440x1080px) would be very welcome.
Both working in non crop mode. I understand that in croped mode the real time preview would be not possible, and I understand the limitation of 1032px (maximum height resolution) with slow motion recording too.
Is possible to solve this?
I understand this mode is more for a anamorphic use than the one I intend to make, but...
Reproduced. ML uses some heuristics to pick between real-time and framing-accurate preview, and looks like you have found some cases where the automatic choice is not the best.
However, this is just the default behavior; you can customize it from the raw video submenu. Should the option be renamed or moved to make it more obvious?
Half-shutter is meant for quick (temporary) changes in preview; you don't have to keep it pressed. If you have to do that, you may want to look in the menu.
1032px - how did you get this value? On 5D3, the closest limit is 1040px at 48 FPS (overridable from menu).
Quote from: hjfilmspeed on April 04, 2018, 03:22:04 AM
So I absolutely love MLV Lite and the Crop rec 4k module. So many awesome things in the newest build. I had a couple of thoughts.
It would be awesome to be able to get into our favorite crop rec 4k mlv lite settings faster like custom presets. Presests specific to MLV lite and crop rec 4K.
Ex: on 5D3 I use 3k x5 mode so if I went into my hypothetical module I would pick 3k and my settings would come up
3072
2.35:1
23.976
Canon 1920x1080 23.976
x5 zoom
canon zoom enabled
1/50 shutter
12bit lossless
framing
sound on
all gets chosen.
Or my 60fps option:
1920
2.35:1 (1920x800)
60fps
Canon 1280 60p mode
1/125 shutter
10bit uncompressed
Real time live view
No sound
All gets chosen.
I threw the shutter option in there because it's just one more step we don't have to take if we are trying to stick with a certain shutter degree ex 180° shutter.
That way we can get in and out of our favorite crop rec 4k MLV lite settings faster when doing some run and gun.
Also it's real tricky monitoring in the framing modes. I see how they help with write speed and its incredible that we even get framing period but I would love to be able to contribute to better live view. Especially for the less intensive settings.
I realize NONE OF THIS IS AND EASY TASK. And I am sure most of this has been mentioned. I am sorry for my brain storming but I thought I would throw it out there. I'm just a humble tester and user so take this all as a grain of salt. I am overjoyed with 1920x1080 24fps RAW with sound and fast start by itself. Just contributing my thoughts.
Which CF card are you using? What would you recommend to use?
Quote from: a1ex on April 30, 2018, 09:28:50 AM
Reproduced. ML uses some heuristics to pick between real-time and framing-accurate preview, and looks like you have found some cases where the automatic choice is not the best.
However, this is just the default behavior; you can customize it from the raw video submenu. Should the option be renamed or moved to make it more obvious?
I don't know if I understand que question because English is not my native language and I don't know if I could understand this context very well, but from the two options I think 'renamed' would be my choice.
Half-shutter is meant for quick (temporary) changes in preview; you don't have to keep it pressed. If you have to do that, you may want to look in the menu.
Will take a better look at the menu to see what I'm missing...
1032px - how did you get this value? On 5D3, the closest limit is 1040px at 48 FPS (overridable from menu).
With slow motion you will only get the real 4:3 aspect if you lower the height pixel dimension to 1032px so you can use 1376px at the larger side. With 1040px it is very close to 4:3 but does not have the perfect aspect ratio.
Thanks for the inputs,
a1ex!
May be I missed something.
After 1 or two or three minutes recording stops with a message: Emergency stop. Movie recording has been stopped automatically. Timer icon is green.
Mark5D3 1.1.3.; magiclantern-crop_rec_4k.2018Mar10.5D3113 crop_rec, ettr, file_man, mlv_lite, mlv_play, mlv_snd, pic_view, raw_twk, silent – on; LV DIGIC peaking- Slightly Sharper, LV brightness- Normal; Global Draw- ON, all modes, Histogram- RAW RGB,Log; Auto ETTR- Press SET; RAW video- ON, 1920x1080, 12-bit lossless, Preview- Auto, H.264proxy- ON, Sound recording- off;
If H.264proxy- off - no problem (Sound recording- ON)
That message appears when the H.264 buffer becomes full. Are you recording both RAW and H.264 on the same card? If yes, try recording H.264 to SD.
No, different cards.
Corrected:
You are right Alex, as usual. I am very sorry. RAW and H.264 were recoded on CF. I do not know why. I did not change settings for cards. Playback card is CF and play separately in Canon menu. ML menu- Card setting - Preferred card-off. Every time I used these settings and RAW and H.264 recorded on two cards. But now they are recorded on CF both.
I can not set camera for two cards recording. How?
Set preferred card to sd card. It will send H.264 to SD while RAW goes to CF card...
Thank you! It works. I have never used this setting before, but recorded on both cards. Very strange.
Changing preferred card in ML menu is not required - all you need is to change the current card in Canon menu. H.264, being a Canon feature, uses that setting. The "preferred card" option only operates at startup to change the selection from Canon menu. It does nothing else - read the help.
On this branch, RAW/MLV always goes to CF if there's one inserted (for speed reasons).
Changing preferred card in ML menu changes preferred card in Canon menu (after reboot). So in order to record on two cards preferred card in Canon menu must be SD.
Quote from: a1ex on April 01, 2018, 10:02:51 PM
Thanks :)
First post recommends something else (real-time uses too much CPU), though I might have something that will help (check twitter).
I'm curious too... Checking Twitter every day but nothing yet.
Have you checked on April 1st? It was there when I've written that. It's still there. What do you expect?
OK, maybe it wasn't interesting enough, judging from other forum posts. Anyway, my CF card is dead from that experiment, so I'm taking a short break.
The source code is online, you can push it forward while I'm taking care of other parts of ML (not video-related).
Aaahhh, ok!! SD card overclocking...
I didn't expect nothing. Just I didn't know what were you talking about.
Thanks for the clarification.
I'm a big proponent of Magic Lantern. I lead the Creative Cow Canon DSLR forum and constantly recommend it as well as work flows.
My question I have for this forum and maybe it's obvious and I'm missing it, but if I install the latest experimental build of the 4k version of ML on my Canon 5D Mark III with 1.3.2 firmware, do I get the same nightly build as "magiclantern-Nightly.2018May01.5D3123" which I currently have installed, WITH the ability to do 3K/4K cropped? or are the normal feature in "magiclantern-Nightly.2018May01.5D3123" missing and it is only focused on3K/4K cropped?
In other words do I get everything the normal nightly build has plus the addition of 3K/4K cropped?
TIA!
Hi a1ex + ML geniuses, thank you for all your amazing work!
Wondering if you can shed some light on a buggaboo I'm having. All of my normal 1080p/24 RAW footage is great. I load the Raw video v2 (mlv_rec) for that and shoot 14bit lossless and it looks fantastic!
However, when I replicate your settings exactly for shooting 1920x818 48p for slow-mo, the footage just comes out with a black preview (or one broken image preview for all dngs) and unrecognizable as a dng for use after conversion (MLV Mystic, MLVFS, raw2dng, Mlrawviewer). Here are my settings:
-magiclantern-crop_rec_4k.2018Mar10.5D3113 (I've also tried 2018May1 build)
-5d Mark III firmware 1.1.3 (I've also tried v1.2.3 - no luck!)
-Modules: crop_rec, mlv_lite
-RAW Video: On, 1920x818, 2.35:1, 14 bit lossless ~50%, preview: Real-time, no Sound recording
(says in green - 62.9MB/s, 120x2.2MB, 48.000p (50%). Continuous recording OK.)
-Crop Mode: 1080p45/1040p48 3x3
-Canon Menu: 1280x720 60fps ALL-I
-FPS override : OFF (automatically shows gray 48.000, Rs.83% which should be great)
-ISO: 100
-Shutter 1/130
-Global Draw: I've tried both On and Off, same result
-CF card for record: SanDisk Extreme Pro 128gb 160mb/s, SD card for firmware: SanDisk Extreme Pro 32gb, 95mb/s
I've tried every Data Format configuration. The only one that works has been 10 bit uncompressed.
Were you able to get 14bit Lossless to work, and if so, could you tell me what I'm missing? I've reloaded all the firmware exactly how it's supposed to be done, tried v.123 as well as v.113 (what I'm using now) and nothing seems to work, the footage is not in a supported format when converted to dng (I've narrowed it down to where it seems the original mlv's are corrupt in some way). When I switch to mlv_rec (v2) I am able to record usable footage so it doesn't seem to be the camera or the cards, but I don't get the same size options as using mlv_lite.
I'd love to be able to shoot RAW slo-mo at 1920x818 2.35:1 48fps if possible. I'd be grateful for any help!!
Thanks! :)
Quote from: pgrandaw on May 07, 2018, 04:59:50 PM
Were you able to get 14bit Lossless to work, and if so, could you tell me what I'm missing?
Maybe it's your method of conversion ? Your MLV conversion must be able to decode the lossless format. I'm using MLVFS but there's an updated version that can convert lossless files. You could try uploading a short clip to see if anyone else can decode it.
14bit lossless is only in mlv_lite.mo not mlv_rec.mo
Have you tried MLV App (https://www.magiclantern.fm/forum/index.php?topic=20025.0) ? same thing?
If it still not working I assume the MLV files is corrupted. Because as a1ex said:
Quote from: a1ex on August 26, 2017, 11:27:03 PM
Regarding 1080p48 - it was pushed a bit over the safe limit, so I've reduced its resolution to 1040. You still have the option to go back to 1080 in the crop_rec submenu (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183399#msg183399); however, you'll get a lot of errors. With some fiddling, you can probably get 1060 (e.g. set Target YRES to 1060 and Delta HEAD3 to -20), but I wouldn't trust it yet.
So maybe you can override it to 45fps or lower the resolution in 48fps ?
EDIT: I saw you have already lowered the resolution to 1920x818, so maybe it's about converting method, Try MLV App.
Thanks theBilalFakhouri and JCut!
Quote from: theBilalFakhouri on May 08, 2018, 10:33:00 AM
Have you tried MLV App (https://www.magiclantern.fm/forum/index.php?topic=20025.0) ? same thing?
EDIT: I saw you have already lowered the resolution to 1920x818, so maybe it's about converting method, Try MLV App.
Switching to the MLV App did the trick. The footage converted just fine! I ran a load of tests today and I'm able to do pull 1920x1040 16:9 in 14bit lossless at 45fps just fine. I tried to do it at 48fps but only got 9 secs. However when I turned Global Draw off, I was able maintain continuous recording. (Also the 1920x818 works great at 48fps.)
Again, many thanks for your help. Sometimes it's the simple things.
One question - when forcing 45fps for 1920x1040, I can't select 1/130th shutter speed but can do 1/80 or 1/180. Which would you guys recommend?
Quote from: pgrandaw on May 08, 2018, 05:00:19 PM
when forcing 45fps for 1920x1040
That should be done from Canon menu, as FPS Override is not fully compatible with crop_rec at the time of writing. Refer to previous posts for details.
Quote
Because as a1ex said:
Quote from: a1ex on August 26, 2017, 11:27:03 PM
Regarding 1080p48 - it was pushed a bit over the safe limit, so I've reduced its resolution to 1040. [...]
[...]
I saw you have already lowered the resolution to 1920x818 [...]
FYI, lowering the resolution in the raw recording menu has absolutely nothing to do with what I said back then. The video mode configuration (including the likelihood of getting corrupted frames) is given by crop_rec settings, so if you have problems with randomly corrupted frames, you should reduce the vertical resolution in the crop_rec submenu, not elsewhere. Reducing the resolution in the raw recording menu simply selects the crop area that ends up recorded, but does not affect the image capture process in any way.
This does not apply to pgrandaw's issue; it's just something to keep in mind when recommending things for troubleshooting.
Quote from: a1ex on May 08, 2018, 05:05:56 PM
That should be done from Canon menu, as FPS Override is not fully compatible with crop_rec at the time of writing. Refer to previous posts for details.
Thanks a1ex, yep I have the Canon menu set to 1280x720/60. I just didn't know if I should manually set the FPS override to 45p but it seems to be capturing 48p with a 2.35:1 ratio so that's awesome. I won't set any override.
I know you've heard it a million times but thank you!
I am experiencing different shutter speed recording in resolutions higher than 1920x1080, mainly in the 3.5k 1:1 mode. I set my shutter at 1/50 for my regular 1080p 14 bit recording but when I change to lets say 2880 resolution, the maigclantern shutter speed suddenly shows 1/30 or so, but also after having imported the footage it says 1/30 while my Canon shutter speed is on 1/50.
Also the whitebalance doesn't work in crop mode 3.5k 1:1. everytime I 'half-press shutter' in 3.5k mode, the liveview has a completely different tone than when not in crop mode
Firmware 1.2.3 and 1.1.3(on this firmware now), 5DIII, komputerbay 1066x 64gb
Is there any discussion or development on a >1080p H264 mode?
With the 40MB/s cameras the limitation is SD write speed, so maybe we can pipe the enhanced resolution of crop mode into a compressed file?
I know we will have other bottlenecks in the system (like reading from the sensor, the digic processor compression, etc etc) and probably can't even do 3K, but even 2.5K H264 mode would be spectacular!
(Many thanks for what the devs have already achieved - in asking this I feel like a kid asking for more presents after already opening a bunch of awesome ones already!)
H.264 encoding for 1080p is done in hardware (blow up from native LV resolution to Full HD, noise reduction, sharpening) and IMO there is not that much knowledge about chip's abilities.
Don't hold your breath.
Is there a quick printable/savable user guide for this experiment version. I keep forgetting the steps to set this up on camera.. Any help appreciated. Thanks!
Video on page 1, post 1 doesn't help?
And storing those settings (config preset) would be helpful, too.
OK relooked at video again. Yes I can do that way. Then save as presents.. Thanks! Been a while since I did the 4K and want for a trip I am about to take. CHEERS!
With the expterimental crop_rec will 5d3 be able to transfer a 4K output through hdmi to an external monitor that supports 4K or is the camera restricted to 1080p even though a monitor would support 4K?
hdmi> same preview like lcd.... :-X
For 700D users let's try the new resolution 2520x1384 ~16:9 @ 23.976
Download the hardcoded adtg_gui (https://drive.google.com/open?id=1-a8jt9y9WQ_o8CyDaSYd5M5kKn7B4X80) for 700D from here
1-Put into modules folder (you must ran a build from crop_rec_4k branch)
2-Enable adtg_gui --> go to adtg_gui submenu --> Advanced --> Enable DIGIC Registers
3-Go to PLAY mode and back again to live view
4-Select 14-bit lossless from mlv_lite (12-8bit lossless not working in higher resolutions @a1ex why and what is the solution?)
5-Go back to adtg_gui submenu again --> Advanced --> then apply the preset "1:1 2.5K 16:9 cope mode (700D)" then magnification and get into x5 crop mode.
Check mlv_lite resolution!
Don't about weird live-view in non crop mode this is normal. The live view is working in x5 mode.
Make your tests and feedback!
Same steps for the 3K 3072x1304 @ 20fps using this adtg_gui (https://drive.google.com/open?id=1fSTEx1X-5IeRj6HXVveh1GPIkG-0Jzz9) --> replace it (don't worry about adtg_gui,, I am trying to implement it to crop_rec module)
The live view is not working in this mode! you must choose Framing in mlv_lite to see what's happening.
And feedback again for corrupted frames :P
Quote from: theBilalFakhouri on May 28, 2018, 06:43:30 PM
12-8bit lossless not working in higher resolutions @a1ex why and what is the solution?
Other image processing modules have to be reconfigured for the higher resolution. The solution is probably somewhere in these huge logs (https://www.magiclantern.fm/forum/index.php?topic=2388.msg197313#msg197313), but I didn't try to decipher them yet.
Quote from: a1ex on February 26, 2018, 12:20:33 PM
That's massive progress btw - two weeks ago I was completely clueless where to start looking; now I believe I've narrowed it down to a few thousand registers :D
Quote from: Levas on February 26, 2018, 12:39:11 PM
O jus a few thousand registers to try, I'm almost there ;D
P.S. To see the source code for the above modules:
cd magic-lantern/modules/
./module_hginfo_dump.sh adtg_gui.mo
Who said the live view is very hard to fix?
The Slicers registers has arrived!
ADTG[807c] Slicing and Tilting
ADTG[8048] Overlapping
ADTG[80a0] Strong Tilting
ADTG[8084] Strong Tilting
ADTG[80a4] Tilting
e.g Let's break the live view by decreasing the resolution a small step: C0F0[6804]=0x4540298 to 0x4540297 (2520x1080 to 2512x1080 on 700D X5 mode)
Now this how it looks the live view:
(https://preview.ibb.co/f8BaAy/LV_000_00000.jpg) (https://ibb.co/mRGmHd)
Let's try to fix it with above registers,, what about ADTG[80a0]=0x1090000 to 0x1080000
(https://preview.ibb.co/e2friJ/LV_001_00000.jpg) (https://ibb.co/fZFP3J)
A lot better?
This is simple example but the ugly news I couldn't do it for increasing the resolution,, the live view freezes when adjusting ADTG[80a0] in the opposite way.
There also registers for stretching the live view.
I will edit the post later.
Would those registers for 700D play nicely with an eos 100D?
@Danne
I don't know if the registers in 700D identical to 100D or not and if it was identical maybe the values will be different.
Make a try!
I had a couple of crashes on my 700D while switching from crop mode to non-crop mode a week or so ago. I got distracted and didn't save log files or anything but if I can replicate them is that something that you guys are interested in?
In which case, what detail do you want?
From memory I'd switch from crop to non-crop (or the other way around - I can't remember now) and when I exited the ML menus the display would be in the previous mode but frozen and camera unresponsive. I think I might have had to take the battery out.
how come everything seems dead?
after a good period it seems that nothing more comes out, no experimetal of any kind all dead?
I do not know, but I fear that from 5d3 down now there is still very little to do and say, the hardware limits have been reached, and more than any small bugfux, I do not think we can push further, including preview.
moreover if I read well above, the main (if not unique) author of all these releases 4k exp. he remained without a suitable CF (which does not cost 2 pennies) and therefore without (fro now) the possibility of testing any further.
I would still say that even if everything stopped here for 5d3 it was a great adventure that has squeezed it a lot!
@leozan: If perceived stall in development is bothering you are heartly invited to join dev team.
@jimiz: Your statement about limits reached is true. As it was one year before. And the year before. And the year before that.
yes, of course, and Santa Claus exists. 8)
What would you still like on 5d3 that is not some bug fix? the preview realtime while recording in 3.5 k without stop and with audio as well ?? or 4k HDMI output in 14 bit? ah, ah, ah .... :D
I'm happy for everything that has been done to date ... and if nothing else comes out, I'm fine with it and I can also understand why accepting it and thanking it anyway!
the only news could be developed on 5d4 that has more hardware potential, but I do not think anything will come out, given the little circular of this machine and who owns it is not 'willing to do experiments.
moreover, the presence of excellent machines (also raw) at very low cost (blackmagic & lumix in the first place) brings people (including Devs) to other interests ...
I agree with you jimi.
Trollfan's answers without logic are not for me
I only saw an obvious general prolonged silence after a lucky period. I do not think in fact abounds of devs ...
and I agree too, that now the limits (and therefore novelty) at least for certain models, such as 5dmark3 have been achieved, except for a few maybe bug fixes. the fact remains (maybe not clear to someone) that the thanks for those who have done all this until now (ML DEVS) , remain and will remain indelible and alive for ever!!!
time will tell'....
I'm already satisfied like that
Hi every expert in ML Forum:
Could you please help me.
I have installed the main build in my 5D3 SD card and want to try 4K raw shooting.
Do I just
1)unzip "magiclantern-crop_rec_4k.2018Mar10.5D3123.zip" in the computer.
2)copy and paste the unzipped files to the original booted SD card in the computer.(It will replace the files with the same file name)
3) Plug the SD card into 5D3 and rotate to update firmware in 5D3.
4)Turn off and on the camera
Is it correct?
Thanks in advance.
Jay in Taipei
Some of us watch this thread for actual developments and not speculation, please have this discussion somewhere else.
Quote from: jpegmasterjesse on June 06, 2018, 03:41:44 PM
Some of us watch this thread for actual developments and not speculation, please have this discussion somewhere else.
This^^
2.5 K raw video on a several year old EOSM with UHS over clock and development has stopped ?! :o
How about, oh I don't know, maybe go and make some films. Cant be nice for hard working devs to read this crap.
I am not a native english-speaker and I do not understand exactly what this is about here...
But somehow I have the impulse at this point -
- to say my deepest thanks to the developers of ML for everything that has been and will be done here.
We are currently in postproduction for our second documentary which will be shown in cinemas.
Shot with 5dmk II & III ML raw, many sequences also in 3K raw.
My conclusion: There is no camera under 20.000,-€ which I would prefer!
It's just great, stunning and wonderful!
The images are not of this shimmering thinness of so many 4K shots.
They have substance, thickness and depth.
I love it!
Thank you!:)
Long time 6d shooter here.
In 1:1 pixel readout the 6d max resolution with magic lantern is 2688 x 958
Now with lossless raw and SD-UHS overclock working on the 6d(giving about 70Mb/s write speed), the time is right to go for 3k/4k on the 6d.
Although I would be very happy with 2.5K in 1:2.40 aspect ratio for now :P
That would be 2560 x 1072, after that 2880 x 1200 would be a nice goal 8)
So at the moment I want more vertical resolution.
Now I have been messing with adtg_gui for a long time, but I just can't get any more resolution, not horizontally and not vertically.
I'm able to make the resolution higher with the '6804' register, but when I change it, to a higher vertical resolution, the last rows of pixels I added is just filled with garbage (black/blue noise, or coloured lines).
I'v also played a lot with the fps timers '6008' and '6014', but not much luck getting the added resolution filled with useful image :-\
Since liveview is messed up very quick with trial and error with these registers, I use ML preview to check if I'm getting any useful image.
How do you guys get higher (vertical) resolution, what's your workflow in finding out ?
Here's an example of a 2688 x 1072 resolution on 6d, see the garbage at the lower bottom of the frame.
How do I get useful image data at the bottom of the frame, which registers to tweak ::)
(https://farm2.staticflickr.com/1731/41840201215_24648dbb01_h.jpg)
It's in cop_rec code. You might want to try these following six adtg2 registers from 5D3 for vertical resolution:
https://bitbucket.org/hudson/magic-lantern/src/0075d0fa70f229ec972d7831aa756c4a9280e625/modules/crop_rec/crop_rec.c?at=crop_rec_4k&fileviewer=file-view-default#crop_rec.c-809
It also seems you have to modify timer a C0F06014 for verticl adjustment
Thanks for the hints Nikfreak
From these 6 registers I haven't seen these 2 on 6d ???
0x8196
0x8197
But I noticed these 4 registers use similar values for vertical resolution as used in 0x6804
0x8178N = same value as vertical 6804 value + 1
0x8179N = same value as vertical 6804
0x82f8N = a little bit less as half the vertical resolution
0x82f9N = same value as half the vertical resolution
Standard values for these registers and the 6804 resolution register:
1080p 5xzoom 1:1
8178N 4e6 3de
8179N 617 470
82f8N 4e5 3dd
82f9N 640 494
6804 4e5 3dd
4e5 translates to 1253 decimal which is vertical resolution in 1080 mode minus the black border
3dd transaltes to 989 decimal which is vertical resolution in crop 1:1 mode minus the black border
As you can see , the 494 of register 82f9N comes close to half the vertical resolution 494 x 2 = 988
The problem for me however is tweaking the 4 registers, they use NRZI values, can't get my head around the NRZI to hexadecimal/decimal values... so I at the moment I haven't successfully altered these values :-\
they show up normal in hexadecimal values in adtg_gui interface.
(https://farm2.staticflickr.com/1724/42691587742_1368513d11_b.jpg)
But once I will alter and override them I see non hexadecimal values:
(https://farm2.staticflickr.com/1743/41841260095_beccfa4dd2_b.jpg)
@nikfreak
Nice progress for 100D :D
Now any suggestions for making united resolutions or presets for small cameras ?
From your screenshots I found out 100D / 700D share almost same values and as you already notice 8-12 bit lossless won't work. Just 14 bit lossless and normal 10,12,14 bit. We can do a preset in crop_rec module by defining some safe deltas for all needed registers but which resolution makes sense in terms of useful recording speed I can't tell atm. Danne maybe can do some tests but 3k width was just a second or so. I am rather interested in this (https://www.magiclantern.fm/forum/index.php?topic=16040.msg202459#msg202459) but not yet there
We really need lossless 10/12bit. Was it a couple of thousands register to search? Maybe set up a lua script going through them all? How, what to look for?
Since these registers seems parallel too existing 2520x1072 we can't get real time live view either? Really mysterious stuff to me :P
Have to get back on what would be considered good ratios and presets. It's good adtg practice either way.
It's continuous in 15 fps at least 8)
@nikfreak
If the register values in 100D identical to 700D ,, Maybe this will helps for full resolution live view I got 5208x3240 you can find the preset in the link:
https://docs.google.com/spreadsheets/d/1TzPxh7ai5JAoEI8zllaiiF1IDF5tAQam2BKRujyIYQU/
10-bit lossless isn't enough for continues 3K @ 24fps we need at least 70mb/s write speed (RAW calculator (http://rawcalculator.bitballoon.com/calculator_desktop)) --> effective write speed using sd_uhs is ~55mb/s in video mode.
So we must get the full overclocked speed sdr_104 @ 208 MHz --> 104mb/s and the effective maybe will be about ~80mb/s
Cool, there's a 208mhz frequency?
Nice spreadsheet by the way.
Clock speed 208 MHz does not mean we have an SDR208/UHS208 mode. See http://hjreggel.net/cardspeed/index.html
@Walter
What I thought too.
@theBilalFakhouri
I tried your registers and could expand to 5796x160 on the 100D. height numbers totally off but verttical seems cool. Need to do some more testing. Fps 6.5 as described in your sheet.
Wow, Danne and Nikfreak! What a remarkable expert tandem! This is a breath taking development that you guys are working on!
Quote from: Danne on June 12, 2018, 11:11:27 PM
We really need lossless 10/12bit. Was it a couple of thousands register to search? Maybe set up a lua script going through them all? How, what to look for?
Since these registers seems parallel too existing 2520x1072 we can't get real time live view either? Really mysterious stuff to me :P
Have to get back on what would be considered good ratios and presets. It's good adtg practice either way.
It's continuous in 15 fps at least 8)
Danne,
When leaving for vacation I was agonizing about which camera to take with me:
1) The 100D with its superb feature set, ease of use, stability, overclocked SD interface but vertical resolution limited to 1072 pixels or
2) The 7D with its old December 1, 2016 build, poor stability, lack of correct framing during filming BUT 1200 pixels vertical resolution.
Finally, I decided to take the 7D with me and I am happy to say that I made the right decision. I film at 2520x1200 resolution all the time now and have to say that the 128 pixels larger vertical resolution, compared to the 100D, really makes a big difference in the vision of the recorded clips. I don't like my clips to look like ribbons and occupy just half of my 16:9 screen. Something close to 16:9 looks so much better on the screen.
Now I see that you have succeeded in achieving continuous recording at 2904x1248 resolution and 24 fps. This vertical resolution is fantastic already! If you could achieve and hardcode 2520x1248 (2,02:1) or even 2209x1248 (16:9) with 10/12-bit lossless compression and SD-overclocking, yielding 5-12 s recording times at 24 fps, that would be GOLDEN! Also, a hardcoded continuous full-sensor-size recording at 1 to 3 fps at 14-bit lossless in a MLV container for smooth, (with no dead time between frames), timelapses would be PLATINUM! I don't think that hunting for more ratios than that or trying to further overclock the interface at the cost of reduced stability would make much sense, at least for now.
I wish, my good old 7D had the current functionality of the 100D. Video quality on the 7D at 2520x1200 resolution and 12-bit uncompressed is absolutely gorgeous!
Keeping my thumbs pressed for you guys to succeed!
Some little progress on the 6d, very slow and many many soft bricks.
I managed to alter the 5x zoom mode, so it has the vertical resolution of normal 1080p mode.
I got 2 MLV's, one in 2688 x 1224 at 24 fps resolution and one in 2688 x 1124 at 24 fps (YES 1:2.39 aspect ratio :D)
Here's te proof:
(https://farm2.staticflickr.com/1723/42078851264_bc97e2f648_b.jpg)
But here's the thing...I can't recreate it :P :-[ :'(
I used the excell sheet of theBilalFakhouri as starting point.
And changed these registers (Don't know exactly which mode I was in when I started altering these registers):
CMOS 7 -> 268
6014 -> 5b4
6804 -> 4e5 0 2c0
8172N -> 4e6(695NRZI)
8178N -> 4e6(695NRZI)
713c -> 516
8000 -> 5
And after that start video mode and press magnify button 2 times so I'm in 5x zoom mode.
Will do some more research...
Cool! Should make a preset like the one I got from Nikfreak so you can go right into action. :)
@Levas
Here´s a adtg_gui.mo version for 6D which gets you into crop registers after selecting digic registers and then select 1:1 3K Crop mode (5D 3) setting. Or use the included adtg_gui helper lua script :). I removed the sd_uhs part...
Download
https://bitbucket.org/Dannephoto/magic-lantern/downloads/adtg_gui_6D_lua.zip
howto:
https://www.youtube.com/watch?time_continue=65&v=GE5na3c0zOQ
commit:
https://bitbucket.org/Dannephoto/magic-lantern/commits/10bc7a9c48eff9728179bdd3435233af6c5a21ef
Some of your registers were a bit fishy looking so the preset has some 700D or 100D starting point but you know how to alter the values.
Thanks Danne, this helps a lot!
Wich registers looked fishy to you?
You do know that the 6d has 25.6 mhz clock speed ?
So A and B timers will look different then 5d3 and 100d/700d.
I'm not sure but I also think 6d has other CMOS numbers usage then other cams.
CMOS 3 is for analog iso value, I know for sure.
CMOS 6 is for horizontal offset, I think ???
CMOS 7 is for vertical offset, I think ???
There's spaces in 6804? Can it be? Anyway if you know how to compile you can add your own numbers in adtg_gui.c code and compile only the specific module from my branch.
Or just send me your latest numbers and I compile it for you when you feel you're on to something :)
Ah the spaces, you're right, I did that on purpose because it actually are 2 numbers for resolution splitted by the 0.
It should be without the spaces indeed 4e502c0
4e5 is the vertical resolution (4e5 -> 1253 decimal )
2c0 is the horizontal resolution x4 columns (2c0 -> 704 x4 = 2816)
2816 x 1253 minus blackborders results in 2688 x 1224 resolution.
You mean there's actually logic in the registrys. I have to sit down with thi :).
Tell me if the module works or if you want any changes to be done when you can.
@Danne, looks like it's working.
These registers are overriden when I load Croprec 3k 5d3:
(https://farm1.staticflickr.com/900/40988372710_c976fca2bf_b.jpg)
Will dig into it and see if I can get some usable combinations for the 6d :D
Looked at crop_rec.c today and found a bunch of registers which are tweaked for the 5d3.
For most of these registers I could find the 6d Values
I summed the values up in 1080-25fps mode, 5xzoom 1:1 mode and photo mode (for full res live view).
So these are the standard values for these registers in these modes.
http://docs.google.com/spreadsheets/d/1iapLI7UrgfCJGwPSFsyhYeKl8fSTY4RVXvDR7MiaOp4/edit?usp=sharing (http://docs.google.com/spreadsheets/d/1iapLI7UrgfCJGwPSFsyhYeKl8fSTY4RVXvDR7MiaOp4/edit?usp=sharing)
For those interested in 6d.
As a bonus, already a column for overriding registers for 3K resolution, in 4.7 fps :P
@Alex,
Doing a lot of trial and error with adtg_gui on 6d.
And the camera doesn't behave consistent on same alterations each time, also getting some memory errors once in a while.
I'm using the magiclantern-crop_rec_4k.2018Mar10.6D116 build from the experiments download page.
And there's something going on with the memory dialog.
When I do a clean start and open up the 'free memory' dialog, the camera crashes immediately in error 70
Crash log file:
ASSERT: hSuite != 0
at RscMgr.c:2228, guess_mem:ff0f41ec
lv:1 mode:3
guess_mem stack: 1e3308 [1e33c8-1df3c8]
0xUNKNOWN @ ea80:1e33c0
0x0045C240 @ 44dde8:1e3380
0x0045C120 @ 45c29c:1e3368
0xFF14A580 @ 45c140:1e3360
0x00001900 @ ff0f41e8:1e3340
0x0044C478 @ 44c4f4:1e3308
Magic Lantern version : crop_rec_4k.2018Mar10.6D116
Mercurial changeset : 57614b3f26e4+02e5918a6ed5+ (crop_rec_4k) tip
Built on 2018-03-10 22:22:24 UTC by jenkins@nightly.
Free Memory : 330K + 984K
Downloaded latest crop rec 10 march build for 5D 123. It should have audio with mlv_snd module activated, no? Normaly then there is the option in the audio menu after activating, or is that only working for 113 build? Thanks!
Quote from: Levas on June 14, 2018, 09:58:51 PM
And there's something going on with the memory dialog.
When I do a clean start and open up the 'free memory' dialog, the camera crashes immediately in error 70
Best guess: raw video (with mlv_lite) enabled; on this branch, it preallocates the entire memory during standby. Try turning it off.
Quote from: Levas on June 14, 2018, 07:54:42 PM
So these are the standard values for these registers in these modes.
http://docs.google.com/spreadsheets/d/1iapLI7UrgfCJGwPSFsyhYeKl8fSTY4RVXvDR7MiaOp4/edit?usp=sharing (http://docs.google.com/spreadsheets/d/1iapLI7UrgfCJGwPSFsyhYeKl8fSTY4RVXvDR7MiaOp4/edit?usp=sharing)
Nice job documenting these. Can you do the same for all other video modes? On 5D3, there were some interesting changes between 1080p 24/25/30 (besides FPS timers). Comparing 1080p with 720p is also interesting, but on 5D3, that can only be done without DIGIC registers enabled (otherwise it locks up).
Edit: 50p no longer locks up after commenting out the patch for EngDrvOut.
@Alex
Updated the spreadsheet with 1080-24fps and 1080-30fps mode.
Weirdly enough, I couldn't get the 6014 register show up in adtg_gui in 1080-30fps mode ?
But I think the interesting part is in the changes in the 8179N and 82f9N registers I guess ???
http://docs.google.com/spreadsheets/d/1iapLI7UrgfCJGwPSFsyhYeKl8fSTY4RVXvDR7MiaOp4/edit?usp=sharing (http://docs.google.com/spreadsheets/d/1iapLI7UrgfCJGwPSFsyhYeKl8fSTY4RVXvDR7MiaOp4/edit?usp=sharing)
At the moment I'm out of battery juice, will check 720p modes soon.
@Camerite -> sound recording with MLV_lite is now visible in raw video menu, at the bottom, same menu where you set raw resolution etc.
@Alex,
Updated the spreadsheet now with 50fps and 60fps 720p modes.
I guess all standard values are in the sheet now.
Please have a look and enlighten me on how to get the good stuff on the 6d ;D
http://docs.google.com/spreadsheets/d/1iapLI7UrgfCJGwPSFsyhYeKl8fSTY4RVXvDR7MiaOp4/edit?usp=sharing (http://docs.google.com/spreadsheets/d/1iapLI7UrgfCJGwPSFsyhYeKl8fSTY4RVXvDR7MiaOp4/edit?usp=sharing)
Interesting found, CMOS 8, not listed in "known registers" but it has another value in full resolution photo mode.
When locked in normal 1080-25fps mode, it messes with the live view, sort of a double ghost image look.
Took a closer look at these registers in 5D3 and matched them with some firmware strings:
1080p24 1080p25 1080p30 1080p24->40 1080p25->40 720p50 720p60 x5 photo full-res LV
0x8178, 0x8196 PowerSaveTiming on 1321 1321 1321 1321 1321 695 695 1383 unused 3933
0x8179, 0x8197 PowerSaveTiming off 2193 1924 1738 1418 1424 924 828 1486 unused 3945
0x82F8 ReadOutTiming 1320 1320 1320 1320 1320 694 694 1382 unused 3932
0x82F9 set to TimerB - 1 2274 1999 1819 1499 1499 999 909 1555 unused 4020
FPS timer A 440 480 440 400 400 480 440 518 792 792
FPS timer B 2275 2000 1820 1500 1500 1000 910 1556 3958 4094
C0F06800 - C0F06804 [ 2080 x 1319 ] [ 2080 x 693 ] 3744x1381 5936x3949 5936x3931
The PowerSaveTiming registers, if misconfigured, are causing artifacts like this (https://www.magiclantern.fm/forum/index.php?topic=9741.msg202696#msg202696). Time units are from 0 to FPS timer B - 1; this covers a complete LiveView frame. For each timer B increment, one line is read out (well, 80D reads two lines per tick, but that's another story).
If the powersave timing registers are set to some value lower than desired vertical resolution, you will not get valid data on the lines in-between. Example: if you set the "on" register to 1320 / 2 = 660, and the "off" register to something a bit higher, e.g. 670, you'll get a thin bar of missing data somewhere in the middle of the image. Canon firmware enables powersaving after one frame was fully read out, until the next frame. That means, the "on" timing is just a bit higher than the desired vertical resolution (including black bars etc) and the "off" timing is just a bit lower than FPS timer B (aka total frame duration, one unit being timer A / MainClock, e.g. 440 / 24MHz = 18.33 microseconds).
If you set both powersave registers to 0, it seems to work just fine, although it might run a little hotter than usual.
TODO: now that we know their meaning, these registers should be set properly to reduce power consumption. In the current source, these were set by trial and error, so there is room for improvement.
Register 0x82F8 affects the image: if you set it to 660 (i.e. nrzi_encode(660) = 0x3de), the bottom half of the image will get darker, with some artifacts near highlights.
Register 0x82F9 had no obvious effect in my quick tests.
On 6D and 700D, the powersave timing registers are 8172/8178 (on), 8173/8179 (off), 82f8 (readout timing), 82f9 (unknown). On 700D, register 82B6 is set to powersave timing "on" - 2. All of them are NRZI-encoded.
@Levas,
Thanks man! Found it now hidden on the bottom somewhere! Appreciate it!
@a1ex
I am trying to implement new resolution in crop_rec.c ,, can you please review this commit (https://bitbucket.org/bilal_fakhouri/magic-lantern-crop_rec/commits/6b2bf96fff96c8b394017ecf93c42c72a781db52?at=crop_rec_4k) .
How I can override C0F0 registers in the same way (case & return) instead of delta things ? Can please give an example or a code for that ?
I tried this but didn't work:
static inline uint32_t reg_override_3k_700d(uint32_t reg, uint32_t old_val)
{
switch (reg)
{
case 0xC0F06800:
return 0x10010; /* raw start line/column, from photo mode */
case 0xC0F06804: /* 1080p 0x528011B, photo 0xF6E02FE */
return 0x5340322;
case 0xC0F06824:
case 0xC0F06828:
case 0xC0F0682C:
case 0xC0F06830:
return 0x3ca; /* from photo mode */
case 0xC0F06010: /* FPS timer A, for increasing horizontal resolution */
return 0x37b; /* from photo mode; lower values give black border on the right */
case 0xC0F06008:
case 0xC0F0600C:
return 0x37b037b;
case 0xC0F06014:
return 0x6d7;
case 0xC0F0713c:
return 0x555;
}
}
static void * get_engio_reg_override_func()
{
uint32_t (*reg_override_func)(uint32_t, uint32_t) =
//(crop_preset == CROP_PRESET_3X) ? reg_override_top_bar : /* fixme: corrupted image */
(crop_preset == CROP_PRESET_3X_TALL) ? reg_override_3X_tall :
(crop_preset == CROP_PRESET_3x3_1X) ? reg_override_3x3_tall :
(crop_preset == CROP_PRESET_3x3_1X_48p) ? reg_override_3x3_48p :
(crop_preset == CROP_PRESET_3K) ? reg_override_3K :
(crop_preset == CROP_PRESET_4K_HFPS) ? reg_override_4K_hfps :
(crop_preset == CROP_PRESET_UHD) ? reg_override_UHD :
(crop_preset == CROP_PRESET_40_FPS) ? reg_override_40_fps :
(crop_preset == CROP_PRESET_FULLRES_LV) ? reg_override_fullres_lv :
(crop_preset == CROP_PRESET_CENTER_Z) ? reg_override_zoom_fps :
(crop_preset == CROP_PRESET_3K_700D) ? reg_override_3k_700d :
0 ;
return reg_override_func;
}
Should I make CENTER_Z preset for 700D ? and as I understood this preset for detecting centered live-view and making live-view centered in x5 ,, right?
Is this right to override adtg ? :
case CROP_PRESET_3K_700D:
adtg_new[1] = (struct adtg_new) {2, 0x81B6, 0x7f4};
adtg_new[2] = (struct adtg_new) {2, 0x8172, 0x77c};
adtg_new[3] = (struct adtg_new) {2, 0x8178, 0x77c};
break;
Any tips I will be appreciated!
+1 for above post.
Thanks for helping out @theBilalFakhouri. Think I got two really nice presets for the 100D going now, 2520x1304 24.006fps and 3096x1320 20fps :). Hopefully we can get them into crop_rec.c soon :P
https://www.magiclantern.fm/forum/index.php?topic=16040.msg202808#msg202808
The latest crop_rec build for 5D3 1.1.3 shows me an incorrect live-view mode. After having imported the footage, they all turned out shifted lower than what I was seeing on my 5D3 screen. Also the aspect ratio crop is wrong. It says to crop it 1.67x but that shows a completely distorted image
Edit: also the exposure is way off. screen on 5D3 shows a way darker image
Quote from: pewpow on June 18, 2018, 04:39:37 PM
Edit: also the exposure is way off. screen on 5D3 shows a way darker image
Use RAW exposure aids like RAW histogram and RAW zebras to accurately measure the exposure of an image, the Liveview does its best at representing a true image, but is in fact more representive of JPEG or h264 capture; this will be evident when enabling RAW zebras for instance. Also if you have your screen brightness set to a lower value than max, that will also make it more difficult to judge accurate exposure, by the image alone.
Update for 6D.
Small step further, find out why the camera isn't consistently with overriding registers...
Yesterday I was messing with adtg_gui and had mlv_lite set on plain 14 bit raw, first thing I did worked, got 2688 x 1222 resolution back :D
Messed some further around and seemed that raw video buffer breaks (ML framing preview) with 12 bit lossless and lower.
So it only seems to work fine in plain 14 bit mode and it seems that 14 bit lossless also works, but 12 bit lossless and lower messes with the raw image buffer :-\
So something needs fixing...not sure, probably in MLV_lite ?
This goes for 4k crop_rec experimental build on downloadpage.
Guess this is only the case for 6d ?
Cropped modes dont work with 10/12 bit lossless on the 5d3
This is new for me, always been that way, did it never work on cropped modes ?
@Levas
Check here regarding other than 14bit-lossless:
https://www.magiclantern.fm/forum/index.php?topic=19300.msg202052#msg202052
Oeps, gotta read the posts better here in this forum :P
Can somebody help with adding an extra crop_rec preset for the 6d ?
Crop_rec_4K branch already has one preset for the canon 6d(50fps with 3x3 readout), I would like to add another preset for the 6d.
Source file - Crop_rec.c - Crop_rec_4K branch:
https://bitbucket.org/hudson/magic-lantern/src/0e38b89bd0ad76a02fa8ef3a89961ddb115eb7bf/modules/crop_rec/crop_rec.c?at=crop_rec_4k&fileviewer=file-view-default (https://bitbucket.org/hudson/magic-lantern/src/0e38b89bd0ad76a02fa8ef3a89961ddb115eb7bf/modules/crop_rec/crop_rec.c?at=crop_rec_4k&fileviewer=file-view-default)
The preset is for 5x zoom mode 1:1 pixel readout.
Possible name for the preset -> 2688 x 1222 - 24fps
The registers to alter are:
CMOS 7 -> 268
8000 -> 5
800c -> 0
8172N -> 695 (695 is the hexadecimal value I enter in adtg_gui -> gives 4e6 value for NRZI)
8178N -> 695 (695 is the hexadecimal value I enter in adtg_gui -> gives 4e6 value for NRZI)
8179N -> 51c (51c is the hexadecimal value I enter in adtg_gui -> gives 617 value for NRZI)
81F8N -> 697 (697 is the hexadecimal value I enter in adtg_gui -> gives 4e5 value for NRZI)
81F9N -> 560 (560 is the hexadecimal value I enter in adtg_gui -> gives 640 value for NRZI)
713C -> 516
7150 -> 4e5
6014 -> 5b4
6804 -> 4e502c0
If somebody can add this preset and share the Crop_rec.c file, I have a good starting point for testing.
And I'm probably savvy enough to alter the preset with other values if I find some other nice values.
Small update - g3gg0 just reworked sound recording support for mlv_lite: https://bitbucket.org/hudson/magic-lantern/pull-requests/837/mlv_snd-and-dual_iso-support-for-mlv_lite
Current experimental build contains his changes (it's build from the crop_rec_4k_mlv_snd branch). I'm not the best person to test it (not exactly a video user), so... please give it a try. Comparisons with the old approach, used in previous builds, are welcome.
Meanwhile I'm working on integrating the changes from Bilal, Levas, Danne and Nikfreak, using the 60D as second camera (almost got full-res LiveView working there as well). The current experimental build doesn't have any of the recent resolution updates (yet).
Already committed an update to adtg_gui that solves lock-ups in 1080p 50/60 on 5D3, and possibly on other models as well.
Nice a1ex!
Reference x5 zoom tweaks tested working:
EOSM:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg202983#msg202983
100D:
https://www.magiclantern.fm/forum/index.php?topic=16040.msg202808#msg202808
@g3gg0
Nice to see progress in your mlv_snd branch. Difficult now to now what to do next. Working with current mlv_lite implementation or focus on your changes and switch branch. Guess tests will tell. Only had very short time to record a few MLV files on a 100D. Short summary:
1 - Records fine on both builds
2 - g3gg0 implementation breaks compatibility with MlRawViewver(metadata related?). Works fine in Mlv App, Switch, MLVFS
3 - rearrangements in mlv sound menu progress lags behind. The sound setting is since recently among RAW video settings which seems more logical. It´s also set to on per default when enabling mlv_snd.mo.
Comparing with this branch:
https://bitbucket.org/hudson/magic-lantern/branch/crop_rec_4k_mlv_lite_snd
What is your take on above branch g3gg0?
Quote from: Levas on June 19, 2018, 01:46:32 PM
Can somebody help with adding an extra crop_rec preset for the 6d ?
CMOS 7 -> 268
8000 -> 5
800c -> 0
8172N -> 695 (695 is the hexadecimal value I enter in adtg_gui -> gives 4e6 value for NRZI)
8178N -> 695 (695 is the hexadecimal value I enter in adtg_gui -> gives 4e6 value for NRZI)
8179N -> 51c (51c is the hexadecimal value I enter in adtg_gui -> gives 617 value for NRZI)
81F8N -> 697 (697 is the hexadecimal value I enter in adtg_gui -> gives 4e5 value for NRZI)
81F9N -> 560 (560 is the hexadecimal value I enter in adtg_gui -> gives 640 value for NRZI)
713C -> 516
7150 -> 4e5
6014 -> 5b4
6804 -> 4e502c0
6804 -> 4e502b0 double checked, this works in adtg_gui
mk11174 made a version of crop_rec.c with an 2K preset for the 6d.
Now fiddling with it, but can't get it to work.
first question is about the NRZI registers, do I write down the NRZI value in crop_rec.c or do I write down the hexadecimal values I put in these registers with adtg_gui ?Furthermore, the crop_rec.c from mk11174 does something to some registers, but not to all.
Looking at the liveview it looks like it does set lineskipping to 0 and CMOS 7 settings are done too, but resolution doesn't change (6804), fps doesn't change (6014) ???
I think it doesn't do the register settings that are starting in line 920 of crop_rec.c
Can somebody take a look at this and tell me where the problem is ?
Link to file:
https://drive.google.com/open?id=1ZJSjmNXS6hMRPTCmkGLiwWuGy_OPAH1T (https://drive.google.com/open?id=1ZJSjmNXS6hMRPTCmkGLiwWuGy_OPAH1T)
#include <dryos.h>
#include <module.h>
#include <config.h>
#include <menu.h>
#include <beep.h>
#include <property.h>
#include <patch.h>
#include <bmp.h>
#include <lvinfo.h>
#include <powersave.h>
#include <raw.h>
#include <fps.h>
#include <shoot.h>
#undef CROP_DEBUG
#ifdef CROP_DEBUG
#define dbg_printf(fmt,...) { printf(fmt, ## __VA_ARGS__); }
#else
#define dbg_printf(fmt,...) {}
#endif
static int is_700D = 0;
static int is_6D = 0;
static CONFIG_INT("crop.preset", crop_preset_index, 0);
static CONFIG_INT("crop.shutter_range", shutter_range, 0);
enum crop_preset {
CROP_PRESET_OFF = 0,
CROP_PRESET_HD,
CROP_PRESET_2K,
CROP_PRESET_ULTRA_HD,
CROP_PRESET_4K,
CROP_PRESET_FullRes,
CROP_PRESET_3x3_1X,
CROP_PRESET_3x3_1X_48p,
NUM_CROP_PRESETS
};
/* presets are not enabled right away (we need to go to play mode and back)
* so we keep two variables: what's selected in menu and what's actually used.
* note: the menu choices are camera-dependent */
static enum crop_preset crop_preset = 0;
/* must be assigned in crop_rec_init */
static enum crop_preset * crop_presets = 0;
/* current menu selection (*/
#define CROP_PRESET_MENU crop_presets[crop_preset_index]
/* menu choices for 700D */
static enum crop_preset crop_presets_700d[] = {
CROP_PRESET_OFF,
CROP_PRESET_HD,
CROP_PRESET_2K,
CROP_PRESET_ULTRA_HD,
CROP_PRESET_4K,
CROP_PRESET_FullRes,
CROP_PRESET_3x3_1X,
};
static const char * crop_choices_700d[] = {
"OFF",
"HD 1920x1080",
"2K 2520x1384",
"UltraHD 3840x2160",
"4K 4096x2560",
"FullRes 4464x2976",
"3x3 720p",
};
static const char crop_choices_help_700d[] =
"Change 1080p and 720p movie modes into crop modes (select one)";
static const char crop_choices_help2_700d[] =
"HD crop (1920x1080 @ 24p, square raw pixels, preview broken)\n"
"2K crop (2520x1384 @ 24p, square raw pixels, preview broken)\n"
"UltraHD (3840x2160 @ 12p, square raw pixels, preview broken)\n"
"4K (4096x2560 @ 9.477p, square raw pixels, preview broken)\n"
"FullRes (4464x2976 @ 6.527p, square raw pixels, preview broken)\n"
"3x3 binning in 720p (square pixels in RAW, vertical crop)\n";
/* menu choices for 6D */
static enum crop_preset crop_presets_6d[] = {
CROP_PRESET_OFF,
CROP_PRESET_2K,
CROP_PRESET_3x3_1X,
};
static const char * crop_choices_6d[] = {
"OFF",
"2K 2688x1222",
"3x3 720p",
};
static const char crop_choices_help_6d[] =
"Change 1080p and 720p movie modes into crop modes (select one)";
static const char crop_choices_help2_6d[] =
"2K crop (2688x1222 @ 24p, square raw pixels, preview broken)\n"
"3x3 binning in 720p (square pixels in RAW, vertical crop)\n";
/* camera-specific parameters */
static uint32_t CMOS_WRITE = 0;
static uint32_t MEM_CMOS_WRITE = 0;
static uint32_t ADTG_WRITE = 0;
static uint32_t MEM_ADTG_WRITE = 0;
static uint32_t ENGIO_WRITE = 0;
static uint32_t MEM_ENGIO_WRITE = 0;
/* video modes */
/* note: zoom mode is identified by checking registers directly */
static int is_1080p()
{
/* note: on 5D2 and 5D3 (maybe also 6D, not sure),
* sensor configuration in photo mode is identical to 1080p.
* other cameras may be different */
return !is_movie_mode() || video_mode_resolution == 0;
}
static int is_720p()
{
return is_movie_mode() && video_mode_resolution == 1;
}
static int is_supported_mode()
{
if (!lv) return 0;
switch (crop_preset)
{
default:
return is_1080p() || is_720p();
}
}
static int32_t target_yres = 0;
static int32_t delta_adtg0 = 0;
static int32_t delta_adtg1 = 0;
static int32_t delta_head3 = 0;
static int32_t delta_head4 = 0;
static uint32_t cmos1_lo = 0, cmos1_hi = 0;
static uint32_t cmos2 = 0;
/* helper to allow indexing various properties of Canon's video modes */
static inline int get_video_mode_index()
{
return
(video_mode_fps == 24) ? 0 :
(video_mode_fps == 25) ? 1 :
(video_mode_fps == 30) ? 2 :
(video_mode_fps == 50) ? 3 :
/* (video_mode_fps == 60) */ 4 ;
}
/* optical black area sizes */
/* not sure how to adjust them from registers, so... hardcode them here */
static inline void FAST calc_skip_offsets(int * p_skip_left, int * p_skip_right, int * p_skip_top, int * p_skip_bottom)
{
/* start from LiveView values */
int skip_left = 72;
int skip_right = 0;
int skip_top = 28;
int skip_bottom = 0;
switch (crop_preset)
{
case CROP_PRESET_HD:
case CROP_PRESET_2K:
case CROP_PRESET_ULTRA_HD:
case CROP_PRESET_4K:
case CROP_PRESET_FullRes:
case CROP_PRESET_3x3_1X:
case CROP_PRESET_3x3_1X_48p:
if (is_720p()) skip_top = 0;
break;
}
if (p_skip_left) *p_skip_left = skip_left;
if (p_skip_right) *p_skip_right = skip_right;
if (p_skip_top) *p_skip_top = skip_top;
if (p_skip_bottom) *p_skip_bottom = skip_bottom;
}
/* to be in sync with 0xC0F06800 */
static int get_top_bar_adjustment()
{
switch (crop_preset)
{
case CROP_PRESET_3x3_1X:
case CROP_PRESET_3x3_1X_48p:
if (is_720p()) return 28; /* 0x1D0017 from 0x10017 */
default:
return 30; /* 0x1F0017 from 0x10017 */ /* 0x1F0017 from 0x10017 */
}
}
/* Vertical resolution from current unmodified video mode */
/* (active area only, as seen by mlv_lite) */
static inline int get_default_yres()
{
return
(video_mode_fps <= 30) ? 1290 : 672;
}
/* skip_top from unmodified video mode (raw.c, LiveView skip offsets) */
static inline int get_default_skip_top()
{
return
(video_mode_fps <= 30) ? 28 : 20;
}
/* max resolution for each video mode (trial and error) */
/* it's usually possible to push the numbers a few pixels further,
* at the risk of corrupted frames */
static int max_resolutions[NUM_CROP_PRESETS][5] = {
/* 24p 25p 30p 50p 60p */
[CROP_PRESET_3x3_1X] = { 1290, 1290, 1290, 960, 800 },
[CROP_PRESET_3x3_1X_48p] = { 1290, 1290, 1290, 1080, 1040 }, /* 1080p45/48 */
[CROP_PRESET_HD] = { 1920, 1728, 1070, 760, 680 },
[CROP_PRESET_2K] = { 1920, 1728, 1384, 760, 680 },
[CROP_PRESET_ULTRA_HD] = { 1920, 1728, 2160, 760, 680 },
[CROP_PRESET_4K] = { 1920, 1728, 2560, 760, 680 },
[CROP_PRESET_FullRes] = { 1920, 1728, 3490, 760, 680 },
};
/* 5D3 vertical resolution increments over default configuration */
/* note that first scanline may be moved down by 30 px (see reg_override_top_bar) */
static inline int FAST calc_yres_delta()
{
int desired_yres = (target_yres) ? target_yres
: max_resolutions[crop_preset][get_video_mode_index()];
if (desired_yres)
{
/* user override */
int skip_top;
calc_skip_offsets(0, 0, &skip_top, 0);
int default_yres = get_default_yres();
int default_skip_top = get_default_skip_top();
int top_adj = get_top_bar_adjustment();
return desired_yres - default_yres + skip_top - default_skip_top + top_adj;
}
ASSERT(0);
return 0;
}
#define YRES_DELTA calc_yres_delta()
/* return value:
* 1: registers checked and appear OK (1080p/720p video mode)
* 0: registers checked and they are not OK (other video mode)
* -1: registers not checked
*/
static int FAST check_cmos_vidmode(uint16_t* data_buf)
{
int ok = 1;
int found = 1;
while (*data_buf != 0xFFFF)
{
int reg = (*data_buf) >> 12;
int value = (*data_buf) & 0xFFF;
if (is_700D)
{
if (reg == 7)
{
found = 1;
/* prevent running in 600D hack crop mode */
if (value != 0x800)
{
ok = 0;
}
}
}
data_buf++;
}
if (found) return ok;
return -1;
}
/* pack two 6-bit values into a 12-bit one */
#define PACK12(lo,hi) ((((lo) & 0x3F) | ((hi) << 6)) & 0xFFF)
/* pack two 16-bit values into a 32-bit one */
#define PACK32(lo,hi) (((uint32_t)(lo) & 0xFFFF) | ((uint32_t)(hi) << 16))
/* pack two 16-bit values into a 32-bit one */
#define PACK32(lo,hi) (((uint32_t)(lo) & 0xFFFF) | ((uint32_t)(hi) << 16))
static void FAST cmos_hook(uint32_t* regs, uint32_t* stack, uint32_t pc)
{
/* also check CMOS registers; in zoom mode, we get different values
* and this check is instant (no delays).
*
* on 5D3, the 640x480 acts like 1080p during standby,
* so properties are our only option for that one.
*/
uint16_t* data_buf = (uint16_t*) regs[0];
int cmos_new[10] = {-1, -1, -1, -1, -1, -1, -1, -1, -1, -1};
if (is_700D)
{
switch (crop_preset)
{
case CROP_PRESET_HD:
cmos_new[5] = 0x380 + delta_head3; /* horizontal centering (trial and error) */
cmos_new[7] = 0xAAA + delta_head4; /* pink highlights without this */
break;
case CROP_PRESET_2K:
cmos_new[5] = 0x302 + delta_head3; /* horizontal centering (trial and error) */
cmos_new[7] = 0xaa9 + delta_head4; /* pink highlights without this */
break;
case CROP_PRESET_ULTRA_HD:
cmos_new[5] = 0x180 + delta_head3; /* horizontal centering (trial and error) */
cmos_new[7] = 0x306 + delta_head4; /* pink highlights without this */
break;
case CROP_PRESET_4K:
cmos_new[5] = 0x100 + delta_head3; /* horizontal centering (trial and error) */
cmos_new[7] = 0xf84 + delta_head4; /* pink highlights without this */
break;
case CROP_PRESET_FullRes:
cmos_new[5] = 0x0DE + delta_head3; /* horizontal centering (trial and error) */
cmos_new[7] = 0x802 + delta_head4; /* pink highlights without this */
break;
case CROP_PRESET_3x3_1X:
/* start/stop scanning line, very large increments */
cmos_new[7] = PACK12(6,29);
break;
}
}
else if (is_6D)
{
switch (crop_preset)
{
case CROP_PRESET_2K:
cmos_new[7] = 0x268 + delta_head4; /* pink highlights without this */
break;
case CROP_PRESET_3x3_1X:
/* start/stop scanning line, very large increments */
cmos_new[7] = PACK12(37,10);
break;
}
}
/* menu overrides */
if (cmos1_lo || cmos1_hi)
{
cmos_new[1] = PACK12(cmos1_lo,cmos1_hi);
}
if (cmos2)
{
cmos_new[2] = cmos2;
}
/* copy data into a buffer, to make the override temporary */
/* that means: as soon as we stop executing the hooks, values are back to normal */
static uint16_t copy[512];
uint16_t* copy_end = ©[COUNT(copy)];
uint16_t* copy_ptr = copy;
while (*data_buf != 0xFFFF)
{
*copy_ptr = *data_buf;
int reg = (*data_buf) >> 12;
if (cmos_new[reg] != -1)
{
*copy_ptr = (reg << 12) | cmos_new[reg];
dbg_printf("CMOS[%x] = %x\n", reg, cmos_new[reg]);
}
data_buf++;
copy_ptr++;
if (copy_ptr > copy_end) while(1);
}
*copy_ptr = 0xFFFF;
/* pass our modified register list to cmos_write */
regs[0] = (uint32_t) copy;
}
static uint32_t nrzi_encode( uint32_t in_val )
{
uint32_t out_val = 0;
uint32_t old_bit = 0;
for (int num = 0; num < 31; num++)
{
uint32_t bit = in_val & 1<<(30-num) ? 1 : 0;
if (bit != old_bit)
out_val |= (1 << (30-num));
old_bit = bit;
}
return out_val;
}
static uint32_t nrzi_decode( uint32_t in_val )
{
uint32_t val = 0;
if (in_val & 0x8000)
val |= 0x8000;
for (int num = 0; num < 31; num++)
{
uint32_t old_bit = (val & 1<<(30-num+1)) >> 1;
val |= old_bit ^ (in_val & 1<<(30-num));
}
return val;
}
static int FAST adtg_lookup(uint32_t* data_buf, int reg_needle)
{
while(*data_buf != 0xFFFFFFFF)
{
int reg = (*data_buf) >> 16;
if (reg == reg_needle)
{
return *(uint16_t*)data_buf;
}
}
return -1;
}
static void * get_engio_reg_override_func();
/* from SENSOR_TIMING_TABLE (fps-engio.c) */
/* hardcoded for 5D3 */
const int default_timerA[] = { 0x20F, 0x27F, 0x20f, 0x27F, 0x20F };
const int default_timerB[] = { 0x9DE, 0x7CF, 0x7E5, 0x3D7, 0x3F2 };
const int default_fps_1k[] = { 23976, 25000, 29973, 50000, 59940 };
/* adapted from fps_override_shutter_blanking in fps-engio.c */
static int adjust_shutter_blanking(int old)
{
/* sensor duty cycle: range 0 ... timer B */
int current_blanking = nrzi_decode(old);
int video_mode = get_video_mode_index();
int fps_timer_b_orig = default_timerB[video_mode];
int current_exposure = fps_timer_b_orig - current_blanking;
/* wrong assumptions? */
if (current_exposure < 0)
{
return old;
}
int default_fps = default_fps_1k[video_mode];
int current_fps = fps_get_current_x1000();
dbg_printf("FPS %d->%d\n", default_fps, current_fps);
float frame_duration_orig = 1000.0 / default_fps;
float frame_duration_current = 1000.0 / current_fps;
float orig_shutter = frame_duration_orig * current_exposure / fps_timer_b_orig;
float new_shutter =
(shutter_range == 0) ?
({
/* original shutter speed from the altered video mode */
orig_shutter;
}) :
({
/* map the available range of 1/4000...1/30 (24-30p) or 1/4000...1/60 (50-60p)
* from minimum allowed (1/15000 with full-res LV) to 1/fps */
int max_fps_shutter = (video_mode_fps <= 30) ? 33333 : 64000;
int default_fps_adj = 1e9 / (1e9 / max_fps_shutter - 250);
(orig_shutter - 250e-6) * default_fps_adj / current_fps;
});
uint32_t (*reg_override_func)(uint32_t, uint32_t) =
get_engio_reg_override_func();
/* what value we are going to use for overriding timer B? */
int fps_timer_b = (reg_override_func)
? (int) reg_override_func(0xC0F06014, fps_timer_b_orig - 1)
: fps_timer_b_orig;
/* will we actually override it? */
fps_timer_b = fps_timer_b ? fps_timer_b + 1 : fps_timer_b_orig;
dbg_printf("Timer B %d->%d\n", fps_timer_b_orig, fps_timer_b);
int new_exposure = new_shutter * fps_timer_b / frame_duration_current;
int new_blanking = COERCE(fps_timer_b - new_exposure, 10, fps_timer_b - 2);
dbg_printf("Exposure %d->%d (timer B units)\n", current_exposure, new_exposure);
#ifdef CROP_DEBUG
float chk_shutter = frame_duration_current * new_exposure / fps_timer_b;
dbg_printf("Shutter %d->%d us\n", (int)(orig_shutter*1e6), (int)(chk_shutter*1e6));
#endif
dbg_printf("Blanking %d->%d\n", current_blanking, new_blanking);
return nrzi_encode(new_blanking);
}
extern WEAK_FUNC(ret_0) void fps_override_shutter_blanking();
static void FAST adtg_hook(uint32_t* regs, uint32_t* stack, uint32_t pc)
{
/* This hook is called from the DebugMsg's in adtg_write,
* so if we change the register list address, it won't be able to override them.
* Workaround: let's call it here. */
fps_override_shutter_blanking();
uint32_t cs = regs[0];
uint32_t *data_buf = (uint32_t *) regs[1];
int dst = cs & 0xF;
/* copy data into a buffer, to make the override temporary */
/* that means: as soon as we stop executing the hooks, values are back to normal */
static uint32_t copy[512];
uint32_t* copy_end = ©[COUNT(copy)];
uint32_t* copy_ptr = copy;
struct adtg_new
{
int dst;
int reg;
int val;
};
/* expand this as required */
struct adtg_new adtg_new[10] = {{0}};
/* scan for shutter blanking and make both zoom and non-zoom value equal */
/* (the values are different when using FPS override with ADTG shutter override) */
/* (fixme: might be better to handle this in ML core?) */
int shutter_blanking = 0;
int adtg_blanking_reg = (lv_dispsize == 1) ? 0x8061 : 0x805F;
for (uint32_t * buf = data_buf; *buf != 0xFFFFFFFF; buf++)
{
int reg = (*buf) >> 16;
if (reg == adtg_blanking_reg)
{
int val = (*buf) & 0xFFFF;
shutter_blanking = val;
}
}
/* some modes may need adjustments to maintain exposure */
if (shutter_blanking)
{
shutter_blanking = adjust_shutter_blanking(shutter_blanking);
}
if (is_700D)
{
/* all modes may want to override shutter speed */
/* ADTG[0x8061]: shutter blanking for 3x3 mode */
/* ADTG[0x805F]: shutter blanking for zoom mode */
adtg_new[0] = (struct adtg_new) {6, 0x8061, shutter_blanking};
adtg_new[1] = (struct adtg_new) {6, 0x805F, shutter_blanking};
switch (crop_preset)
{
/* all 1:1 modes (3x, 2K, 4K...) */
//case CROP_PRESET_HD:
//case CROP_PRESET_2K:
//case CROP_PRESET_ULTRA_HD:
//case CROP_PRESET_4K:
case CROP_PRESET_FullRes:
/* ADTG2/4[0x8000] = 5 (set in one call) */
/* ADTG2[0x8806] = 0x6088 (artifacts without it) */
adtg_new[2] = (struct adtg_new) {6, 0x8000, 5};
break;
}
switch (crop_preset)
{
case CROP_PRESET_FullRes:
/* ADTG2/4[0x8000] = 5 (set in one call) */
/* ADTG2[0x8806] = 0x6088 (artifacts without it) */
adtg_new[3] = (struct adtg_new) {6, 0x800C, 0};
break;
}
/* all modes with higher vertical resolution */
switch (crop_preset)
{
case CROP_PRESET_HD:
case CROP_PRESET_2K:
case CROP_PRESET_ULTRA_HD:
/* adjust vertical resolution */
adtg_new[2] = (struct adtg_new) {6, 0x8172, nrzi_encode(0x88E + delta_adtg0)};
adtg_new[3] = (struct adtg_new) {6, 0x8178, nrzi_encode(0x88E + delta_adtg0)};
adtg_new[4] = (struct adtg_new) {6, 0x82B6, nrzi_encode(0x906 + delta_adtg0)};
break;
case CROP_PRESET_4K:
/* adjust vertical resolution */
adtg_new[2] = (struct adtg_new) {6, 0x8172, nrzi_encode(0xC19 + delta_adtg0)};
adtg_new[3] = (struct adtg_new) {6, 0x8178, nrzi_encode(0xC19 + delta_adtg0)};
adtg_new[4] = (struct adtg_new) {6, 0x82B6, nrzi_encode(0xC10 + delta_adtg0)};
break;
case CROP_PRESET_FullRes:
/* adjust vertical resolution */
adtg_new[4] = (struct adtg_new) {6, 0x8172, nrzi_encode(0xBEB + delta_adtg0)};
adtg_new[5] = (struct adtg_new) {6, 0x8178, nrzi_encode(0xBEB + delta_adtg0)};
adtg_new[6] = (struct adtg_new) {6, 0x82B6, nrzi_encode(0xC6D + delta_adtg0)};
break;
/* 3x3 binning in 720p (in 1080p it's already 3x3) */
case CROP_PRESET_3x3_1X:
/* ADTG2/4[0x800C] = 2: vertical binning factor = 3 */
adtg_new[0] = (struct adtg_new) {6, 0x800C, 2};
break;
}
}
if (is_6D)
{
/* all modes may want to override shutter speed */
/* ADTG[0x8061]: shutter blanking for 3x3 mode */
/* ADTG[0x805F]: shutter blanking for zoom mode */
adtg_new[0] = (struct adtg_new) {6, 0x8061, shutter_blanking};
adtg_new[1] = (struct adtg_new) {6, 0x805F, shutter_blanking};
switch (crop_preset)
{
/* all 1:1 modes (3x, 2K, 4K...) */
case CROP_PRESET_2K:
/* ADTG2/4[0x8000] = 5 (set in one call) */
/* ADTG2[0x8806] = 0x6088 (artifacts without it) */
adtg_new[2] = (struct adtg_new) {6, 0x8000, 5};
break;
}
switch (crop_preset)
{
case CROP_PRESET_2K:
/* ADTG2/4[0x8000] = 5 (set in one call) */
/* ADTG2[0x8806] = 0x6088 (artifacts without it) */
adtg_new[3] = (struct adtg_new) {6, 0x800C, 0};
break;
}
/* all modes with higher vertical resolution */
switch (crop_preset)
{
case CROP_PRESET_2K:
/* adjust vertical resolution */
adtg_new[2] = (struct adtg_new) {6, 0x8172, nrzi_encode(0x4e6 + delta_adtg0)};
adtg_new[3] = (struct adtg_new) {6, 0x8178, nrzi_encode(0x4e6 + delta_adtg0)};
adtg_new[4] = (struct adtg_new) {6, 0x8179, nrzi_encode(0x617 + delta_adtg0)};
adtg_new[5] = (struct adtg_new) {6, 0x81F8, nrzi_encode(0x4e5 + delta_adtg0)};
adtg_new[6] = (struct adtg_new) {6, 0x81F9, nrzi_encode(0x640 + delta_adtg0)};
break;
/* 3x3 binning in 720p (in 1080p it's already 3x3) */
case CROP_PRESET_3x3_1X:
/* ADTG2/4[0x800C] = 2: vertical binning factor = 3 */
adtg_new[0] = (struct adtg_new) {6, 0x800C, 2};
break;
}
}
while(*data_buf != 0xFFFFFFFF)
{
*copy_ptr = *data_buf;
int reg = (*data_buf) >> 16;
for (int i = 0; i < COUNT(adtg_new); i++)
{
if ((reg == adtg_new[i].reg) && (dst & adtg_new[i].dst))
{
int new_value = adtg_new[i].val;
dbg_printf("ADTG%x[%x] = %x\n", dst, reg, new_value);
*(uint16_t*)copy_ptr = new_value;
if (reg == 0x805F || reg == 0x8061)
{
/* also override in original data structure */
/* to be picked up on the screen indicators */
*(uint16_t*)data_buf = new_value;
}
}
}
data_buf++;
copy_ptr++;
if (copy_ptr >= copy_end) while(1);
}
*copy_ptr = 0xFFFFFFFF;
/* pass our modified register list to adtg_write */
regs[1] = (uint32_t) copy;
}
/* these are required for increasing vertical resolution */
/* (used in most other presets) */
static inline uint32_t reg_override_HEAD34(uint32_t reg, uint32_t old_val)
{
switch (reg)
{
/* HEAD3 timer */
case 0xC0F0713C:
return old_val + YRES_DELTA + delta_head3;
/* HEAD4 timer */
case 0xC0F07150:
return old_val + YRES_DELTA + delta_head4;
}
return 0;
}
static inline uint32_t reg_override_common(uint32_t reg, uint32_t old_val)
{
uint32_t b = reg_override_HEAD34(reg, old_val);
if (b) return b;
return 0;
}
static inline uint32_t reg_override_fps(uint32_t reg, uint32_t timerA, uint32_t timerB)
{
/* hardware register requires timer-1 */
timerA--;
timerB--;
/* only override FPS registers if the old value is what we expect
* otherwise we may be in some different video mode for a short time
* this race condition is enough to lock up LiveView in some cases
* e.g. 5D3 3x3 50/60p when going from photo mode to video mode
*/
switch (reg)
{
case 0xC0F06824:
case 0xC0F06828:
case 0xC0F0682C:
case 0xC0F06830:
case 0xC0F06010:
return timerA;
case 0xC0F06008:
case 0xC0F0600C:
return timerA | (timerA << 16);
// case 0xC0F06014:
// return timerB;
}
return 0;
}
static inline uint32_t reg_override_3x3_tall(uint32_t reg, uint32_t old_val)
{
if (!is_720p())
{
/* 1080p not patched in 3x3 */
return 0;
}
/* change FPS timers to increase vertical resolution */
if (video_mode_fps >= 50)
{
int timerA = 400;
int timerB =
(video_mode_fps == 50) ? 1200 :
(video_mode_fps == 60) ? 1001 :
-1 ;
int a = reg_override_fps(reg, timerA, timerB);
if (a) return a;
}
/* fine-tuning head timers appears to help
* pushing the resolution a tiny bit further */
int head_adj =
(video_mode_fps == 50) ? -10 :
(video_mode_fps == 60) ? -20 :
0 ;
switch (reg)
{
/* for some reason, top bar disappears with the common overrides */
/* very tight fit - every pixel counts here */
case 0xC0F06800:
return 0x30010;
/* raw resolution (end line/column) */
case 0xC0F06804:
return old_val + (YRES_DELTA << 16);
/* HEAD3 timer */
case 0xC0F0713C:
return old_val + YRES_DELTA + delta_head3 + head_adj;
/* HEAD4 timer */
case 0xC0F07150:
return old_val + YRES_DELTA + delta_head4 + head_adj;
}
return reg_override_common(reg, old_val);
}
static inline uint32_t reg_override_3x3_48p(uint32_t reg, uint32_t old_val)
{
if (!is_720p())
{
/* 1080p not patched in 3x3 */
return 0;
}
/* change FPS timers to increase vertical resolution */
if (video_mode_fps >= 50)
{
int timerA =
(video_mode_fps == 50) ? 401 :
(video_mode_fps == 60) ? 400 :
-1 ;
int timerB =
(video_mode_fps == 50) ? 1330 : /* 45p */
(video_mode_fps == 60) ? 1250 : /* 48p */
-1 ;
int a = reg_override_fps(reg, timerA, timerB);
if (a) return a;
}
switch (reg)
{
/* for some reason, top bar disappears with the common overrides */
/* very tight fit - every pixel counts here */
case 0xC0F06800:
return 0x30010;
/* raw resolution (end line/column) */
case 0xC0F06804:
return old_val + (YRES_DELTA << 16);
/* HEAD3 timer */
case 0xC0F0713C:
return old_val + YRES_DELTA + delta_head3;
/* HEAD4 timer */
case 0xC0F07150:
return old_val + YRES_DELTA + delta_head4;
}
return reg_override_common(reg, old_val);
}
static inline uint32_t reg_override_HD(uint32_t reg, uint32_t old_val)
{
switch (reg)
{
/* raw resolution (end line/column) */
/* X: (3072+140)/8 + 0x17, adjusted for 3072 in raw_rec */
case 0xC0F06804: return 0x4540202; // 1920x1080 x5 Mode;
case 0xC0F06824: return 0x2ca;
case 0xC0F06828: return 0x2ca;
case 0xC0F0682C: return 0x2ca;
case 0xC0F06830: return 0x2ca;
case 0xC0F06010: return 0x2cb;
case 0xC0F06008: return 0x2cb02cb;
case 0xC0F0600C: return 0x2cb02cb;
case 0xC0F06014: return 0x747;
case 0xC0F0713c: return 0x5C7;
case 0xC0F07150: return 0x59A;
}
return 0;
}
static inline uint32_t reg_override_2K(uint32_t reg, uint32_t old_val)
{
if (is_700D)
{
switch (reg)
{
/* raw resolution (end line/column) */
/* X: (3072+140)/8 + 0x17, adjusted for 3072 in raw_rec */
case 0xC0F06804: return 0x5840298; // 2520x1386 x5 Mode;
case 0xC0F06824: return 0x2ca;
case 0xC0F06828: return 0x2ca;
case 0xC0F0682C: return 0x2ca;
case 0xC0F06830: return 0x2ca;
case 0xC0F06010: return 0x2cb;
case 0xC0F06008: return 0x2cb02cb;
case 0xC0F0600C: return 0x2cb02cb;
case 0xC0F06014: return 0x747;
case 0xC0F0713c: return 0x5C7;
case 0xC0F07150: return 0x59A;
}
}
else if (is_6D)
{
switch (reg)
{
/* raw resolution (end line/column) */
/* X: (3072+140)/8 + 0x17, adjusted for 3072 in raw_rec */
case 0xC0F06804: return 0x4e502b0; // 2688x1222 x5 Mode;
case 0xC0F06014: return 0x5b4;
case 0xC0F0713c: return 0x516;
case 0xC0F07150: return 0x4e5;
}
}
return 0;
}
static inline uint32_t reg_override_ULTRA_HD(uint32_t reg, uint32_t old_val)
{
switch (reg)
{
/* raw resolution (end line/column) */
/* X: (3072+140)/8 + 0x17, adjusted for 3072 in raw_rec */
case 0xC0F06804: return 0x88D03E2; // 2520x1386 x5 Mode;
case 0xC0F06824: return 0x4ca;
case 0xC0F06828: return 0x4ca;
case 0xC0F0682C: return 0x4ca;
case 0xC0F06830: return 0x4ca;
case 0xC0F06010: return 0x43b;
case 0xC0F06008: return 0x43b043b;
case 0xC0F0600C: return 0x43b043b;
case 0xC0F06014: return 0x993;
case 0xC0F0713c: return 0x8E7;
case 0xC0F07150: return 0x8BA;
}
return 0;
}
static inline uint32_t reg_override_4K(uint32_t reg, uint32_t old_val)
{
switch (reg)
{
/* raw resolution (end line/column) */
/* X: (3072+140)/8 + 0x17, adjusted for 3072 in raw_rec */
case 0xC0F06804: return 0xA1C0422; // 2520x1386 x5 Mode;
case 0xC0F06824: return 0x4ca;
case 0xC0F06828: return 0x4ca;
case 0xC0F0682C: return 0x4ca;
case 0xC0F06830: return 0x4ca;
case 0xC0F06010: return 0x45b;
case 0xC0F06008: return 0x45b045b;
case 0xC0F0600C: return 0x45b045b;
case 0xC0F06014: return 0xBD3;
case 0xC0F0713c: return 0xAE7;
case 0xC0F07150: return 0xABA;
}
return 0;
}
static inline uint32_t reg_override_FullRes(uint32_t reg, uint32_t old_val)
{
switch (reg)
{
/* raw resolution (end line/column) */
/* X: (3072+140)/8 + 0x17, adjusted for 3072 in raw_rec */
case 0xC0F06804: return 0xC12047E; // 2520x1386 x5 Mode;
case 0xC0F06824: return 0x56a;
case 0xC0F06828: return 0x56a;
case 0xC0F0682C: return 0x56a;
case 0xC0F06830: return 0x56a;
case 0xC0F06010: return 0x56b;
case 0xC0F06008: return 0x56b056b;
case 0xC0F0600C: return 0x56b056b;
case 0xC0F06014: return 0xDCB;
case 0xC0F06800: return 0x30010;
case 0xC0F0713c: return 0xCA7;
case 0xC0F07150: return 0xC7A;
}
return 0;
}
static int engio_vidmode_ok = 0;
static void * get_engio_reg_override_func()
{
uint32_t (*reg_override_func)(uint32_t, uint32_t) =
(crop_preset == CROP_PRESET_3x3_1X) ? reg_override_3x3_tall :
(crop_preset == CROP_PRESET_3x3_1X_48p) ? reg_override_3x3_48p :
(crop_preset == CROP_PRESET_HD) ? reg_override_HD :
(crop_preset == CROP_PRESET_2K) ? reg_override_2K :
(crop_preset == CROP_PRESET_ULTRA_HD) ? reg_override_ULTRA_HD :
(crop_preset == CROP_PRESET_4K) ? reg_override_4K :
(crop_preset == CROP_PRESET_FullRes) ? reg_override_FullRes :
0 ;
return reg_override_func;
}
static void FAST engio_write_hook(uint32_t* regs, uint32_t* stack, uint32_t pc)
{
uint32_t (*reg_override_func)(uint32_t, uint32_t) =
get_engio_reg_override_func();
if (!reg_override_func)
{
return;
}
/* cmos_vidmode_ok doesn't help;
* we can identify the current video mode from 0xC0F06804 */
for (uint32_t * buf = (uint32_t *) regs[0]; *buf != 0xFFFFFFFF; buf += 2)
{
uint32_t reg = *buf;
uint32_t old = *(buf+1);
if (reg == 0xC0F06804)
{
engio_vidmode_ok =
(old == 0x4540298) ||/* x5 zoom */ (old == 0x4a601d4 || old == 0x2d701d4); /* 1080p or 720p */
}
}
if (!is_supported_mode() || !engio_vidmode_ok)
{
/* don't patch other video modes */
return;
}
for (uint32_t * buf = (uint32_t *) regs[0]; *buf != 0xFFFFFFFF; buf += 2)
{
uint32_t reg = *buf;
uint32_t old = *(buf+1);
int new = reg_override_func(reg, old);
if (new)
{
dbg_printf("[%x] %x: %x -> %x\n", regs[0], reg, old, new);
*(buf+1) = new;
}
}
}
static int patch_active = 0;
static void update_patch()
{
if (CROP_PRESET_MENU)
{
/* update preset */
crop_preset = CROP_PRESET_MENU;
/* install our hooks, if we haven't already do so */
if (!patch_active)
{
patch_hook_function(CMOS_WRITE, MEM_CMOS_WRITE, &cmos_hook, "crop_rec: CMOS[5,7] parameters hook");
patch_hook_function(ADTG_WRITE, MEM_ADTG_WRITE, &adtg_hook, "crop_rec: ADTG[8000,8806] parameters hook");
if (ENGIO_WRITE)
{
patch_hook_function(ENGIO_WRITE, MEM_ENGIO_WRITE, engio_write_hook, "crop_rec: video timers hook");
}
patch_active = 1;
}
}
else
{
/* undo active patches, if any */
if (patch_active)
{
unpatch_memory(CMOS_WRITE);
unpatch_memory(ADTG_WRITE);
if (ENGIO_WRITE)
{
unpatch_memory(ENGIO_WRITE);
}
patch_active = 0;
crop_preset = 0;
}
}
}
/* enable patch when switching LiveView (not in the middle of LiveView) */
/* otherwise you will end up with a halfway configured video mode that looks weird */
PROP_HANDLER(PROP_LV_ACTION)
{
update_patch();
}
/* also try when switching zoom modes */
PROP_HANDLER(PROP_LV_DISPSIZE)
{
update_patch();
}
static MENU_UPDATE_FUNC(crop_update)
{
if (CROP_PRESET_MENU && lv)
{
if (!is_supported_mode())
{
MENU_SET_WARNING(MENU_WARN_NOT_WORKING, "This preset only works in 1080p and 720p video modes.");
}
else if (lv_dispsize != 1)
{
MENU_SET_WARNING(MENU_WARN_NOT_WORKING, "To use this mode, exit ML menu and press the zoom button (set to x1).");
}
else if (!is_720p())
{
if (CROP_PRESET_MENU == CROP_PRESET_3x3_1X ||
CROP_PRESET_MENU == CROP_PRESET_3x3_1X_48p)
{
/* these presets only have effect in 720p mode */
MENU_SET_WARNING(MENU_WARN_NOT_WORKING, "This preset only works in the 720p 50/60 fps modes from Canon menu.");
return;
}
}
}
}
static MENU_UPDATE_FUNC(target_yres_update)
{
MENU_SET_RINFO("from %d", max_resolutions[crop_preset][get_video_mode_index()]);
}
static struct menu_entry crop_rec_menu[] =
{
{
.name = "Crop mode",
.priv = &crop_preset_index,
.update = crop_update,
.depends_on = DEP_LIVEVIEW,
.children = (struct menu_entry[]) {
{
.name = "Shutter range",
.priv = &shutter_range,
.max = 1,
.choices = CHOICES("Original", "Full range"),
.help = "Choose the available shutter speed range:",
.help2 = "Original: default range used by Canon in selected video mode.\n"
"Full range: from 1/FPS to minimum exposure time allowed by hardware."
},
{
.name = "Target YRES",
.priv = &target_yres,
.update = target_yres_update,
.max = 3870,
.unit = UNIT_DEC,
.help = "Desired vertical resolution (only for presets with higher resolution).",
.help2 = "Decrease if you get corrupted frames (dial the desired resolution here).",
},
{
.name = "Delta ADTG 0",
.priv = &delta_adtg0,
.min = -500,
.max = 500,
.unit = UNIT_DEC,
.help = "ADTG 0x8178, 0x8196, 0x82F8",
.help2 = "May help pushing the resolution a little. Start with small increments.",
.advanced = 1,
},
{
.name = "Delta ADTG 1",
.priv = &delta_adtg1,
.min = -500,
.max = 500,
.unit = UNIT_DEC,
.help = "ADTG 0x8179, 0x8197, 0x82F9",
.help2 = "May help pushing the resolution a little. Start with small increments.",
.advanced = 1,
},
{
.name = "Delta HEAD3",
.priv = &delta_head3,
.min = -500,
.max = 500,
.unit = UNIT_DEC,
.help2 = "May help pushing the resolution a little. Start with small increments.",
.advanced = 1,
},
{
.name = "Delta HEAD4",
.priv = &delta_head4,
.min = -500,
.max = 500,
.unit = UNIT_DEC,
.help2 = "May help pushing the resolution a little. Start with small increments.",
.advanced = 1,
},
{
.name = "CMOS[1] lo",
.priv = &cmos1_lo,
.max = 63,
.unit = UNIT_DEC,
.help = "Start scanline (very rough). Use for vertical positioning.",
.advanced = 1,
},
{
.name = "CMOS[1] hi",
.priv = &cmos1_hi,
.max = 63,
.unit = UNIT_DEC,
.help = "End scanline (very rough). Increase if white bar at bottom.",
.help2 = "Decrease if you get strange colors as you move the camera.",
.advanced = 1,
},
{
.name = "CMOS[2]",
.priv = &cmos2,
.max = 0xFFF,
.unit = UNIT_HEX,
.help = "Horizontal position / binning.",
.help2 = "Use for horizontal centering.",
.advanced = 1,
},
MENU_ADVANCED_TOGGLE,
MENU_EOL,
},
},
};
static int crop_rec_needs_lv_refresh()
{
if (!lv)
{
return 0;
}
if (CROP_PRESET_MENU)
{
if (is_supported_mode())
{
if (!patch_active || CROP_PRESET_MENU != crop_preset)
{
return 1;
}
}
}
else /* crop disabled */
{
if (patch_active)
{
return 1;
}
}
return 0;
}
/* faster version than the one from ML core */
static void set_zoom(int zoom)
{
if (!lv) return;
if (RECORDING) return;
if (is_movie_mode() && video_mode_crop) return;
zoom = COERCE(zoom, 1, 10);
if (zoom > 1 && zoom < 10) zoom = 5;
prop_request_change_wait(PROP_LV_DISPSIZE, &zoom, 4, 1000);
}
/* when closing ML menu, check whether we need to refresh the LiveView */
static unsigned int crop_rec_polling_cbr(unsigned int unused)
{
/* also check at startup */
static int lv_dirty = 1;
int menu_shown = gui_menu_shown();
if (lv && menu_shown)
{
lv_dirty = 1;
}
if (!lv || menu_shown || RECORDING_RAW)
{
/* outside LV: no need to do anything */
/* don't change while browsing the menu, but shortly after closing it */
/* don't change while recording raw, but after recording stops
* (H.264 should tolerate this pretty well, except maybe 50D) */
return CBR_RET_CONTINUE;
}
if (lv_dirty)
{
/* do we need to refresh LiveView? */
if (crop_rec_needs_lv_refresh())
{
/* let's check this once again, just in case */
/* (possible race condition that would result in unnecessary refresh) */
msleep(500);
if (crop_rec_needs_lv_refresh())
{
info_led_on();
gui_uilock(UILOCK_EVERYTHING);
int old_zoom = lv_dispsize;
set_zoom(lv_dispsize == 1 ? 5 : 1);
set_zoom(old_zoom);
gui_uilock(UILOCK_NONE);
info_led_off();
}
}
lv_dirty = 0;
}
return CBR_RET_CONTINUE;
}
/* Display recording status in top info bar */
static LVINFO_UPDATE_FUNC(crop_info)
{
LVINFO_BUFFER(16);
if (patch_active)
{
if (lv_dispsize > 1)
{
switch (crop_preset)
{
case CROP_PRESET_HD:
snprintf(buffer, sizeof(buffer), "HD");
break;
case CROP_PRESET_2K:
snprintf(buffer, sizeof(buffer), "2K");
case CROP_PRESET_ULTRA_HD:
snprintf(buffer, sizeof(buffer), "ULTRA HD");
case CROP_PRESET_4K:
snprintf(buffer, sizeof(buffer), "4K");
case CROP_PRESET_FullRes:
snprintf(buffer, sizeof(buffer), "FullRes");
break;
}
}
}
/* append info about current binning mode */
if (raw_lv_is_enabled())
{
/* fixme: raw_capture_info is only updated when LV RAW is active */
if (raw_capture_info.binning_x + raw_capture_info.skipping_x == 1 &&
raw_capture_info.binning_y + raw_capture_info.skipping_y == 1)
{
STR_APPEND(buffer, "%s1:1", buffer[0] ? " " : "");
}
else
{
STR_APPEND(buffer, "%s%dx%d",
buffer[0] ? " " : "",
raw_capture_info.binning_x + raw_capture_info.skipping_x,
raw_capture_info.binning_y + raw_capture_info.skipping_y
);
}
}
if (crop_rec_needs_lv_refresh())
{
if (!streq(buffer, SYM_WARNING))
{
STR_APPEND(buffer, " " SYM_WARNING);
}
item->color_fg = COLOR_YELLOW;
}
}
static struct lvinfo_item info_items[] = {
{
.name = "Crop info",
.which_bar = LV_BOTTOM_BAR_ONLY,
.update = crop_info,
.preferred_position = -50, /* near the focal length display */
.priority = 1,
}
};
static unsigned int raw_info_update_cbr(unsigned int unused)
{
if (patch_active)
{
/* not implemented yet */
raw_capture_info.offset_x = raw_capture_info.offset_y = SHRT_MIN;
if (lv_dispsize > 1)
{
/* raw backend gets it right */
return 0;
}
/* update horizontal pixel binning parameters */
switch (crop_preset)
{
case CROP_PRESET_HD:
case CROP_PRESET_2K:
case CROP_PRESET_ULTRA_HD:
case CROP_PRESET_4K:
case CROP_PRESET_FullRes:
raw_capture_info.binning_x = raw_capture_info.binning_y = 1;
raw_capture_info.skipping_x = raw_capture_info.skipping_y = 0;
break;
case CROP_PRESET_3x3_1X:
case CROP_PRESET_3x3_1X_48p:
break;
}
/* update vertical pixel binning / line skipping parameters */
switch (crop_preset)
{
case CROP_PRESET_HD:
case CROP_PRESET_2K:
case CROP_PRESET_ULTRA_HD:
case CROP_PRESET_4K:
case CROP_PRESET_FullRes:
raw_capture_info.binning_y = 1; raw_capture_info.skipping_y = 0;
break;
case CROP_PRESET_3x3_1X:
case CROP_PRESET_3x3_1X_48p:
break;
}
if (is_700D || is_6D)
{
/* update skip offsets */
int skip_left, skip_right, skip_top, skip_bottom;
calc_skip_offsets(&skip_left, &skip_right, &skip_top, &skip_bottom);
raw_set_geometry(raw_info.width, raw_info.height, skip_left, skip_right, skip_top, skip_bottom);
}
}
return 0;
}
static unsigned int crop_rec_init()
{
if (is_camera("700D", "1.1.5") || is_camera("650D", "1.0.4"))
{
CMOS_WRITE = 0x17A1C;
MEM_CMOS_WRITE = 0xE92D41F0;
ADTG_WRITE = 0x178FC;
MEM_ADTG_WRITE = 0xE92D43F8;
ENGIO_WRITE = 0xFF2C2D00;
MEM_ENGIO_WRITE = 0xE51FC15C;
is_700D = 1;
crop_presets = crop_presets_700d;
crop_rec_menu[0].choices = crop_choices_700d;
crop_rec_menu[0].max = COUNT(crop_choices_700d) - 1;
crop_rec_menu[0].help = crop_choices_help_700d;
crop_rec_menu[0].help2 = crop_choices_help2_700d;
}
else if (is_camera("6D", "1.1.6"))
{
CMOS_WRITE = 0x2420C;
MEM_CMOS_WRITE = 0xE92D41F0;
ADTG_WRITE = 0x24108;
MEM_ADTG_WRITE = 0xE92D41F0;
ENGIO_WRITE = 0xFF2ADE1C;
MEM_ENGIO_WRITE = 0xE51FC15C;
is_6D = 1;
crop_presets = crop_presets_6d;
crop_rec_menu[0].choices = crop_choices_6d;
crop_rec_menu[0].max = COUNT(crop_choices_6d) - 1;
crop_rec_menu[0].help = crop_choices_help_6d;
crop_rec_menu[0].help2 = crop_choices_help2_6d;
}
menu_add("Movie", crop_rec_menu, COUNT(crop_rec_menu));
lvinfo_add_items (info_items, COUNT(info_items));
return 0;
}
static unsigned int crop_rec_deinit()
{
return 0;
}
MODULE_INFO_START()
MODULE_INIT(crop_rec_init)
MODULE_DEINIT(crop_rec_deinit)
MODULE_INFO_END()
MODULE_CONFIGS_START()
MODULE_CONFIG(crop_preset_index)
MODULE_CONFIG(shutter_range)
MODULE_CONFIGS_END()
MODULE_CBRS_START()
MODULE_CBR(CBR_SHOOT_TASK, crop_rec_polling_cbr, 0)
MODULE_CBR(CBR_RAW_INFO_UPDATE, raw_info_update_cbr, 0)
MODULE_CBRS_END()
MODULE_PROPHANDLERS_START()
MODULE_PROPHANDLER(PROP_LV_ACTION)
MODULE_PROPHANDLER(PROP_LV_DISPSIZE)
MODULE_PROPHANDLERS_END()
[\code]
For NRZI registers, the preferred format is with nrzi_encode, so you supply something human-readable (hex or decimal, doesn't matter much). The NRZI format is machine-readable.
For this line:
Quote
8172N -> 695 (695 is the hexadecimal value I enter in adtg_gui -> gives 4e6 value for NRZI)
you would write: nrzi_encode(0x4e6); it will return 0x695.
So this line is correct (delta_adtg0 is 0 by default; it's used to let you fine-tune this from menu).
+ adtg_new[2] = (struct adtg_new) {6, 0x8172, nrzi_encode(0x4e6 + delta_adtg0)};
Tip: for powersave timing registers, you can just set them to 0 for troubleshooting (tested on 5D3 and 60D - 0 works on all resolutions, but the sensor might run a bit hotter). Readout timing registers will affect the image if you set them to 0, but you should still be able to see something.
You can't use adtg_gui to check whether the overrides were correct (they both use the same hooks in Canon code, and only one hook can be installed at a time). You can, however, use the dbg_printf calls to print what is overridden. For example this one:
dbg_printf("ADTG%x[%x] = %x\n", dst, reg, new_value);
Change it to printf so it will go to the console. Turn off raw video, as that will also print a lot of stuff.
Same for cmos_hook and engio_write_hook - each one has a similar dbg_printf.
Caveat: printf is slow; calling it too many times from LiveView tasks (where these hooks are inserted) may lock up the camera. Workaround: enable one printf at a time, or skip printing registers that are changed on every LiveView frame, after making sure they are good.
Thanks for the help Alex,
Still fiddling with the crop_rec.c, I think the problem is that it doesn't set B timer, 6014.
I see some stuff about checking B timer original values and it checks with hardcoded values for 5d3.
Could that be the problem ?
The B timer is visible in the FPS override submenu (don't enable FPS override, just look in the submenu at the timer values).
There is a check at the beginning of engio_write_hook, used to identify the video mode. The values hardcoded there appear to be for 700D. Add some printf to find out the right values, or take them from adtg_gui.
If that check fails, no other ENGIO registers will be set, so the resolution will not be changed at all.
Found the video check, altered it with 6d values:
/* cmos_vidmode_ok doesn't help;
* we can identify the current video mode from 0xC0F06804 SET TO 6D_Values*/
for (uint32_t * buf = (uint32_t *) regs[0]; *buf != 0xFFFFFFFF; buf += 2)
{
uint32_t reg = *buf;
uint32_t old = *(buf+1);
if (reg == 0xC0F06804)
{
engio_vidmode_ok =
(old == 0x3dd02c0) ||/* x5 zoom */ (old == 0x4e501ec || old == 0x29801ec); /* 1080p or 720p */
}
}
But still no luck :-\
Probably found the problem, ENGIO address values are taken from 700d. (CMOS and ADTG are good and for 6d)
What are the correct ENGIO address values for 6d ?
else if (is_camera("6D", "1.1.6"))
{
CMOS_WRITE = 0x2420C;
MEM_CMOS_WRITE = 0xE92D41F0;
ADTG_WRITE = 0x24108;
MEM_ADTG_WRITE = 0xE92D41F0;
ENGIO_WRITE = 0xFF2ADE1C;
MEM_ENGIO_WRITE = 0xE51FC15C;
Found the Engio write address for 6d in the stubs.s file for 6d:
/** Engio **/
NSTUB(0xFF2ADE1C, _EngDrvOut) // AJ_EngDrvOut_1xVar_to_ShadowStruct
NSTUB(0xFF2AE134, _engio_write)
NSTUB(0xFF2ADEB8, shamem_read) // AJ_0x8FB0_engio_struct_n_R0_manipulation_to_get_ptr
// NSTUB( ???, EngDrvOut) /* present on 7D_MASTER.203 */
So ENGIO_WRITE address for 6d = 0xFF2AE134
How do I find the correct address for MEM_ENGIO_WRITE ?
Checking stubs in ROM1.BIN for 700D and 6D seems both shares the same adress:
E51FC15C;
This seems correct for the 6D as stated above(checked stubs):
ENGIO_WRITE address for 6d = 0xFF2AE134
:o ok.
So let's compile...
Yes works! Thanks Danne and Alex :D
Quote from: Levas on June 21, 2018, 10:12:15 AM
mk11174 made a version of crop_rec.c with an 2K preset for the 6d.
:) Def didnt make it, just chopped it up so I can sort of understand what needed to go where, why most of the 5D3 stuff is missing, nothing was changing for me until I chopped it up and looked at what was left, then understood a tiny bit more what needed to go where to make the changes happen like they did in ADTG tool and now it works for 700D.
Was hoping it would help you get the 6D preset you wanted going, But Alex I guess is working on this for many of you, maybe best to wait for his update, you know it will be perfect as usual!
There were this lines that were hard coded though that should be changed for the 6D like I mentions in the PM, so prob try getting all those right first, maybe will work?
444 const int default_timerA[] = { 0x20F, 0x27F, 0x20f, 0x27F, 0x20F };
445 const int default_timerB[] = { 0x9DE, 0x7CF, 0x7E5, 0x3D7, 0x3F2 };
446 const int default_fps_1k[] = { 23976, 25000, 29973, 50000, 59940 };
551 int adtg_blanking_reg = (lv_dispsize == 1) ? 0x8061 : 0x805F;
639 adtg_new[0] = (struct adtg_new) {6, 0x8061, shutter_blanking};
640 adtg_new[1] = (struct adtg_new) {6, 0x805F, shutter_blanking};
1052 (old == 0x4540298) ||/* x5 zoom */ (old == 0x4a601d4 || old == 0x2d701d4); /* 1080p or 720p */
Already changed everything on these lines and it's working now.
First I forgot the video check which is on Line 1052, like you mentioned.
Seems that the only thing that was wrong at the end was the ENGIO_WRITE address.
Took the address from the Stubs.s file for ENGIO_WRITE address and I got my crop_rec working :D
Good job. Could you post your changes to bitbucket Levas? Curious how code differs. Or post the changed crop_rec.c version.
Something else:
Did some math recently and I think the 6d is screwed in terms of high resolution with normal frame rate crop_rec options...
Clockspeed and horizontal column factor determine the options.
5d3 has 24Mhz clockspeed and a column factor of 8
6d has 25.6Mhz clock speed and a column factor of 4
(most?) APS-C cams have 32Mhz clockspeed and a column factor of 4
While the 5d3 has a slow clock speed, it has DOUBLE the column factor.
The 6d has almost the same clock speed as the 5d3, but only a column factor of 4.
And while APS-C have, like the 6d, a column factor of 4, they have at least a much faster clock speed.
Why Canon why, you put the worst of both options in the 6d, slow clock speed and low column factor :P
I did notice that the 'framing' preview from ML works in 14 bit mode, but when I switch to 14bitt lossless it most certainly disappears and doesn't come back. Only after setting it back to 14 bit and turn off and on the camera.
I see in the debug menu it has the following memory error:
No allocator for 675Kb at tweaks.c:3341, live_hiprio_task
Downloaded the newest experimental build Alex mentioned a few post earlier, made on 20 June 2018.
No problem anymore with 14 bit lossless and framing
Quote from: Levas on June 21, 2018, 02:24:46 PM
Already changed everything on these lines and it's working now.
That's great! Glad to hear!
Note, anyone looking at this for 700D, FullRes is not Fullres, it was a work in progress, can't get stable Fullres going in Crop_Rec for some reason like can in ADTG tool, so it is basically set to the highest possible working resolution.
Quote from: Levas on June 21, 2018, 01:38:25 PM
Found the Engio write address for 6d in the stubs.s file for 6d:
/** Engio **/
NSTUB(0xFF2ADE1C, _EngDrvOut) // AJ_EngDrvOut_1xVar_to_ShadowStruct
NSTUB(0xFF2AE134, _engio_write)
NSTUB(0xFF2ADEB8, shamem_read) // AJ_0x8FB0_engio_struct_n_R0_manipulation_to_get_ptr
// NSTUB( ???, EngDrvOut) /* present on 7D_MASTER.203 */
Yeah, that was my bad, I checked the 6D stubs and put in the DrvOut address instead of the write, sorry, why always good to double check thing.
No problem, hate the debugging :P, but learn a lot from it and it's satisfying when it finally works 8)
Quote from: Levas on June 21, 2018, 02:38:46 PM
I see in the debug menu it has the following memory error:
No allocator for 675Kb at tweaks.c:3341, live_hiprio_task[/s]
I've got this one too, when using mlv_lite and adtg_gui (the latter ended up using about 2MB of memory - speed optimizations aren't exactly free).
Workaround: loaded mlv_rec and used that one for grayscale preview.
Quote from: a1ex on June 21, 2018, 03:05:16 PM
Workaround: loaded mlv_rec and used that one for grayscale preview.
Good one to know, cause I need the preview, live view is broken very quick ;D
Does anybody know what is normal behaviour for crop_rec.
The Crop_rec preset I made, which is made for 5x zoom mode and uses the 2K setting route in Crop_rec can be loaded in all video modes.
When I leave Crop_rec option on in ML menu, and turn off camera and turn it on again, it starts in Canon 1080p mode, but crop_rec still applies all the settings I wrote for 5x zoom, so messed up live view in 1080p as long as crop_rec is on.
Not a problem of course, just hit the zoom button until I'm in 5x zoom mode.
But was just wondering if this is normal behaviour.
Not normal, but identifying video modes is not exactly easy. Canon firmware has a notification system (properties), but we get them too late, after the video mode is fully switched. I couldn't rely on properties to tell what video mode is going to be configured next (to know how to override it).
It took me quite a bit of fiddling to get something halfway usable, and there are still edge cases where it doesn't work well...
For checking if camera is in 1080P, 5x zoom or 720p mode, you could read out 800c register value, they're the same for each camera.
800c = 2 -> 1080p mode
800c = 0 -> 5 x zoom
800c = 4 ->720p mode
For an extra check to know if you're in PAL-25/50fps or NTSC-30/60fps or PAL/NTSC-24fps the only way is B timer, 6014.
But those B timer values are camera specific, so need to be write down for each camera.
Noticed the following difference in MLV_Lite between plain vanilla 14 bit raw recording and lossless 14 bit raw.
In experimental Crop_rec_4K build from 20th of June for 6d from download page.
14bit has other max vertical resolution then 14 bit lossless.
Made a Crop_rec preset for 6d, 2760x1200 resolution in 24fps.
-With 14 bit, max vertical resolution = 1200 Wonderfull :D 8) just as I want it to be and pre tested it in adtg_gui :)
-With 14 bit lossless, max vertical resolution =1206, nice those 6 pixels extra resolution, but it's out of live view range, it's in the white bar :-\
This means 14 bit VS 14bit lossless uses different rules for resolution, it's probably in the 'modulo' stuff, rules like, can the pixel count be divided by 16, 8 or 2 that kind of stuff...
Can the resolution rules for 14 bit and 14 bit lossless be made the same, for consistency ?
With crop_rec_4k.2018Jun20.5D3123, though I've got mlv_snd enabled and turned on, and there are audio meters in the gui, I'm not getting audio recorded with clips. If I record with just mlv_lite loaded, I get a 30 second wav file of static regardless of clip length with mlv_dump. If mlv_rec is also loaded, any clips recorded with mlv_rec or mlv_lite don't generate wav files with mlv_dump or mlrawviewer.
resolved, thanks. the MLV file is fine - it was a merge error in mlv_dump which caused the WAV header to be skipped.
I was wondering if this module could also basically being used to take full resolution silent pictures with fast shutter speeds. Never tested it until now! Maybe I should :)
Thanks for the update. Checked sound with crop_rec_4k.2018Jun23.5D3123:
Clips recorded with mlv_lite are now good. I've got WAV files for 14-bit uncompressed and 14-bit lossless via mlv_dump. Yay!
Clips recorded with mlv_rec (14-bit uncompressed) still don't create a WAV with either mlv_dump or mlrawviewer.
Clips recorded with mlv_lite (14-bit uncompressed) and previewed with mlrawviewer playback video but still don't create a WAV. Now that I think about it, I never tested the earlier mlv_lite + mlv_snd builds to see how mlrawviewer would handle sound. Is a break in compatibility expected?
Quote from: MichaelVito on June 25, 2018, 04:49:49 AM
Clips recorded with mlv_rec (14-bit uncompressed) still don't create a WAV with either mlv_dump or mlrawviewer.
forget about mlv_rec, i will drop it soon. at least this is my goal ;)
Quote from: MichaelVito on June 25, 2018, 04:49:49 AM
Clips recorded with mlv_lite (14-bit uncompressed) and previewed with mlrawviewer playback video but still don't create a WAV. Now that I think about it, I never tested the earlier mlv_lite + mlv_snd builds to see how mlrawviewer would handle sound. Is a break in compatibility expected?
did not try this tool, mainly worked with mlvfs
there might some things be different now, yeah. block order for example.
will check tonight.
thanks for feedback
Quote from: g3gg0 on June 25, 2018, 01:36:56 PM
forget about mlv_rec, i will drop it soon. at least this is my goal ;)
Hello g3gg0,
First of all, let me thank A1ex and you for your invaluable work on further improving 4k_crop recording.
I briefly tested the June 23-rd build with the 5D3 of a friend of mine and here is what I found:
1) Sound works synchronously only with the 1920x1080 1:1 and 3x3 modes of the mlv_lite module when frame rate is set to 24 fps;
2) As soon as mlv_lite is switched to 3K 1:1 there is no synchronicity between sound and video - video lags behind by a few tenths of a second regardless of the resolution and bit rate set. The lag is always different and clip length dependent. However, there is no length proportional dependence.
For this reason, it may not be a good idea to drop mlv_rec before sound synchronicity is fixed with mlv_lite. Maybe, returning the old condition, when mlv_rec was generating a synchronous sound file at 10/12/14 bits uncompressed is a better solution for now. As reported above, right now sound does not get recorded at all with mlv_rec.
Keep up this most important work!
hi,
that sync issue was always only 1-2 frames on my tests.
felt like it is quite static, thus is added the option to mlv_snd to correct this offset.
are you sure that mlv_rec was always in sync?
about the mlrawviewer issue, i suspect that the header field for audio frames could be related as it is zero.
but being set to zero means "autodetect" and the player should detect that on its own (as with early mlv_rec headers)
edit:
please check if toggling "sync beep" in mlv_lite menu helps with the variance
Not to get off topic, but just for a reference point, is the original mlv_rec + mlv_snd implementation used in the Nightly builds a solid sync lock? I'd always assumed it was, but never tested and haven't been using it for anything where a variance would have been critical. To further confirm, was this same implementation used in crop_rec on steroids up until 2018 February, when work on expanding mlv_snd support began?
I totally understand getting things streamlined, eventually unifying everything under mlv_lite when sound issues are ironed out, and happy to test as many builds as it takes. But as IDA_ML says, it's helpful to have a fallback to legacy mlv_rec + mlv_snd for when you need it.
If anyone wants to review the code:
- old implementation from ErwinH (https://www.magiclantern.fm/forum/index.php?topic=16650.msg191983#msg191983) et al (crop_rec_4k_mlv_lite_snd (https://bitbucket.org/hudson/magic-lantern/branch/crop_rec_4k_mlv_lite_snd?dest=crop_rec_4k#chg-modules/mlv_lite/mlv_lite.c)): latest build on March 10 (https://builds.magiclantern.fm/jenkins/view/Experiments/job/crop_rec_4k/72/)
- new implementation from g3gg0 (crop_rec_4k_mlv_snd (https://bitbucket.org/hudson/magic-lantern/branch/crop_rec_4k_mlv_snd?dest=crop_rec_4k#chg-modules/mlv_lite/mlv_lite.c) - started back in Feb. 2017): latest build currently on the Experiments page
From what I could tell, there is some divergence between them, although I'm pretty sure Erwin (7c51597 - 5f4ed21, October 2017) started from g3gg0's implementation (4a37b0e - 1ed4731, February-July 2017) or at least looked at it.
Other than a few tests back in February, I didn't record any raw videos with sound. Would be interesting to know whether the issues mentioned above were present in the old implementation (I can test it, but would be time-consuming and I'm not good at "seeing" the sync issues anyway).
I still have a MLV from back then, recorded with mlv_lite + mlv_snd; it does play in MlRawViewer. Some metadata:
File Header (MLVI)
Size : 0x00000034
Ver : v2.0
GUID : 6497533523640690944
FPS : 25.000000
File : 0 / 0
Frames Video: 164
Frames Audio: 39
Class Video : 0x00000021
Class Audio : 0x00000001
Block: RAWI
Offset: 0x00000034
Number: 1
Size: 180
Time: 2.995000 ms
Res: 1920x1290
...
Block: RAWC
Offset: 0x000000e8
Number: 2
Size: 32
Time: 3.005000 ms
raw_capture_info:
sensor res 5760x3840
sensor crop 1.00 (Full frame)
sampling 3x3 (bin 3 lines, bin 3 columns)
P.S. Here's my attempt to fix the audio sync issues (https://bitbucket.org/hudson/magic-lantern/commits/7fcd17733b77543fa771e0dde40f7187f471fc55?at=crop_rec_4k_mlv_lite_snd) (in Erwin's implementation); if that worked, it should probably be included in the newer version as well.
thanks didnt notice that the fixes weren't merged.
will look into them and merge into mlv_snd
Hello all, following this exciting development,
Now that I had the chance to have the 5DMkIII for a few days in my hands, I decided to test the new June 20-th Experimental build from the Experimental download page and thought, it might be useful if I provide some feedback for those who use the 5D3 for shooting RAW video. Here are my first impressions:
Things that I liked:
=============
1) Compared to the previous March 31 build, this one is very stable. No more crashes, glitches, battery pulls, assert logs, etc. It maintains excellent stability even when I switch modes. One of the coolest features is that it is possible to quickly switch from video to photo mode just by flipping the video/photo switch and the mode dial of the camera from M to Av. This can be done in fractions of a second and every time you switch back and forth from one mode to the other, all settings for both modes are stored and maintained. Thus, when you film video and you quickly need to take a few stills or vice versa you can do this in a very fast and hassle free way without missing important moments. Yesterday, I was filming the performance of my daughter's choir and I did this photo/video mode switching several times during the concert, without a single glitch. Also, when filming FHD video with mlv-lite, I switch from 1920x1080 3x3 to 1:1 all the time. This allows me to switch my EF 35/F2 IS lens from 35mm to 105mm in a very fast way, without swapping lenses or using slow zooms. This is equivalent to the "Movie Crop Mode" on the 100D and yesterday, it worked perfectly on the 5D3 too, without a single glitch. EXCELLENT WORK, A1ex !!!
2) The build is very convenient to work with. In all video modes, in-camera previews work very well and scale nicely when resolution is changed in the 4K-crop recording modes. This is very useful for correct framing during filming. The previews are nice and bright even if bit rates are changed from 14 to 11 ... 8. The feature that I absolutely love is that now focusing at 5x and 10x magnification works almost instantaneously. This feature is extremely important especially when using fast primes for video shooting. A quick and precise focus check and adjustment before pressing the record button is a must and it works very well on the 5D3 now, both - with half-shutter and back-button focusing. I did not have a single misfocused clip during yesterday's filming. Also in-camera playback works quite well and is fully usable, despite the fact that you may end up with a corrupt margin at the bottom and the right of the Live View screen if in the 3K 1:1 mode you reduce the resolution.
3) Another cool feature concerns photo mode with the "HOLD" option activated in the Canon menu. Immediately after a still photo is taken, it stays on the screen as usual but a tiny RAW histogram, focus peaking dots and under and overexposed areas appear on it. In this way, just by taking a look at the picture, you know immediately if it is properly exposed and focused. Very handy!
4) All files that I recorded work flawlessly with MLVFS and open nicely in MLVApp and MLVProducer.
Things that I didn't like:
=================
These concern mostly my favorite 3K recording mode with mlv-lite. There are two reasons why I use this mode on the 5D3 more often than any other modes when shooting RAW video:
A. The resolutions of 3072x1920 or 3072x1728 (16:9) at 24 fps, 12 and 8 ... 11 bits lossless provide the highest video quality that I have ever seen even after upscaling to 4k. Period.
B. At its maximum horizontal resolution of 3072p, this mode provides a crop factor of 1.87 which is reasonably small and greatly increases the functionality of the available arsenal of wide angle and normal focal length lenses for video shooting. A 50/1.4 Sigma Art lens is converted into a 94/2.2 lens which provides a breath taking vision in terms of isolation of the main subject from the background, bokeh, smooth tonal transitions, etc. At 12 mm focal length, my old Sigma 12-24 provides very impressive landscape videos at 3072x1728 resolution and 24 fps. Fine detail and colors are just fantastic.
There is one major limiting factor of the 3K mode though - the recording time is too short to be usable. With a well lit scene (0.0 histogram value), Frozen LV and 8 ... 11 bits lossless I barely get 60 frames recorded with sound at 24 fps. If I underexpose by one stop, I get about 85 frames and that's it. If the scene contains a lot of highlights, the number of recorded frames can drop to 25 frames which is just 1s of recording time. These data concern the fastest x1066 CF cards. Of course, I could lower the resolution to say 2560x1440 where I can get continuous recording with sound and I do this all the time. However, this increases the crop factor. Also, I could stay with 3072p resolution and reduce the vertical resolution but then I lose this fantastic full-screen vision and the video starts looking like a ribbon. So, what is the solution? In my opinion, there are two ways to solve the problem - through implementing the good old card spanning feature to crop recording with mlv-lite or through overclocking the CF card interface as this was successfully done with the 100D. Or both ??? I think, more work in this direction is definitely worth considering. A1ex, Danne, what do you think?
Things that need a fix:
================
Those concern the sound sync issues discussed above. I hope, they will be fixed soon. I never had any synchronicity issues with the old implementation of ErwinH applied to mlv-lite or the mlv-rec feature from the Nightly builds. So, I guess, going back to these old sound implementations should fix the sync issues, at least for now, until G3ggO irons out his new implementations.
Bottom line:
=========
The June 2018 builds for the 5DMkIII have turned this camera into a very powerful RAW video recording machine which in my opinion is stable and convenient enough for serious work. After the bugs mentioned above are fixed and problems are ironed out, I am confident that this camera will become the preferred choice for many professional videographers. I wish, I had this stability and functionality in my 7D.
I have to return the 5D3 now and cannot test more features and modes but I will test the June 23-rd build for the 100D and will report on it as soon as I can. I strongly encourage all 5D3 owners in this forum to run their own tests and report their findings here. This will help the developers fix bugs and solve problems easier and faster, so more 5D3 owners can take advantage of this fantastic new functionality and stability of their camera.
thanks a lot for your feedback.
first of all, i missed the point that the branch "crop_rec_4k_mlv_snd" and "crop_rec_4k_mlv_lite_snd" are two different ones :D
so i felt safe to continue work on mlv_snd in crop_rec_4k_mlv_snd, thinking it was the right one.
my wrong, sorry.
i now manually pulled the important things from that - lets call it - bugfix branch.
now the mlv_snd module is being called right from the vsync cbr of mlv_lite, giving minimum chance for desynchronization.
(previously even the module load order could have introduced a frame delay)
@a1ex:
did i miss some important thing to graft?
as said, i want to drop mlv_rec with this branch as soon it is stable.
filmed a few TiB now with mlv_lite and close to zero with mlv_rec.
@all:
please feel free to test the latest build for audio sync issues.
the mlv_snd menu now is in the raw recording menu (https://bitbucket.org/hudson/magic-lantern/commits/e10c7af65495ff5e4dd79cbb00bad9601dd3c82a) in case you miss it.
within that menu there is an option to configure how many frames the audio should be delayed.
if everything is as i expect it, it should be ideal at 1 - make sure you didnt accidentally set it to any other value.
Quote from: g3gg0 on June 27, 2018, 01:36:10 AM
first of all, i missed the point that the branch "crop_rec_4k_mlv_snd" and "crop_rec_4k_mlv_lite_snd" are two different ones :D
Well, at that time, Erwin's version was simpler and worked reasonably well, but at the same time I didn't want to exclude yours by picking the other. So I tried to keep both branches in sync, to decide later.
Quote from: g3gg0 on June 27, 2018, 01:36:10 AM
filmed a few TiB now with mlv_lite and close to zero with mlv_rec.
I've filmed close to zero with both :D (actually one clip during the holidays -> used H.264, since it was half an hour long and I had a 32GB card).
If we drop mlv_rec, I'd suggest renaming mlv_lite back to mlv_rec (feature-wise, it's no longer lite anyway) and port the remaining important features (card spanning, maybe exfat, etc). I'm also thinking to undo the preallocation experiments, as I've got a lot of trouble from them anyways.
Will check later.
Thanks for rounding up all those changes!
Just did a quick check with crop_rec_4k.2018Jun27.5D3123. Now mlv_lite 14-bit uncompressed plays nice with both mlv_dump and mlrawviewer. I'll do a deeper check for other modes and sync soon.
If the plan is to retire mlv_rec, I won't keep asking about it. But you may want to just remove it now, or at least note that sound no longer works.
I checked some of the older builds, and seems mlv_rec sound has been broken since the very first mlv_lite + mlv_snd build. With February 2, recording with mlv_rec I get the message "audio failed to stop, state 0" after hitting stop and no WAV is created with either mlv_dump or mlrawviewer. With March 10, the last ErwinH version, I get the error message as well as an assert referring to mlv_snd_vsync.
With the g3gg0 builds, the errors stopped, though sound is still broken. If sacrificing mlv_rec is the only way to get mlv_snd integration with mlv_lite, I'm ok with that. Maybe a memorial is in order? (^^)
Back in February, mlv_rec worked fine with mlv_snd, but you do have to load only one module at a time (either mlv_rec or mlv_lite, not both). There's no check for that though, just "tribal knowledge".
Quote from: MichaelVito on June 27, 2018, 09:50:13 AM
With the g3gg0 builds, the errors stopped, though sound is still broken. If sacrificing mlv_rec is the only way to get mlv_snd integration with mlv_lite, I'm ok with that. Maybe a memorial is in order? (^^)
to be correct, crop_rec_4k_mlv_snd *is* the branch i started initially 17 months ago.
its goal was to make mlv_snd integrate nicely into mlv_lite, so mlv_rec can get dropped.
it just wasnt finished and certainly still is not.
so there are no "g3gg0 builds" or anything else.
the memorial is "MLV" ;)
I'm working on Crop_rec presets for 6d and aiming for 25fps presets mostly, but how exact does this 25 fps frame rate have to be and does it have influence on audio ?
For instance I have presets with 24.995fps and 25.005fps, is this good enough or do I have to fiddle with A and B timers to get exactly 25.000fps ?
And is ML audio with MLV_lite linked to exact frame rate of the A and B timer settings, or is it linked to PAL or NTSC setting ?
Usually it's easy to get exact values for 25p; you can use a small Python script to do the math. Something like this:
main_clock = 24e6
desired_fps = 25.0
for timer_a in range(500, 600):
timer_b = int(main_clock / timer_a / desired_fps)
actual_fps = main_clock / timer_a / timer_b
print timer_a, timer_b, actual_fps - desired_fps
I'd like to automate this step in the source code, rather than hardcoding all these FPS timers, btw.
Audio sync shouldn't depend on FPS, at least in theory. Audio gets recorded at whatever rate is selected in the menu (44 kHz etc); furthermore, all frames are timestamped.
I see I have still not the right shutter time with the crop mode build I'm using.
It's probably going wrong in the calculations for shutter blanking, the piece of code for shutter blanking is probably optimized for 5d3 or 700d in the code I'm using.
1/50th on top display gives me 1/35th shutter time in ML.
1/100th on top display gives me 1/50th shutter time in ML.
Does anybody know where I have to fix this error in shutter blanking calculations ?
/* adapted from fps_override_shutter_blanking in fps-engio.c */
static int adjust_shutter_blanking(int old)
{
/* sensor duty cycle: range 0 ... timer B */
int current_blanking = nrzi_decode(old);
int video_mode = get_video_mode_index();
int fps_timer_b_orig = default_timerB[video_mode];
int current_exposure = fps_timer_b_orig - current_blanking;
/* wrong assumptions? */
if (current_exposure < 0)
{
return old;
}
int default_fps = default_fps_1k[video_mode];
int current_fps = fps_get_current_x1000();
dbg_printf("FPS %d->%d\n", default_fps, current_fps);
float frame_duration_orig = 1000.0 / default_fps;
float frame_duration_current = 1000.0 / current_fps;
float orig_shutter = frame_duration_orig * current_exposure / fps_timer_b_orig;
float new_shutter =
(shutter_range == 0) ?
({
/* original shutter speed from the altered video mode */
orig_shutter;
}) :
({
/* map the available range of 1/4000...1/30 (24-30p) or 1/4000...1/60 (50-60p)
* from minimum allowed (1/15000 with full-res LV) to 1/fps */
int max_fps_shutter = (video_mode_fps <= 30) ? 33333 : 64000;
int default_fps_adj = 1e9 / (1e9 / max_fps_shutter - 250);
(orig_shutter - 250e-6) * default_fps_adj / current_fps;
});
uint32_t (*reg_override_func)(uint32_t, uint32_t) =
get_engio_reg_override_func();
/* what value we are going to use for overriding timer B? */
int fps_timer_b = (reg_override_func)
? (int) reg_override_func(0xC0F06014, fps_timer_b_orig - 1)
: fps_timer_b_orig;
/* will we actually override it? */
fps_timer_b = fps_timer_b ? fps_timer_b + 1 : fps_timer_b_orig;
dbg_printf("Timer B %d->%d\n", fps_timer_b_orig, fps_timer_b);
int new_exposure = new_shutter * fps_timer_b / frame_duration_current;
int new_blanking = COERCE(fps_timer_b - new_exposure, 10, fps_timer_b - 2);
dbg_printf("Exposure %d->%d (timer B units)\n", current_exposure, new_exposure);
#ifdef CROP_DEBUG
float chk_shutter = frame_duration_current * new_exposure / fps_timer_b;
dbg_printf("Shutter %d->%d us\n", (int)(orig_shutter*1e6), (int)(chk_shutter*1e6));
#endif
dbg_printf("Blanking %d->%d\n", current_blanking, new_blanking);
return nrzi_encode(new_blanking);
}
extern WEAK_FUNC(ret_0) void fps_override_shutter_blanking();
static void FAST adtg_hook(uint32_t* regs, uint32_t* stack, uint32_t pc)
{
/* This hook is called from the DebugMsg's in adtg_write,
* so if we change the register list address, it won't be able to override them.
* Workaround: let's call it here. */
fps_override_shutter_blanking();
uint32_t cs = regs[0];
uint32_t *data_buf = (uint32_t *) regs[1];
int dst = cs & 0xF;
/* copy data into a buffer, to make the override temporary */
/* that means: as soon as we stop executing the hooks, values are back to normal */
static uint32_t copy[512];
uint32_t* copy_end = ©[COUNT(copy)];
uint32_t* copy_ptr = copy;
struct adtg_new
{
int dst;
int reg;
int val;
};
/* expand this as required */
struct adtg_new adtg_new[10] = {{0}};
/* scan for shutter blanking and make both zoom and non-zoom value equal */
/* (the values are different when using FPS override with ADTG shutter override) */
/* (fixme: might be better to handle this in ML core?) */
int shutter_blanking = 0;
int adtg_blanking_reg = (lv_dispsize == 1) ? 0x8061 : 0x805F;
for (uint32_t * buf = data_buf; *buf != 0xFFFFFFFF; buf++)
{
int reg = (*buf) >> 16;
if (reg == adtg_blanking_reg)
{
int val = (*buf) & 0xFFFF;
shutter_blanking = val;
}
}
/* some modes may need adjustments to maintain exposure */
if (shutter_blanking)
{
shutter_blanking = adjust_shutter_blanking(shutter_blanking);
}
Guess: default_timerB needs to be adjusted (currently it's hardcoded for 5D3).
Or, the code needs to be updated to autodetect it (maybe save its value prior to overriding).
I did adjusted the sensor timing table, if that is what you mean ?
Just to be sure, that third column with 29.987fps, is 1080 3x3 binning in NTSC right , or is it 5x zoom ?
/* from SENSOR_TIMING_TABLE (fps-engio.c) */
/* hardcoded for 6d */
const int default_timerA[] = { 0x221, 0x27F, 0x221, 0x27F, 0x207 };
const int default_timerB[] = { 0x7a2, 0x63f, 0x61b, 0x31f, 0x334 };
const int default_fps_1k[] = { 23982, 25000, 29978, 50000, 59964 };
/* adapted from fps_override_shutter_blanking in fps-engio.c */
static int adjust_shutter_blanking(int old)
{
/* sensor duty cycle: range 0 ... timer B */
int current_blanking = nrzi_decode(old);
int video_mode = get_video_mode_index();
int fps_timer_b_orig = default_timerB[video_mode];
Quote from: a1ex on June 28, 2018, 09:28:50 PM
Or, the code needs to be updated to autodetect it (maybe save its value prior to overriding).
If that's the case, that means it's also not working properly in the Crop_rec_4k build, for the 5d3, on the download page, right ?
Out of ideas then; you could try enabling the debug messages from that function.
It works fine on 5D3, but with all these hardcoded values, it's hard to port to other models (that's why I prefer changing it to something more generic).
Wait, the 5d3 has max shutter time of 1/8000th and 6d 'only' has max shutter time of 1/4000th, is that info used somewhere in that piece of code ?
In movie mode it has 1/4000 in Canon UI, but the hardware lets you set the exposure time to 2 units (that is, blanking set to timer B - 2). In microseconds, that would be 1 / (24e6 / timerA / 2). That gives 1/25000 in 50p (timer A = 480), "overclockable" to 1/30000 (timer A = 400), or close to 1/15000 in full-res LV (timer A = 792).
Without crop_rec, you should be able to get these fast shutter speeds from the Movie menu, Image fine-tuning.
With crop_rec, there is an option to remap the shutter speed range (Canon UI lets you dial from 1/4000 to 1/30, but you can choose to remap this range to whatever the hardware allows). Caveat: if you enable shutter remapping ("Full range" in the submenu), you will no longer get "round" shutter speeds, like 1/50; the remapping formula doesn't take care of that.
And yes, 1/4000 is hardcoded in the code you have pasted (250e-6; don't remember why I wrote it like that; 1.0 / 4000 would have been the same).
Quote from: a1ex on June 28, 2018, 10:29:34 PM
And yes, 1/4000 is hardcoded in the code you have pasted (250e-6; don't remember why I wrote it like that; 1.0 / 4000 would have been the same).
250u, thats the scientist in you ;)
Quote from: a1ex on June 28, 2018, 10:29:34 PM
And yes, 1/4000 is hardcoded in the code you have pasted (250e-6; don't remember why I wrote it like that; 1.0 / 4000 would have been the same).
Stupid me :P I should have known that it's the 250e-6.
And the part about the option to remap the shutter speed range is really helpful, helps to understand the piece of code.
I've heard of the crazy shuttertimes in video mode, and saw some evidence in the crop_rec module, but didn't know where it was used.
Thought it was used through full res silent pictures or something.
But anyway, that shutter remapping -> full range option is a hidden gem :D
Gonna dive into crop_rec some more and hope I can solve the shutter mapping problem I have on the 6d.
Trying to figure out how shutter_blanking calculations work, but still don't get it.
Quote from: a1ex on June 28, 2018, 10:29:34 PM
In movie mode it has 1/4000 in Canon UI, but the hardware lets you set the exposure time to 2 units (that is, blanking set to timer B - 2). In microseconds, that would be 1 / (24e6 / timerA / 2). That gives 1/25000 in 50p (timer A = 480), "overclockable" to 1/30000 (timer A = 400), or close to 1/15000 in full-res LV (timer A = 792).
For the 6d :
Clockspeed 25600000
Timer A in 1080_25p = 639
Timer B in 1080_25p = 1599
When I want to calculate the max range in 1080_25p -> 1/25600000 / 639 / 2 = 1/20033
So I assume max shutter speed for 6d in 1080_25p is 1/20000th
But I don't get it, where is the B timer involved, or isn't the B timer involved in shutter speed, because it's the speed where 1 line of pixels is readout ?
And what does the value of the shutterbalnking registers mean , how can they be translated to exposure times ?
At the moment I think the problem is in the line in the middle of this piece of code
({
/* map the available range of 1/4000...1/30 (24-30p) or 1/4000...1/60 (50-60p)
* from minimum allowed (1/15000 with full-res LV) to 1/fps */
int max_fps_shutter = (video_mode_fps <= 30) ? 33333 : 64000;
int default_fps_adj = 1e9 / (1e9 / max_fps_shutter - 250);
(orig_shutter - 250e-6) * default_fps_adj / current_fps;
});
The values 33333 and 64000 are probably hardcoded for 5d3.
Tried to change value 33333 to 20033, it gave other shutter times, but still not the right ones (range goes from 1/34th to 1/120th, no further (1/120th is reached when Canon is on 1/4000).
EDIT, this is not it, I see the 33333 and 64000 values are max shutter time as in 1/33th and 1/64th for video modes...back to the drawing boards again ::)
To get an interval of N units, you write N-1 in the hardware timer register. That means, the timer will count from 0 to N-1, giving a periodic signal every N units. When doing the math, I use N, that is, value read from the hardware register (as displayed by adtg_gui) + 1. So, 25.6e6 / 640 / 2 = 20000 => fastest shutter is, indeed, 1/20000.
Timer A is incremented once by the main clock (so, 25.6 million times per second). When it reaches the value programmed in that register, it goes back to 0 and the image capture hardware moves on to read out the next line. If timer A is "too fast", at first you get some black columns on the right side (line readout does not finish), then the image breaks completely. If timer A is larger than needed, it works fine, you just get some more rolling shutter.
Every time timer A fires, the second timer (B) also gets incremented. When timer B reaches its programmed value, it goes back to 0 and the next frame is read out. Timer B needs to be large enough to read out all the lines (but can be larger).
Things are a bit more complex than that, there are also 4 HEAD timers that I don't fully understand yet (https://www.magiclantern.fm/forum/index.php?topic=22401.0), but for "simple" FPS adjustment, that's basically how it works.
Back to exposure time. For the ADTG register, you no longer measure the time in microseconds; you use timer A ticks for that. That is, time within one frame goes from 0 to timerB - 1. That's the unit you use to program the various timers (HEAD timers, exposure time, powersave timing etc).
Blanking = how much time the sensor is not capturing anything. So, frame_duration = exposure_time + blanking_time (or, exposure_time = frame_duration - blanking_time). That's what the hardware expects: you program how much time the sensor stays "off", rather than "on".
If you have an older model, FPS override at low frame rates will overexpose quite a bit. Why it's that? If you only override the timer registers and leave the ADTG stuff unchanged, the "shutter blanking" register will stay the same. However, the frame duration (aka timer B) will increase, so the exposure time will also increase.
For example, if you cut the FPS in half by doubling timer B, the old method will not be able to get a shutter speed faster than 1 / (FPS/2), even if you dial 1/8000 in Canon menu. Why? Canon code will set the blanking register close to the original timer B, but the frame duration is now double.
If you do the same thing by doubling timer A, the exposure time (and the rolling shutter) will just double (from what you dial in Canon menu). In this case, timer B will not change, so the ratio between blanking and exposure will stay the same, but one unit will now be twice as many microseconds.
BTW, before knowing this stuff, the following was found by writing down shutter values and trying to find an explanation:
// FPS override will alter shutter speed (exposure time)
// FPS "difference" from C0F06014 will be added as a constant term to exposure time
// FPS factor from C0F06008 will multiply the exposure time (as scalar gain)
// TG = base timer (28.8 MHz on most cams)
// Ta = current value from C0F06008
// Tb = current value from C0F06014
// Ta0, Tb0 = original values
//
// FC = current fps = TG / Ta / Tb
// F0 = factory fps = TG / Ta0 / Tb0
//
// E0 = exposure time (shutter speed) as indicated by Canon user interface
// EA = actual exposure time, after FPS modifications (usually higher)
//
// If we only change Tb => Fb = TG / Ta0 / Tb
//
// delta_fps = 1/Fb - 1/F0 => this quantity is added to exposure time
//
// If we only change Ta => exposure time is multiplied by Ta/Ta0.
//
// If we change both, Tb "effect" is applied first, then Ta.
//
// So...
// EA = (E0 + (1/Fb - 1/F0)) * Ta / Ta0
Now it seems to match the above theory.
Quote from: a1ex on June 29, 2018, 10:36:08 AM
Blanking = how much time the sensor is not capturing anything. So, frame_duration = exposure_time + blanking_time (or, exposure_time = frame_duration - blanking_time). That's what the hardware expects: you program how much time the sensor stays "off", rather than "on".
Aha, now I get at least what the shutter blanking value means.
Aslo enabled the debug in the crop_rec.
and enabled a preset I made.
I have set shutter time to 1/50th on Canon top display and ML shows 1/27th.
Console:
[80440] c0f06014: 493 -> 2f2
FPS 25000 -> 25005
Timer B 1599 ->755
Exposure 1128 -> 709 (Timer B units)
Shutter 28217 -> 37555us
Blanking 471 -> 46
ADTG2[805f] = 39
ADTG2[8061] = 39
Now there are a few things wrong here, but first, what is right.
New value of Timer B is right, I have set it to 2f2 in this crop_rec preset.
New FPS value is right, I have 25005 FPS in this preset (Timer A = 54b)
But what is wrong here, I see 2 values used for what Timer B was, the first line says, 493 (which is right, because I'm in 5x zoom, where Timer Normally is 493. )
And further on I see a 'before' Timer B value of 1599 -> comes from plain 1080_25p mode, not from 5x zoom mode.
Furthermore it says FPS was 25000, but that's from 1080_25p mode, 5x zoom has 29921 fps /1000
I can imagine that the this happens when I enable Crop_rec in 1080_25p mode, but the above console values are also showed when I enable Crop_rec preset in 5x zoom mode.
Now I changed the second column in the sensor timing table to values of 5x zoom crop mode (where I first had values for 1080_25p mode)
/* from SENSOR_TIMING_TABLE (fps-engio.c) */
/* hardcoded for 6d */
const int default_timerA[] = { 0x221, 0x2d9, 0x21f, 0x27F, 0x27f };
const int default_timerB[] = { 0x7a2, 0x493, 0x61b, 0x31f, 0x334 };
const int default_fps_1k[] = { 23982, 29921, 29978, 50000, 59964 };
Now the shutter time range behaves a lot better, almost right. exactly right :D (ouch, I had full range option enabled at first)
1/30th top display is 1/25th ML and 1/4000th top display is 1/9400th ML. Values for full range
But if I remember correct, the values in the second column are in the original Crop_rec code for 5d3 from 1080_p video mode and not 5x zoom ?
As seen here:
/* from SENSOR_TIMING_TABLE (fps-engio.c) */
/* hardcoded for 5D3 */
const int default_timerA[] = { 0x1B8, 0x1E0, 0x1B8, 0x1E0, 0x1B8 };
const int default_timerB[] = { 0x8E3, 0x7D0, 0x71C, 0x3E8, 0x38E };
const int default_fps_1k[] = { 23976, 25000, 29970, 50000, 59940 };
Hm, I think you just found something I need to fix. There needs to be an additional entry for the x5 zoom...
Experimental
crop_rec with higher resolutions for 700D (it Should work for EOS M)
The presets are:
- Full-res LiveView 5208x3230 @ 6.5 fps
- 3K 2.35:1 3072x1304 @ 20 fps
- 3.5K 3540x1080 Very Wide @ 22 fps
- 4K 4096x2560 @ 9.5 fps
- UHD 3840x2160 @ 12 fps
- 2520x1386 16:9 @ 23.976 fps
- 1920x1920 @ 23.976 fps (Not working yet)
(https://preview.ibb.co/cyrcNJ/VRAM0.png) (https://ibb.co/kgoP2J)
The code:
https://bitbucket.org/bilal_fakhouri/magic-lantern-crop_rec/commits/a6ea137785ea1d36f417f5db6e020c8447f48545
Thanks for @mk11174 for helping.
Of course the code need to be cleaned and editing somthings (maybe I can't clean the code since I am not programmer :P) --> But for now you can chose the resolution you want in one click instead of adtg_gui steps --> So feedback for current presets and enjoy!
Note: sd_uhs is
only for getting better recording times (by overclocking write speed) --> it's
not necessary to make crop_rec working but it's a good choice.
Use all of new presets
in x5 mode even Full-res (It's working in normal mv1080 but with
vertical stripes) and in 14-bit lossless not 12-8bit lossless (You will have corrupted frames if you chose this).
Download (https://drive.google.com/open?id=1WWpXITJ9_hdkkcdBj75AercVol1Pe5n4)and replace crop_rec under crop_rec_4k build.
For
Full-res LiveView sometimes it's
stuck when selecting it you need to
press PLAY button then
get back to live view then x5 mode (Don't worry about these things it will be fixed). And I think 5208x3230 is the limit for Full-res (With more or less pixels --> if you can getting better result maybe I am wrong tell us that please).
For higher shutter speeds in Full-res --> It's need to adjust something in the code --> Look above maybe @Levas and @a1ex talking about that :P I didn't try yet.
BURN THE CAMERA :D
@theBilalFakhouri -- Excellent!
Thanks @dfort!
Is it working in EOS M ?
Great effort here @theBilalFakhouri
Ran the setting on my eosm and following works:
2520x1384
3072x1304
3544x1080
Corruption:
4096x2304
4096x696(fullres liveview?) can´t remember.
Only briefly tested. More tests ahead.
Cool!
Thanks so much for this @theBilalFakhouri, that makes life so much easier!
Have tested the 2520x1384 setting on the EOS M and it's working flawlessly, tried recording lots of short clips and resulting footage looks incredible - great to have true 16:9 aspect ratio at 2.5k. I also tried a lower resolution with that setting (at 2288x1236) to get longer record times (hope it's ok to do that) - that works fine too.
Hallo, guys! Thank you for your work!
Maybe I'm too late for this party, but until now I used this incredible mode only in FullRes for timelapsing and hyperlapsing.
Now I plan to make some videos of people, with sound and made some tests.
Now feedback:
1. First of all, thanks g3gg0 for 2018-06-26 23:36 build: sound is great, imports to DR automatically in sync.
I didn't shoot anything with sound with mlv_lite before, so i'm unsure, is it fixed, or simply works.
But, need to say, i didn't even used last mlv_dump, but "mlv_dump on steroids".
It has feature to analyse every frame for stripes.
So, i didn't tested last build of mlv_dump, but what i have for converting MLV, works perfectly.
2. Need to be very careful whether to use compressed formats, if you plan to shoot something fine-grained :)
The fun part, the camera keeps shooting after glitch and video continues,
it's just no possible, due to messages on screen.
3. Canon AutoISO works correctly only in 14bit. In other modes it should be disabled with notice for user.
4. Exposure shift in 3.5K 1:1 centered mode. maybe it discussed, i didn't find, sorry.
Look at video from OP: https://vimeo.com/217313287 at 05:35:
as zoom button is pressed and x5 mode entered, exposure changed from 1/50 to 1/35.
It need to be manually corrected every time? Nobody noticed that?
And only this mode supports compressed with reduced bitdepths thus allows maximum resolutions! :(
Quote from: mothaibaphoto on June 30, 2018, 04:48:29 AM
1. First of all, thanks g3gg0 for 2018-06-26 23:36 build: sound is great, imports to DR automatically in sync.
I didn't shoot anything with sound with mlv_lite before, so i'm unsure, is it fixed, or simply works.
But, need to say, i didn't even used last mlv_dump, but "mlv_dump on steroids".
It has feature to analyse every frame for stripes.
you are welcome.
for DR i use MLVFS which works smooth in all situations i had so far.
saves a lot of time and space
Quite exciting times for the ML community! Congratulations, Bilal!
Will your crop_rec_4k build work on the 100D too?
For those who wish to gain an impression on what quality to expect from their Rebels at 2520x1304 resolution, here is some test footage with sound shot on the 100D at 18 fps:
https://we.tl/1MZVexsbjw
At this frame rate, the 100D provides continuous recording in the above resolution. More details here:
https://www.magiclantern.fm/forum/index.php?topic=16040.1425
#IDA_ML evrything looks good and very impressive accept the focus dots. my 70D get lots of moire and color artifacts which is not the case with other cameras like 550D and 650D which i had own in the past and sold out to get a 70D to shoot better raw. but 70D is the worse camera for shooting raw. my 70D can't shoot any tress bcoz of color artifacts. i have unsinstall ML from my 70D few months back. now i think i should get a used 100D as i love shooting raw
Added 100D to the mix in crop_rec.c code. Thanks to @theBilalFakhouri for publishing his code. Could follow and add. Still things to fix like engio stop(disabled atm). Some other stuff to look through as well.
https://bitbucket.org/Dannephoto/magic-lantern/branch/crop_rec_4k_mlv_lite_snd_sd_uhs_HDR_croprec
Settings working in menu(actually 3x3 720p is not working atm):
"OFF",
"2.5K 2520x1304",
"3K 3096x1320",
"4K 4056x2552",
"3x3 720p",
Test version(use at own will and risk):
https://bitbucket.org/Dannephoto/magic-lantern/downloads/magiclantern-Nightly.2018Jun30.100D101_2K-3K-4K.zip
Tip: if settings aren´t applied enter canon menu and go back, also turn raw video on and off if new aspect ratios don´t apply directly...
@Bilal, Levas, Danne, mk11174,
I've just reworked the PowerSaveTiming and ReadOutTiming registers in a generic way, according to this description (https://www.magiclantern.fm/forum/index.php?topic=19300.msg202783#msg202783). That means, you no longer have to hardcode these in your presets - they should work out of the box if you override the other registers like usual. Can you test on your models? Didn't do the integration yet (working on that); just adapt this commit (https://bitbucket.org/hudson/magic-lantern/commits/5ec333c896ffc4d83439befb8e5080a1d322f6cb) to your local branches.
If it works, that would reduce the number of camera-specific bits, making the code more portable.
One additional test:
int readout_end = shamem_read(0xC0F06804) >> 16; /* fixme: D5 only */
+ readout_end -= 10; /* with this added, you should get about 10 bad lines at the bottom of the image */
Updating my working folder with ML. We're working on the 'Crop_rec_4K_MLV_SND' branch, right ?
I see the MLV_Lite is updated 3 days ago, so this is probably the newest branch we're working in for crop_rec_4K
I noticed an error in lossless.c while compiling MLV_lite.
It's in Line 345: (Last two lines in this piece of code)
/* there are a few registers possibly related to RD2/WR2 - we don't use them */
/* 5D3: [0xC0F375B4] <- 0xF6D0B8F, [0xC0F12020] <- 0x18A0127 => slice width / 10, slice height / 10 */
/* 6D: [0xC0F375B4] <- 0xE7B0ADF, [0xC0F12020] <- 0x13400E7 => slice width / 12, slice height / 12 */
/* 70D: [0xC0F37300] <- 0xE7B0ADF, [0xC0F12020] <- 0x13400E7 => slice width / 12, slice height / 12 */
/* 650D, 700D, 100D, EOSM, EOSM2:
* [0xC0F375B4] <- 0xDC70A4F, [0xC0F12020] <- 0x1B80149 => /* slice width / 8, slice height / 8 */
Needs to be fixed to:
/* there are a few registers possibly related to RD2/WR2 - we don't use them */
/* 5D3: [0xC0F375B4] <- 0xF6D0B8F, [0xC0F12020] <- 0x18A0127 => slice width / 10, slice height / 10 */
/* 6D: [0xC0F375B4] <- 0xE7B0ADF, [0xC0F12020] <- 0x13400E7 => slice width / 12, slice height / 12 */
/* 70D: [0xC0F37300] <- 0xE7B0ADF, [0xC0F12020] <- 0x13400E7 => slice width / 12, slice height / 12 */
/* 650D, 700D, 100D, EOSM, EOSM2: [0xC0F375B4] <- 0xDC70A4F, [0xC0F12020] <- 0x1B80149 => slice width / 8, slice height / 8 */
Don't know if it gave any bugs while using it :P
@a1ex
Tested PowerSaveTiming and ReadOutTiming commit and put it here in my latest crop_rec code for 100D(is it even supose to work for this cam out of the box?)
This part:
if (is_100D)
{
switch (crop_preset)
{
/* 3x3 binning in 720p (in 1080p it's already 3x3) */
case CROP_PRESET_3x3_1X:
/* ADTG2/4[0x800C] = 2: vertical binning factor = 3 */
adtg_new[0] = (struct adtg_new) {6, 0x800C, 2};
break;
case CROP_PRESET_2K_100D:
/* adjust vertical resolution */
adtg_new[2] = (struct adtg_new) {6, 0x8172, nrzi_encode(0x87c + delta_adtg0)};
adtg_new[3] = (struct adtg_new) {6, 0x8178, nrzi_encode(0x87c + delta_adtg0)};
adtg_new[4] = (struct adtg_new) {6, 0x82B6, nrzi_encode(0x8F4 + delta_adtg0)};
break;
case CROP_PRESET_3K_100D:
/* adjust vertical resolution */
adtg_new[2] = (struct adtg_new) {6, 0x8172, nrzi_encode(0x77C + delta_adtg0)};
adtg_new[3] = (struct adtg_new) {6, 0x8178, nrzi_encode(0x77C + delta_adtg0)};
adtg_new[4] = (struct adtg_new) {6, 0x82B6, nrzi_encode(0x7F4 + delta_adtg0)};
break;
case CROP_PRESET_4K_100D:
adtg_new[2] = (struct adtg_new) {6, 0x8172, nrzi_encode(0x8FD + delta_adtg0)};
adtg_new[3] = (struct adtg_new) {6, 0x8178, nrzi_encode(0x8FD + delta_adtg0)};
adtg_new[4] = (struct adtg_new) {6, 0x82B6, nrzi_encode(0x8F4 + delta_adtg0)};
break;
}
}
Into this:
if (is_100D)
{
switch (crop_preset)
{
/* assuming FPS timer B was overridden before this */
int fps_timer_b = shamem_read(0xC0F06014) & 0xFFFF;
int readout_end = shamem_read(0xC0F06804) >> 16; /* fixme: D5 only */
/* PowerSaveTiming registers */
/* after readout is finished, we can turn off the sensor until the next frame */
/* we could also set these to 0; it will work, but the sensor will run a bit hotter */
/* to be tested to find out exactly how much */
adtg_new[4] = (struct adtg_new) {6, 0x8172, nrzi_encode(readout_end + 1) }; /* PowerSaveTiming ON (6D/700D) */
adtg_new[5] = (struct adtg_new) {6, 0x8178, nrzi_encode(readout_end + 1) }; /* PowerSaveTiming ON (5D3/6D/700D) */
adtg_new[6] = (struct adtg_new) {6, 0x8196, nrzi_encode(readout_end + 1) }; /* PowerSaveTiming ON (5D3) */
adtg_new[7] = (struct adtg_new) {6, 0x8173, nrzi_encode(fps_timer_b - 1) }; /* PowerSaveTiming OFF (6D/700D) */
adtg_new[8] = (struct adtg_new) {6, 0x8179, nrzi_encode(fps_timer_b - 1) }; /* PowerSaveTiming OFF (5D3/6D/700D) */
adtg_new[9] = (struct adtg_new) {6, 0x8197, nrzi_encode(fps_timer_b - 1) }; /* PowerSaveTiming OFF (5D3) */
adtg_new[10] = (struct adtg_new) {6, 0x82B6, nrzi_encode(fps_timer_b - 3) }; /* PowerSaveTiming ON? (700D) */
/* ReadOutTiming registers */
/* these shouldn't be 0, as they affect the image */
adtg_new[11] = (struct adtg_new) {6, 0x82F8, nrzi_encode(readout_end + 1) }; /* ReadOutTiming */
adtg_new[12] = (struct adtg_new) {6, 0x82F9, nrzi_encode(fps_timer_b - 1) }; /* ReadOutTiming end? */
break;
}
}
Results were bad when viewing with preview set to on compared with the older code. Also tested the readout_end -= 10; and it seemd to shave of bottom part.
Take this with a grain of salt. Pretty tired over here so confirming from others are welcome.
Codebase:
https://bitbucket.org/Dannephoto/magic-lantern/branch/crop_rec_4k_mlv_lite_snd_sd_uhs_HDR_croprec
Almost there - leave your custom definition for 0x800C, since that one is not covered in the generic block.
I'd expect these registers (PowerSaveTiming & ReadOutTiming) to work out of the box on any D5 model (tested on 60D, to some extent), but the other registers still need to be configured manually.
If it's still not working, try to figure out which register is causing issues. To do that, leave the old values for a subset of them, until you narrow down.
Edit: hold a second, found a typo at register 0x82B6 (https://bitbucket.org/hudson/magic-lantern/commits/c692fa66d4ce9adca8c2fb923adade9987194669) (not used on 5D3). Pushed the fix. The above still applies.
@a1ex
The generic way for (PowerSaveTiming & ReadOutTiming) is working in 700D in all presets ;D
Updated code in this commit (https://bitbucket.org/bilal_fakhouri/magic-lantern-crop_rec/commits/eb4ecb96f1585a821d26a316890c469e3bc8a419).
Generic code works on 100D! Good stuff!
https://bitbucket.org/Dannephoto/magic-lantern/commits/6928b96393d12363e95d2d69dbf83e982c33fd2f
https://bitbucket.org/Dannephoto/magic-lantern/commits/97c9679b401965b6727279e139d4c828f488498a
New 100D build:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/magiclantern-Nightly.2018Jun30.100D101_2K-3K-4K.zip
@a1ex
I'd like to fix something about detecting x5 mode and patch or don't patch this mode ,, as I understand you are using CENTER_Z preset (a preset for normal x5 mode when pressing zoom button twice so no registers were overridden except for centering preview) for checking if we in x5 mode or not --> Then apply the preset we chose in the right mode (e.g If I chose UHD preset the message appears I should got into x5 mode to use this preset the problem is this is not happening it's only patch x1 mode or mv1080 ).
Okay what I did (actually @mk11174 did it) for force patching x5 to change this:
/* cmos_vidmode_ok doesn't help;
* we can identify the current video mode from 0xC0F06804 */
for (uint32_t * buf = (uint32_t *) regs[0]; *buf != 0xFFFFFFFF; buf += 2)
{
uint32_t reg = *buf;
uint32_t old = *(buf+1);
if (reg == 0xC0F06804)
{
if (is_5D3)
{
engio_vidmode_ok = (crop_preset == CROP_PRESET_CENTER_Z)
? (old == 0x56601EB) /* x5 zoom */
: (old == 0x528011B || old == 0x2B6011B); /* 1080p or 720p */
}
if (is_700D)
{
engio_vidmode_ok = (crop_preset == CROP_PRESET_CENTER_Z_700D)
? (old == 0x4540298) /* x5 zoom */
: (old == 0x4a601d4 || old == 0x2d701d4); /* 1080p or 720p */
To this:
if (is_700D)
{
engio_vidmode_ok =
(old == 0x4540298) ||/* x5 zoom */ (old == 0x4a601d4 || old == 0x2d701d4); /* 1080p or 720p */
}
And this mean there are a problem with CENTER_Z_700D:
I think it's in this part of code ? 5D3 Uses CMOS 1 too for detecting x5 mode and there are something related to CENTER_Z ?
static int FAST check_cmos_vidmode(uint16_t* data_buf)
{
int ok = 1;
int found = 1;
while (*data_buf != 0xFFFF)
{
int reg = (*data_buf) >> 12;
int value = (*data_buf) & 0xFFF;
if (is_5D3)
{
if (reg == 1)
{
found = 1;
switch (crop_preset)
{
case CROP_PRESET_CENTER_Z:
{
/* detecting the zoom mode is tricky;
* let's just exclude 1080p and 720p for now ... */
if (value == 0x800 ||
value == 0xBC2)
{
ok = 0;
}
break;
}
default:
{
if (value != 0x800 && /* not 1080p? */
value != 0xBC2) /* not 720p? */
{
ok = 0;
}
break;
}
}
}
}
I tried CMOS 7 or 5 for 700D (which have different values in mv1080, mv720 and x5) in the same way above but it didn't work.
Why we just detect current mode by ADTG2[800C] ? and if needed with C0F0[6804] ?
Short question:
How I can patch only x5 mode and get normal view (not patched) in mv1080 and mv720 like 5D3 when crop_rec on ?
Something related:
In old 720p 3x3 preset it's also patch mv1080 for 700D and give RAW detect error (white bar in the bottom) ,, we can make it only to patch 720p mode --> this will be something better for changing between the mods without worry about turning on and off crop_rec in mv1080.
Quote from: theBilalFakhouri on June 29, 2018, 04:47:20 PM
Experimental crop_rec with higher resolutions for 700D (it Should work for EOS M)
This is a great accomplishment in terms of unlocking these higher resolutions of course. Realistically, from my own personal experience with the 700D and a class 3 write speed Sandrisk Extreme Plus SD card, everything is severely limited unless one overclocks the SD bus via Danne's Crop 4K build. I can only yield maybe about 3 seconds of footage before frame skipping, even at the 2.5K preset. Does anyone have any different experience?
Added support for eosm. Generic code confirmed working:
https://bitbucket.org/Dannephoto/magic-lantern/commits/a9eff349f8cd79669ef307d56c88d69987c454bf
in crop_rec menu:
"OFF"
"2.5K 2520x1304"
"3K 3072x1304"
"4K 4032x2558"
"3x3 720p"
Build download:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/magiclantern-Nightly.2018Jul01.EOSM202_2K-3K-4K.zip
@Ballinger
Quote from: Ballinger on June 30, 2018, 11:47:46 PM
This is a great accomplishment in terms of unlocking these higher resolutions of course. Realistically, from my own personal experience with the 700D and a class 3 write speed Sandrisk Extreme Plus SD card, everything is severely limited unless one overclocks the SD bus via Danne's Crop 4K build. I can only yield maybe about 3 seconds of footage before frame skipping, even at the 2.5K preset. Does anyone have any different experience?
No , it's just same thing for all small cameras --> The higher resolutions including 2520x1384 needs at least 80MB/s write speed (a bit more or less depending on the scene) with sd_uhs @160MHz (68MB/s) we can get ~55MB/s effective write speed in 23.976 (https://www.magiclantern.fm/forum/index.php?topic=12862.msg202507#msg202507). You can see in 4K preset 4096x2560 @9.5fps with sd_uhs the write speed ~62MB/s (Drop down the resolution to 3520x1980 16:9 or a bit less from "4K 1:1" preset you will see continuous recording @ 9.5fps and write speed 62MB/s)
The solution is to overclock sd_uhs up to 208MHz (104MB/s) and as I can guess it will make effective write speed at least 80MB/s or more.
And of course lower bit-depths (https://www.magiclantern.fm/forum/index.php?topic=9741.msg202870#msg202870) like 10-bit lossless will help.
BTW -- you can get continuous 2520x1080 @23.976 using FPS override without crop_rec at 11-bit lossless and using sd_uhs.
EDIT: You can use also FPS override with crop_rec to get continuous recording at lower FPS (e.g 2520x1384 @ 18FPS & 4096X2560 @ 6FPS for time-lapses (https://www.magiclantern.fm/forum/index.php?topic=16040.msg203380#msg203380)).
There seems to be a limit for the amount of adtg registers you can override in crop_rec.c
When I want to override more the 10 adtg registers in one preset, everything after 'adtg_new[9]' doesn't get override.
Where can I change the code in crop_rec.c , so it accepts more then 10 adtg registers ?
I want to go further
adtg_new[10]
adtg_new[11]
etc.
@Levas
See the line 727 at this commit (https://bitbucket.org/hudson/magic-lantern/commits/5ec333c896ffc4d83439befb8e5080a1d322f6cb).
Thanks, exactly the line of code I was looking for :)
Quote from: a1ex on June 30, 2018, 04:10:42 PM
@Bilal, Levas, Danne, mk11174,
I've just reworked the PowerSaveTiming and ReadOutTiming registers in a generic way, according to this description (https://www.magiclantern.fm/forum/index.php?topic=19300.msg202783#msg202783). That means, you no longer have to hardcode these in your presets - they should work out of the box if you override the other registers like usual. Can you test on your models? Didn't do the integration yet (working on that); just adapt this commit (https://bitbucket.org/hudson/magic-lantern/commits/5ec333c896ffc4d83439befb8e5080a1d322f6cb) to your local branches.
Can't get it to work here with my version of Crop_rec, tried to alter the crop_rec version of 'Crop_rec_4K_mlv_snd' branch, but got loads of compiling errors.
Maybe if there is a version of crop_rec.c from the 'Crop_rec_4K_mlv_snd' branch with more camera's in it, probably then I could succeed ?
Or a template version of it, with empty presets for the 6d, but code for where to put the presets available.
At the moment I'm using the vertical registers as follows ?
8172 = same value as first three characters of 6804
8178 = same value as first three characters of 6804
81F8 = same value as first three characters of 6804 - 1
8173, 8179 and 81f9 I have set to '0'.
713c = same value as first three characters of 6804
7150 = same value as first three characters of 6804 -1
Not sure if this is logical or correct, but works for me.
Not sure what values to use for registers 8173, 8179 and 81f9.
@Alex, I'm wondering how the digital gain is applied for 12 bit lossless preset ?
I see some references in MLV_lite to putting a digital gain value in bpp_digital_gain.
But I don't see where this translates to settings in register(s).
Which register(s) is/are used for digital gain ?
Like to try what happens if the digital gain is hardcoded in the crop_rec preset.
(I know liveview get's darker, but just want to try and test it)
And do I need to correct white level, and is there a way to do this in Crop_rec ?
Was thinking, maybe we could use crop_rec more as a total filming preset option.
By putting not only resolution and fps in the preset, but also recording type, like 12 bit lossless.
Already have tried hardcoding some lower analog gain values in the crop_rec preset (8882...to...8888 registers), for some small improvement in dynamic range.
But I ended up with pink highlights...does that mean I went to far, or does it mean I have also adjust the white level somewhere ?
I determined the values for analog gain by overexposing the camera and setting the registers back to where it just hit the edge of the magic lantern raw histogram.
Look at Danne's version: https://bitbucket.org/Dannephoto/magic-lantern/src/crop_rec_4k_mlv_lite_snd_sd_uhs_HDR_croprec/modules/crop_rec/crop_rec.c
Drop your values for 0x8172, 0x8179 etc and paste this block somewhere in your adtg_hook. Make sure adtg_new's size is at least 13.
{
/* assuming FPS timer B was overridden before this */
int fps_timer_b = (shamem_read(0xC0F06014) & 0xFFFF) + 1;
int readout_end = shamem_read(0xC0F06804) >> 16; /* fixme: D5 only */
/* PowerSaveTiming registers */
/* after readout is finished, we can turn off the sensor until the next frame */
/* we could also set these to 0; it will work, but the sensor will run a bit hotter */
/* to be tested to find out exactly how much */
adtg_new[4] = (struct adtg_new) {6, 0x8172, nrzi_encode(readout_end + 1) }; /* PowerSaveTiming ON (6D/700D) */
adtg_new[5] = (struct adtg_new) {6, 0x8178, nrzi_encode(readout_end + 1) }; /* PowerSaveTiming ON (5D3/6D/700D) */
adtg_new[6] = (struct adtg_new) {6, 0x8196, nrzi_encode(readout_end + 1) }; /* PowerSaveTiming ON (5D3) */
adtg_new[7] = (struct adtg_new) {6, 0x8173, nrzi_encode(fps_timer_b - 1) }; /* PowerSaveTiming OFF (6D/700D) */
adtg_new[8] = (struct adtg_new) {6, 0x8179, nrzi_encode(fps_timer_b - 1) }; /* PowerSaveTiming OFF (5D3/6D/700D) */
adtg_new[9] = (struct adtg_new) {6, 0x8197, nrzi_encode(fps_timer_b - 1) }; /* PowerSaveTiming OFF (5D3) */
adtg_new[10] = (struct adtg_new) {6, 0x82B6, nrzi_encode(readout_end - 1) }; /* PowerSaveTiming ON? (700D); 2 units below the "ON" timing from above */
/* ReadOutTiming registers */
/* these shouldn't be 0, as they affect the image */
adtg_new[11] = (struct adtg_new) {6, 0x82F8, nrzi_encode(readout_end + 1) }; /* ReadOutTiming */
adtg_new[12] = (struct adtg_new) {6, 0x82F9, nrzi_encode(fps_timer_b - 1) }; /* ReadOutTiming end? */
break;
}
Guess for compiling errors: if you declare variables within a case label, you need { curly brackets }.
Digital gain is applied in raw.c; it's mlv_lite that's asking for different gains.
For the dynamic range stuff, once you get pink highlights, you have reached the limit for that gain you are adjusting (if you lower it further than that, you'll no longer get additional highlight). The idea is to lower these gains just as long as you keep getting additional detail. Once your whites drop before your declared white level, you probably went too far; in this case, you can try adjusting other gains and find a sweet spot. I should probably implement some sort of auto-tuning procedure.
@Alex, the above works!
Could even remove the case crop preset thing, since this works for all crop presets, no need to define these registers per crop_presets
So cleans up a lot of code :D
Quote from: a1ex on July 04, 2018, 10:29:46 AM
Digital gain is applied in raw.c; it's mlv_lite that's asking for different gains.
Looked in raw.c, but only could find the register c0f0 8030, but that one only darkens or brightens the liveview, doesn't affect the raw stream ?
For lower lossless bit depths, the raw stream must be darkened too, right ?
Good point, will do that.
Some older questions regarding video mode detection: identifying it from C0F06804 is not always possible. A comment from my not-yet-committed code:
/* properties are fired AFTER the new video mode is fully up and running
* to apply our presets, we need to know the video more DURING the switch
* we'll peek into the PathDriveMode structure for that
*
* Example: x10 -> x1 on 5D3
* This sequence cannot be identified just by looking at C0F06804;
* some ADTG registers that we need to overide are configured before that.
*
* GuiMainTask: GUICMD_PRESS_BUTTON_SOMETHING(0x0)
* CtrlSrv: DlgLiveView.c PRESS_TELE_MAG_BUTTON KeyRepeat[0]
* Gmt: gmtModeChange
* Evf: evfModeChangeRequest(4)
* Evf: PATH_SelectPathDriveMode S:0 Z:10000 R:0 DZ:0 SM:1
* (lots of stuff going on)
* Evf: evfModeChangeComplete
* (some more stuff)
* Gmt: VisibleParam 720, 480, 0, 38, 720, 404.
* Gmt: gmtUpdateDispSize (10 -> 1)
* PropMgr: *** mpu_send(06 05 09 11 01 00) ; finally triggered PROP_LV_DISPSIZE...
*/
In particular, when switching from x10 to x1, first ADTG[8000] is changed from 5 to 6, then (later) C0F06804 is changed from 0x56601EB to 0x528011B (so if we rely on this register for video mode detection, we miss the ADTG register). Then, other ADTG/CMOS registers are set, but ADTG[8000] is no longer touched.
I think I've found a way around that by looking at this message:
PATH_SelectPathDriveMode S:%d Z:%lx R:%lx DZ:%d SM:%d
Need to check whether it's portable.
C0F08030 affects the raw stream if raw type (https://www.magiclantern.fm/forum/index.php?topic=18393.0) is 0x12, for example (look for entries "scaled by digital ISO" - that's raw data after some preprocessing). The default raw type (CCD), not affected by digital ISO, is, to my understanding, the first in the pipeline (with minimal processing; there is something called "HIV shading" that happens during image capture; I've disabled it at some point by zeroing out the correction data; here's what I think it's a totally raw (https://a1ex.magiclantern.fm/bleeding-edge/nohivshd/IMG_1883.CR2) file).
That's why 14-bit lossless (using raw type CCD) works and 8..12-bit lossless (using raw type 0x12 to darken the image) doesn't work at high resolutions. The latter needs some more stuff reconfigured at the right resolution. No idea what. Changing HIV_V_SIZE didn't help. Locking modified registers from adtg_gui didn't help. Manually resizing the EDMAC transfer used for HIV shading didn't help either. Guess I need to reconfigure all the other image processing modules that affect raw type 0x12; Canon knows which ones :D
Ah I totally missed that there's another raw stream used for lower lossless options.
Raw-type 12, isn't that the one used a few years ago.
Remember some discussions about problems with raw recording and when all went to raw type '10' most of it was solved :P
But, raw type 10 works with lossless compression.
But can't be affected by digital gain...
But it is affected by analog gain :D
Why you're not using analog gain registers (ADTG2 8882,8884,8886 and 8888 plus the ADTG 8,9,a and b) ?
I'm only using full stop iso's and the registers don't change much up too iso 6400, and because of that exactly the usable iso range for video ;D
Quote from: a1ex on July 04, 2018, 02:48:14 PM
I've disabled it at some point by zeroing out the correction data; here's what I think it's a totally raw (https://a1ex.magiclantern.fm/bleeding-edge/nohivshd/IMG_1883.CR2) file).
Looks a lot of what I see when I change values of CMOS 4 on the 6d, not sure, but I think you can see a sort of darkframe, when fiddling with this cmos registers.
Yeah, that could work. IIRC I left the analog gains alone to prevent future conflicts with the ISO stuff. Maybe I was overthinking it :P
Is there an easy way to test this with only crop_rec?
I can already overwrite analog registers in a crop preset, can set them to values so ml raw histograms ends two stops before the end.
But if I do that, I end up with pink highlights.
Is there an easy way to fix the white level?
And can it be done in a crop_preset ?
Or is the only way to fix white level in Mlv_lite or raw.c ?
Try lowering the values until the highlights end up just below the declared white level. Then go back (increase them again) a little bit, so they end up just above that. That's pretty much it for that gain.
If you reduce that gain even further, you won't gain any more DR.
Then play with some other gains, evaluate the DR etc. It's a huge rabbit hole :D
That's the method to optimize DR.
But for now I'm after better lossless compression, and for that I want to lower DR, right?
That's how it works I guess, lower the analog gain 2 stops before the end of ML histogram should be about the same as 12 bit lossless right, or not ???
Yeah, in this case, mlv_lite definitely needs to know about the new white levels somehow. Will think about it, but currently working on something else.
Have some settings where clipping point is before the end of the histogram and want to determine the real white level, or let's say I want to know what the maximum value is for the pixels in a cr2 file.
I have cr2 files with blown out, pink highlights, can I determine the real white level with exiftool ?
Did exiftool plus cr2 file in terminal, without extra command and I get a lot of information, also this:
Normal White Level : 14958
Specular White Level : 15470
Linearity Upper Margin : 10275
Does this tell anything about where the real white level should be ?
Is the maximum value for a pixel in this case, 10275 instead of 16384 ? ???
CR2 is another story; white levels there are different.
From the white level in a MLV: (old_white - 2048) * gain + 2048. In your case, for -2 EV, gain is 1/4.
Getting close to implementing some much-needed refactors, to make the code more generic, but got a small stumbling block at shutter override: the shutter blanking registers on all D5 APS-C models (EOS M, 700D, 650D and 100D). Couldn't figure it out in QEMU.
What you have to do: load adtg_gui.mo and write down the address for registers 805F and 8061 (at the bottom of screen).
I've found these lines in logs from Bilal:
0002805f: 37c (was 21c) ISO=400 Tv=50 Av=35 lv=0 zoom=10 mv=0 res=-1 crop=-1 task=Evf pc=ff56d980 addr=40452d18
00028061: 37c (was 21c) ISO=400 Tv=50 Av=35 lv=0 zoom=10 mv=0 res=-1 crop=-1 task=Evf pc=ff56d980 addr=40452d1c
These registers are defined for EOS M:
#define FRAME_SHUTTER_BLANKING_ZOOM (*(uint16_t*)0x40481B20) // ADTG register 805f
#define FRAME_SHUTTER_BLANKING_NOZOOM (*(uint16_t*)0x40481B24) // ADTG register 8061
#define FRAME_SHUTTER_BLANKING_READ (lv_dispsize > 1 ? FRAME_SHUTTER_BLANKING_NOZOOM : FRAME_SHUTTER_BLANKING_ZOOM) /* when reading, use the other mode, as it contains the original value (not overriden) */
but FRAME_SHUTTER_BLANKING_WRITE is disabled. Don't remember the reason - it should be tested, as I need it enabled for shutter speed overrides in crop_rec.
QEMU shows different addresses, so I'm unable to cross-check them.
Also looking for some basic testing on cameras currently working with crop_rec, just to be sure I didn't break anything with the refactors. The new presets are not included yet. For advanced users only (who can compile from source and understand the code).
Quote from: a1ex on July 04, 2018, 07:26:29 PM
What you have to do: load adtg_gui.mo and write down the address for registers 805F and 8061 (at the bottom of screen).
Short answer:
Register | 700D | EOSM |
805F | 404512F4 | 40481B20 |
8061 | 404512FC | 40481B24 |
Ok--so this confirms the values that were already found for the EOSM but the journey I took to get there was rather interesting so here comes a longer answer.
Reply #1745 (https://www.magiclantern.fm/forum/index.php?topic=19300.msg203623#msg203623) points out some "much-needed refactors" so I assume that is happening in the config_var_refactor branch. That branch was recently merged into the lua_fix branch but there isn't a new build posted on the experiments download page (https://builds.magiclantern.fm/experiments.html) yet so another assumption is that this isn't ready for regular users yet. Tried compiling adtg_gui in the lua_fix branch and it didn't compile. Tried the one posted on the modules download page (https://builds.magiclantern.fm/modules.html) but that didn't work either. Ok, we're only looking for register addresses at this point so I compiled the iso-research branch and got the addresses.
700D
(https://farm2.staticflickr.com/1822/42375257165_70e032fced.jpg) (https://flic.kr/p/27yygCa)
(https://farm2.staticflickr.com/1805/43279262341_94ccd46e6c.jpg) (https://flic.kr/p/28WrwSi)
Something weird happened on the EOSM. I couldn't get to the 805F and 8061 registers at first because the camera was set to 1280x720/60. However, I could see the 805E and 8060 registers. (Note that on the 700D all these registers are visible at the same time.)
EOSM
(https://farm1.staticflickr.com/839/42375855525_22d302758c.jpg) (https://flic.kr/p/27yBkuH)
(https://farm1.staticflickr.com/833/28410512567_4ffd47e7e9.jpg) (https://flic.kr/p/Khxizz)
Switched over to 1920x1080/24 and the registers we're looking for showed up.
EOSM
(https://farm1.staticflickr.com/918/42375429425_3761576059.jpg) (https://flic.kr/p/27yz9Qa)
(https://farm1.staticflickr.com/839/42375855525_22d302758c.jpg) (https://flic.kr/p/27yBkuH)
Note that I might have had to push the zoom button for the registers to show up but I'm not sure if that was necessary.
I did a search for FRAME_SHUTTER_BLANKING_ZOOM / FRAME_SHUTTER_BLANKING_NOZOOM and here is where things started getting interesting. As a1ex mentioned, FRAME_SHUTTER_BLANKING_WRITE is disabled on the EOSM. What I found was that all of the FRAME_SHUTTER_BLANKING constants are disabled on the 100D. In addition, it uses the 805E and 8060 registers instead of 805F and 8061. Also interesting is that the 5D3 uses 805E / 8060 while the 6D uses 805F / 8061 and both of these cameras have all the FRAME_SHUTTER_BLANKING constants enabled.
Looks to me like 805F / 8061 is for mv1080 mode (maybe why FRAME_SHUTTER_BLANKING_WRITE is disabled on the EOSM?) while 805E / 8060 is for mv720 mode. So why are we using one set and not the other? Maybe there should be a way to use both sets of registers depending on the movie settings?
Ok--still need to look into the refactors. Should we be using the lua_fix branch for this? Maybe merge lua_fix into crop_rec_4k? Or should we start with just the config_var_refactor branch?
The refactors are in the crop_rec_4k branch (making things more generic and easier to port on other cameras). Some of them are not yet pushed, as I was waiting for these tests. Just added a few more.
The stuff from other branches is not related to crop_rec or video recording.
I'm still struggling to reliably detect video mode switches. The PathDriveMode stuff described above works pretty well in most cases, except first startup and switching to movie mode from photo mode. The latter was reported to cause blank screen with some demanding presets, but now it fails with even the simplest 3x preset. Besides that, at least on 60D, the PathDriveMode is updated way too late, after the video mode switch is done, so I'm afraid it might be the same on other APS-C models.
QuoteLooks to me like 805F / 8061 is for mv1080 mode (maybe why FRAME_SHUTTER_BLANKING_WRITE is disabled on the EOSM?) while 805E / 8060 is for mv720 mode.
Can you double-check this? My hypothesis was that 5D3 uses 805E / 8060, one in zoom modes (x5, x10) and the other in non-zoom modes (1080p, 720p), while the other cameras use 805F / 8061 for the same purpose; they also set 805E / 8060 to 0, but that's probably unused. I do not see nonzero values for 805E / 8060 in your screenshots.
With the last changeset (https://bitbucket.org/hudson/magic-lantern/commits/d556a102239697c650257b3010981356bbdae64c), I'd expect shutter speeds to be correct (i.e. without this issue (https://www.magiclantern.fm/forum/index.php?topic=9741.msg203805#msg203805)). That should apply to all presets once they are included. Also expecting correct shutter speeds when using FPS override (without crop_rec).
Still looking for addresses of ADTG 805F and 8061 for 650D and 100D. Currently, crop_rec doesn't load on these models.
Quote from: a1ex on July 08, 2018, 11:40:48 PM
Also expecting correct shutter speeds when using FPS override (without crop_rec).
The correct shutter speeds are working in 700D when using FPS override and this is very cool but with a problem showed in this video:
When this happening?
After stopping RAW video in mv1080 but during and before the recording there are no problems and in H.264 as the video showed.
No problems in x5. In x10 as the video.
And also a problem when enabling mlv_lite + mlv_snd in crop_rec_4k branch. It was working right?:
(https://image.ibb.co/htrmZ8/mlv_snd.jpg)
@Bilal: before compiling, merge it with either crop_rec_4k_mlv_snd or crop_rec_4k_mlv_lite_snd (they both provide mlv_snd compatibility with mlv_lite, in a slightly different way).
Will check H.264 later.
Edit: watched again a few times; the issue happens with plain FPS override, as soon as you start recording H.264. If you comment out FRAME_SHUTTER_BLANKING_WRITE, it no longer shows these artifacts, right?
@Alex,
I'v got a full resolution crop_rec mode for the 6d now, but there are way less frames in the buffer memory then expected.
6d has about 250MB buffer.
Not sure if frames that go to the buffer memory are already lossless compressed or not ?
But anyway, if I shoot 5472 x 2290 at 4.8 fps
One uncompressed frame is about 21MB.
I expect that at least 10 frames should fit in the buffer, before automatically stopping recording.
But when I record, recording stops immediately and shows 2 frames writing to memory card out of the buffer (2 when I expect 10 ?).
Is there a logical reason that there are way less frames in the buffer then expected with full resolution crop mode ?
100D
805e -> 41697784
8060 -> 41697ac0
805f -> 41697bc4
8061 -> 41697bc8
what dfort mentioned: 805f and 8061 didn't show up until I switched to 1920/24
Guess different on my 700D?
Also, I was getting an N/A for ML shutter Display until I changed the addresses for my camera, once I did, I get shutter value in ML again.
In movie mode 1080P nothing enabled in ML other then ADTG tool.
FPS Override off, everything default ML settings
They also only stay loaded in ADTG tool for a few seconds then gone until I restart camera.
The values are different in both images because on the 700D, the frame rate jumps around randomly 29.973 and 29.958, has always done this, so these values change with the frame rate jump.
Guess I didnt need to post all these images because I now see these values are only differ cause shutter speed was different, LOL!! OOOPS :-[
1080P @30fps Movie Mode
(https://thumb.ibb.co/fasnRo/VRAM0.jpg) (https://ibb.co/fasnRo)
(https://thumb.ibb.co/kgBQCT/VRAM1.jpg) (https://ibb.co/kgBQCT)
1080P @24fps Movie Mode
(https://thumb.ibb.co/kip96o/VRAM2.jpg) (https://ibb.co/kip96o)
(https://thumb.ibb.co/etHwmo/VRAM3.jpg) (https://ibb.co/etHwmo)
x5 and x10 Mode in Movie Mode all resolution modes
In x5 or x10 mode, these registries don't vanish like they do while not in x5 or x10 mode, they stay in the ADTG tool and if I go back to non x5 or x10 mode, and come back to ADTG tool, these registries vanish after a few seconds until restart.
(https://thumb.ibb.co/m2pbmo/VRAM4.jpg) (https://ibb.co/m2pbmo)
(https://thumb.ibb.co/byLGmo/VRAM5.jpg) (https://ibb.co/byLGmo)
Got it, so these hardcoded addresses can't be trusted.
@Bilal: maybe the address changes while recording H.264?!
Quote from: a1ex on July 09, 2018, 09:36:35 AM
If you comment out FRAME_SHUTTER_BLANKING_WRITE, it no longer shows these artifacts, right?
I didn't get it. Should I do something in the code?
Edit: If you mean deleting this from the code:
#define FRAME_SHUTTER_BLANKING_WRITE (lv_dispsize > 1 ? &FRAME_SHUTTER_BLANKING_ZOOM : &FRAME_SHUTTER_BLANKING_NOZOOM)The shutter speeds no longer affected (like the old builds the range is limited when using FPS override) and there are no artifacts.
Quote from: a1ex on July 09, 2018, 08:15:19 PM
@Bilal: maybe the address changes while recording H.264?!
No it's the same in all modes even while recording H.264.
Made some tests again sometimes the artifacts showed all modes even x5 sometimes it's not shown anything. I couldn't use adtg_gui while FPS override is on I couldn't get the fast shutter speeds --> The higher shutter speeds doesn't affect the exposure when using FPS override and when enabling adtg_gui (memory patches?) .
Also I noticed in @dfort and @mk11174 screenshots I can see the "N" for NRZI,, I don't see "N" in my cam. Edit: this because I am using old adtg_gui . Not a big thing.
If you override the shutter blanking registers 805f and 8061 from adtg_gui, do you still get these artifacts?
If not, I think I can come up with a fix (one that would also fix the old ISOless errors).
I noticed while using crop_rec module with Full Res Live view preset, resized to 4464x2512 16:9 in the preset because anything higher freezes during recording, but anyway, while using this mode, since I can stay in regular mode and don't have to go to x5 mode, I get 6.527fps and I am able to adjust my shutter in ML to 12k down to 1/7 is this really 12k?
If I go into x5 mode, can only adjust to 1/11 down to 1/7.
I also noticed, if you try to set a shutter with ML it reverts back to canon shutter speeds when you goto x5 mode, press shutter button, or go back into ML menu.
So if I goto Full Res live view preset, then goto ML menu and switch to say a non canon value, like 12K, I see the screen get super dark, if I exit ML it holds until I either go back into ML menu or try to switch to x5 mode even though I don't need to in this mode, or if I press half shutter, it will then go to 1/30 again all by itself.
If I switch to the 12k and leave ML menu and go right to recording, it holds until I do any of the above.
Is this to be expected, or not normal with your new update for these shutter blanking registries?
Update! I just tested build before these new changes and I guess this is how it works as far as not holding ML shutter speeds anyways.
I see I def dont have 12k shutter speed though before these changes, so thats new for sure, but is it really that fast, it does get pretty dark?
I believe it's really that fast, but rolling shutter is much larger than exposure time (so it may not be very useful for lightning, for example). Only tested by looking at ML spotmeter and doing the math; didn't check motion blur.
Yeah, def not good for lightning, was just thinking that was pretty fast, which is pretty cool if it can be used for anything, maybe water drops possible.
Just good to know its correct and not just some false value.
Quote from: a1ex on July 09, 2018, 11:50:53 PM
If not, I think I can come up with a fix (one that would also fix the old ISOless errors).
Tested the latest experimental crop rec 4K build from the downloadspage and when trying dual iso I get the isoles error on the 6d. Is this an easy fix, could I be able to fix it in source and test it?
Quote from: Levas on July 10, 2018, 07:54:15 AM
Tested the latest experimental crop rec 4K build from the downloadspage and when trying dual iso I get the isoles error on the 6d. Is this an easy fix, could I be able to fix it in source and test it?
https://www.magiclantern.fm/forum/index.php?topic=7139.msg201800#msg201800
Thaks Sapporo, fixed dual iso, works on my 6d now :D
Was busy the first time you post that :P
@Bilal,
Does this solve the H.264 artifacts on the EOS M when using FPS override?
--- a/src/fps-engio.c
+++ b/src/fps-engio.c
@@ -436,6 +436,9 @@
if (FRAME_SHUTTER_BLANKING_READ != *FRAME_SHUTTER_BLANKING_WRITE)
return;
+ /* read the FPS timer B directly from ENGIO shadow memory to have the latest value */
+ int fps_timer_b = (FPS_REGISTER_B_VALUE & 0xFFFF) + 1;
+
/* sensor duty cycle: range 0 ... timer B */
int current_blanking = nrzi_decode(FRAME_SHUTTER_BLANKING_READ);
int fps_timer_b_assumed_by_canon = fps_timer_b_method ? fps_timer_b : fps_timer_b_orig;
Quote from: a1ex on July 09, 2018, 11:50:53 PM
If you override the shutter blanking registers 805f and 8061 from adtg_gui, do you still get these artifacts?
If not, I think I can come up with a fix (one that would also fix the old ISOless errors).
Yes I can get the artifacts in some values and the other values it working with no artifact --> The values which working in mv1080 it doesn't work in x5 it shows artifacts.
FPS override OFF.
Quote from: a1ex on July 11, 2018, 03:29:09 PM
@Bilal,
Does this solve the H.264 artifacts on the EOS M when using FPS override?
--- a/src/fps-engio.c
+++ b/src/fps-engio.c
@@ -436,6 +436,9 @@
if (FRAME_SHUTTER_BLANKING_READ != *FRAME_SHUTTER_BLANKING_WRITE)
return;
+ /* read the FPS timer B directly from ENGIO shadow memory to have the latest value */
+ int fps_timer_b = (FPS_REGISTER_B_VALUE & 0xFFFF) + 1;
+
/* sensor duty cycle: range 0 ... timer B */
int current_blanking = nrzi_decode(FRAME_SHUTTER_BLANKING_READ);
int fps_timer_b_assumed_by_canon = fps_timer_b_method ? fps_timer_b : fps_timer_b_orig;
The problem is still there,, I am only have 700D so I tested it in 7 0 0 d .
When using "High Jello" in FPS override submenu the problem was solved (In the beginning you will see the artifact but press PLAY and get back to LiveView it's no longer here even while recording H.264) but I noticed some small flickering in some shutter speeds.
Except I couldn't use shutter speeds more than ~1/1500 in "High Jello" at 8FPS (less or a bit more FPS) but in "Exact FPS" I can get up to 1/4000 shutter speed in low FPS.
Quote from: theBilalFakhouri on July 11, 2018, 04:53:11 PM
Yes I can get the artifacts in some values and the other values it working with no artifact --> The values which working in mv1080 it doesn't work in x5 it shows artifacts.
That makes sense if the blanking value is larger than FPS timer B. In x5, timer B is smaller than in 1080p/720p, so with your procedure it's easy to get a blanking value that is too large.
If you do the other way, i.e. lock the blanking register in x5 and go back to 1080p/720p, you should see no artifacts.
If adding that line of code to fps-engio.c does not solve the issue, that might be Canon firmware setting the shutter blanking first, and FPS timer B afterwards. That's more difficult to fix; we may need to know in advance the timer B value that's going to be set by the firmware.
100fps recording on a 6d with Magic Lantern, put on a 25fps timeline 8)
Stopwatch on my phone screen:
https://drive.google.com/open?id=1Chmdf6zciPikRg4NGIGOI1uMZKnphO1M (https://drive.google.com/open?id=1Chmdf6zciPikRg4NGIGOI1uMZKnphO1M)
100fps is possible on the 6d, but not usable.
Camera interface becomes painfully slow, I almost pressed the recording button on the cam the same time as I started the stopwatch on my phone, took almost 30 seconds, before the camera starts recording (30 seconds after start button press :P )
And as you can see in the video, the bottom of the frame is still a little corrupted.
But hey, proof off concept ;D
100 fps? Wtf :o
While playing with resolutions on my 700D, I wanted to see why silent picture will not take a regular silent picture at these custom presets for Fullres LV in crop_rec.
Wanted to tweak with it because even though we can see the frame check preview at the higher res preset, you can't record it without it freezing.
So figured was easier to use regular silent pictures while adjusting.
Here are some configurations I did to see what the limit was trying both sides at max. These all take a silent picture, any higher on either side, just says preparing.
Not sure if this info is useful or not, but figured I would post since it seems there is some sort of limit factor of what we can actually record.
3900x3500 0xdcc03f1 = 0x16C7E8C
3956x3445 0xd9503ff = 0x16BEB07
3960x3444 0xd940400 = 0x16C2E28
5208x2626 0xa620538 = 0x16D31B4
Also, this is max you can record in mlv-lite without it crashing after a few seconds.
And recording as I mean, setting to record while hold HS, so like pressing it quick to grab some frames after a few tries will crash any higher then what these dimensions equal.
5208x2159 0x88F0538 = 0x12C3FC6
Quote from: Levas on July 11, 2018, 08:54:19 PM
100fps recording on a 6d with Magic Lantern, put on a 25fps timeline 8)
wow!
finally :D
Quote from: a1ex on July 11, 2018, 05:31:04 PM
If you do the other way, i.e. lock the blanking register in x5 and go back to 1080p/720p, you should see no artifacts.
Actually it affected only 720p but not like before (a bit better) it's look like this (watch this in 720p @60fps in YT) :
And in 1080p is working fine.
Quote from: Levas on July 11, 2018, 08:54:19 PM
100fps recording on a 6d with Magic Lantern, put on a 25fps timeline 8)
Cool :D
Can you share your settings ? which mode and what the registers have you changed?
@mk11174
You can use the preset "Full-res LiveView" 5208x3230 in this crop_rec (https://www.magiclantern.fm/forum/index.php?topic=19300.msg203316#msg203316) you can get 1 frame in mlv_lite 14-bit lossless sometimes you didn't get any frames (blank MLV) but if you use uncompressed 10-12-14 bit it will become more stable (3 frames in 10-bit). The problem and the solution here (https://www.magiclantern.fm/forum/index.php?topic=19300.msg197870#msg197870) and Here (https://www.magiclantern.fm/forum/index.php?topic=19300.msg197876#msg197876).
In Silent module I noticed "Simple" mode doesn't work (still preparing...) switch it to "Burst" you can shoot uncompressed DNG without problems but L-DNG it will not save the picture.
Quote from: theBilalFakhouri on July 11, 2018, 11:52:09 PM
You can use the preset "Full-res LiveView" 5208x3230 in this crop_rec (https://www.magiclantern.fm/forum/index.php?topic=19300.msg203316#msg203316) you can get 1 frame in mlv_lite 14-bit lossless sometimes you didn't get any frames (blank MLV) but if you use uncompressed 10-12-14 bit it will become more stable (3 frames in 10-bit). The problem and the solution here (https://www.magiclantern.fm/forum/index.php?topic=19300.msg197870#msg197870) and Here (https://www.magiclantern.fm/forum/index.php?topic=19300.msg197876#msg197876).
I am using my own compiled crop_rec module, its updated with all alex new tweaks, I can record at that res too, but I dont feel its stable that high, it freezes randomly during recording, and I need preview to work, so the other bits will not work for me, 14bit and 14bit compressed work just fine with preview.
Not really having an issue with the module, just was adding that info in case it was useful since there seems to be some type of limiting factor.
But thanks for the info. :)
@theBilalFakhouri
The 100fps is done in Canon 720p mode.
I did this one in 5 x zoom mode(Canon 720P mode), so changed CMOS 6 and 7 registers for center crop (little try and error on the CMOS 7 register)
The shutter blanking registers are automated by a piece of code from Alex, sended earlier in this topic.
For the rest I left Horizontal resolution alone, although I did minimize the A timer for standard horizontal resolution.
After that, calculate B timer for 100 fps, adapted 6804 resolution register to that and 713c and 7150 registers.
But camera interface is really slow at these settings.
Just found out that adjusting the B timer slightly the interface can become snappy again. At least at 75 fps.
I first had a slow interface setting for 75 fps, but lowered the B timer just by one and now I have a snappy reacting camera at 75 fps.
Hello, can we please add the following 24p modes?
16:9 24p
(1024*sqrt(10)) x (576*sqrt(10))
3238*1821 = 5,896,398 pixel readout
3238/1821 = 16/9 aspect ratio
Anamorphic 4:3 24p
(1024*sqrt(10))/sqrt(4/3) x (576*sqrt(10))*sqrt(4/3)
2804*2108 = 5,896,812 pixel readout
2804/2108 = 4/3 aspect ratio
Both of these have no issues on the hardware, they fall underneath the 5,898,240 pixel readout limit.
I'm using 5D3 1.1.3 running magiclantern-Nightly.2018May01.5D3113 with 64Gb Lexar 160MB/s and a Kamerar 256GB card.
I was recording fine until It seems I ran out of battery, switched battery and couldn't never record more than 3 frames. at first I thought it was something related to the free space of the card, I formatted them both and cleaned up space in the SD as well. Same thing I couldn't record any more than 3 frames and it stops.
besides that I've just created 3 lua scripts to switch between recording modes, other than that It was working OK.
could it be ML files got corrupted? I even turned on/off the camera several times including removing the battery.
Another weird thing I saw was the "Remaining Storage space" in the top right corner for both cards shows 15GB (both cards formatted exFat)
Reset ML to default and try again. I am sure your Lua scripts messed something up.
Quote from: g3gg0 on July 11, 2018, 10:42:23 PM
wow!
finally :D
And another FAQ entry in troll section (https://wiki.magiclantern.fm/faq#when_will_we_be_able_to_shoot_in_our_canon_100_fps_200_fps_1000_fps) going down the drain:
Quote
When will we be able to shoot in our Canon 100 fps, 200 fps, 1000 fps!?
When you buy this camera (http://www.eoshd.com/content/9973/first-of-the-new-nab-2013-camera-announcements-ft-one-with-global-shutter-4k-900fps)and slap a Canon sticker on it.
Wondering why it hasn't been hyped elsewhere yet ... CR, PP, EOSHD, Slashcam ...
Wouldn't call it usable as it is now, camera interface gets really slow.
Wonder if this 100fps also works on the crop sensor cams.
Could be even more interesting since the crop sensor cams have higher clock speed.
32Mhz vs 25.6Mhz...so who's gonna try :P
Someone could basically make a crop preset for 5x zoom mode in 720p canon mode.
Maybe even let the horizontal resolution and A timer alone, and go right for that B timer :D
Did some testing on crop_rec yesterday and found out I haven't got audio with it.
Further testing revealed that I also get no audio when crop_rec is not used ??? I get 2Kb wav files with mlv_lite...
Downloaded the newest experimental build from downloadpspage, still no audio...but audio meters are working while recording.
Took mlv_lite and MLV_snd module from the 20June crop_rec_4K_build and now I have working audio files with my raw recordings.
Is there something wrong with audio in the newer crop_rec_4K builds from the experiments page ?
Looking back through my files, looks like I got audio on 10th of juli and didn't have audio from 11 Juli until now.
I probably switched to a newer crop_rec_4K build on the 10th of juli, I always use the one on the experiments download page.
Looking at the mlv.txt files from mlv_dump, something went wrong in the crop_rec_4K build made on 3th of Juli and still ongoing.
recorded 128GiB videos *with* audio on 5D3 yesterday using crop_rec_4k_mlv_snd modules.
these modules were involved:
Block: VERS
Offset: 0x000004b0
Number: 12
Size: 180
Time: 514.940000 ms
String: 'mlv_lite built 2018-07-12 23:13:23 UTC; commit f4213dd on 2018-07-03 23:10:22 UTC by g3gg0: mlv_lite: do not write all metadata again on every single chunk '
Block: VERS
Offset: 0x00000564
Number: 13
Size: 80
Time: 522.548000 ms
String: 'mlv_meta built 2018-07-07 21:51:13 UTC; commit (no version)'
Block: VERS
Offset: 0x000005b4
Number: 14
Size: 164
Time: 530.489000 ms
String: 'mlv_play built 2018-06-20 18:58:21 UTC; commit 0e38b89 on 2018-06-18 22:11:42 UTC by g3gg0: mlv_play: use less RAM, proper cleanup on error '
Block: VERS
Offset: 0x00000658
Number: 15
Size: 180
Time: 536.738000 ms
String: 'mlv_snd built 2018-07-03 22:23:18 UTC; commit 9ec9c80 on 2018-07-03 22:20:19 UTC by g3gg0: mlv_lite/mlv_snd: rewrote state machine and placed WAVI writing... '
Block: VERS
Offset: 0x0000078c
Number: 17
Size: 156
Time: 554.623000 ms
String: 'raw_twk built 2018-04-12 13:35:36 UTC; commit 6886061 on 2017-07-01 23:55:44 UTC by g3gg0: raw_twk: rework to support 16bpp input data '
mlv_meta is not public yet, used to count up take number in INFO blocks etc
I was filming for the last 7 days using two most recent firmwares and can confirm, that sound recording works fine on my 5D3. All the videos i made have wav files on place and they are OK. Also I can confirm that I had some issues with sound before, but have no idea why it happened.
All my videos with fine sound were filmed in 1920X1080 29.97 FPS RAW.
Will take a closer look at what's going on here...
Also I noticed that in builds of July 4 and June 27 I have a right white balance after I mount MLV's via MLVFS, but in July 9 build white balance is "cold" - temperature on all videos is way more shifted to the 2500K.
(https://thumb.ibb.co/ffnXOd/Screen_Shot_2018_07_15_at_3_54_07_PM.png) (https://ibb.co/ffnXOd)
(https://thumb.ibb.co/hbS7Od/Screen_Shot_2018_07_15_at_3_54_25_PM.png) (https://ibb.co/hbS7Od)
Check your wb settings for "k" kelvin. If set to awb mlvfs goes for underlying kelvin numbers.
I know that, but it was always AWB, and previous firmware versions sent the right temperature setting to Resolve through MLVFS...
did your change your camera's WB settings since then?
./mlv_dump <mlv-file> -v > out.txt
and check your log files for WBAL block, what does it say?
Quote from: ustik86 on July 15, 2018, 06:13:30 PM
I know that, but it was always AWB, and previous firmware versions sent the right temperature setting to Resolve through MLVFS...
Mlvfs don't compute awb settings. Try again with older "working" builds and see if wb will change or stay the same.
but doesn't it pass thorugh the user's settings which are in WBAL block?
Yes, all other blocks/wb modes are recognized through metadata. Awb isn't. Is it possible to grab wb from awb computations and put the i fo in mlv metadata?
Havn't had much time to test the latest developments in crop_rec_4k yet though.
Quote from: Danne on July 15, 2018, 09:58:30 PM
Is it possible to grab wb from awb computations and put the i fo in mlv metadata?
Possible, but not straightforward and likely model-specific. The coefficients are probably computed by the AeWb task. However, the MLV converters also have the option to implement AWB from scratch. I've implemented two such algorithms (https://www.magiclantern.fm/forum/index.php?topic=10265.msg111203#msg111203) in cr2hdr, but still had to fine-tune their output in practice.
Yes, and algoritm is more precise than canon awb when comparing against whites. However, nice to "get what you see" in liveview as well when selecting half crappy canon awb computations 8)
How about a mlv_dump setting for auto wb like the one in cr2hdr? Actually a good starting point.
one suggestion for the MLV lite submenu,
MLV sound and H264 proxy should never be used together.
Because it results in static noise in the audio.
So how can we prevent this from happing?
Perhaps auto disable mlv sound when h264 is enabled?
or put them together in a sub menu allowing only one active at the time:
MLV sound or h264 proxy
One more related to menu's
I can't seem to add the individual items from the advanced submenu under Raw recording to My Menu.
More people working on wb:
https://github.com/Beep6581/RawTherapee/issues/4650
i am still confused how an algorithm could reliably detect the white balance, not knowing
if the image contains a blue-ish picture on a ivory colored wall, or a window on a bright white wall where you can see the blue sky through :)
Hehe, it probably can't. Question. Cr2 metadata. Is that all gathered after a picture is taken? Is that also the case with mov recordings? I mean, the wb state in liveview will record mov files with awb as seen I suppose so where is that info pending before pressing record button? Aewb task as a1ex says?
Quote from: Danne on July 17, 2018, 04:10:33 PM
Hehe, it probably can't. Question. Cr2 metadata. Is that all gathered after a picture is taken? Is that also the case with mov recordings? I mean, the wb state in liveview will record mov files with awb as seen I suppose so where is that info pending before pressing record button? Aewb task as a1ex says?
i remember that i somewhen looked into this code, but hardly can remember anything.
had some stuff in the register map on wikia, but this was just the engine registers that calculated WB data.
Sorry I've been away for a bit. I installed the crop rec experimental build and compared to the nightly for 5d 1.13. Has it been integrated into the main 1.13 branch? I no longer see the larger shoot options or a crop rec module.
nvm found it under experiments further down.
Quote from: a1ex on July 08, 2018, 11:40:48 PM
Still looking for addresses of ADTG 805F and 8061 for 650D and 100D. Currently, crop_rec doesn't load on these models.
Just following up on this. Seems simple enough. nikfreak found it them on the 100D (https://www.magiclantern.fm/forum/index.php?topic=19300.msg203878#msg203878). Anyone need some help getting these addresses for the 650D?
Was occupied with other things.
Loaded your ISO-research build for 1.0.5 and no problem loading crop_rec.mo + adtg_gui.mo. Sooo ... what to do?
Two new builds, one for eosm and the other for eos 100D:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/magiclantern-Nightly.2018Jul25.100D101_2K-3K-4K_10bit_patch_g3gg0.zip
https://bitbucket.org/Dannephoto/magic-lantern/downloads/magiclantern-Nightly.2018Jul25.EOSM202_2K-3K-4K_10bit_patch_g3gg0.zip
100D crop_rec menu:
"2.5K 2520x1304", only 14bit lossless
"3K 3096x1320", only 14bit lossless
"4K 4056x2552", only 14bit lossless
"3x3 720p", (problematic, probably not working....)
"2K 2520x1080p", 10/12/14bit lossless (Regular x5 mode records in 2520x1078)
"2.5K 10bit 2520x1304", 14bit lossless but will record at 10bit bitrate Thanks to A1ex, Bilal and Levas
eosm crop_rec menu:
"2.5K 2520x1304", only 14bit lossless
"3K 3072x1304", only 14bit lossless
"4K 4038x2558", only 14bit lossless
"3x3 720p",
"3x3_mv1080_EOSM", only 14bit lossless this version will only do 1736x696(don´t know how to do crop_rec exceptions in raw.c code)
"2.5K 10bit 2520x1304", 14bit lossless but will record at 10bit bitrate Thanks to A1ex, Bilal and Levas
Modified a bit in g3gg0 crop_rec_4k_mlv_snd branch:
https://bitbucket.org/Dannephoto/magic-lantern/branch/crop_rec_4k_mlv_snd_lossless_croprec_iso_reg
Crop_rec code from here:
https://bitbucket.org/Dannephoto/magic-lantern/commits/c1588d0719e6bf383964d66fca95c3980f843291?at=crop_rec_4k_mlv_lite_snd_sd_uhs_HDR_croprec#chg-modules/crop_rec/crop_rec.c
Did not have time to test this too much, hopefully working. Please report back how it works or not.
@a1ex
Can you disable crop_rec in x10 magnification by default to use this mode for focusing in higher wider resolutions? and re-enable it automatically in x5 ?
It would be nice trick for making focusing easier.
That's how it's supposed to work, but detecting video mode switches is a bit tricky. The PathDrive stuff works well in this case (x1 -> x5 -> x10), but has some regressions (there are other video mode switches where the current code works better). Still looking into it.
Quote from: Walter Schulz on July 25, 2018, 08:57:44 AM
Was occupied with other things.
Loaded your ISO-research build for 1.0.5 and no problem loading crop_rec.mo + adtg_gui.mo. Sooo ... what to do?
I was also occupied with many other things. Your mission, should you choose to accept it (https://en.wikiquote.org/wiki/Mission:_Impossible), is:
Quote from: a1ex on July 08, 2018, 11:40:48 PM
Still looking for addresses of ADTG 805F and 8061 for 650D and 100D. Currently, crop_rec doesn't load on these models.
So using the adtg_gui module see if you can find the values for those ADTG registers. This is what it looks like on the EOSM:
(https://farm1.staticflickr.com/918/42375429425_3761576059_n.jpg) (https://flic.kr/p/27yz9Qa)
@nikfreak found them on the 100D (https://www.magiclantern.fm/forum/index.php?topic=19300.msg203878#msg203878). Note that you might have to switch settings around before they show up.
I had some problems with the latest Experimental build from 22 July with my 5D MKIII that I'd like to share...
(https://thumb.ibb.co/ngLLM8/M27_1630_000112.jpg) (https://ibb.co/ngLLM8)
Take a look at the pink stain on the right side.
Never experienced this before in 4 years of ML RAW use and more than 40 videos recorded, looked like a random thing occurence, could not reproduce at will.
(I can't verify right now but thinking about it, looks like it only happens with slow motion footage)
Another thing I had was camera freeze after reaching 10x zoom. It blacks out and you have to turn off and on again the camera. This problem is a little annoying when using a gimbal.
I don't know how or where to send the 3 crash reports I have for you developers, please let me know.
Hello guys!
I can confirm the Sganzerla feedback about the blackouts of the 22 july build for 5D MK III.
In my case I use firmware version 1.2.3.
Have experienced the black outs using:
- 3072x1308 2.35:1 24 fps
- 1080 x 818 2.35:1 50 fps
My workaround is to push the MENU button, wait for see the original Canon menus and then immediately push MENU button again and back to Magic Lantern GUI.
No need for shut down or eject the battery.
Luckily is fast enough but please check the "black outs", You don't see a full black screen but a black screen with some color vertical lines that tend to vanish if you wait few seconds becaming all black.
Is like to see the matrix of the sensor..
Anyway the shootings in those resolution are still super freaking awsome!!!!!!!!!
I confirm also, same problem, 5d3 - 123 last exp 22. :(
3072x1320 5DIII
Scaled down to 2560x1440 for Youtube.
https://www.youtube.com/watch?v=U-TZ4MkW3dI&t=28s
I had the same blackouts as Mikv on 1.2.3 with 22 July build for 5D3 on the 3k crop mode.
And I managed to kind of replicate Sganzerla's problem, it happens or it worsens when you go out of the zoom.
Steps to replicate pink pixels/stripes: 22 July build, 5d3, use crop mode 3k or 45/48fps, select 1920x1080 resolution, keep camera cap on, so you get a black screen. Zoom x5, x10, zoom again, you are gonna get pink stripes and other glitches, even on preview.
Steps to replicate blackout/sudden shutdown: 22 July build, 5d3, use crop mode 3k, select 3072x1308 24 fps 2.35:1, zoom 5x, 10x, when you zoom again to reset, shuts down camera.
Crash log on card:
ASSERT: hLvJob->hJpegMemSuite
at ./Epp/Vram/VramStage.c:891, Epp:ff1859cc
lv:1 mode:3
Epp stack: 17dbd0 [17dca8-17d0a8]
0xUNKNOWN @ de48:17dca0
0xUNKNOWN @ 17bbc:17dc78
0x000178B4 @ ff0de67c:17dc58
0xUNKNOWN @ 178e4:17dc48
0xUNKNOWN @ 1796c:17dc28
0x00001900 @ ff1859c8:17dc08
0x00069878 @ 69c1c:17dbd0
Magic Lantern version : crop_rec_4k.2018Jul22.5D3123
Mercurial changeset : c1e44b8e0183 (crop_rec_4k_mlv_snd) tip
Built on 2018-07-22 13:10:52 UTC by jenkins@nightly.
Free Memory : 219K + 3089K
@a1ex
I like Movie crop mode feature in 700D to extend the focal length. Can you enable x10 mode to be used for better focusing when pressing magnification button ?
Quote from: jimiz on July 31, 2018, 07:16:48 PM
I confirm also, same problem, 5d3 - 123 last exp 22. :(
Same Here ::) 5D3 Looking for some love...
Hello, I tried latest developments regarding 3.5K. Latest 22July, 4July and 20June. Had most success with the latest - 22nd July. Using 5dmkIII 113.
This topic is hard to understand or read all the pages. On one hand development is of course the most valuable, but it is a pitty that there is so few feedbacks from testers. Infos on first page and on experimental DL page also did not give me answers. So please answer these questions that I have:
- Which option is used when and what are the differences between them? (options in question are: 1920 1:1 tall/3K 1:1/UHD 1:1/3.5K 1:1 centered x5)
- How can you go above 3584x___?
- If I want to shoot biggest resolution to get smallest crop of sensor as close to continous what settings I should use?
- Why are there limited options of shutter speeds when using crop_rec especially when using lossless compression and how can I get normal options?
- Why I can shoot normally in 3.5K 1:1 in 14 bit lossless, if I choose 12 or 11...8 bit lossless I get underexposed and magentish colored frames? (4 bit lossless works even with 24 FPS override - all perfect frames!)
- Why I get red circle in "memory patches - code: FF28CC3C EA397952 crop_rec"?
Normal versions of HD RAW recording dont work, have corrupted frames. I get that, its bleeding edge.
I recorded successfully in 3520x1320 1,63 crop factor, which is almost continous. If get a bit lower I am on safe side. Crop sensor has much more rolling shutter, so you must have either tripod, video head with slow pan, sliders, IS lenses, gimbals and so on. It is impossible to shoot handheld with manual lens even if using wide lens. But the images are stunning and it is a pleasure to grade them.
I normally use "magiclantern-crop3x.2017Mar21.5D3113" and in that version everything works perfectly stable. Option that I like there is "card spanning", I can get continous 3200x1200 10 bit uncompressed and I can use raw2cdng which is just the best simple converter, then I use resolve and so on (other topic).
Maybe it wouldnt be stupid to actually open specific topic regarding testing crop_rec and maybe even have specific topics for all the models? There is for normal recording, but that is way behind what experimetal builds offer today.
Big THANK YOU to all the developers.
Quote from: rebilith on August 10, 2018, 07:24:08 PM
Steps to replicate pink pixels/stripes: 22 July build, 5d3, use crop mode 3k or 45/48fps, select 1920x1080 resolution, keep camera cap on, so you get a black screen. Zoom x5, x10, zoom again, you are gonna get pink stripes and other glitches, even on preview.
Confirmed.
Quote from: rebilith on August 10, 2018, 07:24:08 PM
Steps to replicate blackout/sudden shutdown: 22 July build, 5d3, use crop mode 3k, select 3072x1308 24 fps 2.35:1, zoom 5x, 10x, when you zoom again to reset, shuts down camera.
Could not reproduce.
Quote from: RÁTNEEK on August 13, 2018, 09:55:18 PM
- Why are there limited options of shutter speeds when using crop_rec especially when using lossless compression and how can I get normal options?
Probably a bug; what limits did you find and how to reproduce?
There was a bug affecting shutter speeds in the centered x5, but it was solved on June 30. Tested again the latest build, just in case, and the fix appears to be working.
Quote
- Why I can shoot normally in 3.5K 1:1 in 14 bit lossless, if I choose 12 or 11...8 bit lossless I get underexposed and magentish colored frames? (it works even with 24 FPS override - perfect frames!)
Sounds like a postprocessing issue - incorrect white level used by your raw converter. If in doubt, you may upload a sample DNG or a very short MLV.
QuoteNormal versions of HD RAW recording dont work, have corrupted frames. I get that, its bleeding edge.
Steps to reproduce?
Just to make sure I understand well: you are able to record 3K without corrupted frames, but you have issues with regular 1080p?!
The other questions are probably best answered by those who actually record video (I don't).
hey guys, in the first post there is tutorial showing how to record continuously at 3072x1308 2:35:1 12 Bit, however in the last builds there is no longer that resolution. It now shows ...2240,2560,2880,3200,3520, what's the one to be used for 3K continuously with sound?
On a side note, when I try to enable the mlv_play.mo and already have the crop_rec, mlv_rec and mlv_sound ML crashes at the start. (crop_rec_4k.2018Feb24) "error: undefined symbol 'lossless_decompress_raw" E. failed to link modules.
thank you.
Quote from: sys73r on August 15, 2018, 01:28:28 AM
hey guys, in the first post there is tutorial showing how to record continuously at 3072x1308 2:35:1 12 Bit, however in the last builds there is no longer that resolution. It now shows ...2240,2560,2880,3200,3520, what's the one to be used for 3K continuously with sound?
You can adapt the preset resolutions in small increments with the scroll wheel (not sure which one on 5dIII, but the one where you can adjust aperture or shutter time with, in manual mode)
Go into Raw video menu and highlight on the resolution and use scroll wheel to adjust.
Took me 3 years or so to find out this option exists in Magic Lantern Raw video menu :P
Alex, thank you for your answers.
I had problems with shutter speeds due to FPS override. FPS override works OK when not using crop mode, regardless of bit rate/lossless.
Colors and exposure were off at lover lossless bitrates because MLVProducer and mlv_play wont show them correctly. Mlv_play sometimes showed totally corrupted frames, but mlv_dump gave clean dng frames. So, it is really cruicial to use latest mlv_dump from experiments DL, at least for now.
For normal HD I retested and now everything seems OK.
There still is a few "bugs". When applying FPS override and turning it off, frame rate goes either to 30.125 or 19.sth, it can be solved by going to Canon menu and back. The other is when going to slowmotion crop mode, and applying FPS camera freezes. It is solved by taking battery out and restarting camera.
I will test more and post my findings.
Quote from: sys73r on August 15, 2018, 01:28:28 AM
hey guys, in the first post there is tutorial showing how to record continuously at 3072x1308 2:35:1 12 Bit, however in the last builds there is no longer that resolution. It now shows ...2240,2560,2880,3200,3520, what's the one to be used for 3K continuously with sound?
On a side note, when I try to enable the mlv_play.mo and already have the crop_rec, mlv_rec and mlv_sound ML crashes at the start. (crop_rec_4k.2018Feb24) "error: undefined symbol 'lossless_decompress_raw" E. failed to link modules.
I use "magiclantern-crop_rec_4k.2018Jul22.5D3113", so it could be different with your version.
You have to load mlv_lite with crop_rec, otherwise modules wont load. I never use sound due to external sound recording, but use FPS override all the time, cause exact 24fps is more managable especially for DCP. So for mlv_snd I dont know.
With recent crop_rec you can not use FPS override.
For "3K 3072x1308 2:35:1 12 Bit continous" I set it this way:
Under "Movie"
- enable crop mode, select 3K
- in raw tab select 2.35:1 and in resolution select 3072 and it goes automatically to 3072x1308, then is also select 14bit lossless, it wont go to 12bit lossless, to get continous (which also depends of ISO and actual scene you are filming).
Another bug: If I set those settings and restart the camera, I have black frame only. I have to set crop mode to OFF and select 3K again, then it works.
Some notes for 700D. Will edit as I'll discover more stuff.
1080p25:
Timer A=640, B=2000
Timer A must keep the same parity (odd value written to register)
Timer A min = 0x207 raw => 520 (same as 100D)
520*4 - 1808 = 272
1080p25 crop:
Mostly the same as above.
Timer A min = 0x217 raw => 536 (100D: 540)
536*4 - 1872 = 272
1080p50:
Timer A=640, B=1000
Timer A min = 0x207 raw => 520 (same as 100D). No change with line skipping factor, unlike 5D3.
520*4 - 1808 = 272
x5:
Timer A = 716, B = 1492
Timer A min = 0x2cb raw => 716 (tight fit in Canon firmware?! very unusual; 100D has 732 default and 724 min)
716*4 - 2592 = 272.
Photo mode:
timer A = timer A min = 0x56b => 1388.
1388*4 - 5280 = 272.
Cross-check with other parameters from raw_res.txt (https://a1ex.magiclantern.fm/bleeding-edge/raw/raw_res.txt).
The above numbers give the timer A requirements for some desired horizontal resolution. That is:
timer A * 4 - horizontal resolution (including black bars) must be at least 272 in all of the above video modes.
The 5D3 does not show this kind of consistency, figure out why.
call("lv_120fps_test") -> does nothing.
GetPathDriveInfo mode map (codenames):
0 = LVx1
1 = LVx1_60fps
2 = LVx5
3 = LVx10
4 = LV_AF_Crop_120fps
5 = LV_FlashExposure
6 = Rec_HD
7 = Rec_720p
8 = Rec_Vga
9 = Rec_DZ_Crop_HD
10 = RecStandby_x1
11 = RecStandby_x1_60fps
12 = RecStandby_x1 (again?!)
13 = LVx1_60fps
14 = LVx1_CreativeFilter
15 = LvServoGaia
16 = LvServoDiana
Funky effects:
call("lv_cfilter_mode", i); // i from 0 to 7
0 = none, 1 = bw, 2 = lighter, 3 = oldstyle, 4 = darker, 5 = fisheye, 6 = saturated/noisy/low-fps, 7 = washed out.
CMOS[7] is vertical position (start and stop scanning line, 6 bits each), equivalent of CMOS[1] on 5D3.
CMOS[5] is horizontal position (mask FC0, max=0xB00) and column binning factor (0x20).
Canon values: 0x380 in 3x crop (active area xres: 1800), 0/0x300/0x600 in x5 zoom, 0x20 in 1080p.
Value used with CROP_PRESET_CENTER_Z_700: 0x300 (active area xres 2520).
This appears to be one-size-fits-all for CMOS[5] and CMOS[7] on 700D. Likely portable with minimal changes.
if (is_700D)
{
/* horizontal and vetical centering, in rough increments */
int xres = get_preset_xres(crop_preset);
int yres = get_preset_yres(crop_preset);
/* the sensor is divided into slices, 120x120 pixels each */
const int slice_size = 120;
/* 700D:
* xpos=0 => capture from the left side
* xpos=44 => black image, no valid data
* sensor xres = 5208; last slice is incomplete
*/
const int x_slices = (sensor_xres + slice_size - 1) / slice_size;
const int y_slices = (sensor_yres + slice_size - 1) / slice_size;
const int extended_xres = x_slices * slice_size;
const int extended_yres = y_slices * slice_size;
/* most presets read out every column;
* a few of them will bin every 3 columns */
int col_binning = 1;
switch (crop_preset)
{
case CROP_PRESET_3x3_1X:
case CROP_PRESET_3x3_1X_48p:
case CROP_PRESET_1x3:
col_binning = 3;
break;
}
/* 700D:
* CMOS[5]: horizontal position mask 0xFC0, column binning 0x020
* 1736: about 14.5 would be ideal, formula gives 14.46 before rounding
* 1800: 14 looks about right, formula gives 14.2 before rounding
* 2520: 12 is Canon default, formula gives 11 (11.2 before rounding)
*/
int xpos = ((sensor_xres/col_binning - xres) * x_slices + extended_xres/col_binning)
/ (2 * extended_xres/col_binning);
int col_binning_flag = (col_binning == 1) ? 0 : 0x20;
cmos_new[5] = PACK12(col_binning_flag, xpos);
/* vertical position in rough increments */
/* CMOS[7]: start/stop scanline, 5 bits each, 120px increments, highest 2 bits unknown */
/* most presets read out every row;
* a few of them will "bin" every 3 rows (actually read one line and skip 2) */
int row_binning = 1;
switch (crop_preset)
{
case CROP_PRESET_3x3_1X:
case CROP_PRESET_3x3_1X_48p:
case CROP_PRESET_3x1:
row_binning = 3;
break;
}
int ystart = ((sensor_yres/row_binning - yres) * y_slices)
/ (2 * extended_yres/row_binning);
int ystop = ((sensor_yres/row_binning + yres) * y_slices)
/ (2 * extended_yres/row_binning);
cmos_new[7] = PACK552(ystart, ystop-1, 0);
dbg_printf("sensor res: %dx%d\n", sensor_xres, sensor_yres);
dbg_printf("capture res: %dx%d\n", xres, yres);
dbg_printf("slices: %dx%d bin %dx%d\n", x_slices, y_slices, col_binning, row_binning);
dbg_printf("x:%d y:%d-%d\n", xpos, ystart, ystop);
dbg_printf("cmos [5]:%x [7]:%x\n", cmos_new[5], cmos_new[7]);
}
I wrote a few pages ago about this theme but could not find any way to solve this question for my needs.
If you put aspect ratio at 1:1,3:2 or 4:3 in mlv_rec, it gives real time preview with croped borders when using at 24fps mode. Everything is perfect.
If you do the same with mlv_lite than it gives the low fps and lower quality preview that we see as if using crop_rec with slow motion. Even when recording at 24fps.
Isn't there really any way to make mlv_lite work as mlv_rec with realtime good image quality preview for different aspect ratios other than 16:9?
After fiddling a bit with a 700D, I've got a better understanding of how resolution overrides should work, and I'm getting closer to reworking the crop_rec code to allow arbitrary resolutions (rather than preset-based). Most of the ADTG register overrides are already generic (used on all models), CMOS registers for H/V centering were almost figured out (see previous post) and FPS timer requirements can be figured out by trial and error (already done for 700D).
Now I'd welcome some suggestions for how the menu should look like. Here's one starting point:
- Column binning factor: 1 or 3
- Row binning/skipping factor: 1 or 3 (hardware allows other values, but don't see why anyone would use them)
- X resolution: from Canon default (1932, 1736, 3520 etc) to full width (5796, 5208 etc)
- Y resolution: from Canon default (1290, 1160, 672 etc) to full height (3870, 3478 etc)
- Frame rate: value from Canon menu / half of that value / max allowed by current resolution / custom?
- Shutter speed: same as in current builds
- custom settings for each video mode selectable from Canon menu?
@Sganzerla
You should look into the options in raw video menu.
There is something called 'Preview' and it has 4 options if I'm right (not the camera with me now)
By default this 'Preview' option is set to some 'auto' setting, which selects indeed the 'ML preview' on 5x zoom mode.
Now change this value to 'real-time', and you get real-time preview, even when using 5x zoom.
@Alex
Sounds like crop_rec is going to become mature :D
How do you want to implement the selectable resolution, since resolution is also set in MLV_lite raw video menu ?
Not sure if this is possible, but would be nice if these options can be put within the already available raw video menu (MLV_Lite).
@Levas
Thank you very much, it solved the situation! I was really unaware of this option.
Using 'framing' option or 'realtime' it gives me the same frame in 24p Full HD setting, 'auto' option could very well relate to 'real-time' instead of 'framing' in this situation.
Alright, please find the first proof of concept for crop_rec with arbitrary resolutions. 700D only; won't work on any other model yet.
It's too much of a hack to commit it in this state, but I have to return the 700D today, so... that's my current snapshot. Please give it a try and write down what works right now, as things will very likely break after I'll integrate the other models, and I won't be able to notice.
crop_rec.c (https://a1ex.magiclantern.fm/bleeding-edge/700D/crop_rec.c)
Beware: might be very buggy. After changing settings, press MENU twice to refresh. Have fun.
I'm having some troubles with mlv_dump and my MLV files on OS X High Sierra. My videos play fine on mlv_play, but while trying to unpack the MLV to DNGs with mlv_dump, I get a segfault:
XREF table contains 20 entries
Processing...
File Header (MLVI)
Size : 0x00000034
Ver : v2.0
GUID : 9216039327435021128
FPS : 2.501000
File : 0 / 0
Frames Video: 4
Frames Audio: 0
Class Video : 0x00000021
Class Audio : 0x00000000
Block: RAWI
Offset: 0x00000034
Number: 1
Size: 180
Time: 0.754000 ms
Res: 4096x3072
raw_info:
api_version 0x00000001
height 3102
width 4248
pitch 7434
frame_size 0x015FDF2C
bits_per_pixel 14
black_level 2047
white_level 16200
active_area.y1 30
active_area.x1 146
active_area.y2 3102
active_area.x2 4248
exposure_bias 0, 0
cfa_pattern 0x02010100
calibration_ill 1
Block: RAWC
Offset: 0x000000e8
Number: 2
Size: 32
Time: 0.761000 ms
raw_capture_info:
sensor res 5760x3840
sensor crop 1.00 (Full frame)
sampling 1x1 (read every line, read every column)
Block: IDNT
Offset: 0x00000108
Number: 3
Size: 84
Time: 0.767000 ms
Camera Name: 'Canon EOS 5D Mark III'
Camera Serial: '4E58BA31B' (21031002907)
Camera Model: 0x80000285
Block: EXPO
Offset: 0x0000015c
Number: 4
Size: 40
Time: 0.773000 ms
ISO Mode: 0
ISO: 1600
ISO Analog: 104
ISO DGain: 0/1024 EV
Shutter: 19016 microseconds (1/52.59)
Block: LENS
Offset: 0x00000184
Number: 5
Size: 96
Time: 0.794000 ms
Name: '35-35mm'
Serial: '000000000' (no valid S/N)
Focal Len: 35 mm
Focus Dist: 0 mm
Aperture: f/2.80
IS Mode: 0
AF Mode: 3
Lens ID: 0x00000028
Flags: 0x00000000
Block: RTCI
Offset: 0x000001e4
Number: 6
Size: 44
Time: 0.810000 ms
Date: 30.08.2018
Time: 05:16:09 (GMT+0)
Zone: ''
Day of week: 4
Day of year: 241
Daylight s.: 0
Block: WBAL
Offset: 0x00000210
Number: 7
Size: 44
Time: 1.446000 ms
Mode: 1
Kelvin: 5200
Gain R: 491
Gain G: 1024
Gain B: 638
Shift GM: 0
Shift BA: 0
Block: VERS
Offset: 0x0000023c
Number: 8
Size: 152
Time: 599.243000 ms
String: 'crop_rec built 2018-07-09 21:48:22 UTC; commit c76ef16 on 2018-07-04 12:58:01 UTC by alex: crop_rec: fix case fallthrough warnings '
Block: VERS
Offset: 0x000002d4
Number: 9
Size: 148
Time: 774.878000 ms
String: 'ettr built 2018-02-17 21:49:58 UTC; commit 5aa7ec6 on 2018-02-16 23:44:52 UTC by alex: Merged 70D_merge_fw112 into crop_rec_4k '
Block: VERS
Offset: 0x00000368
Number: 10
Size: 176
Time: 809.217000 ms
String: 'file_man built 2018-02-17 21:49:56 UTC; commit bcbabb3 on 2018-02-17 20:28:28 UTC by alex: Moved timer functions to timer.h (always included from dryos.h) '
Block: VERS
Offset: 0x00000418
Number: 11
Size: 180
Time: 999.310000 ms
String: 'mlv_lite built 2018-07-03 23:13:14 UTC; commit f4213dd on 2018-07-03 23:10:22 UTC by g3gg0: mlv_lite: do not write all metadata again on every single chunk '
Block: VERS
Offset: 0x000004cc
Number: 12
Size: 164
Time: 1038.454000 ms
String: 'mlv_play built 2018-06-20 20:32:09 UTC; commit 0e38b89 on 2018-06-18 22:11:42 UTC by g3gg0: mlv_play: use less RAM, proper cleanup on error '
Block: VERS
Offset: 0x00000570
Number: 13
Size: 160
Time: 1223.783000 ms
String: 'mlv_rec built 2018-07-22 13:10:25 UTC; commit c1e44b8 on 2018-07-21 21:31:09 UTC by g3gg0: mlv_dump: pass INFO blocks as image description '
Block: VERS
Offset: 0x00000610
Number: 14
Size: 160
Time: 1296.649000 ms
String: 'pic_view built 2017-08-19 10:52:04 UTC; commit fca6399 on 2017-08-19 06:39:25 UTC by alex: pic_view: lossless DNG support (experimental) '
Block: VERS
Offset: 0x000006b0
Number: 15
Size: 148
Time: 1331.059000 ms
String: 'silent built 2018-07-09 21:48:10 UTC; commit 3d26697 on 2018-06-30 16:48:59 UTC by alex: lossless.c: fix warning in comment '
Block: VIDF
Offset: 0x00000800
Number: 16
Size: 13493760
Time: 2310.348000 ms
Frame: #0000
Crop: 152x30
Pan: 152x30
Space: 32
LJ92: Decompressing
LJ92: 2048x3072x2 14 bpp (25165824 bytes buffer)
LJ92: 22020096 -> 22020096 (100.00% ratio)
Thread 2 received signal SIGSEGV, Segmentation fault.
dng_unpack_image_bits (input_buffer=0x10a2d5000, output_buffer=0x10c000000, max_size=<optimized out>,
bpp=14) at dng/dng.c:739
739 uint32_t uncorrected_data = *((uint32_t *)&packed_bits[bits_address]);The faulting video (54MB): https://s3.sherlockphotography.org/posts/2018/M30-0516.MLV
The video is 4096x3072 recorded at 2.5 FPS on the 5D III.
If I build mlv_dump using "USE_X64=1", some of my videos will decode without crashing, but some still crash, including this one. Running under Valgrind, I get:
==17989== Invalid read of size 4
==17989== at 0x10000FE1F: dng_init_data (dng.c:739)
==17989== by 0x1000084C6: main (mlv_dump.c:3486)
==17989== Address 0x10bc1003e is 22,020,094 bytes inside a block of size 22,020,096 alloc'd
==17989== at 0x1007243E4: realloc (vg_replace_malloc.c:788)
==17989== by 0x1000061F8: main (mlv_dump.c:2917)
==17989==
==17989== Invalid write of size 4
==17989== at 0x100010313: dng_save (dng.c:772)
==17989== by 0x10000932D: main (mlv_dump.c:3632)
==17989== Address 0x10d11103e is 22,020,094 bytes inside a block of size 22,020,096 alloc'd
==17989== at 0x100723A36: malloc (vg_replace_malloc.c:302)
==17989== by 0x10000FD75: dng_init_data (dng.c:823)
==17989== by 0x1000084C6: main (mlv_dump.c:3486)
==17989==It looks like dng_pack_image_bits is writing 2 bytes beyond the end of its buffer, because it writes 4 bytes at a time and that write can begin 2 bytes before the end of the buffer. I've submitted a PR to fix that by adding 2 bytes of padding to the allocated buffer size:
https://bitbucket.org/hudson/magic-lantern/pull-requests/944/mlv_dump-fix-segfault-while-writing-dngs/diff
dng_unpack_image_bits has a problem with invalid reads too, but I'm not sure of the best way to fix that one.
EDIT: The culmination of this morning's shooting with crop_rec, thanks for the great software!:
I have a 5d3 to play around with for the next couple days.
Can't seem to figure out crop rec, I'm interested in being able to record UHD.
Can somebody please tell me their working settings to get continuous'ish high res 24fps raw video?
Cheers
Quote from: tonij on August 31, 2018, 12:22:42 PM
I have a 5d3 to play around with for the next couple days.
Can't seem to figure out crop rec, I'm interested in being able to record UHD.
Can somebody please tell me their working settings to get continuous'ish high res 24fps raw video?
Cheers
First look at first post of this topic, there is a good and relevant video tutorial. And read text also. It is recomended that you read lets say last 10 pages also.
UHD is not continous. You will probably get something around 3520x1320 on 11...8bit lossless. That is under 3.5K in crop_rec menu.
If you will set FPS override, you will experience altered shutter speeds in crop mode, but you can fine tune even that, it works.
Your question is probably not being answered because this is experiments development page, but I can totally get you, I spent a few days just browsing through forum and didnt get it completely, but then after testing it for another few days... Latest developments here are even hard to follow and test, so huge respect to all of the people here, who actually move this developments forward.
It takes alot of time but it is defenitely worth it.
Now:
3072x1304 2.35:1 @ 23.976 & 2520x1440 @ 23.976 Is just working fine in 700D without any corrupted frames I didn't know why I got frames corruption before. Here the new settings in the spreadsheet (https://docs.google.com/spreadsheets/d/1TzPxh7ai5JAoEI8zllaiiF1IDF5tAQam2BKRujyIYQU/edit#gid=0) and crop_rec (https://drive.google.com/open?id=1ZXcqPUu-d6i1vpR8j869e7ghVqovjkg8)module for testing.
I didn't go for 2560 instead of 2520 because the LiveView will be scrambled for only 40 pixels not a big deal, isn't it? Maybe if we could solve the distorted LiveView mystery the 2560 will be a great thing. 2520x1420 is True 16:9 in this case (in Camera 2520x1418 I don't know why but you can go up to 1440).
Welcome back Bilal!
@Bilal,
Have you managed to test the source from reply #1824 on 700D?
Mind highlighting the differences between current and previous configuration?
@Bilal
If the 700d works about the same as the 6d, I'll think you can get even more vertical resolution in 2520 mode.
6d needs the B timer(6014) to be about 4F(hexadecimal) higher then the vertical resolution in 6804.
Can you try what happens if you make a small change in the 2520x1440 crop ?
Change the following in your 2520x1440 preset:
c0f06804 from 5BC0298 to value 6F80298
c0f0713c from 635 to value 6F9
This would give a vertical resolution of ~1753, only problem now is, you need to line out CMOS 7 value for this vertical resolution, to get normal frames again.
You can do this from within the crop_rec menu, use q button to enter crop_rec menu and change delta_head 4 timer.
Change delta head 4 timer 1 increment up and check your liveview/preview, if not correct, do another increment, check one increment at the time and see if your liveview/preview is back to normal again.
When liveview/preview is normal again, check the value in the menu you have for delta head 4.
Now to fix the crop_rec preset, you can change the value for CMOS 7 by adding the value of delta head 4.
Remember CMOS 7 value is hexadecimal and delta head 4 value is decimal.
So if you need delta head 4 value of 10. You can fix CMOS 7 by 0xaa9 + 10(decimal) = 0xab3
Curious if this works.
If this works, it means you probably also can get 3072 x 1440 resolution.
Thanks @Danne :D
@a1ex
Everything is working Just fine you are genius! Wow the new source code is a awesome and the way you can change the resolutions and mods is Perfect! I didn't notice any problem with all mods and there are no corrupted frames at all the only thing is you have to refresh LiveView manually as you mentioned.
The centering is almost if not PERFECT I will do more tests for this one (I noticed the centering change in x10 compared to other mods but not a big deal) .
Something is a bit annoying when pressing x5 mode the FPS stays 30fps I want to keep 2520 for not destroying LiveView and I want to increase vertical res I can get only 2520x1230 in 30fps and there is no way to decrease FPS to 23.976 for getting more vertical the solution is to exit x5 back to 1080p24 now the FPS is 23.976 with increased height up to 2520x1604 but the LiveView is scrambled (In this case I have to change X resolution to 2520).
It should be an option to set 23.976 FPS in x5 mode.
I tried also the max resolution in 3K (X res to 3072 Y res to 2000) I got 3072x1302 the same as what I got in adtg_gui using trial and error method. The only thing is missing is the higher SD write speeds and the lower bit-depths.
Alex.. You have unlocked the Canon cameras like a Monster! merciless! :D
@Levas
We don't have to override each register manually anymore! The new way is by Entering the X and Y Resolution you want and crop_rec will do the calculations for each register to get that resolution and if the resolution and FPS reach the limits it will not increase anymore and you will be in the safe frames area!
I can get with above way up to 2520x1604 @ 23.976 :D and the Max resolution in x5 16:9 is 2700x1500 @ 23.976.
I forgot about the experimental crop_rec Alex made for the 700d.
Have you tested max framerate in 720p mode ?
I got best results in 5x zoom mode in 720p50 canon mode.
In 1080p mode the 6d softbricks ad about 38 fps.
Tell us, can you get 75 fps, 100? :D
No luck with high FPS on 700D from my side, but didn't try too hard. Long answer (https://www.magiclantern.fm/forum/index.php?topic=19336.msg205641#msg205641).
At least I'm starting to know where the limits come from.
Nailed it, 96fps, clean raw stream on the 6d 8)
It's done in 3x5 readout (Not in 5x zoom mode but in full view), resolution at 96 fps is 1440 x 360 -> which after correction for the squeeze factor, results in 1440 x 600 resolution.
Got it working by lowering some image registers, only the first will do. Added the other three registers later on for trying to lower A timer, but that didn't work :-\
case 0xC0F08184: return 0x1;
case 0xC0F08188: return 0x1;
case 0xC0F08194: return 0x1;
case 0xC0F08198: return 0x1;
Two examples I shot today, 1440x360 base resolution at 96 fps, upscaled to 1920 x 800 on a 25 fps timeline:
I see the youtube compression does horrible things, better download the files for better view :P Probably because 360p is only available at start, now 1080p available.
https://drive.google.com/open?id=1gbBmBiJ59Z75UeKc8tuTKb05a3gcwKfZ (https://drive.google.com/open?id=1gbBmBiJ59Z75UeKc8tuTKb05a3gcwKfZ)
https://drive.google.com/open?id=1qHHEr9C2PPEhHcB20VbUbDuBbOCJa2K4 (https://drive.google.com/open?id=1qHHEr9C2PPEhHcB20VbUbDuBbOCJa2K4)
Here are 2 framegrabs from DaVinci Resolve
(https://farm2.staticflickr.com/1845/30822588048_2fca1b8e61_o.jpg)
(https://farm2.staticflickr.com/1884/29755547377_d056d0a6ed_o.jpg)
Awesome!
Mind sharing the full register configuration, or source code?
If you weren't able to lower timer A, I'd expect the horizontal resolution to be full width (1824 iirc).
You may also use other line skipping factors on the Y direction: 3, 5, 7, 9...
Have to go to work now, will send source later.
:o
@Levas. Amazing, amaaaazing!!
Getting 5D3 video quality (No alisaing!) in 700D :D
Let's start with some samples: (There are no video processing or scripting in all samples 1736x736 2.35:1 @23.976 )
(https://preview.ibb.co/eyTBXK/Before.png) (https://ibb.co/geniez)
(https://preview.ibb.co/mYwC6e/After.png) (https://ibb.co/bsVGXK)
Video:
Link (https://youtu.be/9E4H1I3CObs)
Real life video samples: (I added sharpness more than usual for YT compression)
Link (https://youtu.be/Z3f2llcJYHI)
This another and
better example 1280x720 (Actually 1280x2160 in 1x3) @ 23.976 for better recording times:
Original file: Download (https://drive.google.com/open?id=1ytJ4jJBJOqTsSpquS7f2TQDRTdTNLynm)
A close up look:
(https://preview.ibb.co/cjeS6e/Building.jpg) (https://ibb.co/kqm76e)
(https://preview.ibb.co/kTjume/Notebook.jpg) (https://ibb.co/n0KS6e)
Details:1x3 Sampling (Binning/Line skipping whatever) it was only for 5D3 I tried to get it using adtg_gui about three months ago and it worked and it was perfect with no aliasing like the samples above the problem was too wide aspect ratio 1736x386 (actually 1736x1160 but you have to stretch it to get the correct aspect ratio) I didn't realize I can increase the vertical resolution so I can get more vertical I was thinking 1x3 like 3x3 nothing to do with resolution..
Until a1ex made the new way for using crop_rec you can
select a binning mode:(https://preview.ibb.co/hDqiUz/Pixel_binning.png) (https://ibb.co/h57mhK)
(https://preview.ibb.co/hijA9z/Pixel_binning_choices.png) (https://ibb.co/eWZowe)
And X , Y
resolution you want:(https://preview.ibb.co/eSF3we/Y_resolution.png) (https://ibb.co/b7d1hK)
I thought okay let's make a try to increase vertical in 1x3 and it worked then I realized I was wrong and there is no vertical lines skipping so you can get the original height in sensor which is stretched 3478 pixels. To get 1736x736 2.35:1 the vertical should be 2220 in Y res you actually shooting 1736x2220 @ 23.976 FPS then stretch it down in post to get the right aspect which is 1736x736.
(https://preview.ibb.co/nGmAbe/MLV_lite.png) (https://ibb.co/bxkQ9z)
The problems are the write speed now about
80mb/s in 14-bit lossless (1736x736) so no continuous recording but this is the small problem and with some more tries to get higher SD writes speed and using Analog gain to get 10-bit lossless working in increased resolution will become not a big deal.
Annoying one: LiveView is
very stretched! you can use ML Framing instead which is slow unfortunately.
@Levas
Nice! :D as a1ex said no much luck on 700D. Great Job!
Amazing developments. One of the biggest drawbacks to these cameras, aliasing, has been conquered!
I just had a facepalm moment. At first I thought that the 1x3 was reading every pixel horizontally and skipping vertically. That's because I'm more familiar with the way image sizes are expressed. Take 1920x1080 for example, always horizontal size first by vertical size second. The way ML is showing binning/skipping patterns is line by column.
For example, the mv720 pattern is described as:
sampling 5x3 (read 1 line, skip 4, bin 3 columns)So that means sampling 5 vertically by 3 horizontally.
I haven't taken a close look at the crop_rec 1x3 pattern on the 700D yet but I'll update this post when I do.
[EDIT]Block: RAWC
...
sampling 1x3 (read every line, bin 3 columns)
...
Block: VERS
...
String: 'crop_rec built 2018-09-15 16:23:15 UTC; commit c7130e1 on 2018-09-13 14:55:07 UTC by Dan: crop_rec: 700D test module from a1ex '
[/EDIT]Ok--back to why 1x3 is working so well. All of the ML ported cameras are blending (binning) the pixels that are lined up horizontally but only the 5D3 bins the pixels that are lined up vertically (columns) so unless you are using a 5D3 you are probably going to experience some pretty serious aliasing, especially with the mv720 setting that samples every 5th pixel vertically. Now a head scratching moment--does the 5D3 always bin the columns even when in mv720 mode?
Quote from: a1ex on January 21, 2016, 08:52:22 AM
...I now have very strong reasons to believe that 5D3 does a 3x3 binning in 1080p, and a 5x3 binning in 720p (5 lines, 3 columns).
(if you shoot 720p on 5D3, the desqueezing factor - to correct the aspect ratio of your footage- is therefore exactly 5/3 = 1.67x)
Back to what this means for non-5D3 users is that we should be able to get the same 5D3 raw image quality by shooting 1x3 then correct the aspect ratio of your footage in a way that it smoothly blends--no pixel skipping.
Very nice! I was used to Octave notation (line/column) when writing the binning notation (5x3 etc), but relating these to resolution seems more logical; too late to swap them?
Now to figure out how to use the Eeko routines (https://www.magiclantern.fm/forum/index.php?topic=13408.0) in LiveView and compress the footage to 3x3 (or at least 2x3) before writing it to card.
Binning: I believe all cameras are either reading out every column, or binning 3 columns of the same color. Some models apparently can bin more than 3 columns. Column binning is always configured from CMOS registers (model-specific).
Wow, bilal, a1ex. Epic. 700d, how about the eosm? Closely related registers?
Likely identical; please give it a try (enable the is_700D flag in the experimental source from #1824 and let me know how it goes). 650D might work as well. 100D might need minor tweaks or might work out of the box, too. Need to check the CMOS registers for differences, as these are usually model-specific.
/* default FPS timers are the same on all these models */
if (is_EOSM || is_700D || is_650D || is_100D)
If timer A ends up too small (broken image or missing data on the right side), adjust it here:
if (is_700D)
{
target_timerA = (272 + target_xres + skip_left) / column_factor;
}
I forgot a crosshair in that bleeding-edge crop_rec source :) (used it for verifying the centering formulas)
EOSM is not happy using the settings @theBilalFakhouri recommends for the 700D:
ML ASSERT:
0
at crop_rec.c:327 (get_default_skip_top), task PropMgr
lv:0 mode:3
PropMgr stack: 17ce20 [17cf68-17bf68]
0xUNKNOWN @ bac08:17ce90
0x00B2666C @ b26d20:17ce88
0x0009EBB4 @ b26c50:17ce50
0x0009E558 @ 9ec10:17ce20
Magic Lantern version : Nightly.2018Sep15.EOSM202
Mercurial changeset : c7130e1e4c49 (crop_rec_4k_700D_test) tip
Built on 2018-09-15 17:02:43 UTC by rosiefort@RosieFoComputer.
Free Memory : 226K + 3266KQuote from: a1ex on September 15, 2018, 05:41:10 PM
I was used to Octave notation (line/column) when writing the binning notation (5x3 etc), but relating these to resolution seems more logical; too late to swap them?
Probably not necessary. Besides, it makes the "desqueezing factor" super easy to figure out -- 5/3 = 1.67x
@dfort: enable the is_700D flag for the EOS M, or turn these conditions into is_700D || is_EOSM.
@Bilal: in the emulator I've got a vertical resolution limit of 2306 pixels for 1080p24 1x3; wondering why you've got 2220 as the upper limit (this screenshot (https://preview.ibb.co/hDqiUz/Pixel_binning.png)). I've got 2200 in 25p.
[crop_rec] source: 1736x1160 @ 0.000 fps # ignore this number; emulator bug
[crop_rec] target: 1736x2306 @ 23.976 fps
[crop_rec] ylimit: 1160,2306
[crop_rec] timerA: 528 -> 520
[crop_rec] timerB: 2527 -> 2566
[crop_rec] delta : X+0 Y+1146 [z=0]
[crop_rec] source: 1736x1160 @ 0.000 fps
[crop_rec] target: 1736x2200 @ 25.000 fps
[crop_rec] ylimit: 1160,2200
[crop_rec] timerA: 640 -> 520
[crop_rec] timerB: 2000 -> 2461
[crop_rec] delta : X+0 Y+1040 [z=0]
May I ask what timer values did crop_rec pick for your preset? Or maybe you tweaked the 260 from these formulas?
desired_fps_x1000 = (uint64_t) fps_main_clock * 1000ull
/ (uint64_t) target_timerA / (desired_yres + 260);
...
int max_yres = (target_timerB - 260) & ~1;
@a1ex Yes this is correct but 2220 is stable and it's the closest number to get 1736x738 2.35:1 @23.976FPS (in 2306 you can get 30 more pixels not something huge) if I used the full 2306 I got some messages and black frame/s:
ML ASSERT:
0
at mlv_lite.c:2695 (compress_task), task compress_task
lv:1 mode:3
compress_task stack: 1b1150 [1b11e0-1b01e0]
0x0007F5D0 @ b2328c:1b1180
0x0007EF78 @ 7f62c:1b1150
Magic Lantern version : Nightly.2018Sep11.700D115
Mercurial changeset : d556a1022396+ (crop_rec_4k)
Built on 2018-09-11 08:15:57 UTC by Bilal4Fakhouri@WORKS.
Free Memory : 124K + 3142K
ML ASSERT:
((mlv_vidf_hdr_t*)slots[slot_index].ptr)->frameNumber == (uint32_t) slots[slot_index].frame_number - 1
at mlv_lite.c:3421 (raw_video_rec_task), task raw_rec_task
lv:1 mode:3
raw_rec_task stack: 1acd10 [1acdd0-1abdd0]
0x0007F5D0 @ b31970:1acd40
0x0007EF78 @ 7f62c:1acd10
Magic Lantern version : Nightly.2018Sep11.700D115
Mercurial changeset : d556a1022396+ (crop_rec_4k)
Built on 2018-09-11 08:15:57 UTC by Bilal4Fakhouri@WORKS.
Free Memory : 124K + 3139K
I don't like to push it to the limits,, this is safer :D
Quote from: a1ex on September 15, 2018, 07:34:06 PM
May I ask what timer values did crop_rec pick for your preset? Or maybe you tweaked the 260 from these formulas?
desired_fps_x1000 = (uint64_t) fps_main_clock * 1000ull
/ (uint64_t) target_timerA / (desired_yres + 260);
...
int max_yres = (target_timerB - 260) & ~1;
timer A=520
timer B=2566
I didn't tweak anything ::)
Okay, got it. These timer B limits are difficult to find because they are not hard (you don't see the image breaking as soon as you cross some limit, like it happens with timer A). Vertical resolution can usually be pushed further (i.e. closer to timer B) in low FPS modes. Not sure what's the best formula to use here. Writing down various resolutions and their outcome (stable or not) should reveal a pattern.
Selecting a higher margin (i.e. using 280 or 300 instead of that 260) may be a better option for high FPS modes, but might be suboptimal for e.g. full-res LiveView.
I did not try to record anything during my tests, only tried to get clean grayscale preview in mlv_rec.
Did think about this solution for aliasing too, it's just a matter of setting the line skipping factor to 0 (register 800c) in non zoom mode and expand the vertical resolution.
But because it's 3 times more data that needs to be recorded, so no continuous recording...I didn't see any use in it, hell I was just getting continuous recording times, because of the lossless development :D
But I must say, the 1x3 files look a lot better, much more detail.
Can you share some sample DNG's of these examples. Both an 1x3 file and 3x3 dng file from the same subject ?
Curious how much the difference in detail is in a raw editor.
@Alex
Here is the source crop_rec.c file with 96 fps preset in it, it's the preset called HD:
https://drive.google.com/file/d/1n-hUn8Oo815K4meJhvbAXrj4Fbcft6Zy/view?usp=sharing (https://drive.google.com/file/d/1n-hUn8Oo815K4meJhvbAXrj4Fbcft6Zy/view?usp=sharing)
Here are the most important pieces of the 96fps source, I used Preset HD for 96 fps:
case CROP_PRESET_HD:
cmos_new[6] = 0x400;
cmos_new[7] = 0xb + delta_head3;
break;
if (is_6D)
{
switch (crop_preset)
{
case CROP_PRESET_HD:
case CROP_PRESET_2K:
case CROP_PRESET_ULTRA_HD:
case CROP_PRESET_4K:
case CROP_PRESET_5K:
case CROP_PRESET_FullRes:
case CROP_PRESET_HD2:
case CROP_PRESET_3x3_1X:
/* all modes may want to override shutter speed */
/* ADTG[0x8061]: shutter blanking for 3x3 mode */
/* ADTG[0x805F]: shutter blanking for zoom mode */
adtg_new[0] = (struct adtg_new) {6, 0x8061, shutter_blanking};
adtg_new[1] = (struct adtg_new) {6, 0x805F, shutter_blanking};
/* adjust vertical resolution */
/* assuming FPS timer B was overridden before this */
int fps_timer_b = (shamem_read(0xC0F06014) & 0xFFFF) + 1;
int readout_end = shamem_read(0xC0F06804) >> 16; /* fixme: D5 only */
/* PowerSaveTiming registers */
/* after readout is finished, we can turn off the sensor until the next frame */
/* we could also set these to 0; it will work, but the sensor will run a bit hotter */
/* to be tested to find out exactly how much */
adtg_new[2] = (struct adtg_new) {6, 0x8172, nrzi_encode(readout_end + 1) }; /* PowerSaveTiming ON (6D/700D) */
adtg_new[3] = (struct adtg_new) {6, 0x8178, nrzi_encode(readout_end + 1) }; /* PowerSaveTiming ON (5D3/6D/700D) */
adtg_new[4] = (struct adtg_new) {6, 0x8196, nrzi_encode(readout_end + 1) }; /* PowerSaveTiming ON (5D3) */
adtg_new[5] = (struct adtg_new) {6, 0x8173, nrzi_encode(fps_timer_b - 1) }; /* PowerSaveTiming OFF (6D/700D) */
adtg_new[6] = (struct adtg_new) {6, 0x8179, nrzi_encode(fps_timer_b - 1) }; /* PowerSaveTiming OFF (5D3/6D/700D) */
adtg_new[7] = (struct adtg_new) {6, 0x8197, nrzi_encode(fps_timer_b - 1) }; /* PowerSaveTiming OFF (5D3) */
adtg_new[8] = (struct adtg_new) {6, 0x82B6, nrzi_encode(readout_end - 1) }; /* PowerSaveTiming ON? (700D); 2 units below the "ON" timing from above */
/* ReadOutTiming registers */
/* these shouldn't be 0, as they affect the image */
adtg_new[9] = (struct adtg_new) {6, 0x82F8, nrzi_encode(readout_end + 1) }; /* ReadOutTiming */
adtg_new[10] = (struct adtg_new) {6, 0x82F9, nrzi_encode(fps_timer_b - 1) }; /* ReadOutTiming end? */
break;
case CROP_PRESET_HD:
adtg_new[11] = (struct adtg_new) {6, 0x8000, 6};
adtg_new[12] = (struct adtg_new) {6, 0x800C, 4};
break;
static inline uint32_t reg_override_HD(uint32_t reg, uint32_t old_val)
{
switch (reg)
{
/* raw resolution (end line/column) */
/* case 0xC0F06804: return 0x33f01ec; // 1440 x 360 6d 96fps; */
case 0xC0F06804: return 0x186018a;
case 0xC0F08184: return 0x1;
case 0xC0F08188: return 0x1;
case 0xC0F08194: return 0x1;
case 0xC0F08198: return 0x1;
case 0xC0F06824: return 0x202;
case 0xC0F06828: return 0x202;
case 0xC0F0682C: return 0x202;
case 0xC0F06830: return 0x202;
case 0xC0F06010: return 0x204;
case 0xC0F06008: return 0x2040204;
case 0xC0F0600C: return 0x2040204;
case 0xC0F06014: return 0x203;
case 0xC0F0713c: return 0x187; // Same as 6804 res +1 ?; */
}
return 0;
}
Quote from: Levas on September 16, 2018, 12:41:17 AM
Did think about this solution for aliasing too, it's just a matter of setting the line skipping factor to 0 (register 800c) in non zoom mode and expand the vertical resolution.
Of course there are no aliasing! another example (1280x720):
https://drive.google.com/open?id=1S2lM18EpDPSnRx06_j6W1orh54jjn3IS
The remaining aliasing in 1x3 is also can shown in 1x1 mode, so this depending on the subject too. But it's 90% better than 3x3 if not 100% you must try it on 6D :D
This is the most hated moire pattern has gone:
Quote from: Levas on September 16, 2018, 12:41:17 AM
But I must say, the 1x3 files look a lot better, much more detail.
Can you share some sample DNG's of these examples. Both an 1x3 file and 3x3 dng file from the same subject ?
Curious how much the difference in detail is in a raw editor.
I can't see much more details except for aliasing areas the image now smoother, Here is two samples (https://drive.google.com/open?id=11_WddnlZnH0RDGrU6FYF9sPQihY-NLEV).
@a1ex
Quote from: a1ex on September 15, 2018, 05:41:10 PM
Now to figure out how to use the Eeko routines (https://www.magiclantern.fm/forum/index.php?topic=13408.0) in LiveView and compress the footage to 3x3 (or at least 2x3) before writing it to card.
Is this means the files size will be reduced ? lower write speeds?
Interesting developments. Wonder how 1x3 performs with that moving patter we used a while back on the focus pixel discussion (https://www.magiclantern.fm/forum/index.php?topic=16054.msg196313#msg196313).
Speaking of focus pixels, I don't see any on these tests. Are they gone or is our focus pixel removal tool working with 1x3? Taking a close look at some of the tests I shot in 1x3 and processed in mlv_dump with the --no-fixfp option I don't see any focus pixels.
Haven't had much luck with the EOSM yet. Maybe we need to add Danne's mv1080 crop_rec settings (https://www.magiclantern.fm/forum/index.php?topic=9741.msg203530#msg203530)? As I recall there was a problem handling the EOSM exception so the module can't handle both mv720 and mv1080.
Another benefit for using 1x3 Binning is:
There is
no aliasing in Dual ISO video! Take a look:
(https://preview.ibb.co/kbDOzz/3x3.jpg) (https://ibb.co/eC9wKz)
(https://preview.ibb.co/cuVgme/1x3.jpg) (https://ibb.co/jiSizz)
(https://preview.ibb.co/cMjkCK/1x3_Dual_ISO.jpg) (https://ibb.co/isWAez)
Video:
Example: (1280x720 Upscaled to 1080p)
YouTube did horrible compression in the details so here is the original file (encoded via Grass Valley HQX maybe you should install Grass Valley codecs):
https://drive.google.com/open?id=17ob5AOkirAwq3TRwBGGY1qIyNfZwfj-f
I am not sure about the image details which is better normal 3x3 or 1x3 Dual ISO I will make more tests in future. Even if it was aliasing in some cases using 1x3 + Dual ISO it's a lot better than normal 3x3 aliasing. This huge!
Hey man. Is this seriously working with dualiso as good as it looks? I mean, is this already working on the 5D mark III already or only 700D? I could test this of course but the way my life swings atm I have a hard time finding time to even recharge my cam batteries. To me it seems this image quality is somewhere touching some sort of pinnacle in terms of file quality. The nonexistent aliasing on the keyboard footage is insane.
Thanks so much for sharing your findings Bilal!
Personally I like to test this on eosm, 100D and a 5d mark III. Will find time soon.
So no aliasing and greatly increased dynamic range -- Fantastic!
It makes total sense that dual iso works better in 1x3 sampling mode. You loose up to half the vertical resolution with dual iso and that is a very big deal with 3x3 sampling and pretty much unusable with 5x3 sampling but since the vertical resolution is going to be resized anyway in 1x3 mode there should be no quality loss when using dual iso.
Now how long are your recording times? I can only get a second or so. Need some more experimenting with the sd_uhs module.
Insane work for 96fps and remove aliasing 1x3, even for dual iso !!
This is fantastic and if the bitrate remains the same, it means that all camera could benefit of these function !
No need for expansive vaf anymore !
Envoyé de mon iPhone en utilisant Tapatalk
@Danne
Yes it's that good and YouTube did horrible compression in the details so here is the original file (encoded via Grass Valley HQX maybe you should install Grass Valley codecs):
https://drive.google.com/open?id=17ob5AOkirAwq3TRwBGGY1qIyNfZwfj-f
In 5D3 3x3 Binning mode using Dual ISO the aliasing will be more visible than 1x3 which work a lot nicer even maybe more than 1x1 mode!
@dfort
In 1736x738 @ 23.976 FPS 2.35:1 (Actually 1736x2214 in 1x3) I can get Max 2 seconds or a bit less. So I am using 1280x544 (1280x1632 in 1x3) 2.35:1 aspect ratio to get continuous recording times using sd_uhs of course :D.
Smart idea, the extra vertical lines with 1x3 mode are indeed a good use for dual iso 8)
Now I have take a look at it again for the 6d, I see why I stopped before, 6d isn't fast enough, only 25.6 Mhz. instead of the 32Mhz of most other cams.
1824 is the maximum horizontal resolution. This needs an A timer value of 515.
25600000 Hz / (515+1) / 25FPS = 1984, so that leaves a value of 1984-1= 1983 for the B timer.
With a little safety margin to prevent corrupted frames, that probably gives about 1900 vertical resolution at max.
After scaling correction, 1824 x 636 resolution is left... ::)
Luckily I know my ways to get rid of color moire, so still going strong on 3x3 ;D
Quote from: Levas on September 17, 2018, 01:04:59 AM
Luckily I know my ways to get rid of color moire, so still going strong on 3x3 ;D
And what is your method ? =)
I use Rawtherapee as a raw editor to develop the DNG's.
Most of the color moire is already gone by using LMMSE method for debayering, which RawTherapee can do.
I saw it recently also be used in MLV APP, but that version was compiled by somebody on this forum, under development, not sure if it has already LMMSE debayering option in the current build ?
To get rid of some more color moire there's a second trick that can be done in RawTherapee, by the use of 'defringe', which in most raw editors only removes purple glow, but in RawTherapee you can adjust it to remove more colors than only purple.
The LMMSE debayering can give a pixelated look on some patterns, in that case, AMAZE debayering including defringe can also be an option.
And be gentle with the defringe, dragging the colors too high and you will notice unsaturated lines around colored subjects.
Made a quick tutorial 2 years ago:
https://www.youtube.com/watch?v=MweH3RAssXs (https://www.youtube.com/watch?v=MweH3RAssXs)
Thank you Levas ! Yes LMMSE available on RawTherapee. I tried with a dual iso footage from 5DmkIII but it wasn't working well.
Oh my God, am I dreaming? Or are we all in the middle of new extremely exciting times for Magic Lantern? You guys are WIZARDS !!! Thank you so much for your amazing work!
Recently, I shot a lot of landscape footage at 2520x1304@18fps and 14-bits lossless on the 100D. Beautiful footage, excellent fine detail, huge dynamic range, smooth transitions. Bilal, could you please answer the following question for me: Would it be possible to achieve the following goal on the Rebel cameras (700D, 100D, EOS-M and maybe other Rebel models too)?
Resolution: 2560x1440 (this resolution would provide very good quality on Vimeo and Youtube)
Bit depth: 10-bit with lossless compression
Frame rate: 24 ps
Recording: Continuous or at least 10-12 sec. recording time
Preview: WySiWyG, even if it is slow
And what about D4 cameras (7D, 5D2)? If the answer to that question is YES, then this would be a real break through for these cameras and a revolutionary achievement for the ML community !!!
Quote from: Levas on September 17, 2018, 09:36:25 AM
I saw it recently also be used in MLV APP, but that version was compiled by somebody on this forum, under development, not sure if it has already LMMSE debayering option in the current build ?
Yes, it is included in MLVApp v1.0.
Quote from: 12georgiadis on September 17, 2018, 10:49:32 AM
I tried with a dual iso footage from 5DmkIII but it wasn't working well.
LMMSE has little effect on dual_iso, because it's already altered to bring both iso's to the same level.
But I see the defringe option does work very well. Did bring the threshold for defringing to zero in this case (13 by default).
Example of dual iso, with amaze debayering and without defringing. (click to expand to full res)
(https://farm2.staticflickr.com/1879/43828029075_ddd1e93f77_o.jpg)
And here the same, with defringe option in Rawtherapee, all colors lifted a little as seen in the youtube video and this time, threshold set to a value of zero.(click to expand to full res)
(https://farm2.staticflickr.com/1842/44018438344_44a9bf27c5_o.jpg)
Thank you for the example Levas. I can see the improvement for defringing but my concern is also aliasing and I can't get rid off in my try. On the other hand, this 1X3 binning mode seem to be the solution, especially if it's done before going in post, isn't it ?
Quote from: theBilalFakhouri on September 16, 2018, 01:39:12 PM
I can't see much more details except for aliasing areas the image now smoother, Here is two samples (https://drive.google.com/open?id=11_WddnlZnH0RDGrU6FYF9sPQihY-NLEV).
Checked out your samples, removed the aliasing in the 3x3 samples and rescaled the 1x3 samples and must conclude you're right, can't see much more detail, probably none, in the 1x3 samples.
By the way, this 1x3 method could, besides dual iso, also be good for low light shooting.
The final image is rescaled to 1/3 of it's original size, that should give about 1.33 stops of noise reduction. So shooting iso 3200 with iso 1250 results...
@12georgiadis
You're right that the options I mentioned only work for color aliasing, it doesn't work for annoying moire patterns.
Shooting 1x3 should fix this in 99%(I guess, because even camera's that don't do binning and lineskipping can suffer from moire patterns, since the pixels on the CMOS are pretty straight aligned in rows and columns)
The problem for now is that shooting in 1x3 takes up 3x the amount of resolution and thus 3x the amount of data to be recorded.
Quote from: a1ex on September 15, 2018, 02:09:40 PM
If you weren't able to lower timer A, I'd expect the horizontal resolution to be full width (1824 iirc).
You may also use other line skipping factors on the Y direction: 3, 5, 7, 9...
Alex, you're right, I could have gone full width with these settings, 1824 pixels wide.
Did try to set lineskipping to 6 instead of 4, but that gave me a weird preview, which I can record (so it's not corrupted, it's just not what you would expect)
(https://farm2.staticflickr.com/1896/44737205041_f5a9663406_o.jpg)
I've got the original DNG frame here:
https://drive.google.com/open?id=1RxwkrQ69UVU69oFhsfIVOFs7EaRy2fML (https://drive.google.com/open?id=1RxwkrQ69UVU69oFhsfIVOFs7EaRy2fML)
The weird thing is, all other options work, lineskipping value 2,3,4 even a value of 5 works, of course colors are a bit weird :P, but that is expected with an uneven value.
Could it be caused by your automated shutter blanking and shutter speed registers ?
Quote from: Levas on September 17, 2018, 02:20:10 PM
@12georgiadis
You're right that the options I mentioned only work for color aliasing, it doesn't work for annoying moire patterns.
Shooting 1x3 should fix this in 99%(I guess, because even camera's that don't do binning and lineskipping can suffer from moire patterns, since the pixels on the CMOS are pretty straight aligned in rows and columns)
The problem for now is that shooting in 1x3 takes up 3x the amount of resolution and thus 3x the amount of data to be recorded.
Ok, good to know...
Quote from: dfort on September 16, 2018, 08:51:49 PM
..but since the vertical resolution is going to be resized anyway in 1x3 mode there should be no quality loss when using dual iso.
Quote from: Levas on September 17, 2018, 02:11:37 PM
Checked out your samples, removed the aliasing in the 3x3 samples and rescaled the 1x3 samples and must conclude you're right, can't see much more detail, probably none, in the 1x3 samples.
I can confirm There is no quality loss when using Dual ISO in 1x3! (@Levas No much details in normal 1x3 but the details has been restored when using Dual ISO)
1x3 Binning made for Dual ISO! no price to pay when using dual iso in video now!
Quote from: a1ex on July 16, 2013, 06:33:50 PM
Price to pay
- Half resolution in highlights and shadows
- Aliasing and moire - of course, in highlights and shadows
- You can no longer check critical focus when zooming in
:D
Magic Lantern Cinema Camera:
Link: https://youtu.be/cP6DjS4QgcE
Original file in Google Drive: Download (https://drive.google.com/open?id=1juv12PM7F1_6J4kh35bJXEG5EPj5_30e)
Details: https://www.magiclantern.fm/forum/index.php?topic=22818.msg206134#msg206134
Quote from: Levas on September 17, 2018, 02:11:37 PM
By the way, this 1x3 method could, besides dual iso, also be good for low light shooting.
The final image is rescaled to 1/3 of it's original size, that should give about 1.33 stops of noise reduction. So shooting iso 3200 with iso 1250 results...
You are right there is improvement in the shadows:
ISO 100 3x3: (Pushed 4 stops)
(https://preview.ibb.co/c8zcBe/ISO_100_3x3.jpg) (https://ibb.co/ivCxBe)
ISO 100 1x3: (Pushed 4 stops)
(https://preview.ibb.co/jyTFxK/ISO_100_1x3.jpg) (https://ibb.co/bWwPre)
Dual ISO 100/800 3x3: (Pushed 5 stops)
(https://preview.ibb.co/kzLW4z/Dual_ISO_100_800_3x3.jpg) (https://ibb.co/m9nB4z)
Dual ISO 100/800 1x3: (Pushed 5 stops)
(https://preview.ibb.co/cbFLxK/Dual_ISO_100_800_1x3.jpg) (https://ibb.co/jBV9re)
@IDA_ML
Quote from: IDA_ML on September 17, 2018, 11:22:16 AM
..Bilal, could you please answer the following question for me: Would it be possible to achieve the following goal on the Rebel cameras (700D, 100D, EOS-M and maybe other Rebel models too)?
Resolution: 2560x1440 (this resolution would provide very good quality on Vimeo and Youtube)
Bit depth: 10-bit with lossless compression
Frame rate: 24 ps
Recording: Continuous or at least 10-12 sec. recording time
Preview: WySiWyG, even if it is slow
And what about D4 cameras (7D, 5D2)? If the answer to that question is YES, then this would be a real break through for these cameras and a revolutionary achievement for the ML community !!!
We already have more than 2560x1440! we can get up to 2700x1500 @ 23.976 fps (https://www.magiclantern.fm/forum/index.php?topic=19300.msg205975#msg205975) without any corrupted frames! This thing has been done by using the new method of crop_rec (https://www.magiclantern.fm/forum/index.php?topic=19300.msg205546#msg205546) only for 700D for now but I think it's very soon for all D5 cams!
10 Bit-lossless is already working also when using Analog gain. Maybe @Danne have a build for 100D. (I noticed a vertical stripes appears in 700D when using Analog gain and maybe it's fixable but for now you can fix it in MLVApp).
Recording times: we have to figure out how to get full SD write speed, 10-bit lossless isn't enough for continuous recording in this resolutions! and maybe we are not sure if we can see effective 80MB/s write speed in small cameras.
Preview is working in 700D. and maybe in all other cameras if you have a problem you can ask the developers :D .
Now 5D2 has reached two big things 10/12 bit depths working and 3K (2880x1080) RAW video recording. This is very good sign for D4 Cameras !
- How can I turn on 4096x1440 @ 25p (1:1 crop) on my 5dMk3?
I've installed the latest experimental built
magiclantern-crop_rec_4k.2018Jul22.5D3113.
(https://thumb.ibb.co/dRYbNK/5dmkIII.jpg) (https://ibb.co/dRYbNK)
While I've sucessfully recorded in
3.5K 1:1 centered x5, I can't figure out how to turn on
4096x1440 @ 25p (1:1 crop).
No combination of
Crop mode and
RAW video in the ML menu seemed to achieve that resolution.
Did I forget to make some settings in a certain module?
Thanks a lot for this important clarification, Bilal. Your findings are really amazing, especially those concerning Dual ISO. I have always loved Dual ISO, not only because of the high dynamic range but also for much better and noiseless results in low-light and night videography. Hopefully, Danne will be able to provide a working build for the 100D soon. In that case, I will test and provide feedback as soon as possible.
By the way, did you try shooting at the maximum resolution but at 18 fps? For the types of landscape shots that you provided, this reduced frame rate should provide excellent results and you will get reasonably long recording times.
Keep up the good work !!!
Working against time here but added EOSM by including is_700D || is_EOSM in a1ex crop_rec.c code. Following Bilals settings vertical resolution is stuck at 696, so probably raw.c height restriction needs to be erased and doing that opens up another can of worms creating image corruption. In theory maybe the newly mv1080p register settings would make this work but can´t get any decent 1x3 binning results yet. For anything to record uncorrupted I have to lower fps to 17 and then I can record 1x3 1736x696, probably not valid files.
I will make wish for more time...
Take your time, Danne, there is no rush! It is much more important that there is serious work going on on the Rebels and D4 models and that these new amazing findings have found proof of concept already. Why is that important? Because this keeps the interest of Rebel and D4-camera owners in Magic Lantern alife and stops them from selling their cameras and transfering to SONY for example.
By the way, there is a lot of excitement among ML users in my country at the moment. So please, A1ex and other developers, don't give up on these fantastic new developments. Our cameras may be 10 years old but are fully up to them and deserve your attention! My 7D still writes at 90 MB/s without CF-card overclocking!
question.... but there is not little confusion about what you can do with 700 D , not same of 5D3 and viceversa , al all the another models, use, the same experimental file?
I'v read some set's on 700d not working on 5d3 with same results or for nothing... when experimental firmware file is update, is good think put all new togheter for any model ? like dual iso x no aliasing seem not work same of 700d on 5d3 or another model... so, meaby better compile source firmwares only for 700d , for 5d3 and so x alother models? or not problem to put all togheter on one?
The idea is to get one file (the crop_rec module) that runs on all cameras.
Simple said, most of the routines used for these new resolutions are the same for all cameras, with the exception that each camera has his own limits and starting values.
So there are pieces in the code where all limits and specific values for each cam are written down, and the rest of the code is basically routines for getting these new resolutions.
By doing it this way, all cameras are using the same routines, so if new development is done, you can do it at once for all camera's.
thanks Levas for the explanation ;)
so there 's no risk that some rotines specific to a model x, can create malfunctions to another model y, maybe on things that instead in the previous version were fine? for example on the 5d3 with the latest version, and I wondered if this could also depend on insertions of things for the 700d , 100d ecc..
and of course also vice versa ..
just out of curiosity'..
There is always a small chance that in a new version things are broken for other camera(s) or even for all cameras :P.
But that's what the forum and the user base is for, testing testing testing ;)
The advantages of doing it this way in one module file, is that new cameras (maybe 5dIV in the future) or older ones like the 5dII, can be added to the code, without writing the whole thing from scratch again.
And other users who can't write code from scratch are sometimes still able to alter the code for other cameras, by entering the limits and starting values for their specific camera.
Quote from: Litewolf on September 18, 2018, 10:44:21 AM
- How can I turn on 4096x1440 @ 25p (1:1 crop) on my 5dMk3?
I've installed the latest experimental built magiclantern-crop_rec_4k.2018Jul22.5D3113.
(https://thumb.ibb.co/dRYbNK/5dmkIII.jpg) (https://ibb.co/dRYbNK)
While I've sucessfully recorded in 3.5K 1:1 centered x5, I can't figure out how to turn on 4096x1440 @ 25p (1:1 crop).
No combination of Crop mode and RAW video in the ML menu seemed to achieve that resolution.
Did I forget to make some settings in a certain module?
I was confused with that when I tried this experimental build at first too.
There is no 4096x option in latest builds, even if it says so in the first post and in text next to DL options. Even if it was, you wouldn't get many frames. But I think it will be at some point if 11...8 bit lossless combined with CF overclocking works one day. There is so much developing going on, that it is almost impossible to even follow or fully understand what is going on.
So my question here: If I want to try 1X3 binning Dual ISO on 5DmkIII 113, where is it possible to download such crop_rec module and how to set it; did anybody try it on 5DIII?
@a1ex
I noticed flickering with new crop_rec it becomes stronger when using high shutter speeds such as 1/500 1/1000 etc and very visible in Full range shutter speeds 1/32K I thought it was from dual iso but it's look like from shutter speeds.
Is this also in 5D3?
No, it doesn't flicker. Can you record a video?
Are we talking daylight or artificial lighting (50Hz/60Hz electricity) ?
@Levas
Hmm no it's like this:
Link: https://youtu.be/MHwYdmaadb8
Weird.
Did a quick test with max shutterspeed on the 6d, 1/25000 of a second, no flicker.
Are you sure there's no light source in the frame, maybe computer monitor or something ?
And are you in full Manual mode, there's no chance autoexposure tries to compensate with iso or aperture ?
On 700D, the registers used for shutter speed override, ADTG[805F/8061], are toggled back and forth between two values.
If you override the shutter speed from adtg_gui, do you get similar flicker?
If not -> it's a roundoff error that we should fix in crop_rec
If yes -> we may need to flip that register back and forth in crop_rec or figure out some other trickery.
Overriding from adtg_gui give similar flicker. Also I think the problem in the build not in crop_rec I chose from Canon menu 1080p24 then set the shutter speed to 1/4000 there is no flicker and I can see in ML menu the shutter speed swing between two values with really tiny distance 1/4328.80 and 1/4329 so nothing noticeable and this the normal from Canon.
But once you turn on FPS override and override FPS to 23.976 same as Canon menu the flicker appears and the distance increased between the values 1/4328.80 to 1/4040.20 and it causing the flicker without using or even loading crop_rec. I am using the build from crop_rec_4k branch which is support shutter blanking.
Edit: @Levas
No, everything is normal no flicker in all shutter speeds until using fps override or crop_rec :P
Finally got to test the new MLV lite module with sound. JUL22, 5D3 113. Those are the first bug that I notice that impact my workflow:
1) Rec indicator works only on elapsed time.
2) Sync bip doesn't work.
3) Changing ISO provokes various anomalies on the liveview.
Also the silent picture modules doesn't seem to honour the exposure settings. And sometimes it recorded in the CF card and sometimes in the SD.
1) H.264 only; same as previous builds
2) disable sound recording
3) ?!
4) - with plain full-res, known limitation (https://www.magiclantern.fm/forum/index.php?topic=12523.0); otherwise, the silent picture module won't alter exposure on its own
- if using crop_rec or FPS override, double-check submenu settings (some options will change shutter speeds)
- with full-res LV, exposure looks OK to me (matches LiveView brightness and metadata shows exposure values dialed in Canon menu).
5) it uses the card selected in Canon menu.
Quote from: zalbnrum on September 22, 2018, 01:19:30 PM
There is no 4096x option in latest builds
In the crop_rec menu, look for 4K instead of 4096. Just tested and it's still working.
1-2) Right! I kind of remembered about it. Too bad.
3) When I tried to change ISO while recording the ISO overlay came up, then didn't want to disappear etc.
4) I need to spend more time in this. I didn't use plain full-res. I *think* I've used full-res LV, I mean I've selected it from the silent pic menu and then shoot in photo mode LV. Brightness didn't match and values neither. But I'll try to be more precise tomorrow.
5) Weird, I hadn't change it while testing. I'll try again.
@Alex: @Litewolf asked about 4096x1440 @ 25p (1:1 crop); in my menu there is only 4K 1:1 half fps - 12.5fps. Am I missing something?
Quote from: togg on September 29, 2018, 09:44:23 PM
I *think* I've used full-res LV, I mean I've selected it from the silent pic menu and then shoot in photo mode LV
Right, this might be the issue. In photo mode, Canon firmware selects exposure settings that are - most of the time - very different from the settings dialed in Canon menu; they are chosen to give the same overall brightness (that's called exposure simulation). Otherwise, photo mode LV on 5D3 is pretty much the same as 1080p30, but crop_rec only looks at video settings, so... to use this trick in photo mode, it looks like you need the video mode set to 1080p30 (that's a bug). Otherwise there will be some exposure mismatch.
Besides, Canon's exposure simulation choices are not always obvious. For example, for ISO 100 1/50 dialed in Canon menu, they picked ISO 100 1/120 for the LiveView exposure.
No idea why. It
might be appears to be their underexposure bug with manual lenses (I've used one during the test). Edit: re-tested with 50/1.8 STM; got 1/120 at f/2.8 (the setting when powering on) and 1/50 when opening the aperture to f/1.8.
Enabling exposure override in ML menu switches to our own ExpSim algorithm, which should be a little more predictable. In M mode, it tries to use dialed settings, unless it requires out-of-range exposure times; in this case, it switches the ISO. It doesn't touch aperture at all (at least at the time of writing).
Quote from: a1ex on September 30, 2018, 08:26:25 AM
Right, this might be the issue. In photo mode, Canon firmware selects exposure settings that are - most of the time - very different from the settings dialed in Canon menu; they are chosen to give the same overall brightness (that's called exposure simulation). Otherwise, photo mode LV on 5D3 is pretty much the same as 1080p30, but crop_rec only looks at video settings, so... to use this trick in photo mode, it looks like you need the video mode set to 1080p30 (that's a bug). Otherwise there will be some exposure mismatch.
Besides, Canon's exposure simulation choices are not always obvious. For example, for ISO 100 1/50 dialed in Canon menu, they picked ISO 100 1/120 for the LiveView exposure. No idea why. It might be appears to be their underexposure bug with manual lenses (I've used one during the test). Edit: re-tested with 50/1.8 STM; got 1/120 at f/2.8 (the setting when powering on) and 1/50 when opening the aperture to f/1.8.
Enabling exposure override in ML menu switches to our own ExpSim algorithm, which should be a little more predictable. In M mode, it tries to use dialed settings, unless it requires out-of-range exposure times; in this case, it switches the ISO. It doesn't touch aperture at all (at least at the time of writing).
Ok thanks, this is clear now.
- Use video mode for fullres LV.
- Use exposure override for manual lenses.
It makes sense, my mistake was that I remembered from the past that you should be in photo mode for silent pic. Probably a text when selecting it would help.
This is one of my favourite new features honestly, 48fps is nice but silent shooting on a dslr is so important for so many situations. The only problem now is the strong lag in order to take the picture. Wouldn't be possible to have some kind of pre recording working in the module? A burst mode would help as well.
Looking at the 14lossless option I've noticed that it says ISO <100. Why this advice? Does it change anything if I shoot ISO 1600 in 14bit uncompressed or lossless?
High ISO will produce more noise affected pixel. Those random pixels will not work well with compression. Worst case: "Compressioned" pic will be bigger in size than uncompressed.
Yeah, video mode is the best option for full-res LV right now.
Pre-recording: there are a few options (https://www.magiclantern.fm/forum/index.php?topic=19300.msg182324#msg182324), but none of them is what I'd call user friendly. That's mostly because I didn't use the camera too much since introducing this feature (unfortunately), so I didn't really feel their shortcomings.
I might add one more:
- crop_rec.mo, silent.mo
- crop mode: full-res LV; silent mode: burst, end trigger, lossless DNG
- press and hold half-shutter; release to save the last frame (fixme: should be able to save 3 frames...)
- mlv_rec.mo for preview (not very nice, but...)
I did try to implement a full-shutter trigger, with only partial success; still triggered a shutter actuation every now and then. Managed to save a CR2 as well, but decided not to publish it, as it was too much of a hack. Also managed, to some extent, to change the video resolution on the fly, but required a few seconds to recalibrate the vertical stripe corrections, so... it wasn't exactly usable for a real-time preview with small shutter lag in full-res mode. These happened somewhere before April 1st (I had a bunch of pretty interesting proof of concepts, but none of them was exactly ground-breaking, and only the SD hack worked reasonably well...)
ISO < 100: that means 14 bits may be a little overkill even for ISO 100 (12-bit should be nearly just as good in theory, although I didn't test much). The message also anticipates the reduced ISOs from the CMOS/ADTG research thread, where the additional dynamic range may require a higher bit depth when recording. First post has a colored matrix which links to some more details.
Quote from: Walter Schulz on September 30, 2018, 03:01:09 PM
High ISO will produce more noise affected pixel. Those random pixels will not work well with compression. Worst case: "Compressioned" pic will be bigger in size than uncompressed.
Daamn. But what about the quality? Is it exactly the same according to your test? I'll try to test how much bigger does it get...
Quote from: a1ex on September 30, 2018, 03:15:18 PM
Yeah, video mode is the best option for full-res LV right now.
Pre-recording: there are a few options (https://www.magiclantern.fm/forum/index.php?topic=19300.msg182324#msg182324), but none of them is what I'd call user friendly. That's mostly because I didn't use the camera too much since introducing this feature (unfortunately), so I didn't really feel their shortcomings.
I might add one more:
- crop_rec.mo, silent.mo
- crop mode: full-res LV; silent mode: burst, end trigger, lossless DNG
- press and hold half-shutter; release to save the last frame (fixme: should be able to save 3 frames...)
- mlv_rec.mo for preview (not very nice, but...)
I did try to implement a full-shutter trigger, with only partial success; still triggered a shutter actuation every now and then. Managed to save a CR2 as well, but decided not to publish it, as it was too much of a hack. Also managed, to some extent, to change the video resolution on the fly, but required a few seconds to recalibrate the vertical stripe corrections, so... it wasn't exactly usable for a real-time preview with small shutter lag in full-res mode. These happened somewhere before April 1st (I had a bunch of pretty interesting proof of concepts, but none of them was exactly ground-breaking, and only the SD hack worked reasonably well...)
ISO < 100: that means 14 bits may be a little overkill even for ISO 100 (12-bit should be nearly just as good in theory, although I didn't test much). The message also anticipates the reduced ISOs from the CMOS/ADTG research thread, where the additional dynamic range may require a higher bit depth when recording. First post has a colored matrix which links to some more details.
I can see how complex it is to balance the feature between multiple modules. But seems like the capability is there at least! I've read again your first post, looks like you wanted to makes this a feature used mostly for stills, which makes sense to me.
"Fast half-shutter trigger (with negative shutter lag)" is a decent solution that I should try to set up and test. The new one you propose looks maybe even better.
Quote(That means, 14-bit lossless will give raw data identical to uncompressed 14-bit (100% identical, not one single bit different, contrary to some earlier report)
"=> there's nothing to gain by using a lower bit depth at a higher ISO + underexposure."
Oooh, ok! Of course. Now I get it. Why should anybody do that? Clearly noisier images are more difficult to compress. My concern was about a change in quality, if there's none I don't care if sometime compression change expected size, I'll work under the assumption that recording time isn't fixed.
I don't trust 12bit enough, I had see some difference in other people testing.
Hello, guys, I want to participate in "full-res LV in video mode" discussion.
My ultimate goal is to solve 2 main (for me) silent timelapse shortcomings:
1. No aperture control on USM/STM lenses.
2. Long exposure limitation.
And "full-res LV in video mode" looks as the best solution for rise the timelapse capabilities of ML: Video mode brings control for aperture on its own. And full-res LV adds exposure control. That was my idea.
Spoiler: After spending all day i come to conclusion, that full-res LV is very interesting proof of concept, but barely usable in it's current stage. It need correct exposure readings, integration with intervalometer and Auto ETTR.
Details. I loaded crop_rec, ettr, silent and start experimenting in manual video mode. crop_rec_4k.2018Jul22.5D3113, EF 35mm f/2 IS USM
1. Auto ETTR disabled due "This feature requires you shooting raw". So no Auto ETTR.
2.I dial to shortest exposure - 1/33 and press half shutter - and got brighter image with 1/7 on preview. Preview has no histogram, no way to estimate exposure.
1/50 corresponds to 1/11, 1/130-1/29, 1/1000-1/310.
3. I started intervalometer. It takes pictures, but if canon's image review is "hold" you need to press half shutter yourself every time, and if you make image review "Off" intervalometer gets disabled - "Auto ETTR: enable image review". But keeps working, even with preview.
4. Ok, exposure is wrong, but controllable, is it possible to get longer than 1/7 exposures? I started to activate crop mode "Full res LiveView" and FPS override in various combinations. Crop mode always makes preview distorted. FPS override affects only on preview, but not actual image taken with full-res LV.
5. Finally, I unloaded crop_rec and changed silent mode on full-res. Yes, it warns that works only on photo mode and not works with half shutter. But, works via intervalometer. Surprisingly, Auto ETTR with intervalometer works too, and histogram on preview. Sadly, it looks like silent full-res in video mode suffer from all the shortcomings: no long exposures due to video mode and no short exposures due to silent full-res limitations.
Yeah, this is precisely what I meant by "not what I'd call user friendly".
Long exposure: FPS override works to some extent, but it triggers after the video mode change, so it won't work with the relatively quick trigger from the silent module. Using mlv_lite + half-shutter trigger might be a better option; you get some sort of preview, too. Use a Lua script to generate periodic half-shutter triggers, something like this (not tested):
-- Half-shutter intervalometer
interval = 2000 -- ms
t_next = dryos.ms_clock -- will add "interval" later in the loop
while true do
key.press(KEY.HALFSHUTTER)
msleep(500) -- time to hold half-shutter pressed
key.press(KEY.UNPRESS_HALFSHUTTER)
-- wait for next interval
t_next = t_next + interval
while dryos.ms_clock < t_next do
msleep(10)
end
end
ETTR doesn't work while recording because... it's known to cause large jumps until it settles. It's not aware of recording triggers and other recent stuff. Canon's auto exposure might work; didn't try.
Ok, Thank you, finally, if ETTR lost anyway, I found simple working variant: Av, autoISO, mlv_lite, crop_rec, Full-res LiveView, FPS override. Works pretty well, except everything gets slow(and autoISO) the less FPS you choose.
Sorry if it was commented before.
When I'm in croprec in 1920_50p 3x3 mode and use SET button to enter in 5x to focus, all is ok but in the next SET press to go to 10x, liveview freezes and I'm forced to stwich off camera (5D3 1.1.3 with external monitor connected via hdmi) Last 22 july build.
Canon menu is in 50p mode and camera records images in 50p with no problem.
Somebody has the same problem?.
Thanks!!!
Similar behavior here with 1920 50p 3x3 on 5D3123. I don't use SET, just the default zoom button (magnifier glass). Did you remap yours? I can get to 10x, but it freezes when I hit zoom again to return to normal view. If I try to sneak out of zoom mode by hitting Menu, I get a black screen when returning to Live View.
QuoteDid you remap yours?
No, by default the SET button act as magnifier (in my case)...
I returned to a previous build and all is ok now.
the last on 5d3 have problems , read back the comments...
try setting half-shutter press to Clear Overlays and hold the half shutter as you do the zoom in, see if it bypasses the problem. In 60p 3x3 this works for me.
@Kharak
Thanks I will try!!!
Bonus Info: Holding half-shutter press for clear overlays while recording 50/60p/3.5k etc increases record times substantially.
I have been using the crop_rec (5D Mark III) for a project last year, and the short film is now in the festival circuit and has several official selections, so I am very grateful. There was however a problem using the crop, where there where lines or complete error segments once in a few seconds (see attached). I first thought it was the specific version of the crop_rec firmware, but now that I have the latest version installed (in the experiments folder) for a new film project, I have these errors again. I haven't tested it yet in the non-crop_rec version, but is this something common? I am probably doing something wrong. Thanks for any help.
BTW I have been using the 3.5K centered x5, 2.35:1, 3072 x 1308, and have the errors in 14bit lossless, and if I recall well, also in 12bit lossless and 10bit lossless.
(https://thumb.ibb.co/dAd5iA/Screen-Shot-2018-10-26-at-15-09-27.png) (https://ibb.co/dAd5iA)
Quote from: theBilalFakhouri on September 24, 2018, 10:30:34 PM
I noticed flickering with new crop_rec it becomes stronger when using high shutter speeds such as 1/500 1/1000 etc
700D, 1080p24, 1/4000:
FPS timer A = 528, B = 2527/2528
ADTG[8061]: 0x9d1/9d2 = 2513/2514
Locking just one of them in adtg_gui to some value introduces flicker.
Locking both of them gets rid of the flicker.
In adjust_shutter_blanking, current_blanking is going back and forth between two adjacent values; since our timer B is fixed in crop_rec, we may need to get rid of these variations for current_blanking, too.
Why is that? We need to get rid of variations in exposure time, i.e. timer B - blanking (where one unit = timer A / FPS clock). If you do the math, 14 vs 15 units of exposure time means ~ 0.1 EV of flicker. At large exposure times, this is no longer noticeable.
Draft patch for the bleeding edge crop_rec:
--- a/modules/crop_rec/crop_rec.c
+++ b/modules/crop_rec/crop_rec.c
@@ -558,4 +558,11 @@
int current_blanking = nrzi_decode(old);
+ static int previous_blanking = -1;
+ if (ABS(current_blanking - previous_blanking) == 1) {
+ current_blanking = previous_blanking;
+ } else {
+ previous_blanking = current_blanking;
+ }
+
int video_mode = get_video_mode_index();
Homework: point out the bug from the above code (yes, there is one).
Quote from: MartijnR on October 26, 2018, 03:20:43 PM
I have been using the crop_rec (5D Mark III) for a project last year, and the short film is now in the festival circuit and has several official selections, so I am very grateful. There was however a problem using the crop, where there where lines or complete error segments once in a few seconds (see attached). I first thought it was the specific version of the crop_rec firmware, but now that I have the latest version installed (in the experiments folder) for a new film project, I have these errors again. I haven't tested it yet in the non-crop_rec version, but is this something common? I am probably doing something wrong. Thanks for any help.
BTW I have been using the 3.5K centered x5, 2.35:1, 3072 x 1308, and have the errors in 14bit lossless, and if I recall well, also in 12bit lossless and 10bit lossless.
(https://thumb.ibb.co/dAd5iA/Screen-Shot-2018-10-26-at-15-09-27.png) (https://ibb.co/dAd5iA)
I've never used any of the lower lossless bit depths with 3.5k or 3k crop_rec and I've never had footage corrupt like that. Are you sure this also happened with 14 bit lossless 3.5k/3k?
I am on fir 1.2.3
Edit: how fo you process the MLV's? What program do you use?
Good stuff @a1ex
Quote from: a1ex on October 28, 2018, 10:24:21 PM
Homework: point out the bug from the above code (yes, there is one).
I have a homework also for you; there is a bug in the generic code when using other PowerSaveTiming registers that are not necessary for 700D, keep ADTG 8172 8178 82b6 and comment out the others ADTG.
I have used 1x3 Binning setting Y res to 2214 and Dual ISO 100/3200.
Just kidding :P what I am saying I am not a coder at least for now to do coding homeworks hahaha but the problem is there; It's some kind of flicker(Low contrast frame/High constrast frame something like that) I thought it was from shutter speed bug or aggressive 100/3200 Dual ISO but using same Dual ISO and without crop_rec the problem is gone. That's how I found the problem wasn't from Dual ISO and shutter speed bug (shutter speed was set to lowest value so no flicker was happening) then I tried to comment out other ADTG registers and the problem has been fixed.
I didn't search to find out which specific ADTG registers causing this but the point is not to use unnecessary registers for all cameras maybe that isn't good thing. For small cameras (700D, 650D, 100D and EOS M) the necessary ones is 8172, 8178 and 82b6. I don't have my cam and laptop right now posting examples next week.
Also after commenting out these ADTG registers some new focus pixels showed in 1x3 Binning mode.
Interesting; I've also noticed ADTG 8196 and 8197 have different values and poked them last night to look for side effects, but could not notice any. Will try to reproduce, just not sure what to look for.
Is it enough to comment out ADTG 8196 and 8197? 8173 and 8179 are required according to my theory, and should be noticeable at high vertical resolution and might also have an impact on sensor power consumption and temperature (I did not do any measurements; it's just what I think they might do). 82F8/F9 are only present on 5D3, so keeping them in the source would have no effect on other models.
Here's a (draft) graphical view of LiveView configuration for 700D, where you can see these outliers:
(https://a1ex.magiclantern.fm/bleeding-edge/lv-resolution/700D.png) (https://a1ex.magiclantern.fm/bleeding-edge/lv-resolution/700D-crop.png)
Same for 5D3:
(https://a1ex.magiclantern.fm/bleeding-edge/lv-resolution/5D3.png)
60D:
(https://a1ex.magiclantern.fm/bleeding-edge/lv-resolution/60D.png) (https://a1ex.magiclantern.fm/bleeding-edge/lv-resolution/60D-crop.png)
5D2:
(https://a1ex.magiclantern.fm/bleeding-edge/lv-resolution/5D2.png)
5D4 (from here (https://www.magiclantern.fm/forum/index.php?topic=17695.msg206730#msg206730)):
(https://a1ex.magiclantern.fm/bleeding-edge/lv-resolution/5D4-lv.png)
These graphs were created from io_trace_full logs; will prepare some test builds later for other models.
Hi Everyone,
when using the 3.5K centered mode (2640x1320), i cannot set the shutter speed above 1/140e is there something i'm missing?
i'm on 1.2.3 with ML from july using PAL 25P
Could not reproduce (1.1.3 with vanilla July 22 build).
With default settings, I could get speeds between 1/33 and 1/4300 in centered x5 zoom mode at 24/25/30p. With "full range" shutter speed enabled, I could select between 1/FPS (i.e.1/24 or 1/25 or 1/30) and 1/23000.
To get exactly 1/140, I had to use shutter fine-tuning. I couldn't get the "e" suffix; assuming typo.
Didn't measure these shutter speeds against a calibrated reference, so take the printed numbers with a grain of salt. I've only tried to match them with Canon readings (https://www.magiclantern.fm/forum/index.php?topic=16814.msg164062#msg164062) and made an educated guess from there.
Quote from: Kharak on October 28, 2018, 11:24:25 PM
I've never used any of the lower lossless bit depths with 3.5k or 3k crop_rec and I've never had footage corrupt like that. Are you sure this also happened with 14 bit lossless 3.5k/3k?
I am on fir 1.2.3
Edit: how fo you process the MLV's? What program do you use?
Hi Kharak,
Thanks for your comment. Yes, I am sure it happened on 14bit lossless. I have done a lot of testing to seek for the best image quality and recorded in different test settings. In the end I went for 14bit lossless because the quality seemed the best. However, I am now not so sure anymore if I actually recorded the way I wanted (crop in highest image quality setting). I am on firmware 1.1.3.
As for MLV process software I have used a lot, cr2hdr, Footage, MLV App and others, and they all show the same errors on the same timepoint in the clips. However, somehow I didn't, and still do not, seem to get the MLV Dump specifically for the crop running. Not in Terminal (Mac), and also not linked to other software. Maybe this is the problem.
I would be very thankful if you (or somebody else) could process the clip (macro of walnut) with the specific crop MLV Dump software in my dropboxlink. If you also have the errors around second 1 and second 4, then at least I know it is in the MLV files themselves, and not in the process software.
Also, I wonder if you can see if this small clip has indeed the high quality of the crop setting you can obtain (lens is otus (reversed), and lighting was fairly good (iso 1600 though), so the image quality should be high).
https://www.dropbox.com/sh/au5lkf1cfwq1qp7/AADdRbNh5jBK_s5zx4r8X6pva?dl=0
Thanks!
Could be related:
https://www.magiclantern.fm/forum/index.php?topic=22818.msg207853#msg207853
Might have found out where the FPS timer B overhead comes from, on 5D3:
(https://a1ex.magiclantern.fm/bleeding-edge/lv-resolution/5D3-fast.png)
Previously, I've noticed timer B overhead (i.e. difference between vertical resolution and minimum timer B that still gives clean image) was a bit hard to figure out; it wasn't a plain number that I could just hardcode in crop_rec. I could notice the overhead is larger in high-FPS modes and smaller at large resolutions (such as 4K half-FPS), but could not quantify it, so I had to tune each preset manually. Now I've confirmed a hypothesis I had since looking into 5D2: the overhead for timer B is, indeed, constant, but measured in milliseconds.
The red line shows the timer B limit found experimentally in the old FPS override code. Notice it matches the last red marker.
Between the red markers, EvfState executes its state transition functions; in particular, evfSetParamInterrupt and evfReadOutDoneInterrupt. During the last transition (last pair of red markers), Canon code sets up the hardware for capturing the next frame. This process is quite complex and... takes a significant chunk of execution time on the ARM processor (about 2.3 ms).
According to that graph, the 1080p60 can be easily pushed to 70 FPS without sacrificing resolution. Whether that can be recorded, I did not test.
If that software reconfiguration can be somehow optimized (e.g. by caching the register values configured by Canon code, and just replaying them for every frame), the 720p mode can - in theory - be pushed to about 83 FPS. Probably not worth the effort, as the lossless encoder is likely to be unable to keep up, the rolling shutter would be very close to 100%, and so on.
Our raw video "vsync" hook is placed right at the last red marker (it's executed right after Canon code does its evfReadOutDoneInterrupt transition, i.e. it extends the duration of that), so whatever we do there might also limit the maximum frame rate, to some extent.
Also notice the evfSetParam event sometimes starts to be processed *after* the HEAD3 interrupt (which signals the evfReadOutDoneInterrupt event). That might result in corrupted frames (unconfirmed hypothesis). Reducing the value of HEAD4 (which triggers evfSetParamInterrupt) *might* fix these corrupted frames (again, unconfirmed hypothesis).
Here's what happens if one reduces HEAD4 from 1320 to 1000 in 1080p24, timing-wise:
(https://a1ex.magiclantern.fm/bleeding-edge/lv-resolution/5D3-fast-h4.png)
In other words, it appears to help, but that's not the only cause of delayed frames.
I like this 8)
Hi, it possible to now run on the 5d2 record 3k video?
Quote from: MacMillan on November 06, 2018, 12:24:04 PM
Hi, it possible to now run on the 5d2 record 3k video?
Yes it is. https://www.magiclantern.fm/forum/index.php?topic=19336
Some more LiveView timings:
(https://a1ex.magiclantern.fm/bleeding-edge/lv-resolution/5D3-edmac0.png)
Quote from: DeafEyeJedi on April 13, 2017, 03:59:09 PM
Let's keep tinkering down on this one please. 1080p 48p is crucial. Especially for HDR video kind of work.
Based on the above picture, I'd suggest the following settings:
- current July 22 build from Experiments page
- crop_rec, 1080p45/1040p48 preset, 60 FPS in Canon menu
- Target YRES: dial 1080 there
- Delta HEAD4: -100 (value not critical; -50 or -200 should work just as well)
- Delta HEAD3: fine-tune between 0 and -20
- press MENU twice to apply the new settings
With DH4=-100 and DH3=-10 I've got clean clips at YRES=1100 (i.e. slightly higher than the desired target of 1080). You don't have to use 1100; the goal is stable recording, without corrupted frames, at 1080p48. I've tested at 1100 just to ensure a small safety margin.
Please check for corrupted frames in recorded videos and report back.
Are you able to get continuous recording at 1080 48fps?
I'd like to know what to expect before I begin the test.
Why not just do the test and see what you get @70MM13 8)
I got 1920x1080p 48fps working with a1ex proposed settings. How awesome is that!
No corruption with Delta Head3 set to -20. Started out at -14 and -10 but gave me corrupted frames and logs on the screen.
EDIT: Lovered registers directly in here and got it working as well:
/* HEAD3 timer */
/* 2E6 in 50p, 2B4 in 60p */
case 0xC0F0713C:
return 0x2B4 + YRES_DELTA + delta_head3;
/* HEAD4 timer */
/* 2B4 in 50p, 26D in 60p */
case 0xC0F07150:
return 0x26D + YRES_DELTA + delta_head4;
to:
/* HEAD3 timer */
/* 2E6 in 50p, 2B4 in 60p */
case 0xC0F0713C:
return 0x2A4 + YRES_DELTA + delta_head3;
/* HEAD4 timer */
/* 2B4 in 50p, 26D in 60p */
case 0xC0F07150:
return 0x22D + YRES_DELTA + delta_head4;
And of course:
[CROP_PRESET_3x3_1X_48p] = { 1290, 1290, 1290, 1080, 1040, 1320 }, /* 1080p45/48 */
to:
[CROP_PRESET_3x3_1X_48p] = { 1290, 1290, 1290, 1080, 1048, 1320 }, /* 1080p45/48 */
Nice!
It works for me also, starting at -17.
-16 gave a black frame, and a screen full of errors, but it kept recording anyway.
I only get about 1000 frames before it can't keep up.
I need Delta HEAD3 to be -30 - then only a dozen of first frames are "jumping" left-right, and record continues without artifacts. With any other value(-20, -17, -15, -10) - frames corruption, flickering, console messages about corrupted frames and compression errors. :(
@mothaibaphoto
What happens if you do changes straight in crop_rec.c code like here?
https://www.magiclantern.fm/forum/index.php?topic=19300.msg208153#msg208153
Same s**t:
ML ASSERT:
0
at mlv_lite.c:2723 (compress_task), task compress_task
lv:1 mode:3
compress_task stack: 1ae538 [1ae5c0-1ad5c0]
0x0006A030 @ ae1174:1ae568
0x00069868 @ 6a09c:1ae538
Magic Lantern version : Nightly.2018Oct09.5D3113
Mercurial changeset : 41153b1177ce+ (crop_rec_4k_mlv_snd) tip
Built on 2018-10-09 14:52:06 UTC by VOLD@VOLDMONSTER-II.
Free Memory : 211K + 2983K
By the way, your code example with error: 1048 instead of 1080 in preset. But your repo code is correct.
To minimize further confusion, I've decided to move Danne's experiments to a new topic (https://www.magiclantern.fm/forum/index.php?topic=23041.0).
As long as these builds contain mislabeled ISO settings, I am not going to support them, sorry.
I will erase the iso stuff when I get the time. They have no valid reason for being in there. Focus in this build should be 1x3 settings and playing with analog gain registers getting most out of the settings. Accepted build?
Bugs are still valid though. Mv720p stuck preview and lack of framing preview while using hdr or having adtg_gui module enabled.
Done:
https://www.magiclantern.fm/forum/index.php?topic=23041.msg208432#msg208432
Commit:
https://bitbucket.org/Dannephoto/magic-lantern/commits/4e18649356067e49e15134e432025acdb2b6f0a7
Would like to understand more about shutter blanking and see if we can fix a bug. In 100D when including following in consts.h:
#define FRAME_SHUTTER_BLANKING_ZOOM (*(uint16_t*)0x41697784) // ADTG register 805e
#define FRAME_SHUTTER_BLANKING_NOZOOM (*(uint16_t*)0x41697ac0) // ADTG register 8060
#define FRAME_SHUTTER_BLANKING_READ (lv_dispsize > 1 ? FRAME_SHUTTER_BLANKING_NOZOOM : FRAME_SHUTTER_BLANKING_ZOOM)
#define FRAME_SHUTTER_BLANKING_WRITE (lv_dispsize > 1 ? &FRAME_SHUTTER_BLANKING_ZOOM : &FRAME_SHUTTER_BLANKING_NOZOOM)
I can compile and use a build but shutter is always set to 1/24 on overlay screen. Does it indicate incorrect definitions or are there more places in code to defoine maybe? Thanks.
Yeah, these definitions appear to be dynamic on some models (on 700D it's also wrong). Back then, I've found them in some property blocks, so at least I should be able to find a fixed pointer to these addresses. Unable to look into it right now, sorry :(
Ok, thanks. I think I found the correct ones with adtg_gui when in photo mode! Seems to be working now.
#define FRAME_SHUTTER_BLANKING_ZOOM (*(uint16_t*)0x41697bc4) // ADTG register 805f
#define FRAME_SHUTTER_BLANKING_NOZOOM (*(uint16_t*)0x41697bc8) // ADTG register 8061
Yes, these are valid for you (or for your current startup sequence, or whatever causes these addresses to change), but they may be not be valid for all other 100D users. The previous ones were found in the same way.
O, I see, dynamic numbers. Well, well...
I tested the latest crop module last night on my 6D and I'm getting 2/3 of the image with a magenta color:
(https://i.imgur.com/gsb3huf.jpg)
I used mlv_lite and mlv_rec both with default settings. Both give the same results.
Any idea what could be happening?
Best guess: some sort of misconfiguration for ADTG powersaving registers.
Please define "latest crop module", mention the settings used for crop_rec (including resolution and frame rate in Canon menu) and the number of affected lines (the low-resolution screenshot doesn't help much).
Quote from: a1ex on November 27, 2018, 09:07:54 AM
Best guess: some sort of misconfiguration for ADTG powersaving registers.
Please define "latest crop module", mention the settings used for crop_rec (including resolution and frame rate in Canon menu) and the number of affected lines (the low-resolution screenshot doesn't help much).
I'm using magiclantern-crop_rec_4k.2018Jul22.6D116.
- Settings for crop_rec: default, I touched nothing
- Canon menu settings: 1280x720 @ 50fps, Low comp (ALL-I)
- Number of affected lines: 540px
Here's the full-res image: https://i.imgur.com/ZbvYcPq.png
Did this work any better in previous builds? If yes, can you find the last good build?
https://builds.magiclantern.fm/jenkins/view/Experiments/job/crop_rec_4k/
Quote from: a1ex on November 29, 2018, 12:05:24 AM
Did this work any better in previous builds? If yes, can you find the last good build?
https://builds.magiclantern.fm/jenkins/view/Experiments/job/crop_rec_4k/
This is the first time I'm using it. I'll test previous builds and see how that goes. Will report back after testing.
I tested a few builds and went back to the first build that supports the 6D (magiclantern-crop_rec_4k.2018Mar10.6D116.zip) and even that one doesn't work properly.
I also reseted the camera to factory settings before testing.
Quote from: Danne on November 17, 2018, 05:03:37 PM
Mv720p stuck preview and lack of framing preview while using hdr or having adtg_gui module enabled.
I assume this didn´t go unnoticed. Anyway. Would be great to get some insights to why mv720p and crop_rec gets stuck when trying to get out of x10 mode. Sorry my nagging here but 1920x1080 48fps setting is golden.
On the bugside let´s not forget this:
https://www.magiclantern.fm/forum/index.php?topic=22963.msg208129#msg208129
I also noticed that dualiso module will report the diso metadata into files even if dualiso isn´t set. Bug.
Latest Nightly Build: 2018-07-03
Experim. Latest Build (2018-07-22)
what happen's ?!? afraid....
@leozan The lastest build is good to go works perfectly on my 5D III.
to read lasts back post for nightly and more in exp. , seem not be all perfect.
but later many updates , now are months of silence.
I think we are in the near end or? :(
changed another place in power time registers:
- int fps_timer_b = (shamem_read(0xC0F06014) & 0xFFFF) + 1;
+ int fps_timer_b = (shamem_read(0xC0F06014) & 0xFFFF);
When fps override set this is giving corruption(pulsating minor stripes) to footage with certain presets.
commit:
https://bitbucket.org/Dannephoto/magic-lantern/commits/aa687c72d837ce56511e566db19941d7c5a18522
Is it needed? Not sure what it´s for so erased it for the time being.
Quote from: a1ex on August 29, 2018, 04:08:39 PM
Alright, please find the first proof of concept for crop_rec with arbitrary resolutions. 700D only; won't work on any other model yet.
Now that I've got crop_rec_4k working again on my camera (this commit (https://bitbucket.org/hudson/magic-lantern/commits/58d67357be15201ba9b4cd08c47628c3cede3f8d) broke it and this was the solution (https://bitbucket.org/hudson/magic-lantern/issues/2122/fps-override-eosm#comment-49628652) for my 700D) I thought I'd try this out. Didn't get very far, but it looks promising.
Quote from: a1ex on August 29, 2018, 04:08:39 PM
Beware: might be very buggy. After changing settings, press MENU twice to refresh. Have fun.
Yes it is! First few times I used it I got a warning, "Card's write protect switch is set to lock" and had to reformat the card to get rid of it. Tried to follow Reply #1839 (https://www.magiclantern.fm/forum/index.php?topic=19300.msg206040#msg206040) from @theBilalFakhouri but somehow wasn't getting the same results.
Yeah, I know--not much of a report.
An easy trick how to fix scrambled LiveView in higher horizontal resolutions :
"For real-time preview in higher resolution mods. Do you mean how to solve the scrambled broken LiveView?
Using new crop_rec I can get 1x1 mode in mv1080 without getting into x5 mode. Simply I can select 1x1 Binning and The sensor will be cropped to 1x1 1736x1160 .
The real-time preview isn't broken yet but If I increased the horizontal resolution to 2520 it will break, to fix it Now I can press magnification button and get into x5 mode and the LiveView will be working again.
Okay so we can get benefit from this trick to lock the registers between this two mods and find the right ones by trial and error and apply it in mv1080 then if the LiveView worked like x5 mode this mean we found the right ones and now we can play with them to fix LiveView in higher horizontal resolutions."
There is something weird going on with lossless and or crop_rec recording. (At leats on 6d that is :P )
With the higher resolution crop modes I always felt that I got less recording time then expected considering frames size and buffer memory.
But since we have lossless compression, frame size varies so I thought it was caused by the variation in compression.
But it still bothered me, so did some testing.
I have a crop mode setting which records in 5464x3072 at 5.6 FPS.
This resolution and frame rate setting is very heavy, lots of data, so the SD card doesn't have much time to get on speed and I think we're actually measuring used buffer size with this setting.
The buffer on the 6d is 255MB.
When I use plain 14 bit recording I get a video file of 235MB
When I use plain 10 bit recording I get a video file of 189MB
When I use lossless 14 bit recording I get a video file of 119MB
The only one I can explain with 255 buffer size in mind is plain 14 bit recording.
The 10 bit and 14 bit lossless video files are much smaller then buffer size, the 14 bit lossless is actually 2 times smaller then the buffer size ??? How can it be ?
The 14 bit file size of 232MB makes sense:
One frame in plain 14 bit is about 29MB, 255MB buffer divided by 29 = 8.8, since you want whole frames, this is rounded to 8 frames. 8 times 29MB = 232 MB which is exactly my video file size I get with normal 14 bit recording :D
The 10 bit file size of 189 is weird.
One 10 bit frame is about 21MB, 255MB buffer divided by 21 = 12 whole frames. 12 x 21MB = 252MB. But I get only 189MB ???
The 14 bit lossless is even more weird.
I extracted the MLV file with MLV dump to DNG files and let it recompress the frames. Frame size appears to be around 18MB. 255 buffer memory divided by 18MB = 14 whole frames and 252MB in video size. But I get only 119MB, that's half of what I expected ???
Another weird thing is that MLV_dump doesn't seem to extract all DNG's out of the file.
14 bit recording file of 232MB gives 6 DNG's of 29.4MB each and a audio file less then 0.5MB, which is one frame less then expected according to filesize.
10 bit recording file of 189MB gives 7 DNG's of 21MB each and a audio file less then 0.5MB, which are two frames less then expected according to filesize.
14 bit lossless file of 119MB gives 4 DNG's of 18.2MB each and a audio file less then 0.5MB, which are two frames less then expected according to filesize.
Conclusion:
With 14 bit lossless compression I get 4 frames, where I expect about 14 frames (according to buffer memory and frame size)
P.S. this is all done with te experimental 4K build off 22 June 2018 from the downloads page and a customised crop_rec module.
Starting to wonder, how are frames written to the buffer when lossless compression is used?
I would expect that frames are stored compressed in the buffer, but I'm in doubt if that really is the case? Are frames in the buffer memory compressed or uncompressed when lossless recording is selected ?
Sorry for a bit of an offtop but I've just recently found a few evenings to have a look at my MLV files recorded since May 2018, do some postprocessing and editing. These are the first "real" videos in my life and first experience with video editing ever. This forum was actually the place, which made me aware that recording with 24..25 fps matters, that a 180° shutter matteras, why is a vario-ND filter a good idea and all that basic stuff. I've never really felt the video thing is for me and focused on stills only. Now it is just about to change, mainly thanks to you!
Saying it shortly: the spendid quality of a final image, workflow (MLVApp!), all the tricks & hacks you have developed are amazing. It's stunning how far you've pushed the cameras, how much you are able to squeeze out of them, despite it all needs a hard reverse-engineer work. These are great, terrific achievements! I am truly amazed of your work and fully appreciate it!
Yes, we can't say that often enough! I have the same feeling of real deep gratitude for what other people are doing for me somewhere in the world... Most of the video aesthetics you see with the new cameras is not my thing. Only through Magic Lantern I get into a visual territory where it starts to be fun. What ARRI does with their cameras is a kind of a benchmark. They put the aesthetic image over tech specs and pixel counting. With the 5dIII ML raw I have a camera that feels like a cousin of the Arris:). We brought two movies to cinema and honestly I feel such a loyalty and love for MLraw that I prefer the compactness and the inconspicuousness over an ARRI - also for the future:).
Cheers & a lot of thanks!:)
Volker Schmidt
Okey,
when i record crop 3x3 it's all ok, but when i zoom X5 to get 3.5k the images lag à lot in the liveview and MLV app too ! I don't know why. I have the same settings as in the installation video.
My CF card is a Lexa Pro 64Gb 1066x UDMA7. Shoot with 5D mrk iii
A hint / wish of a happy! 5dIII ML raw user:
Canon's Zoomin button on the left side contradicts any logic. If once magic zoom/full screen - focus control would be possible via half press shutter, it would be a big deal!:)
The focus control via focus ring unfortunately means that the sharpness has to be changed in order to control it.
For those that are curious, I had decided to record this set-up of myself (Thanks for the tip @BouncyBall) going through varieties of modes while utilizing HDMI output to an external monitor (so you can see what works and what doesn't) as far as 5D3.123 is concerned when it comes to using latest experimental 4K_crop_rec stuff. ;)
That was really informative, but you should be speaking! I wanted to know your thoughts about what we're seeing...
What a treat to see no lag on the external monitor. I'm guessing the 5d3 doesn't support anything beyond 1080?
It would be interesting to see if it's possible to do accurate monitoring in 1080 of the 1*3 binning somehow?
Sorry if these are already answered, this is all new to me, but it definitely looks appealing!
It feels like a puzzle that has some pieces missing though :P
Hopefully not for long!
It's a sad side effect of the march of technology that so much potential is lost by instant obsolescence. Magic lantern and its excellent community is the rebel fleet...
thanks for the video!
Where can I find the special build of silent.mo that allows returning to liveview after full-res silent pictures? I looked on Dfort's downloads page but can't see anything related to it. Any help appreciated!
Thanks!
@Danne
Can you compile crop_rec_4k_mlv_snd_isogain_1x3_presets_2019Feb14 also for 5d3 123 ?
thanks lot !!!
Quote from: kamranjon on February 12, 2019, 04:06:37 PM
Where can I find the special build of silent.mo that allows returning to liveview after full-res silent pictures?
Are you using a 7D? That camera doesn't work on the crop_rec_4k branch so you are a bit off topic here. It should be working in the unified branch. This was the fix (https://bitbucket.org/hudson/magic-lantern/commits/c46ffb21e7a7bb2fab56f83f97b1fd121a8dfcd6?at=unified).
To all crop_rec_4k users:I'm currently working in extending and porting manual_lens_info (https://www.magiclantern.fm/forum/index.php?topic=18083.0) to crop_rec.
Made a PR here (https://bitbucket.org/hudson/magic-lantern/pull-requests/940/manual-lens-info-support-to-crop_rec_4k/diff) and uploaded a compiled version for all supported camera models at this download page (https://bitbucket.org/aprofiti/magic-lantern/downloads/).
I need your help with testing because my camera is not supported on this branch, please report you findings!
Some notes:
- It include ELNS block in metadata, so most editor may fail to open to MLV files.
@Ilia3101 and all others devs of MLV processing tools. May be a good chance to add support for it or make editor robust and skip unknown blocks - Verify metadata with mlv_dump compiled from the new branch
- Config.lua is different and use .lcf extension instead of .cfg.
Please verify if there are some conflicts with any existing script (api should be the same).
It now support multiple separate menus and config in the same script - mlv_lite was the biggest diff to check after merging in this branch, so check also for problems or missing features
Please test overal stability especially when recording raw videos!!!
Quote from: aprofiti on February 23, 2019, 06:11:37 PM
Some notes:
- It include ELNS block in metadata, so most editor may fail to open to MLV files.
@Ilia3101 and all others devs of MLV processing tools. May be a good chance to add support for it or make editor robust and skip unknown blocks
Done for MLVApp. But it is 100% untested. Need such a MLV file. Some frames are enough.
Quote from: DeafEyeJedi on February 09, 2019, 10:38:36 AM
For those that are curious, I had decided to record this set-up of myself (Thanks for the tip @BouncyBall) going through varieties of modes while utilizing HDMI output to an external monitor (so you can see what works and what doesn't) as far as 5D3.123 is concerned when it comes to using latest experimental 4K_crop_rec stuff. ;)
Thank you for sharing this is very interesting! I assume that there is no recording?
Also did you test the UHD mode?
If someone worked out how to hack the cf to a ssd this camera would be unstoppable!
Could someone please clarify if anyone has managed too pull off 3/3.3/3.5K capture with a properly framed (not 5x) HDMI output? I'm assuming not, but wanted to be certain. Any bit-rate. Cheers.
Quote from: masc on February 23, 2019, 07:31:43 PM
Done for MLVApp. But it is 100% untested. Need such a MLV file. Some frames are enough.
Here is a mlv file (https://ufile.io/1njb8) done with Full-Res Silent Pictures, with 2 different lenses selected from the script.
For your interest:
When printing lens name from LENS block in "Clip Information Windows", it's also printing the serial number at the end of the string; this was fixed in mlv_dump (https://bitbucket.org/aprofiti/magic-lantern/commits/4387bc272e400bd1472b90217eff5832eafba7d6?at=manual_lens_info) and it's because of the missing null terminator character of the string (it's optional as described in MLV specification and it's not inserted when strncpy a name (https://bitbucket.org/aprofiti/magic-lantern/commits/cc10a4324b69fc811e74cc9f2fc357f294c03b2e?at=manual_lens_info) with more than 31 chars, due to lens_info.lensName being truncated)
Thanks for the clip. I added some macro's to the code for easy testing via qDebug() function, if someone likes. Seems to work fine.
Quote from: aprofiti on February 26, 2019, 06:17:17 PM
For your interest:
When printing lens name from LENS block in "Clip Information Windows", it's also printing the serial number at the end of the string; this was fixed in mlv_dump (https://bitbucket.org/aprofiti/magic-lantern/commits/4387bc272e400bd1472b90217eff5832eafba7d6?at=manual_lens_info) and it's because of the missing null terminator character of the string (it's optional as described in MLV specification and it's not inserted when strncpy a name (https://bitbucket.org/aprofiti/magic-lantern/commits/cc10a4324b69fc811e74cc9f2fc357f294c03b2e?at=manual_lens_info) with more than 31 chars, due to lens_info.lensName being truncated)
:o This was never an issue in past, as I can remember. But okaaay. So what should be the correct name of your lens used? "Samyang 8mm f/3.5 UMC Fish-Eye C" (32 chars) or "Samyang 8mm f/3.5 UMC Fish-Eye " (31 chars)?
Edit: I think I got it fixed. Also for DNG export.
Quote from: masc on February 26, 2019, 07:51:43 PM
:o This was never an issue in past, as I can remember.
This issues was potentially already present (https://bitbucket.org/hudson/magic-lantern/commits/f58e8f343d5331583689c9935ff386e322aa6c4a) but not noticed before (it's related to the issue reported from JohanJ in this thread (https://www.magiclantern.fm/forum/index.php?topic=18083.msg177972#msg177972) but present in all other branches), because probabily none of the chipped Lenses have long name.
I have previously modified LENS block (https://bitbucket.org/aprofiti/magic-lantern/commits/cc10a4324b69fc811e74cc9f2fc357f294c03b2e?at=manual_lens_info) to safely add termination char as showed in previous links, but then g3gg0 suggested to don't use it as he prefer to not have mandatory "\0" in the string of a MLV block, so stick with this method (you can find it in all the other branches).
In practice: LENS block is without termination character with lens name >= 32 byte (otherwise should be copied from lens_info.lensName when assigning is done but
only with the a1ex's fix linked before applied to source code), while ELNS will have it as safety in mlv.c when building block due to snprintf() (https://bitbucket.org/hudson/magic-lantern/pull-requests/939/dynamic-length-elns-block/diff#Lmodules/mlv_rec/mlv.cT88).
Quote from: masc on February 26, 2019, 07:51:43 PM
So what should be the correct name of your lens used? "Samyang 8mm f/3.5 UMC Fish-Eye C" (32 chars) or "Samyang 8mm f/3.5 UMC Fish-Eye " (31 chars)?
It should be none of the above but "Samyang 8mm f/3.5 UMC Fish-Eye CS II", assuming you want to keep info showed in the same way (ie. not replicating some infos in separate textbox).
I would suggest you to retrieve lens name from ELNS block; well... always use it if available :)
It would contain the complete name or the exact name if less than 32 bytes length (corner case of lens with 31 chars + "/0" added from ML code) compared to the one in LENS.
Quote from: masc on February 26, 2019, 07:51:43 PM
Thanks for the clip.
Thank you for adding support to MLV App!
Quote from: aprofiti on February 26, 2019, 09:10:25 PM
Thank you for adding support to MLV App!
No prob. But now we should think about what we could do with that new info. Until now it is just there (in memory and code).
Do somebody know what can i do with this? I can play this in camera with mlv_play. But on PC it look like broken. Before i used raw2cdng. But it isnt work with 4k. I use 5dm3 123. SanDisk 128gb, 160mb\s. i tried use different settings for record. For this i used mvl_lite and crop_rec. Same results for everything.
(https://i.ibb.co/6Y0zX6H/cds.jpg) (https://ibb.co/6Y0zX6H)
Mind if you could share your original trimmed MLV file?
Have you tried MLV App for Windows yet @Koltoshow?
@DeafEyeJedi yea, thank - MLV App work fine. https://www.magiclantern.fm/forum/index.php?topic=20025.0
Sorry if slightly offtop here: :-X
I'm getting often pink / blue frames and tearing if I turn from PAL to NTSC and select 1280x720/60fps – then crop mode 1080p45/48 3x3. Info shows 48fps. I get running for about 10secs with the pink/tearing then stops and runs screen errors "warning double buffers...et al"Have to restart and batt-pull. This doesn't happen if I change back to PAL and select 1280x720/50fps.
I'm using Danne's build of crop_rec_4k here: https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_isogain_1x3_presets_2019Feb15.5D3123.zip (https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_isogain_1x3_presets_2019Feb15.5D3123.zip)
Is this expected behaviour?
This is what we call "corrupted frames". This may happen for EOS M for bigger resolutions or faster fps. Always make sure to double press canon menu button after having chosen your setting. For 46/48fps disable global draw for less corrupted frames.
@masc Thanks! :) – I'm on 5D3 1.2.3. Does the same fix apply?
For those who want to shoot Full HD+ 16:9 ratios (only) in crop_rec_4k build(s) – and are having trouble visualising what the various crop modes do to field of view. I know I do! :o
I made a graphic showing the 16:9 ratios I've been able to use so far (5D3 1.2.3). Please let me know errors or if I've missed something vital!
:)
(https://i.ibb.co/FqpjWWQ/ML-crop-non-crop-HD-16-9-ress.jpg) (https://ibb.co/FqpjWWQ)
I've lost myself in a sea of random resolutions and aspect ratios! ;D
Can anyone please describe the exact steps to set the following 'standard' HD resolutions? I'd be very grateful:
2K(QHD) 2560x1440 / 24fps /16:9 ***Edit I have achieved this now:
1) I select Crop Mode: 3K1:1
2) I go into RawVideo and select the res 2560x1440 by fine tuning with the joystick/jog wheel/top wheel
3) Don't use the Zoom/Magnify button!
4) Turn GD off, Sound off,
The Preview is pink scrambled noise – but hey! I now get Orange rec times 30s+**** :P
3K 3072x1728 / 24fps /16:9 *** This still eludes me...***
I seem to see them in the menu occasionally – then when I look again they seem to have changed themselves back to something different! – usually limited to 1320 vert res. *** This still happens occasionally ***
Many thanks.
5D3 1.2.3
Danne's crop_rec_4k_mlv_snd_isogain_1x3_presets_2019Feb15.5D3123 build
KB1066x 128gb CF card
Have updated the 16:9 graphic to show 2K(QHD).
ANY HOPE to see later long time update of :
ML Nightly Build and ML crop_rec_4k for 5D3 (123) ?
...with Latest Lua updates + fixes 2019-03-20 ?
no way to do alone , bad....sorry.
thanks
Please tell us in detail which of the problems you are struggling with requires a merge of crop_rec_4k and lua_fix because it cannot be handled by either version alone?
Ceterum censeo experimental build page esse delendam!
Quote from: Walter Schulz on March 20, 2019, 07:10:01 PM
Please tell us in detail which of the problems you are struggling with requires a merge of crop_rec_4k and lua_fix because it cannot be handled by either version alone?
Ceterum censeo experimental build page esse delendam!
e98cf27: api_test.lua: test for all items from the Overlay menu
(to make sure they can be safely turned on or off with menu.set)
TODO: also check their submenu options
4c6cde3: Waveform: fix crash when enabled outside ML menu
(e.g. from Lua, https://www.magiclantern.fm/forum/index.php?topic=23673)
Issue: initialization on first use + race condition.
abdd1e2: api_test.lua: on EOS M, the INFO button may get us out of LiveView
(fixes API test about ML overlay status)
085b79c: EOS M: fixed display_idle() outside LiveView
4aa1ba0: Zoom on HalfShutter: allow triggering when AF is enabled
(only works when AF is not actually triggered by half-shutter, *, AF-ON etc)
df8d088: Zoom tweaks: updated for recent models
- option to disable the "x1" zoom (i.e. to go directly into x5/x10)
- compatibility with touch-screen models (still rough)
- api_test.lua updated (need to disable these tweaks in order to pass the zoom test)
3a5bade: api_test.lua: skip checking for "Play mode actions"
(feature not present on EOS M; requires serious cleanup)
400d73f: EOS M: enabled rack focus and focus stacking
(they appear to work, at least with regular EF lenses)
0060b26: editor.lua: disabled debugging features
(FIXME: they don't seem to work)
252a4b7: EOS M, 100D: fix Q/SET short press outside ML menu
(ETTR trigger and many others)
3f5aa76: editor.lua: use the MENU key as primary method for triggering the menu
(Q is still used, but it's not working on all models)
b213791: EOS M: align long-press symbols for delete and Q/SET
(minor)
20628f1: EOS M, 100D: fix long-press symbol remaining visible after Q/SET
https://www.magiclantern.fm/forum/index.php?topic=14828.msg210032#msg210032
f40dacf: Arrow key shortcuts: minor cleanups
- removed bogus ARROW_MODE_TOGGLE_KEY definition on EOS M
- removed unused definitions on 650D and 1100D
- require ARROW_MODE_TOGGLE_KEY if and only if FEATURE_ARROW_SHORTCUTS is defined
9e54731: EOS M: fix PlayMain_handler stub
7000D, 650D, 100D: comments for PlayMain_handler / PlayMovieGuideApp_handler
7b7a9a3: Color scheme: fix conflict with fast zebras
684ff6b: Menu: fix typo regarding FEATURE_JUNKIE_MENU
(7469c34)
91c84fd: exit_play_qr_menu_mode: wait until mode switch is finished
(fixes GUI mode switching test in api_test.lua on EOS M, maybe others)
7256e17: EOS M: fix GUIMODE_ML_MENU to allow operation outside LiveView
(required for passing the Lua menu API tests outside LV)
sorry can't help you.
I don't know how update api_test.lua inside ML and recompile.
but seem most fox for EOS M I think....not 5d3, or meaby not...bhoooooo?!
the rest seem dead..
Thanks , no problem , WAS only a question, no afraid.... ;)
if you like try something more "new" for 5d3 , look here...take time... https://bitbucket.org/Dannephoto/magic-lantern/downloads/
Quoting dmilligan (https://www.magiclantern.fm/forum/index.php?topic=14779.msg143039#msg143039):
How about a 1000% refund of the purchase price of ML? I will pay that to you personally, let me know where to send the check.
FYI - 99% of my spare time (i.e. when I'm not at work and not with my family) already goes to ML. Lately I've been focusing on other areas (https://bitbucket.org/hudson/magic-lantern/commits/all?), such as new ports, or just researching and documenting Canon internals. I don't record video, and the pictures I've been taking for myself are sitting in the waiting queue for years (guess why). Honestly, I don't know how to do better. OK, let's say I could focus more on the video side, or on some particular camera model, but that will be at the expense of neglecting other areas of ML (which unfortunately have their own share of trolls). FYI, there are areas in ML that did not get their fair "round robin" time slice (i.e. my attention) for years.
If you think the progress is too slow, the
only way to speed it up is to join the development efforts. Trolling, like you are doing here, only serves to drive me away from this topic (can't speak for others, but for me it's not funny to read such things after an intensive debugging session, or after a full day at work). It's not the first time you are complaining for what you've got for free, so please take it as a warning. Be sure to check this guide (http://catb.org/~esr/faqs/smart-questions.html#not_losing) before replying.
Yeah, I could have fixed another bug in ML instead of writing this.
QuoteCeterum censeo experimental build page esse delendam!
OK, what to do with it? Let users build from source? Just include everything in mainline without testing? Other ideas?
Anyone who thinks they can do a better job is welcome to take over maintaining ML (release management, keeping stuff up to date, making sure things work on all supported models and so on), as it's already taking 110% of my time.
Quote from: a1ex on March 20, 2019, 08:54:53 PM
... Other ideas?
Anyone who thinks they can do a better job is welcome to take over maintaining ML (release management, keeping stuff up to date, making sure things work on all supported models and so on), as it's already taking 110% of my time.
while we are
offtopic and I am at 109% myself (just family and work not counting ML) I may just add my 2cents for those
supported models you mentioned earlier as a proud owner of unsupported models (got it?) ;D :P
As harsh as it may sound I would announce some kind of "code freeze":
- Merge as much as possible of those open PRs (don't forget mine)
- announce dropping support for Digic IV and older cameras (yes those oldie goldies like 7D, 5D2, 50D, 60D, 1x00D)
- release a "final" build before dropping support for those cameras
- Do some code cleanup afterwards (yes cleanup code so anyone wanting support for those cameras has to use a code freezed branch on btbucket / github or whatever). Just point them with links over there. Anyone's free to pick up work for it.
- focus on digic 5/6/7/8 and onwards do exactly the same for digic V announcing to drop support for it but keep 5D3 because it has been built to survive. It's THE DIGIC V camera to keep / pick up.
- Afterwards focus on digic 6/7/8
Nothing will be lost but official support will be simply discontinued to free up resources for future work on new cameras. I know it's a hard decision. You mention yourself being not able to handle more cameras yourself. I guess dust spreaded all over your cameras (i know what I am talking about) but you need to get rid off 'em in terms of developing / testing ML on 'em. Just use them as a photographer but don't develop / test them. I know it's hard. You have been spending thousands of countless hours with them but I think ML is nothing w/o you as the lead developer and if you don't make the first DIGIC 6 / 7 / 8 port no one else seems to be capable to do it but everyone is expecting it atm. Once accomplishing the first working digic 6/7/8 port others will be able to follow your work. I don't think "someone" will
suddenly appear and do the first DIGIC 6 / 7 / 8 port for ML. You have been asking others to pick up work for it for so long already. Maybe later "someone" could appear but I personally think that chances are much higher that most users will just switch away and use their smartphones or whatever tool to take their stills and 8k videos.
At least think about it. Something like the 5D4 would fit everyone's need for the next years.
Now let's return back to topic please :P
forgot to mention: On April 1st please, as usual ;D
:D well ... a little life on the topic ... ;)
(https://i.ibb.co/gm3XLWX/assertlog.jpg) (https://ibb.co/gm3XLWX)
Should I be taking notice of all these files on my SD (ML install) card – or simply deleting them?
Thanks.
If you can find a way to reproduce these errors, that would be even better.
The CRASH*.LOG files are errors coming from Canon firmware (i.e. ERR70), usually caused by ML. The others are from ML.
@a1ex Ok, thanks. So in this example, what information should I note, and how to reproduce? :
ML ASSERT:
0
at mlv_lite.c:2712 (compress_task), task compress_task
lv:1 mode:3
compress_task stack: 1b0940 [1b09d0-1af9d0]
0x0006A01C @ ac1e5c:1b0970
0x00069878 @ 6a084:1b0940
Magic Lantern version : Nightly.2019Feb15.5D3123
Mercurial changeset : d60f7852c937 (crop_rec_4k_mlv_snd_isogain_1x3_presets) tip
Built on 2019-02-15 15:16:20 UTC by
[email protected].
Free Memory : 211K + 2954K
If you look into the source code (changeset d60f7852c937, line 2712 in mlv_lite.c), it had trouble compressing a frame to LJ92 while recording.
I'd expect a corrupted frame in the clip you have recorded when that message appeared.
Should I be looking for source code changeset d60f7852c937, line 2712 on Danne's page – or the main ML page? Sorry to be a bitbucket noob...
Quote from: a1ex on April 01, 2019, 01:17:25 PM
If you look into the source code (changeset d60f7852c937, line 2712 in mlv_lite.c), it had trouble compressing a frame to LJ92 while recording.
I'd expect a corrupted frame in the clip you have recorded when that message appeared.
I have general question about corruped files. Is there a way to reduce these corrupted frames besides lowering resolution or fps, disabling global draw etc? Are there other "tweaks" on values, let´s say lj92 compression for instance which could affect outcome? Anything in mlv_lite.c or maybe something memory-related?
Another one is how much do gui code affect recording times and overhead on cpu? Disabling global draw has clearly an effect on recording times but there´s a lot of other things going on under the hood. So thinking one could go hardcore disabling almost everything but basic recording indicator but skip everything else to optimize in a maximized way.
Think I fixed an issue causing corrupted frames with the 48fps setting on the 5DIII:
https://bitbucket.org/Dannephoto/magic-lantern/commits/c7c0ebf6f6489347d8148b0779a79b1c4bb748ef
Downloads here:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/
Or compile yourself here:
https://bitbucket.org/Dannephoto/magic-lantern/branch/crop_rec_4k_mlv_snd_isogain_1x3_presets
Using crop_rec_4k to digitize movie film - test results
My movie digitization machine came off the back burner recently and was fitted with a hall sensor for reliable half press triggering (each 1/2 press captures 1 frame into an mlv file). Here are the long overdue test results. It works great! I have been running the projector at 10 fps without any blurred frames in the 3k crop mode at 2560 x 1920 14 bits lossless continuous recording on a 5D3 1.1.3. Magic Lantern crop_rec_4k is amazing software for this purpose.
https://drive.google.com/open?id=184kg_8hzHTJwUDwKXIsQlGjavjSka6dy (https://drive.google.com/open?id=184kg_8hzHTJwUDwKXIsQlGjavjSka6dy)
While the resolution may seem higher than needed for 16mm movie films, I have found that it's much easier to crop off the extra pixels in post rather than trying to make fine adjustments to the camera position during capture. Also, different movie cameras mask the image area differently with some movie cameras having image area running out to, and sometimes, beyond the film sprocket holes. I used a file on the film mask in my projector so I can record almost to the edge of the film.
a few raw frames which are h and v flipped
https://drive.google.com/open?id=1GLMwmbsmOTAPqvqUHhfwDQg2TI9QxLPz (https://drive.google.com/open?id=1GLMwmbsmOTAPqvqUHhfwDQg2TI9QxLPz)
Timing between ML the projector film advance is the key to avoiding blurred frames. Running a capture and then looking through a few thousand frames to check for frames with blurs was not efficient so I set up a test to capture the projector main shaft position when ML captures the frame from the buffer.
example blurred frame
https://drive.google.com/open?id=1k-NNk0ZmYOVdCzdq45So0YJ3f1K7sMsD (https://drive.google.com/open?id=1k-NNk0ZmYOVdCzdq45So0YJ3f1K7sMsD)
Here is a great video about how a projector works, it may help with understanding my test video below.
How a Film Projector Works by the engineerguy
https://youtu.be/En__V0oEJsU (https://youtu.be/En__V0oEJsU)
Running the projector at about 1/2 the Canon camera frame rate seems to be the most reliable to avoid catching the frame while the film is advancing. The film advances within about 1/8 of a turn of the main shaft. My projector shutter has been removed so I have the other 7/8 of a turn to capture the frame at a standstill. I timed the hall sensor activation so it activates just before the film advances since the half shutter activation captures a frame already in the camera buffer. Here's the video, hopefully, it makes sense!
https://vimeo.com/327622574 (https://vimeo.com/327622574)
I find it amazing that the half-press triggering at 20fps works. I ran a previous test in 3k crop mode at 2560 x 1920 14 bits lossless for 6000 frames (the exact number expected) but there were lots of blurred frames.
On one hand, the 3x3 pixel binning at 48fps looks very workable though the lens would need a lot more extension tubes which will complicate things. The more macro magnification the harder it is to set and maintain focus due to even more shallow depth of field.
Running at 10 FPS is more than enough for my project, that said, running at full projector speed of 18 to about 24fps would benefit other projects like this since the projector drive would not need to be re-worked, which is a time-consuming task.
If there are any setting that I could change to get the 3k crop mode at 2560 x 1920 14 bits lossless to run at a higher fps please make a suggestion. I have tried FPS override, it crashes if I change to higher frame rates.
I captured a film through the projector this weekend as an example. The mlv file was converted to an image sequence then globally graded and sharpened in Davinci Resolve. This reel was shot by my Grandfather during the 1969 United States Grand Prix at Watkins Glen.
https://vimeo.com/327447552 (https://vimeo.com/327447552)
Quote from: miket on April 02, 2019, 02:42:35 AM
I find it amazing that the half-press triggering at 20fps works. I ran a previous test in 3k crop mode at 2560 x 1920 14 bits lossless for 6000 frames (the exact number expected) but there were lots of blurred frames.
Nice! However, that's expected, as the half-shutter trigger simply selects the last captured frame in buffer, for saving. That's all it does. Image capture still runs at the LiveView frame rate, i.e. not in sync with half-shutter events.
I wanted to prepare a real half-shutter trigger that would actually start the exposure on each external event (that's called genlock, right?), but... debugging the M50 took way longer than I've initially expected.
Quote
If there are any setting that I could change to get the 3k crop mode at 2560 x 1920 14 bits lossless to run at a higher fps please make a suggestion.
That's technically possible; I "just" need to find the time to sit down and finish the bleeding edge crop_rec. Maybe Danne already has a preset for it (didn't try).
I'd expect the frame rate to go up to about 40 FPS in 2.5K according to my theory about FPS timers (3596->2560 => timer A 518->388), but for some reason I couldn't achieve that in practice, last time I tried.
Quote from: Danne on April 01, 2019, 06:23:17 PM
Think I fixed an issue causing corrupted frames with the 48fps setting on the 5DIII:
https://bitbucket.org/Dannephoto/magic-lantern/commits/c7c0ebf6f6489347d8148b0779a79b1c4bb748ef
@Danne. Thanks! 45fps PAL tested: seems fine (Trying NTSC 1280x720 60fps always results in crashes and super dark preview/shots with very pink highlights.) – with no crashes or corrupted frames so far. However, on this build and previous (15Feb2019) the 1920x1920 1:1 preset appears broken and only records straight mv1080. ?
Cheers.
I can´t reproduce your issues. Could others share if builds are working or not? Also uploaded two new version, small changes:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/
This version let´s you exit x10 zoom in 48fps mode but you´ll need to update with canon menu button for regs to reset. I noticed that Either lowering resolution before coming back to 48fps setting or lowering fps will get you out of the stuck liveview. Not sure how to temporary change fps while in x10 mode and switch back when in x1 mode but that would possibly solve this issue.
@Danne. Thanks.
Quote from: Danne on April 02, 2019, 01:44:24 PM
I can´t reproduce your issues. Could others share if builds are working or not? Also uploaded two new version, small changes:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/
...in x1 mode but that would possibly solve this issue.
Which issue? NTSC 60fps or 1920x1920 1:1 ?
There's no 1920x1920 preset among my presets. There's x3zoom(digital zoom).
@Danne. Sorry! Confused myself there. 1280x720 NTSC 60fps throws these errors:
(https://i.ibb.co/DMfmBCF/IMG-3046.jpg) (https://ibb.co/DMfmBCF)
@a1ex - Its a dream come true as it is, if you do find time to work on it I would love to give it a try!
QuoteI'd expect the frame rate to go up to about 40 FPS in 2.5K according to my theory about FPS timers (3596->2560 => timer A 518->388), but for some reason I couldn't achieve that in practice, last time I tried.
I'll do some testing and let you know.
Mike
@timbytheriver
What happens if you set Kill global draw to ON under RAW video menu and turn off audio? I suspect you really need to downgrade your firmware version to 1.1.3. Not the same power horse with later firmware. Also try reducing resolutin to something like 1728x972.
@Danne. Thanks for your help, but none of that works with 48fps / NTSC/60fps selected. Still get those code errors the instant I hit record. Interesting about v 1.1.3 ... But I use HDMI out ... :(
Gone back to 'Experiments page' 4k crop_rec build 22Jul2018.
Can't dial in any vert resolution above 1320 in 3K, 3.5K, or UHD. Tried all sorts of combinations. I'm sure I had 3072 x 1728 working at one time... :o
Any suggestions pls?
Is there a more stable build than 22Jul2018? I think I read somewhere on the forum that this build had problems.
Thanks.
@timbytheriver, did you try setting a higher resolution crop mode then exit and return to live view before returing to your preferred crop mode? Sometimes that works for me. Also try a 24 fps mode in the canon menu if you don't already have that set.
Thanks. Have tried that – and restored ML to defaults etc.
If I set say, UHD and set res to 2880x1536, it shows that, but when I record and playback, the info shows it has only recorded 2880x1320. It's like it's got itself locked to a max vert res of 1320!
Any help appreciated.
CF Card KB1066x (been fine...)
To answer my own question and for anyone else having this problem:
I downgraded my firmware to 1.1.3 (from 1.2.3) and installed Danne's 4K crop_rec build from 2/4/19 https://bitbucket.org/Dannephoto/magic-lantern/downloads/
I can now get resolutions up to 1536 vert so far. Getting dialled in resolutions to 'stick' after setting still seems to need a bit of dial-voodoo – press Canon MENU button a couple of times, press zoom (x5, x10) exit, or restart.
An observation: Sometimes it's not crystal clear whether a 'Crop Mode' absolutely requires the Zoom (x3) button to be set. Sometimes there's a help text saying so, others not. I'm sometimes confused until I playback as to what mode I'm exactly recording in. Is there a way to make this clearer/automatic?
***Is there a shutter speed limitation in x5 crop mode when shooting 3k? If I am at 1/1250th for example in HD non-crop mode and then I go into crop mode then the shutter speed will say 1/110 and it gets really bright. Is there a limit on how high the shutter speed can go in crop mode? I searched, but couldn't find the answer. Thanks!
***UPDATE: I was having this issue in the March 10th build. In the latest build I can use any shutter speed I want.
Before someone else will answer i can say that i suspect 3k is implemented as fullscale silent pictures modification and silent pictures are taken with very low shutter speeds only like 1/10 , you cant use 1/50 with them.You can read more about it in silent pics module thread.
Probably highest mode with free shutter speeds is 2.5k or anamorphic modes.
Before anyone will believe it's true, need to say thanks god NO, x5 crop mode has nothing to do with FRSP and it's exposure limitations.
If so, that will be completely unusable for any realtime video, only for timelapses.
x5 mode originally intended to LCD only, so it 30 fps despite your actual settings and exposures are different too.
Need to play with FPS/Exposure override to mutch x5 to non-crop.
Good day! Am I wrong or is this firmware in mlv recorded without sound? On my 650Д even have mlv sound menu although the module is included!
Do you mind telling us which "firmware" you are using? BTW: ML is not firmware ...
And which settings you use?
Modules you loaded?
http://www.catb.org/~esr/faqs/smart-questions.html#beprecise
Magic Lantern crop_rec_4k.2018Jul22.650D104
Camera : 650D
Firmware : 104
Changeset: c1e44b8e0183 (crop_rec_4k_mlv_snd) tip
Built on : 2018-07-22 13:12:39 by jenkins@nightly
___
Loaded modules:
adv_int
crop_rec
mlv_lite
mlv_play
mlv_rec
mlv_snd
pic_view
raw_twk
silent
____
When enabled RAW video (At use MLV_dump there is a blank audio file), And when RAW video (MLV) only the sequence DNG no audio file.
Don't load mlv_lite and mlv_rec at the same time. They will conflict with one another.
With the 650D and crop_rec_4k_mlv_snd try loading just mlv_lite and mlv_snd and report back.
Unfortunately check it is not possible! Downloaded from the website of newly unpacked set. when you turn on any modules after you restart the screen " failed to link modules "
updating Movie Tweaks -> Movie Loading
updating Movie Tweaks -> Time indicator
and tcc: error: undefined symbol 'lossless_decompress_raw'
Ask for advice, when supplied with the solution, deny it ...
Been taking a dive into shutter blanking to understand how to fix an issue. Whenever selecting x5 zoom all is read and showing fine shutter readings. Until I change timer b for preset reasons. For instance, setting fps to 24 instead of keeping native (around)29.997 when going into x5 zoom is cruicial for higher resolutions. Changing timer b in this case will fool shutter readings still thinking I´m in 29.997? At least that´s how I understand this particular issue. Looking into shutter blanking routines in crop_rec.c and fps-engio.c it´s all very, very hard to find out what numbers to work on. I found out it all ends up in lens.c and then gets presented onto the display but under the hood. Very hard.
Issue example:
shutter showing 1/50 staying coherent to canon presentation which also shows 1/50. Going into adtg gui pushing timer b so now we have 24 fps instead of 29.997. Shutter suddenly lands on 1/35. Canon still in 1/50, hm.
A bit confused here. Is magic lantern presenting the correct shutter but canon is fooled here by the reg change? Would be great if I could get confirmation to wether this is a bug or not.
Thank you!
EDIT: The same shutter discrepancy seems to occur in x1 mode after changing timer b with different shutter readings.
EDIT2: Also tested to undefine shutter blanking code(also erase stuff in crop_rec.c) and run some older routines and that way all shutter stays coherent. On the other hand I get other problems with some presets giving med flickery output already at at shutter 1/100 or 1/200.
As far as I know, the ML reported shutter time is the right one.
Although I must say that the shutter blanking register stuff is a big black box to me :P
Explanation how B timer affects shutter time.
You are in 29.997fps, 5x zoom and shutter time is 1/50th.
Now you mess with B timer to get 24 fps, as you know, this means literally that a smaller portion of the sensor is read out. Or to be more specific, less vertical lines.
Reading out less vertical lines is faster, so shutter speed changes, 24fps/29.997fps x 1/50 = 1/40th shutter speed.
Whooops :o
Edit:
Now you mess with B timer to get 24 fps, as you know, this means literally that a bigger portion of the sensor is read out. Or to be more specific, more vertical lines.
Reading out more vertical lines is slower, so shutter speed changes, 24fps/29.997fps x 1/50 = 1/40th shutter speed.
Canon is still reporting 1/50th, because it apparently doesn't know we are messing with B timer :P
The Shutter blanking register stuff in crop_rec works with offsets, the code needs to know which mode we are coming from (5x zoom or non zoom etc.) to know what the standard values of the registers are. And when the code knows in which (Canon) mode it is, and it knows what you have done with the registers, it can calculate the offset.
At least, that's my understanding of this black box.
Here comes the next question on that topic. In crop_rec preset I choose the preset mv1080p 1736x976 46/46fps. In the preset I manipulated timer A and B to start in 46fps but this time we´re in x1 mode. Shutter readings stays the same both for canon and magic lantern. By this logic something is not right here?
Back to x5 mode. Selecting a preset with fps starting at 24fps, namely my 2.5k 1:1 centered. Readings now show the newly calculated shutter different from canon. So a test increasing shutter speed to 1/500 in canon results in 1/98 in ml shutter report. Is that a correct shutter too? Increasing shutter from 1/500 here seems to have no effect so I guess roof is around 1/100 for x5 zoom? I shutter number scaling when increasing to 1/500 correct here?
Quote from: Danne on May 08, 2019, 11:46:09 AM
Here comes the next question on that topic. In crop_rec preset I choose the preset mv1080p 1736x976 46/46fps. In the preset I manipulated timer A and B to start in 46fps but this time we´re in x1 mode. Shutter readings stays the same both for canon and magic lantern. By this logic something is not right here?
Have to think about that one ???
Quote from: Danne on May 08, 2019, 11:46:09 AM
Back to x5 mode. Selecting a preset with fps starting at 24fps, namely my 2.5k 1:1 centered. Readings now show the newly calculated shutter different from canon. So a test increasing shutter speed to 1/500 in canon results in 1/98 in ml shutter report. Is that a correct shutter too? Increasing shutter from 1/500 here seems to have no effect so I guess roof is around 1/100 for x5 zoom? I shutter number scaling when increasing to 1/500 correct here?
This is happening with the 6d too when feeding wrong information to the shutter timing black box.
The build Dfort made, (merge in your build), doesn't have the fix for video mode.
The 6d behaves the same in that build, weird shutter time readings and not able to go higher then 1/200 1/500 or so.
Also, one of my presets didn't even came up in my build (no preview, all black), one of the higher full sensor width presets.
So, if you are able to feed the right information to the shutter blanking black box part of code, you are getting rewarded with normal shutter time behaviour, which is, ML and Canon are about the same and you can go up to 1/4000th( which ML reports as 1/4600th on the 6d if I remember correct).
When full shutter range option is activated, it can go as high as 1/15000th or so for shutter speed (that is exposure/shutter speed per horizontal line, you still get awful rolling shutter :P )
Ok, so if I erase the newly set timer B which is set to 24 in 2.5k 1:1 centered I now have native 29.997 set and now I can modify shutter to whatever. So the bug is happening with x5 zoom and code expects 29.997 or else we get punished with the bug? Hopefully we get word from a1ex here if it´s really a sensor_timing_table issue or if it´s a safeguard or something else to look for.
Is the issue in fps-engio.c? Seems like it.
In crop_rec.c:
/* from SENSOR_TIMING_TABLE (fps-engio.c) or FPS override submenu */
static int fps_main_clock = 0;
static int default_timerA[11]; /* 1080p 1080p 1080p 720p 720p zoom crop crop crop crop crop */
static int default_timerB[11]; /* 24p 25p 30p 50p 60p x5 24p 25p 30p 50p 60p */
static int default_fps_1k[11] = { 23976, 25000, 29970, 50000, 59940, 29970, 23976, 25000, 29970, 50000, 59940 };
hm...modifying these numbers have no effect.
Try altering the numbers in that table from 1080/30fps, the second column.
If I remember correct there was something wrong with the numbering of that table.
So put standard canon 5x zoom values in that table at the place where now numbers for 1080/30 are.
Testing:
/* from SENSOR_TIMING_TABLE (fps-engio.c) or FPS override submenu */
static int fps_main_clock = 0;
static int default_timerA[11]; /* 1080p 1080p 1080p 720p 720p zoom crop crop crop crop crop */
static int default_timerB[11] = { 1863, 1863, 1863, 1863, 1863, 1863, 1863, 1863, 1863, 1863, 1863 };
static int default_fps_1k[11] = { 23976, 25000, 29970, 50000, 59940, 29970, 23976, 25000, 29970, 50000, 59940 };
No effect.
Also trying:
/* from SENSOR_TIMING_TABLE (fps-engio.c) or FPS override submenu */
static int fps_main_clock = 0;
static int default_timerA[11]; /* 1080p 1080p 1080p 720p 720p zoom crop crop crop crop crop */
static int default_timerB[11] = { 0x727, 0x727, 0x727, 0x727, 0x727, 0x727, 0x727, 0x727, 0x727, 0x727, 0x727 };
static int default_fps_1k[11] = { 23976, 25000, 29970, 50000, 59940, 29970, 23976, 25000, 29970, 50000, 59940 };
No effect...
I guess something is reset somewhere else.
EDIT: I can modify fps from with crop_rec sub menu so lowering timer B so I get 30 fps shutter blanking routines start to work again so obviously some code break...
Looking closer at that table, looks weird :o
And mainclock = 0 ? Shouldn't that be 32000000 for eos-m ?
Try giving those variables one value each, the ones for 5x zoom.
Not near a computer right now, but Look in the 6d topic, I posted that table yesterday. I think this table is just not right.
Too many mysteries and assumptions for one day. A break.
Gottcha!
/* some modes may need adjustments to maintain exposure */
if (shutter_blanking)
{
/* FIXME: remove this kind of hardcoded conditions */
if ((crop_preset == CROP_PRESET_CENTER_Z && lv_dispsize != 1) ||
(crop_preset != CROP_PRESET_CENTER_Z && lv_dispsize == 1))
{
shutter_blanking = adjust_shutter_blanking(shutter_blanking);
}
}
For now I add all cams that is not a 5D3 as well in there:
/* some modes may need adjustments to maintain exposure */
if (shutter_blanking)
{
/* FIXME: remove this kind of hardcoded conditions */
if ((crop_preset == CROP_PRESET_CENTER_Z && lv_dispsize != 1) ||
(crop_preset != CROP_PRESET_CENTER_Z && lv_dispsize == 1) ||
(!is_5D3))
{
shutter_blanking = adjust_shutter_blanking(shutter_blanking);
}
}
Problem solved. Shutter stays coherent.
Quote from: Danne on May 08, 2019, 06:34:00 PM
Gottcha!
off topic: damn, this (https://youtu.be/Pt7kmByAPxI?t=40) came to mind when I read your "Gottcha!", haha. Dropped hard here. Good to see progress, though.
Quote from: g3gg0 on September 23, 2017, 05:43:19 PM
could you possibly genlock your steampunk device's motor to the camera?
so the camera's card LED blinks a msec whenever a MLV frame was captured, signalling the stepper should advance one frame?
then the projector runs as fast as the camera is recording...
I can do that in my DIY projector! How can I blink that LED? Which file (or some kind of hint where to look) must I modify to build a version that blinks the card LED?
how is 4K RAW continuous recording so far? is it achieved?
Quote from: dfort on May 06, 2019, 07:14:20 PM
Don't load mlv_lite and mlv_rec at the same time. They will conflict with one another.
With the 650D and crop_rec_4k_mlv_snd try loading just mlv_lite and mlv_snd and report back.
Does the same rule apply with 5d3 firm 123?
Are there any cropmarks I can use that shows the correct framing on my LCD while recording in 3.5K 1:1 centered x5 in 16:9 and 2.35:1?
Quote from: erek on May 19, 2019, 06:03:08 PM
how is 4K RAW continuous recording so far? is it achieved?
I wil test and let you know soon. I have bought a 5d amrk III but havent started testing as i dont have any cf cards with me atm
Worked on my 5D3 crop_rec.c code in this branch:
https://bitbucket.org/Dannephoto/magic-lantern/branch/crop_rec_4k_mlv_snd_isogain_1x3_presets
Commits:
https://bitbucket.org/Dannephoto/magic-lantern/commits/c89688d60f2c0da642c9460deb1cf1cf79553148
https://bitbucket.org/Dannephoto/magic-lantern/commits/0d200f7cb6d6158d83d38c9092e4bcdef208d76f
- Refined anamorphic presets so it now gives continuous:
1808x2300(2:35.1) = 5425 x 2300
1360x2300(16:9) = 4080 x 2300
- Also adressed the battery pull issue happening when in 48fps or mv720p 3x3 mode. Workaround allows for looping through x5,x10 and back into x1 mode.
- cine.lua included
- set_25fps in crop mode sub menu working with anamorphic mode
- bitdepth modes in crop mode sub menu working for all presets
- not included sd_uhs(don´t see why sd patching would be needed with mlv_lite anyway)
Downloads:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_isogain_1x3_presets_2019May23.5D3113.zip
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_mlv_snd_isogain_1x3_presets_2019May23.5D3123.zip
In terms of image quality and performance the anamorphic mode on this camera outruns anything coming from eosm, 100d etc...
@Danne
Just copied over only new crop_rec.mo (Is that ok?)
Anamorphic mode:
Getting slow saves. Cannot playback: 'Invalid RAW: A:/DCIM/100EOS5D/M23-1032.MLV
Any ideas?
Thanks!
Quote from: timbytheriver on May 23, 2019, 12:37:00 PM
@Danne
Just copied over only new crop_rec.mo (Is that ok?)
Why would that work? Use the complete build. There´s plenty going on at other places in code as well.
@Danne
Just checking! ;) Thanks, will try again.
Still the same errors with complete new build:
In Anamorphic: Sometimes Hi contrast Green / Black display
Getting slow saves. Cannot playback: 'Invalid RAW: A:/DCIM/100EOS5D/M23-1032.MLV
CF card reports FULL when not.
My CF card now isn't recognized by mac. 'Initialize / Eject'.
Reverted to old build, all fine expect CF card still not being recognized by mac.
Oddness! Any ideas?
Cheers.
Could you give exact settings? What firmware version?
Are you recording to SD card? CF card problem persists maybe from first test where you ran only the crop_rec.mo module.
Others tested yet?
Everything works here on 1.1.3.
Quote from: timbytheriver on May 23, 2019, 01:51:45 PM
In Anamorphic: Sometimes Hi contrast Green / Black display
this one is confirm i too get green black preview. image is clean though after recording
i was trying for some slowmotion video. i set to ntsc 30fps 1080p and increase the fps til 40 as it cant go beyond 40.
and tried shooting in 10bit. i get lots of pink/corrupted frames. i tried both with dannes's build and the build from the experimentel page
if i set to lossless i get error msg. i donkt know if it is known issue
here im sharing a mlv file https://ufile.io/6o30xfal
Greenish preview will occur when crop mode sub menu bitdepth is lowered. Expected because I lower the analog gain. Workaround.
Overriding to 40 fps in mv1080p will probably not work. That´s why you have the mv720p 3x3 preset. Or why not use the crop_rec 48 fps preset?
Ok. Reverted to old build – working.
Tried another card reader – which seems to work now:
Formatted CF card in Mac to ExFat / Master Boot Record scheme.
Re-installed clean version of today's build on SD card. Loaded minimal modules.
Set to straight 1080p. Pressing stop doesn't stop recording. Eventually stops itself (20second approx).
Try to playback file. Get a flash of a yellow question mark on screen, then "Cannot playback: 'Invalid RAW: A:/DCIM/100EOS5D/M23-1032.MLV'
Then CARD FULL message. (Even though it is fresh) Looking on the CF card in mac shows no .mlv saved.
I'm on 1.1.3 recording to CF.
:o
As I said. It works over here.
I still don´t know full settings set up so cannot try to reproduce. "Loaded minmal modules"? Set straight to 1080p? Maybe you have a faulty CF card. Sounds like it.
Quote from: Danne on May 23, 2019, 02:18:03 PM
Greenish preview will occur when crop mode sub menu bitdepth is lowered. Expected because I lower the analog gain. Workaround.
its greenish and black so nothing can be seen while recording
Quote from: Danne on May 23, 2019, 02:18:03 PM
Overriding to 40 fps in mv1080p will probably not work. That´s why you have the mv720p 3x3 preset. Or why not use the crop_rec 48 fps preset?
i tried fps overiding in 1080p because all other presets have problem with preview. not possible to shoot a moving object.
fps override in 1080p is pretty good with the correct framing and real time preview. it wil be fantastic if there is a way to get rid of those pinkcorrupted frams
Quote from: andy kh on May 23, 2019, 02:52:50 PM
its greenish and black so nothing can be seen while recording
i tried fps overiding in 1080p because all other presets have problem with preview. not possible to shoot a moving object.
fps override in 1080p is pretty good with the correct framing and real time preview. it wil be fantastic if there is a way to get rid of those pinkcorrupted frams
its a mistake
Quote from: Danne on May 23, 2019, 02:46:41 PM
I still don´t know full settings set up so cannot try to reproduce. "Loaded minmal modules"? Set straight to 1080p? Maybe you have a faulty CF card. Sounds like it.
@Danne
modules loaded: crop_rec, mlv_snd, mlv_lite, mlv_play
res: 1920x1080p /14bit lossless (not crop mode)
CF (KB1066x) card seems to be working/saving fine when I revert to a previous 4K_crop-rec Danne build 2/4/19 :)
Still odd...
Let´s continue here:
https://www.magiclantern.fm/forum/index.php?topic=23041.new#new
now it works im not sure what i had done in my first test. now i just clean install and turn on the anarmophic preset in crop rec and no more greenish black preview. i have a very slow 4gb cf card atm. wil test more as soon a i recieved another card
Just thinking here.
Everyone likes buzzwords, right?. So wouldn't it be cool to change some of the Crop_rec presets' names for the 5D3? Like 3x Crop mode is as close to Super 16 as it gets. (12,35x7,5) / (12,4x7) 36x24:3 = 12x8.
3.5k mode equals 1.65 crop. 35 mm Film is about 1.7'ish crop not to be confused with Super 35's 1.5 crop which ML also can achieve, but in not so favorable recording times.
With all the talk I see in youtube videos about emulating film or sticking to "true" cinema etc. I think adding some buzzwords could boost the popularity of ML further.
'Super 16 Mode' and '35 mm Film'?
I've been shooting a lot with a ultra wide vintage lens in "super 16" mode and the look is just fantastic, kinda feels like a little camera within the camera ;)
Anyways, just an idea I've been contemplating.
@Kharak I agree with idea behind this. Labelling presets with an established filmic title makes them more meaningful within the context, and easier to spot in the menu. The problem occurs not with the established film conventions such as Super 35, Super 16, 2.35:1, etc., but with the still developing plethora of digital conventions. Despite an extensive Wikipedia entry I see confusion (myself included) in many conversations as to what exactly UHD is Vs 4K, or 3K (which isn't really any fixed standard) or heaven help: QHD. Just sayin...
Got the mlv_lite lossless aspect of this working nicely on a 70D!
Out of the box the crop_rec_4k_mlv_snd branch didnt work - symptoms were soft-freezing when entering the ML menu video tab in video mode, and inability to start recording. After a copy+paste of these changes, it works perfectly: https://bitbucket.org/hudson/magic-lantern/commits/91593cabf6c9a97541b7f7de5f633b21ff75ad83?at=crop_rec_4k
As this is my first post I'd like to say absolutely excellent work to all involved! I will try to find some time to contribute properly
your opinion.... we can use a CF adapter (type II) > SD card ... to take advantage of the excellent fast now, even from 256 GB with prices significantly lower than 128Gb CF, compatible to recording continuos the ML raw HD 14bit (5d3)
like the Computerbay x1000..ecc...
maybe someone has already tried
Short story: Most CF/SD adapter are crap. Strongly recommend not to use this. We had lots of incompatibility issues and performance is ... well ... crap, too.
so ...no way to go out of the absurd price of the outdated and LIMITED CF ?!?
one step away from 2020 is not acceptable to pay a computerbay x1000 from only 128 gb..160 euro!
same price of 4 years ago !!! :'(
incredible there is not a perfect CF / SD adapter for the 5d3 ...
or at least a CF adapter> HD SSD even better .... with all the adapters and chinoiseries that exist !! >:(
Amazon.de: Komputerbay 128GB, 1066x for 79€.
Hone your google-fu ...
+sp 90 euro.... and no tested! mah...
and cf adapter ? must wait 2099 meaby...
Quote from: jimiz on June 21, 2019, 02:40:37 PM
+sp 90 euro.... and no tested!
Sorry, you lost me. What does "+sp 90" mean", what does "and no tested" mean?
Quote+sp 90 euro
Maybe donation for developer? 8)
TRUE!! computerbay x1000 and 1066 must be accurate tested with 14bit raw of ML canon , because they are squeezed at maximum write speed, and many don't match the declared speed, so you get errors and blocks
for this reason I ordered them from a shopkeeper who, before sending them, did not have amazon prices, they cost just under 160 euros, but they are totally reliable and without bad surprises.
eye with amazon that won't pay you back if they work for them!
and also 79 euro plus spedition (not free) ,,,they are not very few, since they should now cost 30-40 euros at most! 128 gb ... not 1280 !!!
I really don't get it. Komputerbay's 1066x cards are with us for some time and have been tested extensivly without failure (means: performance is consistent, nothing said about endurance)). Look here: https://www.magiclantern.fm/forum/index.php?topic=12630.0
There is no guarantee they won't change internals of course. So baselining your cards after purchasing is - IMO - mandatory (for all manufacturers and card types).
Disclaimer: No experience with 1000x. They tend to cost more with lower performance (compared to 1066x) so it makes no sense to me to purchase one of those.
ofcourse...but crazy there are not way/possibility to connect with a special CF adapter, fast SD or better SSD HD....
for working better (space) and price/gb ...with canon ML raw...
Quote from: jimiz on June 21, 2019, 08:27:32 PM
...with canon ML raw...
This has nothing to do with ML raw. It is just the adapters.
I say..."for recording with ML raw"...not is the cause!
a good adapter port CF , for connect a fast little SSD hd , was the best way for cut the problem of space recording....(20 minute in 128gb...) think on a 1Tb SSD.. and easy fast way also for edit (just ready)... cheap as well as dozens of CF cards
ok you need a supply adapter extra..but this is the minor problem.
my question was easy, in a world where in this sector you will find thousands and thousands of cards for connectors, adapters for everything and more ...
no one, did an adapter be used in this manner on CF canons? maybe will not cheap ... but don't tell me it's impossible. Such a pity.
dear jimi....summer talking :D...
your question (already treated in this forum even if died in nothing ...) pertinent and very shareable, in so many canon ML fans and not novices, they have already done it in 2013 (search google and you'll see) already then the problem of enormous convenience in terms of space / registration as well as economic are raised , to use harddisk instead of the "small" expensive cf to record in raw! and in addition to experiments with cables and wires, there are also flowers of the cf (pcmcia) card adapters where theoretically it was possible to connect small IDE hardisk ... but all the speeches died in nothing ... we are in 2019, at the gates of 2020, and you're absolutely right !! impossible that today with all the news, fast ssd, cards and chips of all kinds and modes, no one has finally churned out a suitable adapter to connect the small ssd finally allowing good use of the elite RAW canons, as well as in the canon compressed formats of course! ....
in fact one thing is very clear .. it is certainly not the CF connector and their internal circuits to the canon, the problem ... but only and exclusively external. that is, you need an adaptation that allows you to power and connect a SSD hard disk.
why nobody did it? probably because CANON itself never wanted to have it come, tapping its mouth and experiments on the spot ... pity that not even the nonconformists without Russian and Chinese rules, have not given us anything today. I believe that by now you can forget something like that, and of course ... that a cf128 gb still costs 80 or more euros ... it is a huge great shame as well as the price with which panasonic continues to sell P2 (and not mini!) ... the world is rotten for that too!
amen! also IF we do'nt like, we must look on another way (blackmagic) :'(
Quote from: jimiz on June 22, 2019, 12:35:15 PM
... but don't tell me it's impossible. Such a pity.
It is possible in terms of space... that's the crap you can buy. It is technically impossible in terms of speed. The adapter must convert between parallel and serial transmission. This will always be slower! And additionally it will never be faster than the limit of the CF interface in the camera. So why one should need this, when loosing resolution or bitdepth, just to record some more minutes?
Buy 2 cards and a little HD with CF interface (e.g. NextoDi Nd2901), so you can record until your 2 or 4 TB are full, with just some seconds break to change between the cards.
this can be a good help ... thanks for the info ..
Nexto DI NPS-10-CF
with a good ssd 1tb you can archive about 7 or little more CF of 128 gb ..for more of 2 hours/half
but if you have only one CF 128 and you are outside and you need 1 hour of more of recording and you have no stres to continuative recording, can be a good way.
question HOW much time need to transfer 128 gb inside the ssd ?
ND2901 saves 70-80MB/s, same speed like my 5D2 records. So a 64GB card is full/empty in about 15min.
Quote from: a1ex on October 28, 2018, 10:24:21 PM
Draft patch for the bleeding edge crop_rec:
--- a/modules/crop_rec/crop_rec.c
+++ b/modules/crop_rec/crop_rec.c
@@ -558,4 +558,11 @@
int current_blanking = nrzi_decode(old);
+ static int previous_blanking = -1;
+ if (ABS(current_blanking - previous_blanking) == 1) {
+ current_blanking = previous_blanking;
+ } else {
+ previous_blanking = current_blanking;
+ }
+
int video_mode = get_video_mode_index();
Homework: point out the bug from the above code (yes, there is one).
This patch works nicely, no flicker at all and no more of two values for shutter speeds toggling.
I couldn't find the bug you are talking about ?
@a1ex
I want to override C0F0 register in new crop_rec so I added these:
/* for Brighten LiveView in lower analog bit-depth */
static inline int brighten_liveview(uint32_t reg, uint32_t old_val)
{
switch (reg)
{
/* Brighten LiveView for lower bit depths */
case 0xC0F42744:
return 0x4040404;
}
return reg_override_common(reg, old_val);
}
/* for Brighten LiveView in lower analog bit-depth */
static void * get_engio_reg_override_func()
{
uint32_t (*reg_override_func)(uint32_t, uint32_t) =
(binning_mode == BINNING_1x3) ? brighten_liveview :
0 ;
return reg_override_func;
}
And the bottom part of this:
static void FAST engio_write_hook(uint32_t* regs, uint32_t* stack, uint32_t pc)
{
static int prev_zoom = 0;
static int prev_fpsm = 0;
int current_zoom = PathDriveMode->zoom;
int current_fpsm = PathDriveMode->fps_mode;
if (current_zoom != prev_zoom ||
current_fpsm != prev_fpsm)
{
/* video mode changed? */
fps_update_params();
cmos_update_params();
prev_zoom = current_zoom;
prev_fpsm = current_fpsm;
}
if (!is_supported_mode())
{
/* don't patch other video modes */
return;
}
for (uint32_t * buf = (uint32_t *) regs[0]; *buf != 0xFFFFFFFF; buf += 2)
{
uint32_t reg = *buf;
uint32_t old = *(buf+1);
int new = reg_override_generic(reg, old);
if (new)
{
//~ if (reg == 0xc0f06014 || reg == 0xc0f06008 || reg == 0xc0f06804)
//~ printf("[%x] %x: %x -> %x [z=%d]\n", regs[0], reg, old, new, PathDriveMode->zoom);
*(buf+1) = new;
}
}
/* for Brighten LiveView in lower analog bit-depth */
uint32_t (*reg_override_func)(uint32_t, uint32_t) =
get_engio_reg_override_func();
if (!reg_override_func)
{
return;
}
for (uint32_t * buf = (uint32_t *) regs[0]; *buf != 0xFFFFFFFF; buf += 2)
{
uint32_t reg = *buf;
uint32_t old = *(buf+1);
int new = reg_override_func(reg, old);
if (new)
{
*(buf+1) = new;
}
}
}
This affect 1x3 Binning mode; for brighten Live View because I am using Analog gain for lowering bit-depth, So everything is working properly until I change Y resolution the Screen goes Black and I can't enter ML menu only Canon menu, and after undo Y resolution changing, it goes normal, I think I should add something in the code above, because engino registers are conflicting when overriding it ? (The resolution ones Vs brighten live view register) something wrong happening in the buffer because of my code :P
Delete please.
Quote from: masc on June 22, 2019, 02:46:22 PM
It is possible in terms of space... that's the crap you can buy. It is technically impossible in terms of speed. The adapter must convert between parallel and serial transmission. This will always be slower! And additionally it will never be faster than the limit of the CF interface in the camera. So why one should need this, when loosing resolution or bitdepth, just to record some more minutes?
Amen.
KomputerBay has been dreadfully spot on over the years -- even their customer services are flawless!
Quote from: masc on June 22, 2019, 02:46:22 PM
Buy 2 cards and a little HD with CF interface (e.g. NextoDi Nd2901), so you can record until your 2 or 4 TB are full, with just some seconds break to change between the cards.
Straight up. Done deal. Great call, @masc! 8)
Sorry, I didn't read all the thread and don't know if you wrote about that before, but how about 3:1 or 1:3 binning for shooting 4k (2,35:1 for example) without stopping? The same way as Reddeercity did it for 5d mark II.
This is my shoot on 5d2 and old Nikkor lens using 4k with binning:
https://youtu.be/nM7hfO5dkcM
I think the image is pretty detailed.
PS: I don't have 5d3, just wrote this as a suggestion.
Quote from: Igor_Braun on September 08, 2019, 11:27:35 PM
Sorry, I didn't read all the thread and don't know if you wrote about that before, but how about 3:1 or 1:3 binning for shooting 4k (2,35:1 for example) without stopping? The same way as Reddeercity did it for 5d mark II.
This is my shoot on 5d2 and old Nikkor lens using 4k with binning:
There is no 4K binning (1x3) for 5D2 released. There is just 4K skipping (3x1).
Quote from: masc on September 09, 2019, 04:29:52 PM
There is no 4K binning (1x3) for 5D2 released. There is just 4K skipping (3x1).
@reddeercity There is so much potential in a 4k binning mode!
So at this point in development, is the 4K firmware pretty stable? I've been shooting RAW with the ML last nightly build and it has been pretty rock solid. I just haven't moved up the the 4K builds yet.
Quote from: unfilmde on October 31, 2019, 06:29:12 PM
So at this point in development, is the 4K firmware pretty stable? I've been shooting RAW with the ML last nightly build and it has been pretty rock solid. I just haven't moved up the the 4K builds yet.
Check the 1'st post on the 1'st page :)
Quote from: Igor_Braun on November 13, 2019, 05:01:28 PM
Check the 1'st post on the 1'st page :)
Doesn't really answer my question. For those who are using it. Is it pretty stable. I've been using the ML RAW version for several years and it has been rock solid. Is the 4K version there as far as stability? Can I switch over to the 4K version and stop using the 'regular' RAW version?
Quote from: unfilmde on November 14, 2019, 07:40:02 PM
Doesn't really answer my question. For those who are using it. Is it pretty stable.
its stable
Quote from: unfilmde on November 14, 2019, 07:40:02 PM
Can I switch over to the 4K version and stop using the 'regular' RAW version?
edit: i dont think you should stop using the regular RAW version i hope you mean nightly build. you can switch over from time to time however i have stopped using the nightly build
Thanks. I'm not using the nightly build I'm just using the Last full released ML RAW version. I'll give the 4K a try.
So you're using the 4K version full time?
I hope one of the friendly ( :D) video experts can throw a little education/insight my way.
Now that I have access to MLV App to experiment with non-ND Long Exposure still photography, I wish to see what (shutterless) FF (MLV wrapped) Silent can do.
Currently, I can take x cr2s and bring them into MLV App for processing down to an 'averaged' single DNG. Great.
I see the other route, and I hope the better one, is is to create an MLV FF silent.
I see the exposure as being anything between, say, a 1/2s to 10s, as an example.
Framerate is not that important, but frame to frame 'gap' should be minimised.
The 4K raw video recording; lossless compression experimental branch has a Full-resolution LiveView: 5796x3870 at 7.4 fps (128ms rolling shutter).
But, as some know, I find all this video stuff/talk a little confusing ::)
By question is, if I wish to create a FF silent MLV, that I can post process in MLV App, ie to simulate an LE image, what modules and settings should I best use?
Once I have proven to basic idea works, I intend to write an LE Lua script to allow LE (simulated) images, ie without having access to the correct ND. This approach should allow me to dial in an LE time I want, albeit at the cost of creating a (large) MLV for processing (which with card sizes these days is not an issue).
Bottom line: as the coding experts progress, I wish to start developing workflow and scripts in parallel. I hope I can get some insight, as I say, I do stills :D
...just to add to my last post.
'Randomly' playing about I seem to be able to take an MLV with full res, ie silent, but how do I fix the shutter to what I want, eg 1s.
I can't get longer than 1/60s.
:'( ::) ??? :o :(
Ok I think I'll give up for now.
Despite reading many posts, because I'm not use video, I can't seem to achieve what I'm after, namely:
- long shutter, say, 1s
- FPS override on to account for the shutter speed
- capture Full Res LV in an MLV wrapper
I don't even know if this is possible.
1) FPS override to 1 FPS, low light
2) reply #85, paragraph #2
Caveats:
- FPS override doesn't play nice with crop_rec, but the low light preset appears to work, to some extent
- camera is fairly unresponsive at this low frame rate - you need some patience
- the full-range shutter doesn't seem to work with FPS override, but can be tricked if you also enable expo override (set exposure time to 1/25 via expo override, and afterwards enable FPS override and crop_rec)
- if anything goes wrong, MENU twice to refresh LiveView.
Quote from: garry23 on November 17, 2019, 02:47:13 PM
...just to add to my last post.
There's an Edit button, available for this purpose ;)
@a1ex
I truly thank you for your reply/reminder.
I've tried playing with the settings (5D3) but get errors, eg card/memory related, plus can't work out how, say, to set the shutter I need, ie 0.5s shutter captured at 1FPS.
BTW do I need a super fast card if I'm only doing low FPS capture?
When I go into video mode, having put crop_rec to FLV, I can't set the shutter to anything longer than 1/60.
I need to spend some time trying to understand all the 'video stuff' ;)
Cheers
Garry
QuoteWhen I go into video mode, having put crop_rec to FLV, I can't set the shutter to anything longer than 1/60.
You need to read it from ML interface, not from Canon's; we don't change their numbers. Also, you might be in 720p50/60; while this should also work, it's generally a better idea to avoid extreme overrides, i.e. stick to 1080p if you don't need the higher frame rates.
In 1080p25 with FPS override to 1 second, I've got 0.5" mapped to 1/50. ML tries to remap the available shutter range in movie mode (1/33 ... 1/4000) to 1/FPS ... fastest_shutter (source (https://bitbucket.org/hudson/magic-lantern/src/91593cabf6c9a97541b7f7de5f633b21ff75ad83/modules/crop_rec/crop_rec.c#lines-692)), but - for some reason - it remapped 1/33 to 0.7" or 0.8" instead of 1" in my tests; likely caused by conflicts between crop_rec and FPS override. With FPS override turned off, shutter speed mapping appears to work as expected.
@a1ex
Once again, thanks for taking time to (re)educate me.
I've two developments underway at the moment.
I'm rewriting my Tilt Shift script's UI, but need to work out how to clear my drawing stuff each refresh, ie as I move lines around on the screen.
The hardest is to clear each line, ie with a transparent version, before drawing the line in the new location. Obviously I use the draw function.
If a use display.clear, the ML menus flash on and off, which surprised me.
I could also use display.rect, with a transparent colour and limited to my drawing areas.
The other project I have is the idea of getting an LE shutterless mlv workflow up and running.
Once again, thanks for your support.
Cheers
Garry
Ok education failing and my experimentation is driving me insane.
I've put my 5D3 in canon 1080 mode.
I set the shutter to 0.5s in ML.
I set FPS override to 1.
I switch on raw video MLV and touch nothing.
I switch crop mode to flv.
I wait for the screen to stabilise, showing a cropped pink image.
In video mode I press play and get the error message "failed to set up. Card/ram full.
The card is not full.
I'm still not getting it, ie how to record an flv mlv at, say, 1FPS and a shutter of 0.5s.
Any hints from a kind video-savvy person, to put this humble stills photographer out of his misery.
Cheers
Garry
Did you test what is actually working? Seems you are pushing the limits.
Try and find the full range option in crop rec and enable that. I don't movie mode can have lower shutter than fps override. Ie 12 fps equals 1/12 shutter.
@Danne
I think I've followed guidance from elsewhere.
As I say, my objective is to get an MLV FLV at, say, 1-2 FPS, with a shutter up to, say, 0.3-0.5 sec, according to FPS.
I did set FLV in crop ;)
Works here so just put up a screen recording of what you do so we can follow your steps more clearly.
Did you set shutter range to full range?
@Danne
Here are screen shots of my set up:
(https://pictshare.net/7vfrvv.jpg)
(https://pictshare.net/nf0joo.jpg)
And here are a couple of images of the screen taken with my iPad:
(https://pictshare.net/phhie2.jpg)
(https://pictshare.net/rfkj22.jpg)
(https://pictshare.net/ir0dxc.jpg)
I don't see anything wrong. I assume full range is set or else search for it.
You migh want to enable framing preview and read at the bottom when enabled.
Good luck.
@Danne
Thanks for the reassurance.
BTW, yes I had full range set.
Oh well.
@garry23 -- just to be sure we're all on the same page you are using @Danne's latest experimental build from his bitbucket page, correct?
No the one from the experimental download area !
Ah, please use @Danne's builds whenever you can. It's been doing wonders!
No matter how bleeding edge it seems, it's just as stable if not better. 8)
https://bitbucket.org/Dannephoto/magic-lantern/downloads/
@DeafEyeJedi
Many thanks for the heads up.
BTW do you recommend a 5D3 version from the download page.
Cheers
Garry
I did some tweaking to get pass some bugs. The more severe one sometimes happening around power timing registry.
Other than that some half finished presets like anamorphic and in higher fps. Didn't touch flv preset and I expect whatever issue you are having will be the same in my build but who knows...
@Danne
I did have a look, but things are just going down hill.
The build I downloaded didn't appear to have MLV 2, so I used MLV lite that comes with the build.
This time, I can't seem to change the shutter :-(
I'll give it a rest for a while.
Pity, I was really looking forward to nailing an MLV LE workflow.
Oh well, such is life.
BTW went back to the experimental build on the ML download page and, yes, still get the card/ram full error :(
It's the weekend and I'm going to have another go to see if I can get 4K FLV up and running on my 5D3.
I'm sure it's operator trouble, but I can't see where I'm going wrong or why I get the card/ram full error message, ie the card is not full.
Has anyone else seen this error message?
Cheers
Garry
Quote from: a1ex on November 18, 2019, 10:11:17 AM
2) reply #85, paragraph #2
Quote from: garry23 on April 04, 2017, 01:47:28 PM
In particular, how do I set up, run and process the full Res 'photo mode'. Once I know how to get things running, I can test.
Quote from: a1ex on April 04, 2017, 06:25:18 PM
Continuous full-res recording at 4-5 FPS:
- crop_rec.mo, mlv_lite.mo
- crop mode: full-res LV; raw recording: max resolution, lossless compression
- FPS override: 4 or 5 FPS, low light (important)
- press record
- mlv_dump foobar.mlv --dng
Quote from: garry23 on November 20, 2019, 06:37:34 PM
Here are screen shots of my set up:
When all else fails, try following the instructions...
If you are wondering, I did test this scenario before posting, starting from a fresh config of both Canon and ML (fresh installation, default settings). I didn't have to change anything else.
If these instructions are not clear, first post contains a video tutorial.
@a1ex
Have repeated your instructions and don't get the error :)
But end up with a cropped mlv capture at video res.
As I say, I'll keep playing around until I 'get it' ;)
Thanks for your continuing support.
I hope a video guru can help me out.
Assuming I set an override shutter as x sec, say, for example, 0.2s.
I'm shooting FLV with this crop_rec module and I wish to capture y sec of video, say 10s.
The question is, what FPS override should I use?
Should I add a few ms on to my shutter speed and work out a suitable FPS?
Obviously, in theory I could use an FPS of about 4-5 in the example above.
I would welcome any guidance.
Did you read a1ex answer ;)?
Anyway. If you don't touch fps override, does it work? Isn't anything working using flv? We need more contextual references.
Sorry, should have been clearer.
Working well :)
What I'm trying to do now is craft a Lua script that takes the long exposure time required as an input and sets up the crop_rec etc
The shutter will be set up for the scene.
So I'm trying to work out what settings are 'best' to minimise the frame frame gap.
I'm not worried about 180 degree shutter rules, I just wish to set up FPS override, based on the shutter.
As I say, if my shutter is 0.2s, what should I use as the FPS?
Five, or would it be best to lower it a bit.
Is there an algorithm that helps? That is takes the shutter, adds a 'bit' and thus leads you to the FPS override.
Cheers
Garry
Ok, so then just go from what's working and by trial and error see what works or not pushing limitations? It's all what looks good in the end is it not? It's not like anybody with certainty can tell you what to do here. Not me at least ;).
@Danne
Thanks.
I was just checking that there wasn't a 'right way'.
Let the experiments begin :)
Cheers
Garry
@Danne
Hope you can help.
I have made great progress and now have a script running that does what I want, ie sets up the FLV correctly, lets you check it and takes a FLV MLV of the length specified.
What I can't work out is how to direct the mlv capture to the second card in my 5D3 (the sd card).
I have it selected in the Canon menu, but I only get the mlv on card A.
I've looked through the ML menus, but can't see how to direct the recording to the second card.
Why would you?
I think these builds are recording raw files automatically to cf card if present and I think you are the first ever to ask for it not to. Well, I'm not that surprised ;).
I guess if you remove your cf card it will record your mlv to sd card. A workaround.
@Danne
I have ML on my CF card, a 64gb card.
I have a 256gb sd that I was hoping to record the larger MLFs to.
As you say the work around is to put ML on the sd card and not have a cf card loaded.
I'll look at that tomorrow.
Without overclocking 5D3's SD-card interface is limited to 25 MByte/s (theoretical) bandwidth.
@Walter Shultz
Damn! Just 'wasted' a few pounds on a 256Gb SD card ;)
Oh well, such is life.
BTW, looking at the numbers, I believe the sd card would be good for FLV at, say, 1-2 FPS, but not at the 'max' of 7 FPS.
Looks like I need to buy a cheap 256Gb CF card that will cope with the 7 FPS FLV needs I have, ie about 54Mb/s. Does anyone have any recommendation as I'm not seeking to max out the 4k recording, 'just' handle the FLV feature.
Cheers
Garry
Quote from: garry23 on November 27, 2019, 07:44:48 AM
Damn! Just 'wasted' a few pounds on a 256Gb SD card ;)
You can use it to record proxies. It's handy to preview the recorded material in a camera. Unfortunately, it doesn't work with all resolutions.
Quote from: garry23 on November 27, 2019, 07:44:48 AM
Damn! Just 'wasted' a few pounds on a 256Gb SD card ;)
Perhaps you can scoop up an EOSM and utilize this SD. Ha.
@DeafEyeJedi
Yes, no problem in using it.
BTW I already have three EOSMs, one IR converted.
Cheers
Garry
Oh, that's right. Thanks for the reminder, @garry23! :P
Noob 5D3 questions coming up ...
How important is having a working SD card slot for the 5D mark III in 3K/4K?
Does having magic lantern on the SD increase performance of the CF? Can you even install ML on the CF instead of SD?
Also does the feature where it uses both cards at once for increased write speed still exist?
Ok I did get 5D3 with broken SD slot. ML works great. Gonna replace it myself, the part is 25$ on aliexpress.
I have downloaded the newest crop_rec_4k build, which is from july 2018, and all the presets are great fun to test out, however I have noticed the UHD 1:1 mode is not centered vertically:
(https://i.ibb.co/s9HbHZ5/Screenshot-from-2019-12-07-19-10-48.png)
With swirly bokeh lenses this is quite noticable :D
Can anything be done about this? I tried the advanced settings in crop_rec, but only succeeded in making the image purple and glitched. I am ready to investigate, make changes in code and recompile if needed.
Nice Ilia3101!
Nice one @Ilia3101! 8) Please do share your experiment w the SD slot replacement when it comes. Exciting times are ahead for you!
Good catch on the UHD 1:1 mode not being centered. Have you tried using @Danne's recent update on his Bitbucket download page (https://bitbucket.org/Dannephoto/magic-lantern/downloads/%20page)yet?
yes, Ilia, definitely share your experience with your camera repair!
it would definitely be great to use both slots while filming to get more captured! long overdue! overclocking the slots would be nice too ;)
good luck with your camera!
I've been experimenting with different build sources for my Q-iso menu (AKA 'TNT' build) https://bitbucket.org/rivertim/magic-lantern-danneclone/downloads/ because I want to nail down my hunch that there are subtle exposure differences between modes – and builds itself. Have typically used: an old @a1ex build from Dec17, iso-research build, and various @Danne's with success.
Yesterday I tried the most recent crop_rec_4k from the 'Downloads' page on the ML site. 22Jul2018. There appear to be some issues with this build:
1) Laggy response shifting through main/sub menus.
2) Top half of LiveView often fills with pink 'mist' – looks like preamps settling (without any 'regs' enabled). This can suddenly fade. Or it can persist and record into the file. Haven't seen this before.
Anyone have any suggestions on which recent build from this branch is a better bet?
Many thanks.
PS Should I be looking to this page for clues? https://builds.magiclantern.fm/jenkins/view/Experiments/job/crop_rec_4k/changes
Is it posible to have 1920X804 60P in the RAW video mode, not the crop mode.
The 1.67 strech blurs the image a lot.
I use 5DIII 1.13 and magiclantern-crop_rec_4k.2018Jul22.5D3113 build by the way.
Is it possible to get ML-preview (half-press) to work on an external monitor and how?
Hi, Any updates to this module?
Quote from: Levas on September 13, 2018, 05:35:04 PM
Tell us, can you get 75 fps, 100? :D
Nope, but
74 FPS @ 1736x390 --> 1736x650 after 1.67 correction, of course on 700D :D
(https://i.ibb.co/Ht69NXw/74-FPS-on-700-D.png) (https://imgbb.com/)
The Timer:(Slowed Down from 74 FPS to 23.976 FPS)
https://www.youtube.com/watch?v=HR3vo1qh-mw
The settings are done in 720p60 , I turned off adtg_gui then adjusted the settings in photo mode, after that I turned on adtg_gui and went to video mode again, to avoid crashes when overriding the resolution
C0F06804 0x1a701d4
C0F06014 0x332
C0F0713C 0x1a8
I used mlv_rec instead of mlv_lite, the ML preview didn't work with mlv_lite when adtg_gui is loaded, memory error.
I noticed some hiccups in this preset, it might be fixable with C0F0713C, the LiveView is frozen but the camera didn't crash, I guess applying lower resolution and higher FPS at the same time will prevent the camera from crashing, the higher FPS is giving more data to LiveView to feed it to make up for the decreasing of data when lowering the resolution ?
I also managed to get working 78.607 FPS preset at 1728x186, I can get 80 FPS also at above settings but the camera is very slow at 80 FPS, can't start the recording, and if I applied 80 FPS in the working 78.607 FPS preset, as soon as the recording starts, the camera will crash.
Working 78.607 FPS presetC0F06804 0xd701d4
C0F06014 0x302
C0F0713C 0xd8
Note: 6D has DIGIC 5+ processor and higher Memory, 700D has slower DIGIC 5 processor with less memory, this might be the cause that 6D Can handle up to 96 FPS easier than 700D.
Didn't play harder with more higher FPS, also didn't try to decrease Horizontal resolution, I am not planning to fine-tune or get a higher stable FPS preset, since the Camera is Very Slow and these higher FPS mods unusable.
It's just Proof of Concept.
Have fun on other cams :D
77 FPS @ 1736x390 (1736x650) Works too, the camera crashes at some point,
The camera also becomes very slow when hitting 80 FPS, it freezes when trying to get above 80 FPS, I think the main reason is the camera is trying to process 80 FPS in liveview, it can't handle it, if we can limit the FPS preview in LiveView or disable LiveView completely maybe it will work a bit nicer.
Fortunately I was wrong, it seems applying the settings from adtg_gui make the camera crashes and will perform more corrupted frames and will make the camera slower, ported the last preset to crop_rec with a little adjustment to FPS timer B and we have stable:
80 FPS @ 1736x390 (1736x650) ;D
(https://i.ibb.co/H2WZwZ5/Canon-700-D-80-FPS.png) (https://imgbb.com/)
didn't have any crash yet on 700D
Yes I had crashes at some point, same preset works @ 92 FPS, there is a line on top of the frame, it might be fixable, good news there are no corrupted frames, but the camera crash faster at 92 FPS.
Quote from: a1ex on April 01, 2017, 11:15:41 AM
* Canon preview is broken in most modes; you need to use the grayscale preview in the raw recording module.
Not today:
Fixed broken LiveView in Higher resolutions:
https://youtu.be/DSSc-1arTjQ
My understanding is improving about broken horizontal resolutions, here is the one register that fixes it:
C0F38024 Its Value: 0x4530287, there is a factor between this register and resolution register which is C0F06804 = 0x4540298
You can show it using adtg_gui between e.g mv1080 and x5 mode.
You need to increase C0F38024 as you increase C0F06804 for resolution, e.g on 700D start increasing C0F06804 from 0x4540298 to 0x4540299 and set C0F38024 from 0x4530287 to 0x4530288.
Max value before C0F38024 freeze the LiveView is 0x45303c9 which can fix resolution of 3808x1080 (0x4540298 = 45403da) on 700D, there is a limit we should figure out where it comes from, I expect on 5D3 can fix more than this resolution.
Note: focus pixels showed in LiveView, also some barely visible lines.
@a1ex @Danne @Levas
There is an easy task to do :D
@theBilalFakhouri
The wizard. Well done!
Quote from: theBilalFakhouri on August 10, 2020, 05:56:27 AM
Max value before C0F38024 freeze the LiveView is 0x45303c9 which can fix resolution of 3808x1080 (0x4540298 = 45403da) on 700D, there is a limit we should figure out where it comes from, I expect on 5D3 can fix more than this resolution.
The limit comes from these Known Preview registers:
C0F383D4
C0F383DC
Its values change when moving preview box around in x5, these values allowed me to fix up to 3896x1080 resolution preview instead of 3808x1080:
C0F383D4 tweaked to 0x1b00af
C0F383DC tweaked to 0x3d401b7
Some values of these two registers won't allow you to fix high resolution preview like 3072x1080, it will limit C0F38024 range to fix more high resolutions, be aware for this.
Also it seems from all three registers above at some values I can
lower resolution in mv1080 and mv720 without frozen LiveView, not lowering it too much yet, but it does something :D
You my friend, are a genius.
How did you figure out to reduce these for real time preview in even higher resolutions?
C0F383D4 tweaked to 0x1b00af
C0F383DC tweaked to 0x3d401b7
Anyway. Applied cropped real time preview for all 3k modes on eosm so far. Looks like this in eosm code. Check the EngDrvOutLV
3k modes.
static inline uint32_t reg_override_3K_eosm(uint32_t reg, uint32_t old_val)
{
EngDrvOutLV(0xc0f383d4, 0x1b00af + reg_83d4);
EngDrvOutLV(0xc0f383dc, 0x3d401b7 + reg_83dc);
if (ratios == 0x0)
{
EngDrvOutLV(0xC0F38024, 0x4530307);
switch (reg)
{
case 0xC0F06804: return 0x5b90318 + reg_6804_width + (reg_6804_height << 16); // 3032x1436 x5 Mode;
case 0xC0F06014: return (get_halfshutter_pressed() && zoomaid && !RECORDING) ? 0x839: 0x83a + reg_6014;
case 0xC0F0713c: return 0x5b9 + reg_713c;
case 0xC0F06824: return 0x3ca;
case 0xC0F06828: return 0x3ca;
case 0xC0F0682C: return 0x3ca;
case 0xC0F06830: return 0x3ca;
case 0xC0F06010: return 0x34b + reg_6008;
case 0xC0F06008: return 0x34b034b + reg_6008 + (reg_6008 << 16);
case 0xC0F0600C: return 0x34b034b + reg_6008 + (reg_6008 << 16);
/* reset dummy reg in raw.c */
case 0xC0f0b13c: return 0xf;
}
}
if ((ratios == 0x1 || ratios == 0x2) && !set_25fps)
{
EngDrvOutLV(0xC0F38024, 0x45302ff);
switch (reg)
{
/* will change to 19fps for continous action */
case 0xC0F06804: return 0x5190310 + reg_6804_width + (reg_6804_height << 16);
case 0xC0F0713c: return 0x519 + reg_713c;
case 0xC0F07150: return 0x514 + reg_7150;
case 0xC0F06014: return 0x767 + reg_6014;
case 0xC0F06824: return 0x3ca;
case 0xC0F06828: return 0x3ca;
case 0xC0F0682C: return 0x3ca;
case 0xC0F06830: return 0x3ca;
case 0xC0F06010: return 0x34b + reg_6008;
case 0xC0F06008: return 0x34b034b + reg_6008 + (reg_6008 << 16);
case 0xC0F0600C: return 0x34b034b + reg_6008 + (reg_6008 << 16);
}
}
if (ratios == 0x1 && set_25fps)
{
EngDrvOutLV(0xC0F38024, 0x45302b5);
switch (reg)
{
/* will change to 24fps for continous action 2.39:1 */
case 0xC0F06804: return 0x49b02c6 + reg_6804_width + (reg_6804_height << 16);
case 0xC0F0713c: return 0x49b + reg_713c;
case 0xC0F07150: return 0x514 + reg_7150;
case 0xC0F06014: return 0x62c + reg_6014;
case 0xC0F06824: return 0x3ca;
case 0xC0F06828: return 0x3ca;
case 0xC0F0682C: return 0x3ca;
case 0xC0F06830: return 0x3ca;
case 0xC0F06010: return 0x34b + reg_6008;
case 0xC0F06008: return 0x34b034b + reg_6008 + (reg_6008 << 16);
case 0xC0F0600C: return 0x34b034b + reg_6008 + (reg_6008 << 16);
}
}
if (ratios == 0x2 && set_25fps)
{
EngDrvOutLV(0xC0F38024, 0x45302b5);
switch (reg)
{
/* will change to 24fps for continous action 2.35:1 */
case 0xC0F06804: return 0x49b02c6 + reg_6804_width + (reg_6804_height << 16);
case 0xC0F0713c: return 0x49b + reg_713c;
case 0xC0F07150: return 0x514 + reg_7150;
case 0xC0F06014: return 0x62c + reg_6014;
case 0xC0F06824: return 0x3ca;
case 0xC0F06828: return 0x3ca;
case 0xC0F0682C: return 0x3ca;
case 0xC0F06830: return 0x3ca;
case 0xC0F06010: return 0x34b + reg_6008;
case 0xC0F06008: return 0x34b034b + reg_6008 + (reg_6008 << 16);
case 0xC0F0600C: return 0x34b034b + reg_6008 + (reg_6008 << 16);
}
}
if (ratios == 0x3)
{
EngDrvOutLV(0xC0F38024, 0x45302ff);
switch (reg)
{
case 0xC0F06804: return 0x9d50310 + reg_6804_width + (reg_6804_height << 16);
case 0xC0F0713c: return 0x9d5 + reg_713c;
case 0xC0F06824: return 0x4ca;
case 0xC0F06828: return 0x4ca;
case 0xC0F0682C: return 0x4ca;
case 0xC0F06830: return 0x4ca;
case 0xC0F06014: return 0xbe3 + reg_6014;
case 0xC0F06010: return 0x36b + reg_6008;
case 0xC0F06008: return 0x36b036b + reg_6008 + (reg_6008 << 16);
case 0xC0F0600C: return 0x36b036b + reg_6008 + (reg_6008 << 16);
}
}
return reg_override_bits(reg, old_val);
}
4k/5k still awaits and of course. 5d3. In time...
Great news!
(YouTube video is unavailable.)
Can't wait to test it ^^ Very good hunting job !
Quote from: Danne on August 10, 2020, 04:17:38 PM
How did you figure out to reduce these for real time preview in even higher resolutions?
C0F383D4 tweaked to 0x1b00af
C0F383DC tweaked to 0x3d401b7
I couldn't reproduce the fixable cropped LiveView @ 3072x1080, I stuck with max fixable cropped LiveView with 2696x1080 resolution, I was pretty sure I didn't do something wrong, until I started playing with arrows while in x5 mode, noticed I can get fixable 3072 again, I looked for what registers has changed using adtg_gui and saw the above two guys :P
These values can fix up to 4034x1080 on 700D
C0F383D4 to 0x100bf
C0F383DC to 0x2e401c7
Quote from: Danne on August 10, 2020, 04:17:38 PM
Check the EngDrvOutLV
I tried to override the three registers in crop_rec.mo, but didn't make any effect, it didn't patch them, have you added
ENG_DRV_OUT_FUNC = 0xFF2C29E8; but for EOS M in crop_rec ? like in adtg_gui, Did you add a patch hook for that?
Could you share your code please
Quote from: domasa on August 10, 2020, 05:04:35 PM
(YouTube video is unavailable.)
Strange, it's working here, here is the link:
www.youtube.com/watch?v=DSSc-1arTjQ
Syntax in my post above:
EngDrvOutLV(0xc0f383d4, 0x1b00af);
EngDrvOutLV(0xc0f383dc, 0x3d401b7);
and example:
EngDrvOutLV(0xC0F38024, 0x4530307);
Nice stuff! Will test some more later.
Quote from: theBilalFakhouri on August 10, 2020, 06:49:35 PM
Strange, it's working here, here is the link:
www.youtube.com/watch?v=DSSc-1arTjQ
Link works.
Quote from: Danne on August 10, 2020, 07:02:50 PM
Syntax in my post above:
EngDrvOutLV(0xc0f383d4, 0x1b00af);
EngDrvOutLV(0xc0f383dc, 0x3d401b7);
and example:
EngDrvOutLV(0xC0F38024, 0x4530307);
Yes, I added these lines, still no effect:
static inline uint32_t reg_override_3K_700d_gwb(uint32_t reg, uint32_t old_val)
{
switch (reg)
{
/* raw resolution (end line/column) */
/* X: (3072+140)/8 + 0x17, adjusted for 3072 in raw_rec */
case 0xC0F06804: return 0x5340322; // 3072x1306 x5 Mode;
case 0xC0F06824: return 0x3ca;
case 0xC0F06828: return 0x3ca;
case 0xC0F0682C: return 0x3ca;
case 0xC0F06830: return 0x3ca;
case 0xC0F06010: return 0x35b;
case 0xC0F06008: return 0x35b035b;
case 0xC0F0600C: return 0x35b035b;
case 0xC0F06014: return 0x60f;
case 0xC0F0713c: return 0x535;
/* case 0xC0F383D4: return 0x1b00af;
case 0xC0F383DC: return 0x3d401b7;
case 0xC0F38024: return 0x4530311;*/
EngDrvOutLV(0xc0f383d4, 0x1b00af);
EngDrvOutLV(0xc0f383dc, 0x3d401b7);
EngDrvOutLV(0xC0F38024, 0x4530311);
}
return 0;
}
Is there something I am missing ?
Try adding them outside switch (reg) function.
This isn´t working right?
/* case 0xC0F383D4: return 0x1b00af;
case 0xC0F383DC: return 0x3d401b7;
case 0xC0F38024: return 0x4530311;*
Quote from: Danne on August 10, 2020, 07:11:46 PM
Try adding them outside switch (reg) function.
Nice, it worked, Thanks!
Quote from: Danne on August 10, 2020, 07:11:46 PM
This isn´t working right?
/* case 0xC0F383D4: return 0x1b00af;
case 0xC0F383DC: return 0x3d401b7;
case 0xC0F38024: return 0x4530311;*
Yes, this is right
Already had the liveview fix with C0F38024 register in the latest 6d builds.
But there is indeed a certain limit I couldn't pass.
On 6d I could fix liveview for 2880x1200, but couldn't get it to work on higher resolutions then that, like 3072x1152.
It also has a minimum limit for vertical resolution, if I want to fix liveview with presets below a vertical resolution something about 960, it also doens't work.
Didn't know about the C0F383D4 and C0F383DC, so maybe messing with those I can stretch it a little.
@theBilalFakhouri
For extreme high fps presets, it worked for me to set the following 4 register to value 1.
Liveview is gone when doing that, but the camera doesn't freeze (most of the times :P)
C0F08184
C0F08188
C0F08194
C0F08198
I use this messy code to have set these registers to value 1 for presets >60fps, it's put in the same block of code where you override resolution and A and B timer registers etc.
//720P_HighFPS_6D
static inline uint32_t reg_override_720P_HighFPS_6D(uint32_t reg, uint32_t old_val)
{
if (CropReadoutPresetChoice_720P > 5 )
{
F08184 = 0x295;
F08188 = 0x3be;
F08194 = 0x1ef;
F08198 = 0x3be;
}
else
{
F08184 = 0x0;
F08188 = 0x0;
F08194 = 0x0;
F08198 = 0x0;
}
switch (reg)
{
// fix for >60fps will destroy liveview
case 0xC0F08184: return 0x296 - F08184;
case 0xC0F08188: return 0x3bf - F08188;
case 0xC0F08194: return 0x1f0 - F08194;
case 0xC0F08198: return 0x3bf - F08198;
// case 0xC0F06804: return 0x5be02a0 - SelectableResolutionFix + (((((Horizontal_Res_720P/3) - 2560)/4)/4)*4) + ((((Vertical_Res_720P - 1440)/4)*4) << 16) ;
case 0xC0F06804: return 0x5be02a0 - SelectableResolutionFix + (((((Horizontal_Res_720P/3) - 2560)/4)/4)*4) + ((((Vertical_Res_720P - 1440)/4)*4) << 16) ;
case 0xC0F06824: return 0x2b0 + (((Horizontal_Res_720P/3) - 2560)/4);
case 0xC0F06828: return 0x2b0 + (((Horizontal_Res_720P/3) - 2560)/4);
case 0xC0F0682C: return 0x2b0 + (((Horizontal_Res_720P/3) - 2560)/4);
case 0xC0F06830: return 0x2b0 + (((Horizontal_Res_720P/3) - 2560)/4);
case 0xC0F06010: return 0x2b5 + (((Horizontal_Res_720P/3) - 2560)/4);
case 0xC0F06008: return 0x2b502b5 + (((Horizontal_Res_720P/3) - 2560)/4) + (((Horizontal_Res_720P - 2560)/4) << 16);/* for dec 690*/
case 0xC0F0600C: return 0x2b502b5 + (((Horizontal_Res_720P/3) - 2560)/4) + (((Horizontal_Res_720P - 2560)/4) << 16);
case 0xC0F06014: return 0x603 + (((Vertical_Res_720P - 1440)/4)*4) + SafetyMargin_720P; /* dec 1543*/
case 0xC0F0713c: return 0x5b8 + (((Vertical_Res_720P - 1440)/4)*4) + Head_3_timer_offset_720P; //5b8 works best for 5472
case 0xC0F07150: return 0x5b8 + (((Vertical_Res_720P - 1440)/4)*4) + Head_4_timer_offset_720P; //5b7 works best for 5472
}
return reg_override_bits(reg, old_val);
}
Quote from: Levas on August 10, 2020, 07:58:36 PM
Already had the liveview fix with C0F38024 register in the latest 6d builds.
Didn't know that before, good work
Quote from: Levas on August 10, 2020, 07:58:36 PM
but couldn't get it to work on higher resolutions then that, like 3072x1152.
Didn't know about the C0F383D4 and C0F383DC, so maybe messing with those I can stretch it a little.
Now you can!
Guess Whose Back!
https://youtu.be/1JiJMThS-Dk
Link:
https://youtu.be/1JiJMThS-DkDownload Guess Whose Back
crop_rec.mo , only for 700D. Edit: Deleted download link.
**(Edit 6-6-2021) You will find 3K Preset with fixed preview in this build:
https://www.magiclantern.fm/forum/index.php?topic=25784.msg231049#msg231049
**
Quote from: Levas on August 10, 2020, 08:06:06 PM
@theBilalFakhouri
For extreme high fps presets, it worked for me to set the following 4 register to value 1.
Liveview is gone when doing that, but the camera doesn't freeze (most of the times :P)
C0F08184
C0F08188
C0F08194
C0F08198
Yes I took a look on your old 96 FPS preset, and noticed you mentioned this thing, but the main problem the camera is crashing, I don't have enough time to start recording e.g. @ 96 FPS on 700D, the camera is slow and can't record immediately, and LiveView is frozen I will have a camera crash before the recording starts, I am trying to fix frozen LiveView to prevent the camera from crashing, then I will try to get more high FPS :D , not mentioned the write speed drops to 15 MB/s or 9 MB/s @ these high FPS mods :P .
Have 4k working up to 4040 now on eosm. I guess next level would be 5k. Gotta do some new pixel maps with slightly reduced width but that's quite ok considering what we get in return.
It is sure very interesting, if you search for someone I'm willing to test your next MKIII build ^^
There is a simple trick to semi-fix Canon preview @ 5K , valid for focusing:
Normal x5 Mode @ 2520x1080:
(https://i.ibb.co/KqBszXS/Normal-x5.jpg) (https://ibb.co/C749szD)
5K @ 5208x1080 Full-Width Before:
(https://i.ibb.co/KWqrhKT/5-K-Mode-5208x1080.jpg) (https://ibb.co/Rc0DbjR)
5k@ 5208x1080 Full-Width After:
(https://i.ibb.co/9cw3sM3/5-K-Mode-5208x1080-Semi-Fix.jpg) (https://ibb.co/tQZP8dP)
I called it Half-Way fix, we can get semi valid Canon preview image if we set C0F38024 to fix half of the desired resolution, e.g:
C0F38024 Original value in x5 (2520x1080) is: 0x453x287, I tweaked it to 0x453x293 to semi-fix 5208x1080 resolution, the tweaked value 0x453x293 should fix 2604x1080 resolution and give clean preview which is half width of 5208x1080, that's the trick we can make a semi-fix for higher resolution than 4K by finding a fix for the half of desired resolution, you maybe should tweak it a bit more or higher to find the sweet spot.
am using nightly build july2018 on my 5d3 it only record 2sec on 3k or 2k videos and also cant Fing the option which I can adjust the bit rate please can anyone help me
Any news for m50?
I am using the latest 1.2.3 experimental build for 3.5k / compression. When I select record indicator (REC Indicator) and select "remaining time" it is still showing elapsed time I believe. Is there a way to get an estimate on remaining card?
BTW I just used this for professional work (with an HDMI backup) and it worked flawlessly with 12 bit lossless. Thank you!!
Is there a build with 60+ FPS somewhere ?
Some GoPro camera clones, record up to 120FPS at 720p and its awesome.
I wish I had this option on ML :(
Quote from: Quentin on October 13, 2020, 11:16:40 AM
Is there a build with 60+ FPS somewhere ?
Some GoPro camera clones, record up to 120FPS at 720p and its awesome.
I wish I had this option on ML :(
It's up to Canon, not ML
Happy Holidays all! Is there any chance someone may have the copy of the August252020 Dannes' build here for download? I am trying to get a hold of it and I can't seem to locate the file. Any assisstance is appreciated. Thank you thank you!
In this thread it's possible to find builds and ask questions: https://www.magiclantern.fm/forum/index.php?topic=23041.msg233207;topicseen#msg233207
Recommended to use latest builds.
Quote from: hungrygeneraiton on September 06, 2020, 06:45:33 PM
I am using the latest 1.2.3 experimental build for 3.5k / compression. When I select record indicator (REC Indicator) and select "remaining time" it is still showing elapsed time I believe. Is there a way to get an estimate on remaining card?
I haven't been able to find any way to show "remaining time" on the 5Dmk3 with the latest 1.2.3 build. Nor "elapsed time".Back when I was using ML on the 7D it didn't work then either. I thought a few years after others have requested this simple feature multiple times surely it would be implemented, but no such luck?
No matter which setting is selected under the "REC indicator" in the Movie menu, it doesn't work. "Elapsed time" and "Remaining time" don't show either of those things.
"Avg Bitrate" and "Instant Bitrate" also don't seem to have any effect.
The bitrate (which is far less practically useful to creators) appears to be the only possible default option with the recording indicator. Whether it's "instant" or "average" I can not say. Maybe someone would like to see this in a debug mode for troubleshooting, but as long as the card status is "green" while recording, as a film maker I don't need the bitrate taking up my screen real estate. But I definitely need to know how much (approximately) recording time is left on the card, before messing up a critical long take because the card ran out of room.
The only indicator available is
total GB used on the card, in the upper righthand corner
BUT it DOES NOT UPDATE unless the whole camera is rebooted. This is very problematic for film work."Time Remaining on Card" feature has been requested multiple times, going back to 2013. Is there some reason why "time remaining" presents a particular challenge for the ML team, or do the developers just not realize how important this is for indie film makers, and therefore it hasn't been a priority?
My cast is ready to strangle me with how many times I've had to say "....OOPS, CARD RAN OUT..." in the middle of a great take. It's very demoralizing to performers to have equipment failures get in the way of performances.
That said, I don't want to sound like I'm complaining. As a tiny no-budget indie film producer, I don't take what you've done for us lightly. We are beyond grateful for all the work you've put into ML - and I hope this feedback is helpful in making ML even more amazing for the creators it was designed to empower.
Cheers to you all,
~Aria
That's the kind of post making devs think about dropping out from the project
It's more complicated than that. It all depends on what resolution, what bit, if raw or h264... with or without sound... all of that use different amount of space. It's not possible to quantify it as in "time".
Plus ML is based on pushing the limits of memory writing.
For example with experience I know more or less that 30 sec of 3.5K in 14 bits loss less is about 1GB.
You have to think in "space available "not in "time" and simply buy more stock of cards... I have 3 CF (2x64 GB and 128 go) and a 128 GB SD for card split record.
@Walter Schulz: correct. And there are not many left.
@ariaelf: to what devs do you write? Nobody is working on these parts of ML since a longer time. But you could be the one... surely you can implement "this simple feature" soon.
And if it is so problematic: you should use something else for production work.
"The only indicator available is total GB used on the card, in the upper righthand corner BUT it DOES NOT UPDATE unless the whole camera is rebooted.This is very problematic for film work."
Buy a larger size CF and reboot the camera to check remaning free GB when you suspect almost full card.
I am surprised at the extremely cranky replies given that my last paragraph was nothing but praise for ML and the hard work of the development team.
@masc I am offering feedback from the front lines - this is essential to creating anything great. If you can't handle honest feedback from actual users, then don't make anything.
Telling me to do it myself is childish. I am an indie artist, not a programmer.
ML was designed to empower "no-budget" artists and creators just like myself who are getting started, and don't have the $$$ for high end professional film cameras. The beauty of ML was that it can do so much of what far more expensive cameras can do, while costing nothing. But this is one of those critical features that makes it highly impractical for production work.
Wouldn't it be better to fix it, so that creators COULD use it on their productions without driving everyone crazy?
If I had to do it over again, maybe I would have gone a different route, but switching cameras and workflows mid-production isn't really an option either. When I started, I had no idea the features which were LISTED IN THE MENU simply didn't work!
Why list them as features in the recording indicator menu if they don't exist or are impossible to implement? ML should be transparent about what it can and can not do. You have no idea how many hours of my life I have wasted searching these forums for an answer as to why the listed features didn't work, and wondering if I was the only one with this problem.
ML was created by Trammel Hudson because he was unpleased with his 5D2's options.
Now be quiet, troll!
Quote from: Walter Schulz on January 08, 2021, 11:49:55 AM
That's the kind of post making devs think about dropping out from the project
If my respectful, honest feedback about a broken, miss-advertised feature
which included heaps of praise and appreciation makes devs want to quit, then they
probably should find something else to do.
Why list something as a feature if it doesn't work? I spent countless hours looking for information and answers on these forums, wondering if I somehow had the wrong settings or a bad build... Everytime the exact same question came up by a user on these forums, it was ignored and skipped over. Shame on you guys... just remove the non-existent feature from the menu, and be honest that it doesn't exist instead of explaining how something works in the help documents which in actual fact does NOT work for anyone! What is the point of pretending it's "a thing" if it isn't??
And a second one for the twit filter.
Quote from: ariaelf on January 08, 2021, 11:13:17 PM
@masc I am offering feedback from the front lines - this is essential to creating anything great.
Where´s your portfolio? Please share all your work for us to see.
Rest assure that if something was easy work it would have been in the ml builds already. Even if it´s hard work it would be.
Also:
http://www.catb.org/~esr/faqs/smart-questions.html
Quote from: wib on January 08, 2021, 11:56:57 AM
It's more complicated than that. It all depends on what resolution, what bit, if raw or h264... with or without sound... all of that use different amount of space. It's not possible to quantify it as in "time".
Yes, I understand that it would be very difficult to make it exact, but users have been asking for even a rough ballpark (plus or minus a minute would be totally fine for my workflow) since 2014.
I know that a 64G card will hold roughly 12 minutes of footage for me. Just having a simple time counter on the display keep track of how many minutes/seconds of
actual recorded takes on the camera so far would be a game-changer. Instead I see how many seconds long each individual take is while it's recording, but then the information disappears. I'm asking for those numbers to be added together and a running total be visible for the user.
There is a lot that happens between takes and it's impossible for me to try to keep track in my head whether 30 short takes in we're close to the limit of the card or not, whether the recording was accidentally left to run after saying "cut" or whatever. No one is going to stand there with a stopwatch to keep track of how many minutes and seconds have added up between each "action" and "cut" so that we don't screw up a sequence when the card finally runs out.
Therefore a ballpark timer of live recording time elapsed on the ML menu would be extremely helpful, so that takes aren't potentially ruined when the card is near capacity.
If no one wants to work on this, or it's for some reason impossibly difficult, at the very least,
remove everything in the recording indicator menu and the help documents that doesn't actually exist as a real feature, so creators are not mislead about what ML can actually do. "Elapsed time" "Remaining Time" flat out don't work, but instead of admitting that fact, I'm being called "a troll" by devs.
There's no excuse for this. Grow up. Be honest. Fix mistakes. Listen to users. Move forward. ML was an amazing community when I first started using it, it's sad to see this is the current state of affairs.
Quote from: Danne on January 08, 2021, 11:24:52 PM
Where´s your portfolio? Please share all your work for us to see.
Rest assure that if something was easy work it would have been in the ml builds already. Even if it´s hard work it would be.
Also:
http://www.catb.org/~esr/faqs/smart-questions.html
I'm perfectly happy to share my work, but it would be off topic in this thread.
We've done incredible things in spite of the limitations of hacked ML cameras, learned lots along the way, and no doubt captured an image quality that would have been impossible without it. Do I need to restate this every time for the delicate egos? I had no idea that respectfully pointing out a critical flaw would cause such a defensive, emotional backlash from the team. This isn't personal. It's rational feedback from a film maker.
Chill. Yes, I understand it "isn't easy". So then
remove the fake feature from the menu and stop pretending it exists when it never has. Or is that also impossible to implement??
What fake? It works for H.264.
The way you describe and command your selfish wishes related to your work has nothing to do with what is going on here. Even if that is your strong belief.
Open source initiatives here going on for years with nothing expected in return but maybe respect and some gratitude. Please read the link I provided. It explains most, if not all, what to expect in return.
And no thanks. No pm. Create a forum post and present your work there. I´d rather look at indie "greatness" than troll around.
Edit: Also consider taking the time to "clean" up all inappropriate info you can find by testing each and every feature systematically. Write down your findings and create a list of changes you would suggest for the community. This would be nice and effective user feedback probably useful for a lot of users. This place is in need of users digging in more actively.
Broken features have nothing to do with "my work", or it's relative "greatness" either way. And unlike straight answers to users questions about the remaining time and elapsed time functions, my work is easy enough to find online with a simple search.
Fact are facts. If you can't bear to hear rational feedback, without ranting and raving about supposed lack of "appreciation," and trying desperately to bully the messenger, that's your problem.
No one is getting ML to record in H.264. The "elapsed time" and "remaining time" recording indicator feature is broken for RAW video recording and should be clearly listed as such in the help files, and removed from the menu when RAW video is "ON".
Those who ask questions on the forums about why these features aren't working in RAW mode should be given a straight answer, instead of being dismissed or ignored. At least now they can find my comments on the matter, and not waste as many hours as I did wondering why selecting "elapsed time" or "remaining time" did absolutely nothing, searching through these forms for non-existent answers, reading help documents, trying different builds, etc.
And I think you've made it very clear to everyone how end users are treated here.
Quote from: ariaelf on January 09, 2021, 01:19:07 AM
...my work is easy enough to find online with a simple search.
Code is to be found with a simple search. That´s also a fact.
Aria,
The issue here wasn't with the feedback you provided. That was actually welcome - reporting bugs properly (https://www.chiark.greenend.org.uk/~sgtatham/bugs.html) is something that actually helps - especially considering that those who write the code might not be using it for filmmaking, so they might not have ran into your issue. (I, for one, record H.264 for my own needs - and that happens like 2 or 3 times per
year).
The issue was with your expectations: you seem to think we somehow have a duty to fix what's not working for you. That is, the part about
demanding that we should do something about it - whether it's removing a menu item, or fixing the feature, or whatever. That's unreasonable in an open source community, and no amount of praise would change that.
Here's a long answer:
https://mikemcquaid.com/2018/03/19/open-source-maintainers-owe-you-nothing/Quote
- The way the software is today is all that the maintainers ever agree to provide you (bugs and all)
- The maintainers provide no assurances that the software will ever work for any user or use case (even documented ones)
- The maintainers are never liable for any problems caused by any use of the software (including damages that require you to pay for repairs)
- You must agree with the above to have any right to use the software
Quote from: Danne on January 08, 2021, 11:56:53 PM
Open source initiatives here going on for years with nothing expected in return but maybe respect and some gratitude.
Exactly this - here's an even longer answer:
https://snarky.ca/setting-expectations-for-open-source-participation/Quote
Everything in open source should be a series of kindnesses
To reiterate, just because open source software is free for you doesn't mean someone else hasn't paid some price on your behalf to get you that code.
Since open source is paid for by others in effort and time, that would suggest that there really shouldn't be any expectations on your part from what you get out of an open source project that you choose to use. You could view everything in OSS as a kindness that someone has done for you and others. Putting things into that perspective takes away the feeling of expectation and entitlement. This frames open source participation as someone doing something nice for the community and project, as someone having done a kindness for you.
As soon as you start demanding or expecting something from open source you have stopped viewing it as it was intended, and that distortion can be poisonous. When I choose to donate my precious time to open source I do it voluntarily as a nice thing that I enjoy doing. I didn't do it because someone demanded it of me, and the instant I feel that my time is not appropriately appreciated as the gift that it is, I stop enjoying doing open source. And when someone stops enjoying their contributions to open source, they burn out and quit.
Taking the altruistic view of open source keeps things grounded and healthy. Viewing open source as a kindness someone else has done for you gives the appropriate perspective that this is something nice and no one has any expectations from it. It's like when I hold the door open for someone. Ultimately I don't expect anything in return (although a "thanks" is always appreciated). And the person passing through the door doesn't expect anything else from me either. But when someone in open source makes demands it's like the person passing through the door criticizing how I held the door open. It's pointless and simply leads to people no longer being willing to hold open doors for others.
To reiterate, you should view open source as a series of kind acts people have done altruistically. That means you cannot make any demands or hold any expectations of an open source project. Or as Evan Czaplicki, the creator of Elm, put it, "Remember that people do free work because it is fun, not to get stressed by strangers".
BTW, you've actually received useful feedback on what you - as a non-programmer -
can do to help the project:
Quote from: Danne on January 08, 2021, 11:56:53 PM
Edit: Also consider taking the time to "clean" up all inappropriate info you can find by testing each and every feature systematically. Write down your findings and create a list of changes you would suggest for the community. This would be nice and effective user feedback probably useful for a lot of users. This place is in need of users digging in more actively.
And you actually did, to a small extent - and I appreciate that. Lack of good feedback over what works and what not, has been a problem here for years. But it should have been done without demanding that we should do something about it. From the same article:
QuoteAnd this extends to bug reports and feature requests. It's a kindness when you report a bug so that I have a chance to try and fix it. But just because you reported a bug doesn't mean I specifically owe you anything. I should try to thank you, but there is no contract here that says I have to address your bug or feature request. My kindness was to provide my time and effort for the open source project up until now, but there's no expectation beyond that of my time and effort. Open source really should be viewed as self-service unless I choose to help further. If something doesn't work for you then you are free to take the code and modify it to meet your needs; that's a key tenet of open source. So the idea that any open source project owes anyone anything is a misunderstanding of what open source fundamentally is.
Hopefully the above makes it clear where the backlash comes from. Also, you may want to review our older ML's Goal (https://www.magiclantern.fm/forum/index.php?topic=7486.0) thread - which feels surprisingly up to date, even if it's from 2013 - and this particular piece (http://www.catb.org/~esr/faqs/smart-questions.html#not_losing) as well.
However, I do agree the community could have handled it a lot better, rather than starting with a dismissive answer, without any context. Hr, hr ;)
A small bit of history: yes, there are some features that were written before raw video was even imagined in our community. Some of them still work with raw video, others don't. Somewhere in 2014, I've actually started to disable the menu features that are not compatible with raw video (menu-raw-yuv-features) - but for some reason, the changes weren't well received by the reviewers at that time (one of them even commented that I'm trying to "dumb down" the user interface - yes, I remember these words very clearly even today). So, it quickly went to the back burner, and since I wasn't hitting this particular issue during my regular use of the camera, it hasn't been a priority - for me. YMMV.
I am also well aware of card free space not updating on its own (https://builds.magiclantern.fm/lua_api/modules/dryos.html#card.free_space), at least on some camera models. I know Canon firmware has some way to update it (iirc, if you open the Format dialog), but currently I have no idea how to call or trigger this procedure. One might be able to reproduce the issue in QEMU and figure out how to trigger the free space update from there, but... it does require programming skills and time for research - code doesn't grow on trees.
Hi All,
Very basic question here. I've scoured the forum and I might have just missed it. I'm looking to move from magic lantern nightly 2018 Jul03 build to crop_rec_4K (5D31.1.3). The information that I seem to have found suggests that I simply copy and replace the crop_rec_4K ML folder, autoexec.bin and MLsetup.fir onto the SD card. Is that correct? If anyone could point me in the direction of a post or resource that would explain the process I would be very grateful. Many thanks and apologies for the beginner question.
Quote from: nomadfilm on January 10, 2021, 06:07:21 PM
Hi All,
Very basic question here. I've scoured the forum and I might have just missed it. I'm looking to move from magic lantern nightly 2018 Jul03 build to crop_rec_4K (5D31.1.3). The information that I seem to have found suggests that I simply copy and replace the crop_rec_4K ML folder, autoexec.bin and MLsetup.fir onto the SD card. Is that correct? If anyone could point me in the direction of a post or resource that would explain the process I would be very grateful. Many thanks and apologies for the beginner question.
yes totally, just take the new files and delete what was in your card. you will probably have to adjust the settings again, according to you taste, as the pref file will be new too.
Thanks you so much! I assume that the same applies to all of Danne's 4K build, right? Again, greatly appreciate the response.
Hello everyone!
I have been filming for many years - also professionally, exclusively with Magic Lantern.
I am extremely grateful to the whole team here!!!
I have a question about a small problem and maybe someone can help me.
(Since I'm not English-speaking (deepL) there's a good chance that the question has already been covered enough
- however, I can't find it.)
I am filming with
5dmk3
1.1.3
crop rec 4k Build 2018-07-22 13:09
mostly 3072x1320 crop (5x) or FF-HD
12Bit losless
The thing: When I filmFF-HD in 12Bit losless it comes to "vagabond" pixel errors at prominent - edgy spots that look like digital fireflies:)
If I use 14 Bit or 14Bit Losless instead (with FF-HD) then I have the problem that the ColorScience tilts clearly to reddish from ISO 800...
In short: Which setting (or build) is best for a clean FF HD shot.
Although 12 bit losless is a bit coarser from the noise, I prefer the sunnier, fresher tint and it would be my choice....
I hope I'm not getting on anyone's nerves :) - maybe someone has a hint.
A thousand thanks for the great work here!
Volker Schmidt
https://vimeo.com/user10183773
Try Danne's Build, it is the newest Built and can much much more.
https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899
For example :
- Cardspanning: simultaneously record on CF and SD Card
and SD Card overclocking - writespeed in my Test up to 139mb/s!
- up to 5.7K Anamorphic mode (FF)
- 4K 1x1 (3840x1536)
- 3.5K, AR: nearly 2:1 and possible 5x zoom preview ( without global draw)
... And much more
Hi. I'm sorry for such a silly question but is pixel binning real?
I saw a video recently where a guy is talking about how he is shooting RAW full sensor wide with pixel binning on 700D and there is no aliasing. I've been scanning some related threads here but I didn't find a details on how this feature works, on what cameras and on what builds. Could anyone please clarify this?
What are the requirements for this? I see that there is an older crop_rec build and this newer one with 4K. Do they both support pixel binning?
What cameras support this feature and what camera (among the affordable) would be a good option for this? I remember 50D being a great bang for the buck thanks to fast CF write speeds and low price. Is this still the case? Or maybe a newer camera is a better option now? Perhaps some newer Digic features are more important now than card recording speed?
Does pixel binning also work for h264 movies and for HDMI out? Does the preview work OK?
Is it now really possible to have (almost) artifact free full sensor raw video on the cheap?
Would you mind linking any source?
Generally speaking: If there is any person telling me there is a way to avoid aliasing without exceptions and without additional filters my BS alarm is ticking.
Sure. I hope this is not fake
https://www.youtube.com/watch?v=OyOO_AhTPas
To simplify, this is achieved with an "anamorphic" recorded image.
(like the "5K preset" in Danne's build or like in this video where he uses the Canon 700D, it's with Bilal's build)
But to clarify, you are not necessarily free of aliasing. In my experience this depends of the process used to "desqueeze" the image.
For example if you are using MLV App, or DNGs in Davinci Resolve... the scaling algorythm is different and the result will be different too.
TBH I'm watching the youtube video with a 4K monitor and it was aliased anyway, but sure very much less in the shots in 1x3 than 3x3, but moire still present a little bit to an advised viewer's eyes.
That was my impression too, "much less". That could be a difference between garbage and usable footage.
So I followed the links in the description and found some more info.
But still no idea about the current state of support of this feature across cameras/builds.
Could anyone please clarify this? Is pixel binning only available in this newer crop rec on steroids build?
Do older/cheaper cameras have a chance of getting this feature?
You'll find this information so often in this forum... but again: all ML cameras do a line skipping and a column binning. Only 5D3 does also line binning. So if you find a build which supports 1x3 (anamorphic) recording, you will make use of "only" pixel binning and no skipping. As far as I know, this exists for 5D3, M, 700D, 100D and 6D (maybe a few more, maybe not).
Why does the raw module (the one without lossless) has only 3.5k?
I was trying to use
Full-resolution LiveView: 5796x3870 at 7.4 fps (with frame skipping)
Is there frame skipping on the lossless raw module?
I can only get a square 3.4k x 3.4k instead of 5.9 x 3.4