The EOSM cannot currently record in mv1080 mode though it is within the capability of the camera and it seems to be a "possible" Magic Lantern feature request.
Quote from: a1ex on January 30, 2016, 09:04:10 AM
Quote from: dfort on January 30, 2016, 02:58:37 AM
I'm wondering if there is any possible way to get mv1080 on the EOSM but I take it that has been tried before.
Quote from: a1ex on January 30, 2016, 09:04:10 AM
It wasn't tried too hard ;)
Looking a bit in forum history, it seems like in the very first implementations of raw video, there was a (undocumented) way to do it: http://www.magiclantern.fm/forum/index.php?topic=5860 and http://www.magiclantern.fm/forum/index.php?topic=12022
Maybe I should revisit it, but that would be a different topic.
Quote from: dmilligan on February 06, 2016, 05:10:05 AM
Quote from: dfort on February 06, 2016, 03:31:19 AM
a1ex explained that there are other ways (http://magiclantern.fm/forum/index.php?topic=16054.msg161356#msg161356) to capture raw video. Perhaps one of these alternate methods can capture the data after it has gone through the focus pixel annihilation process?
I wouldn't really say there are "other ways" to capture raw video, there are just other possible video modes. Capturing them is the same, you just have to figure out how to get Canon firmware to put the camera in those modes. The EOSM is unique b/c Canon firmware doesn't put it into mv1080 until you start recording (H.264). So if you want to record raw video on mv1080, you have to do it while also recording H.264 at the same time (ML prevents you from doing this now a days, but it was possible in some old versions when raw video just came out). That or do some reverse engineering to figure out how to put the camera in mv1080 without starting recording. All other cameras go into the video mode you selected in the Canon menu immediately, so actually mv1080 is pretty much the "default".
Let me try to sell this feature request by stating some of the reasons I believe why the lowly EOS-M is perhaps just as important to moving Magic Lantern forward as the 5D mark III.
The camera is the least expensive of all the ML capable cameras which makes this the ideal model to experiment without being too concerned about bricking an expensive piece of photographic equipment. It is so affordable that multiple bodies can be had for less than the cost of higher end models. In fact you can get 10 EOSM bodies for the price of one 5D3--I myself have 4 bodies, yeah I'm crazy.
At first the EOSM may seem radically different from the other Canon models because it is a mirrorless but the internals are very similar to the DSLR's. The advantages include not having to lock up a mirror to get into Live View and a shorter lens flange distance so many lenses can be adapted for use on this camera.
Canon EOSM with C-mount lenses
(https://farm2.staticflickr.com/1607/24557694310_136815b315.jpg)
Though the EOSM may feel like a toy compared to the DSLR's it actually has quite a rugged metal body and can stand up to the riggers of a documentary project I'm working on. I planned to use it as a spare body for a 5D3 setup but it proved capable enough as the 'A' camera. (When I was a professional still photographer back in the day I would never consider going on a job without a backup camera body.)
EOSM documentary rig
(https://farm2.staticflickr.com/1574/24557694160_054dd01baa.jpg)
The EOSM can also be an ideal replacement for a director's viewfinder and with a PL adapter the camera can be used to look through professional cine lenses. Adding mv1080 also gives the capability of shooting much higher quality tests than with H.254 video.
EOSM with PL mount adapter
(https://farm2.staticflickr.com/1689/24853746505_db231d4f22.jpg)
Adding mv1080 on the EOSM will also elevate this platform to the same level as the rest of the Magic Lantern capable cameras though filmmakers may want to dress it up a bit in order to be taken seriously.
EOSM Shoulder Rig with Follow Focus
(https://farm2.staticflickr.com/1594/24485578289_afa133e3ea.jpg)
There are disadvantages to the EOSM. One problem for video shooters is the short battery life though extra batteries are very affordable and it is very simple to rig up external power supplies. Another problem is that it responds slower than other cameras in still photo mode though it is just as quick as the other cameras in video mode. Speaking of still photo mode, there's that pesky shutter bug when using Magic Lantern but that's a different topic.
Reeeeeeaaaal quick, dumb question... What is mv1080?
The name ML code uses to refer to the video mode the camera is when recording 1080p video. It's full frame with line skipping and/or binning, and the actual raw resolution is slightly less than 1920x1080 on most cameras (except 5D3).
Ahh thank you. So the idea is to (re)enable 1080p raw video on the eos-m?
Looks like I forgot to explain what mv1080 is for users though developers, who are the target audience for this topic, know exactly what it is.
Thanks dmilligan for that concise explanation. Just to be complete, here is a table showing all the Magic Lantern video modes.
Mode Buffer Size Notes
mv1080 - 1808x1190 - not available on the EOSM.
mv720 - 1808x727 - needs to be stretched by 1.67x vertically - up to 50/60 fps except on EOSM.
mv640 - 1808x1190 - basically the same as mv1080 mode.
mv1080crop - 1872x1060 - highest resolution and largest sustainable frame sizes on all cameras.
mv640crop - - not possible.
zoom - 2592x1108 - same 5x magnification as mv1080crop but can pan around sensor. Performance not as good as mv1080crop.
This was determined by experimentation, not just reading code which is something I still need to learn. Note that the table shows the raw buffer sizes, the final image is cropped down even further. The maximum possible resolution for these APS-C cameras is 1728x1158. That's too high of a data rate for raw video but it is not a problem when the frame is run through the on board H.264 compressor.
When using the full un-cropped area of the sensor the Canon firmware skips pixels in order to reduce the 18 megapixel resolution down to HD video which is about 2 megapixels. In mv1080 mode it does this by using only every third pixel horizontally and every third pixel vertically. In mv720 mode it uses every third pixel horizontally but then it takes every fifth pixel vertically.
Another anomaly of the EOSM is that m720 raw doesn't inherit the frame rate of H.264 1280x720 50/60p. I don't see that as a problem because that high of a frame rate requires reducing the image size or throttling back the frame rate but that disables audio recording. The problem for the EOSM is that the mv1080 image quality is much better than mv720.
When shooting in crop mode the EOSM matches the other APS-C cameras but then why get a large sensor camera and just use a tiny area? You're missing out on the filmic quality that made shooting on DSLR cameras popular in the first place.
As a side note there has been several discussions (http://www.magiclantern.fm/forum/index.php?topic=16428.msg159735#msg159735) about taking the full resolution of the sensor and running it through the on board JPEG compressor to achieve 4k video. That's apparently how the EOS-1D C and 1D X Mark II handle 4k video. Let's not ask for that on the EOSM just yet. ;)
Couple of nitpicks:
Quote from: dfort on February 08, 2016, 05:04:18 AM
The maximum possible resolution for these APS-C cameras is 1728x1158.
The buffer sizes of the video modes varies greatly from camera to camera. Some cameras don't have mv1080crop, I think only 60D has mv640crop. I wouldn't actually say that mv1080 "- not available on the EOSM". It is available, you just can't record raw video in that mode currently.
I think there's a chart around somewhere with all these sizes for all cameras (and other things too like FPS capabilities).
Having this implemented would be really sweet and I'm motivated to look into it.
We have to figure out a way to set the EOSM to mv1080 when starting recording in the mlv module. A starting point would be to take a look at the 650D (is this the most similar firmware/hardware?) and see what it does when switching to 1080p mode in the canon menu and see if a similar function could be called in the EOSM firmware.
And I think in a couple of places inside the mlv module it is assumed that mv1080 mode on EOSM is actually only mv720 mode - this needs fixing if we ever manage to actually switch modes correctly.
@dmilligan
Thanks for being nitpicky. Details count when you're doing quality work. My ML world is currently limited to the EOSM/100D/700D and 5D3 but it is rapidly expanding. I think this is the chart though some of the data for the cameras that I tested seems off.
http://www.magiclantern.fm/forum/index.php?topic=6215.0
@rbrune
The 650D is very similar but the issue seems to be that the EOSM is always in Live View and doesn't switch into mv1080 mode until it starts recording. Looking over your posts it looks like you have the skills to do this and you've got an EOSM. Please let me know if there's anything I can do to help. I bookmarked your bitbucket page, mine is https://bitbucket.org/daniel_fort/
[EDIT: Looks like you were aware about this issue long before me! (http://www.magiclantern.fm/forum/index.php?topic=14763.msg142818#msg142818)]
Sure, I vote for this!
I would love for this to happen as well.
+1 ... Got my vote on this one as well!
please please make this happen :D
+1
love this little camera
+1!
The EOS M became a great replacement for my old t3i!
Quick question:
Where can i research about c-mount lenses for the EOS M?
I'm looking for cheap cctv lenses, but i can't find any details about how they would work on a eos m...
This keeps coming up. Basically all you need is a EF-M to C-mount adapter (https://www.amazon.com/Fotasy-Movie-Mirrorless-Camera-Adapter/dp/B00ACYYUWI/ref=sr_1_1?ie=UTF8&qid=1474055974&sr=8-1&keywords=ef-m+to+c-mount). Some lenses like the Fujian 35mm (http://www.ebay.com/itm/Fujian-35mm-F1-7-CCTV-Lens-for-Micro-4-3-NEX-Nikon-1-EOSM-Fujifilm-Pentax-Black-/271756426240?hash=item3f45f1fc00:g:05MAAOSwepJXYMjX) will cover the whole sensor while shorter lenses will vignette in full frame but still work fine in crop mode video.
http://magiclantern.fm/forum/index.php?topic=15660
http://magiclantern.fm/forum/index.php?topic=10036.0
Hey, thanks for the reply
I have some more questions about adapted lenses.
Does the C-mount work with CS-Lenses?
And what about D-mount lenses, is there any way to adapt them to work with eos-m and keep infinity focus?
Thanks once again, and sorry if i'm changing the focus of this thread.
Yes we are getting off topic.
In any case, CS mount has a shorter flange distance than C mount so even though a CS mount lens will screw into a C mount adapter it won't work properly--you lose the ability to focus at infinity. It isn't an issue going the other way. I once had a CS mount video camera and it could use C mount lenses by using a 5mm adapter ring. Of course that doesn't help us EOSM users. D mount lenses were made for 8mm movie cameras. Again, no adapters that I know of. However, if you really want to mount these lenses on your camera you might be able to get a machine shop make you one. Though it is probably not worth the cost.
[EDIT] So it turns out that there are D-mount to C-mount (as well as C to D) adapters available on ebay (http://www.ebay.com/sch/i.html?_from=R40&_trksid=p2050601.m570.l1313.TR0.TRC0.H0.XD-mount+to+C-mount+adapter.TRS0&_nkw=D-mount+to+C-mount+adapter&_sacat=0) and amazon (https://www.amazon.com/dp/B00YCPOPD2/ref=wl_it_dp_o_pC_S_ttl). They are simple and inexpensive and at first it seems like it would do the trick but there seems to be a problem. The flange distance (https://en.wikipedia.org/wiki/Flange_focal_distance) is shorter on D-mount and the adapter doesn't compensate for this so focusing at infinity is not possible. Too bad, there are some rather interesting D-mount lenses out there.
I found this 3d printer model of the C-mount adapter:
(http://i.imgur.com/1VNtOSI.png)
And then i tried modifying the model, by pushing the lens mount area 5mm closer to the sensor, like this:
(http://i.imgur.com/rAStfG6.png)
i think the adapter will fit into the eos-m body without hitting anything.
I'm looking after some places here to print it and test it with CS lenses.
Interesting, the only thing I would caution you about is that you might get a CS mount lens that is wide at the flange. This is an extreme example of one of my C-mount lenses:
(https://c6.staticflickr.com/6/5781/30242419525_c0bb38f4f3_o.jpg)
In some cases you can grind of a piece of the lens but it would be better if you can design your adapter so you can avoid this as much as possible.
http://videoproducent.blogspot.com/2011/10/return-of-15-pentax-6mm-12-succes-story.html
(http://1.bp.blogspot.com/-n1oLEDk4h2A/TodGLhEQP8I/AAAAAAAAAuM/Iay1fMn7Jpc/s400/Pentax+success+008.JPG)
To be honest I can't tell if my c mount lenses are slightly out of focus at infinity or if I'm just running into resolution limitations with the eos m. Mv1080 please!!
@Teamsleepkid - thanks for getting the discussion back on topic!
Yeah, some of my cheaper C-mount CCTV lenses are quite soft. I guess that's part of the attraction to these lenses. However, I'd prefer to use my Canon EF-M lenses in non-cropped mv1080 mode but as you know the EOSM shoots mv720 in non-cropped mode.
yeah. we need mv1080 like the desert needs the rain. can't you ask some of your friends dfort?
Quote from: Teamsleepkid on October 18, 2016, 06:16:33 AM
...can't you ask some of your friends dfort?
That's the point of this topic. We're all friends here--well, sort of.
Thing is that ML development is a volunteer effort. Each developer has a different priority over what they want to work on. I've contributed some minor fixes but re-writing the code to get mv1080 on the EOSM is way beyond my abilities. I am reading up on programming and trying to get familiar with the ML code but the hints I've been getting from my developer friends is like explaining the theory of relativity to a cat.
"the easiest way would be to record RAW and H.264 simultaneously. This one fits into easy coding tasks." Alex said that. I think he can do it...
Hey @Teamsleepkid thanks for the hint.
Quote from: Teamsleepkid on October 23, 2016, 03:25:53 AM
Alex said that. I think he can do it...
Alex can do many things but he's a busy guy. In addition, this is a community so he doesn't do everything by himself. Let's take a closer look at his statement. The quote comes from the topic, REC COMMAND via the HDMI trigger - Atomos (http://www.magiclantern.fm/forum/index.php?topic=12022.msg118321#msg118321). This is mainly for 5D3 users and in a later post he suggests, "The easiest way should be to select the SD card from Canon menu (since H.264 will use that), and force RAW/MLV recorder to use the CF drive regardless of the current card choice." Obviously something that doesn't apply to the EOSM. Still, there's hope and the best clues came here:
QuoteSince the H.264 encoder is a hardware chip, I don't expect a significant performance drop. Here's a hint from somebody who tried: http://www.magiclantern.fm/forum/index.php?topic=5860.msg42534#msg42534
It even looks like this was a bug in some versions: http://magiclantern.fm/forum/index.php?topic=10259
So, definitely a task for anybody who wants to jump in and code something easy.
Since it was considered a bug a while back maybe there was a comment in a commit to fix it so I searched the commits on bitbucket for H.264 (https://bitbucket.org/hudson/magic-lantern/commits/all?search=H.264) and found some interesting things. There were a couple commits made back in March 2014: raw_rec/mlv_rec: if you somehow managed to start recording H.264, let it stop (https://bitbucket.org/hudson/magic-lantern/commits/41adc6425276247428dca06a8d1cbcd3b91fb3e9) and EOS-M seems able to change frame iso / shutter timer only while recording H.264. ETTR and other tools that depend on these settings should fall back gracefully (https://bitbucket.org/hudson/magic-lantern/commits/a6ff18c6aa443164b08a135980ec9911a0213ac5). Both of these were made to prevent recording H.264 and raw/mlv simultaneously so there must be a way to force simultaneous recording. It might be possible to stop the H.264 stream once raw recording is triggered so that it doesn't impact the SD card write performance for the raw video but that should probably be an option so that triggering an external recorder while recording raw would also be possible--killing two birds with one stone so to speak, one for the EOSM and another for the 5D3.
I got ya. honestly I was just talking some trash to keep the thread alive. I'm more of a camera guy than a coder. On the main eos m thread looks like the new guy can code..wildstray. Is mv1080 even worth it? Is the resolution gained that much? I've never tried it cause I only have the M..Thanks Dfort you're awesome and you always have tons of useful knowledge and help.
Quote from: Teamsleepkid on October 25, 2016, 08:38:46 AM
...I was just talking some trash to keep the thread alive...
Thanks. You brought up a good point though. Is mv1080 really worth the effort? Since the 700D has basically the same sensor I thought I'd run some comparisons between mv1080, mv720 and mv1080crop just for good measure.
mv1080
(https://c7.staticflickr.com/6/5505/30270012590_49b1940600_z.jpg)
mv720
(https://c1.staticflickr.com/6/5617/30452112912_9084f21a73_z.jpg)
mv1080crop
(https://c8.staticflickr.com/6/5725/30481008751_7c81ccd523_z.jpg)
Ok, not too dramatic yet. A few notes, I used a 17-55mm lens zoomed all the way in to 55mm on the mv1080 and mv720 shots and all the way out to the other extreme at 17mm for the mv1080crop shot. The EOSM can do mv1080crop just fine but it means shooting on a small part of the sensor and losing that "film look" from the APS-C sensor's depth of field. Shooting in crop mode with the same focal length as non cropped mode results in a 3x crop.
mv1080crop showing 3x crop factor
(https://c8.staticflickr.com/9/8622/30569020055_326652f848_z.jpg)
To really see the difference between mv1080 and mv720 let's look at a small section of the frame and play it in motion.
Both the mv1080 and mv720 shots were framed at 1280x720 but the mv720 shows a drop in resolution and increased aliasing.
What are the advantages of mv720? On other cameras the default frame rate for mv720 is 60 or 50 frames per second depending if you are set in NTSC or PAL mode. On the EOSM it defaults to 24, 25 or 30 fps depending on the settings in the Canon menu. Yeah, the EOSM should be able to do raw 60p just like the DSLR's but that's a different topic.
Looks a hell of a lot sharper on that video you posted. Think the grain will be less pronounced in low light? Thats where i seem to be struggling the most with my M. Not enough sharpness and too much grain. If I add sharpening to make up for the 720p image the grain gets sharper and more pronounced. If we shoot mv1080 crop maybe we don't have to add sharpness and the grain won't look so pronounced.
Yeah, the best setting for recording raw video on the EOSM is mv1080crop. As far as grain, it should be more pronounced in low light. To make mv720 video look good it is probably better not to add any sharpness. If anything, keeping the image soft might hide some of the aliasing. If you shoot 720p at 60fps it looks better but we can't go there on the EOSM.
After a long time I finally had time to fiddle around a little bit.
Using adtg_gui I overrode the ADTG 800C register that controls the vertical line skipping to switch from 3x5 (0x4) to 3x3 (0x2) line skipping mode. To successfully record an mlv file I also changed mlv_rec.c to ignore the special squeeze factor calculation that assumes the EOSM is always in mv720 mode.
To get the camera to accept the overridden register one has to use the INFO button to switch into Canons menu and display modes and back into magic lantern. The preview image should be stretched when it worked.
Here example 1x1, 3x5 and 3x3 frames from the exact same spot, with the same 22mm lens and the same mlv_rec resolution settings.
What I'm not sure about is how to properly implement this in a way that is conveniently usable - and acceptable for the main branch version of Magiclantern.
single 1x1 crop mode frame recorded with mlv_rec
(http://rbrune.de/ml/eosm-1x1.jpg)
single 3x5 mv720 squeeze mode frame recorded with mlv_rec
(http://rbrune.de/ml/eosm-5x3.jpg)
single 3x3 frame recorded with mlv_rec overriding ADTG2[800C]=2
(http://rbrune.de/ml/eosm-3x3-override-adtg2-800C-2.jpg)
Very nice. Sounds like a great addition to the crop_rec module.
The code there is be fairly generic - you can see the actual mode patches are done under "if (is_5D3)" blocks. It's written that way to make it easier to accommodate other models.
Quote from: a1ex on October 26, 2016, 09:59:23 PM
Very nice. Sounds like a great addition to the crop_rec module.
The code there is be fairly generic - you can see the actual mode patches are done under "if (is_5D3)" blocks. It's written that way to make it easier to accommodate other models.
Great, I'm looking through the module right now. Some questions popup.
In crop_rec ADTG_WRITE for the 5D3 is defined as 0x11640 but in adtg_gui ADTG_WRITE_FUNC is 0x11644 - since the idea is to place the hook, how would I determine a good position, I assume the difference is based on that?
And what is the purpose of MEM_ADTG_WRITE / the second argument (orig_instr) to patch_hook_function? Following the code I assume this is just a safety measure to check the memory content before patching?
0x11640 is good. I usually do these adjustments are to avoid cache conflicts (not the case here, because this address is in RAM) or to avoid backend quirks (for example, older versions of patch hooks could not be placed on a branch). Here, both addresses will work, but 0x11640 is the actual start address for the function.
MEM_ADTG_WRITE is just a safety check.
Quote from: a1ex on October 26, 2016, 11:45:27 PM
0x11640 is good. I usually do these adjustments are to avoid cache conflicts (not the case here, because this address is in RAM) or to avoid backend quirks (for example, older versions of patch hooks could not be placed on a branch). Here, both addresses will work, but 0x11640 is the actual start address for the function.
MEM_ADTG_WRITE is just a safety check.
That works really well. I made a pull request into the crop_rec branch.
Issues so far:
- I had to deactivate the cmos_hook, even without it changing values it results in black preview data from the sensor
- mlv_rec assumes that the EOS is in 3x5 mode when calculating the squeeze factor. I removed that for now. Maybe we can check the crop.preset value there?
- the preview in 3x3 override mode is of course now stretched. Not sure if that can easily be fixed
In case someone wants to play around with it:
https://drive.google.com/file/d/0B5FJDsqHlkJ6TVNXS1Y0MWF0dXM/view?usp=sharing (https://drive.google.com/file/d/0B5FJDsqHlkJ6TVNXS1Y0MWF0dXM/view?usp=sharing)
Activate the crop_rec and mlv_rec module. Under video set the crop to 3x3 mode, switch to play mode or to a canon menu for a second to make the override work and then go ahead and record mlv. Be aware that the preview is stretched.
Interesting idea. I took a quick look and it does seem to be recording in mv1080 mode or something like that. Seems that the maximum vertical resolution is 692 so you can't record 1280x720 16:9 video. Thanks for posting a binary so more EOSM users can check it out.
heros
Rafael Brune is the EOSM hero of the day.
Thanks for sharing Rafael and perhaps we should remind those that aren't aware ... please do not activate the autofocus fine-tuning (dot_tune.mo) otherwise you will see wall of errors scrolling up on your LiveView LCD as it hasn't been fixed (yet).
Finally EOS-M is just about to become little video beast on steroids :) Thank you, Rafael, for your work and time!
P.S. Unfortunately I wasn't able to open that google drive link :(
Right--it looks like there are some problems with build and link rbrune posted. According to his latest pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/762/crop_rec-add-limited-eosm-support-to/diff#comment-25948042) comment he won't be near his camera until Monday evening so for anyone eager to try it out over the weekend I posted a build on my bitbucket download area:
https://bitbucket.org/daniel_fort/magic-lantern/downloads
Also hoping to get this into the Crop mode recording (crop_rec.mo) (1:1, RAW/H.264, 25/30/50/60 fps) (http://www.magiclantern.fm/forum/index.php?topic=17021.msg173917#msg173917) topic.
@rbrune
Sorry for some offtopic
As I understood you managed to switch subsampling inside of mv720? So it might be possible now to make 48/50/60fps RAW on 5D3 without stretching using same method?
it,s better. It affects mv1080p footage. Great work rbrune.
Got a file from dfort. Seems it,s also free from focus pixels? Could this be right?
Quote from: madz on October 28, 2016, 05:10:32 PM
@rbrune
Sorry for some offtopic
As I understood you managed to switch subsampling inside of mv720? So it might be possible now to make 48/50/60fps RAW on 5D3 without stretching using same method?
Check out the crop_rec module thread linked in the post above you. That does exactly that - but don't credit me, it is not my work.
I just tried if we can get 3x3 working in the mv720 mode on the EOSM and after a1ex pointed me towards the crop_rec mode I just added some preliminary support for the EOSM. After the weekend I will try to tackle the remaining issues of which the most important one is probably to center the recording area.
Quote from: rbrune on October 28, 2016, 09:23:58 PM
After the weekend I will try to tackle the remaining issues of which the most important one is probably to center the recording area.
I helped a 60D user with this problem a while back. Here's how we fixed it:
http://www.magiclantern.fm/forum/index.php?topic=16912.0
yes i noticed the recording area is a little off. didn't want to sound like i was complaining. thank you.
Quote from: dfort on October 28, 2016, 09:57:15 PM
I helped a 60D user with this problem a while back. Here's how we fixed it:
http://www.magiclantern.fm/forum/index.php?topic=16912.0
It's a little different in this case.
mlv_rec tries to center the recording area inside the area of the sensor that is read out. What is off here is that the area of the sensor that is read out is not centered anymore after switching to 3x3 binning. What needs to be done is getting the cmos_hook working and changing the start read out line for the read sensor area.
Quote from: rbrune on October 28, 2016, 10:58:47 PM
It's a little different in this case.
It certainly is. I got a chance to do some quick tests. Please check that I'm doing this right because there aren't really any instructions for this yet.
Let's start with the Canon menu. Make sure to have the camera in movie recording mode. We're going for mv1080 video (3x3 line skipping) so the camera should be set up for 1920x1080 24fps.
(https://c2.staticflickr.com/6/5512/30630699145_ee99f63475.jpg)
Everything else is set up via the Magic Lantern menu starting with making sure both the crop_rec and mlv_rec modules are loaded.
(https://c4.staticflickr.com/6/5476/30630699075_e416d4d25a.jpg)
Turn on the crop_rec module and pick the
3x3 720p (1x wide) option. Yeah, it is confusing because the Canon menu is set to 1080p and the crop_mode is set to 720p but it will all make sense in a few steps.
(https://c2.staticflickr.com/6/5545/30630699025_f9480671d1.jpg)
The frame size that I really wanted is 1280x720, that's a 16:9 aspect ratio but the maximum vertical resolution is 692 which corresponds to a 1.85:1 aspect ratio. Ok, close enough and it is a standard cinema aspect ratio so I'm good with that.
(https://c8.staticflickr.com/6/5444/30630698975_e8305bf959.jpg)
When you go to LiveView everything looks "normal" if you've ever shot raw video except maybe the crop doesn't look quite right and there's a little warning sign at the bottom of the screen. If you look back a couple of screenshots you'll see a message saying, "After leaving ML menu, press PLAY twice to enable crop mode."
(https://c1.staticflickr.com/6/5441/30330953920_232369e79f.jpg)
This is what it looks like after pressing PLAY twice. Wow, that's quite a stretch!
(https://c6.staticflickr.com/6/5453/30630698845_cf88762494.jpg)
The default settings are for most of the screen overlays to disappear when recording. I find that the best way to frame your shot is to make some marks right on the screen with tape or a marker. Lo-tech but it works.
(https://c8.staticflickr.com/6/5475/30630698695_70ccccd896.jpg)
I haven't shot any video's worth sharing using this module yet but here's a frame of a quick shot using the above settings. The actual frame size is 1280x692 and the EOSM seems to have no problem recording continuously at this setting. That wild moiré going on in the background is due to a window screen. This is "normal" with raw video and these ML enabled Canon cameras (with the notable exception of the 5D3).
(https://c6.staticflickr.com/6/5561/30630698485_c1a2485f1c_c.jpg)
So why 720 (or actually 692) and not 1080 and isn't this supposed to be mv1080? The frame full buffer when recording 1080p video on the EOSM is 1808x1190. The camera is skipping every 3rd pixel horizontally and vertically and the maximum resolution of the sensor is 5184 x 3456, give or take. In mv720 mode it is a 3x5 pattern skipping 3 pixels horizontally and 5 pixels vertically with a buffer size of 1808x727. When recording H.254 the frame is scaled up in 1080p mode and in 720p mode it is scaled down but only in the horizontal axis. Ok so in mv1080 mode the ML menu should go up to 1728x1158 on the EOSM like it does on it's big cousin, the 700D (the full buffer is cropped when recording video). However, there's no way to record that much data with the camera's hardware so we're limited to a frame size of about 1280x720. You can vary the dimensions somewhat, it is the size of the frame in MB that matters. Make sense? Good, now can someone explain it to me because I'm not sure I got this right. As far as I know this isn't really documented all in one place.
Long story longer, @rbrune has figured out a way to get mv1080 raw video out of the EOSM. What I'm finding very intriguing is that there are no focus pixels showing up on the frames. Just to make sure I'm not seeing things I used mlv_dump to create the DNG frames because MLVFS automatically masks out the focus pixels. Now how did that happen? Maybe there is an answer to dealing with focus pixels (http://www.magiclantern.fm/forum/index.php?topic=16054.0) in camera?
I have focus pixels on my footage. I'm using 5:3 aspect ratio though 1.66.. maybe since the framing is off they are underneath of your footage?..maybe I'm doing something wrong but Ive got em. I did the same thing you outlined dfort except i used 1080p30 with fps override for 23.976.
Quote from: dfort on October 29, 2016, 05:43:43 AM
So why 720 (or actually 692) and not 1080 and isn't this supposed to be mv1080?
On EOS M, in standby, the LiveView is configured as 720p, even if you have selected 1080p in Canon menu (see first post). Once you start recording H.264, the LiveView is reconfigured to what you have selected in Canon menu (resolution, fps). Of course, it goes back to 720p once you stop recording H.264.
Tip: you can see the default LiveView configuration from the FPS override submenu, without turning it on.
QuoteOk so in mv1080 mode the ML menu should go up to 1728x1158 on the EOSM like it does on it's big cousin, the 700D (the full buffer is cropped when recording video). However, there's no way to record that much data with the camera's hardware so we're limited to a frame size of about 1280x720.
FYI, the card speed never affects what options appear in menu. And, iirc, the two cameras have nearly identical write speeds.
Quote from: a1ex on October 29, 2016, 11:22:43 AM
On EOS M, in standby, the LiveView is configured as 720p, even if you have selected 1080p in Canon menu...
Let's see if I got this right. So that means that this bit of code in src/raw.c is never true in standby:
int mv1080 = mv && video_mode_resolution == 0;
Because of this in src/propvalues.h:
extern int video_mode_resolution; // 0 if full hd, 1 if 720p, 2 if 480pHowever, this
can be true:
int mv1080crop = mv && video_mode_resolution == 0 && video_mode_crop;
Does this mean that LiveView on the EOSM is configured as 1080p when Movie crop mode is active? (This is also true in ZOOM mode, right?)
Quote from: a1ex on October 29, 2016, 11:22:43 AM
Tip: you can see the default LiveView configuration from the FPS override submenu, without turning it on.
I'm a little lost here. Where can I see the LiveView configuration?
(https://c8.staticflickr.com/6/5574/30551212151_d210957804.jpg)
Digging into src/fps-engio.c there's this:
#elif defined(CONFIG_EOSM)
#define TG_FREQ_BASE 32000000
#define FPS_TIMER_A_MIN (ZOOM ? 666 : MV1080CROP ? 532 : 520)
#undef FPS_TIMER_B_MIN
#define FPS_TIMER_B_MIN ( \
RECORDING_H264 ? (MV1080CROP ? 1750 : MV720 ? 990 : 1970) \
: (ZOOM || MV1080CROP ? 1336 : 1970))
So are the timer values a hint at what the LiveView configuration is? There doesn't seem to be a definition for MV1080, is that because it just doesn't happen on the EOSM unless it is actually recording H.264?
Sorry if I'm asking too many questions. I'm just hoping something that I bring up sparks an idea.
The EOSM and 700D (and 650D) are very similar in many respects. They even have the same size LCD and show the same focus pixel patterns in raw video. I keep thinking there must be a way to get the EOSM to record mv1080 without having to simultaneously record H.264 but it is beyond my limited knowledge of this stuff.
Quote from: Teamsleepkid on October 29, 2016, 06:44:27 AM
I have focus pixels on my footage. I'm using 5:3 aspect ratio though 1.66..
How is that possible? When I try to set a 5:3 aspect ratio I get a warning, "Could not get 5:3. Max vertical resolution: 692." Are you using 1280 for your horizontal resolution? Could you upload a short sample MLV so I can check the focus pixel pattern and see if the tools to remove them are working?
Quote from: dfort on October 29, 2016, 03:54:22 PM
Where can I see the LiveView configuration?
Actual FPS. The resolution of the raw buffer (active area) can be seen under Rolling Shutter, if raw recording is enabled. Or, in the error messages from the raw recording submenu (if you try to select a higher resolution).
The full resolution (including black borders) can be seen on the console if you enable RAW_DEBUG in raw.c, or by taking a silent picture (which contains the full raw buffer) and developing it with "dcraw -4 -E". Its size is autodetected from EDMAC registers (it used to be hardcoded in raw.c, but it's no longer the case).
Quote from: dfort on October 29, 2016, 03:54:22 PM
So that means that this bit of code in src/raw.c is never true in standby:
int mv1080 = mv && video_mode_resolution == 0;
Correct.
Quote
However, this can be true:
int mv1080crop = mv && video_mode_resolution == 0 && video_mode_crop;
Does this mean that LiveView on the EOSM is configured as 1080p when Movie crop mode is active?
IIRC yes (you can double-check).
Quote
There doesn't seem to be a definition for MV1080, is that because it just doesn't happen on the EOSM unless it is actually recording H.264?
No, mv1080 is handled on the "else" branch. But notice the EOS M uses RECORDING_H264 for choosing FPS_TIMER_B_MIN, and the reason for this is what you stated above.
QuoteI keep thinking there must be a way to get the EOSM to record mv1080 without having to simultaneously record H.264 but it is beyond my limited knowledge of this stuff.
This one is also beyond my knowledge, although not exactly in the "pipe dream" category (probably doable, given enough time and motivation).
shot in 1152x692 1.66 aspect ratio 720 3x3 mode. I realize this looks like crap it was shot at iso 3200 with like a 30 watt light bulb in my room. you can see white focus pixels on the pillow in the middle. I also boosted contrast so they are easier to see. You can also tell by the vignette on the lens that the picture isn't centered its more towards the top of the frame. link to video
https://vimeo.com/189469270 (https://vimeo.com/189469270)
Quote from: Teamsleepkid on October 30, 2016, 12:27:28 AM
...you can see white focus pixels on the pillow in the middle...
Yes I can see that and I was also able to reproduce the issue over here with your 1152x692 frame size. MLVFS unsqueezed the frame so the aspect ratio is wrong and it didn't removed the focus pixels while Danne's new cr2hdr.app has the right aspect ratio but didn't remove the focus pixels either. Darn it, no focus pixels was too good to be true. I think the frame I was using was so far off center that it missed the focus pixel area in the center of the sensor.
@a1ex -- thanks for taking the time to answer my questions. Here's what I discovered with the various MLV settings. I didn't check H.264 or raw v1.0. I'm using the 700D as a baseline and hopefully people reading this won't confuse Movie Crop Mode with the Crop Record module.
700D:
Canon menu set at 1080/24p - Actual FPS = 23.970 Rolling shutter : 19.1 ms at 1736x1158
Canon menu set at 720/60p - Actual FPS = 59.946 Rolling shutter : 11.5 ms at 1736x695
Canon menu set at 1080/24p - Actual FPS = 23.970 Rolling shutter : 17.6 ms at 1800x1082 <- Movie Crop Mode
Canon menu set at 1080/24p - Actual FPS = 29.954 Rolling shutter : 24.2 ms at 2520x1080 <- ZOOM Mode
EOSM:
Canon menu set at 1080/24p - Actual FPS = 29.973 Rolling shutter : 11.4 ms at 1734x693
Canon menu set at 720/60p - Actual FPS = 29.973 Rolling shutter : 11.4 ms at 1734x693
Canon menu set at 1080/24p - Actual FPS = 23.970 Rolling shutter : 17.5 ms at 1798x1026 <- Movie Crop Mode
Canon menu set at 1080/24p - Actual FPS = 29.975 Rolling shutter : 24.0 ms at 2518x1074 <- ZOOM Mode
EOSM using crop_rec.mo:
Canon menu set at 1080/24p - Actual FPS = 29.973 Rolling shutter : 11.4 ms at 1734x693
Canon menu set at 720/60p - Actual FPS = 29.973 Rolling shutter : 11.4 ms at 1734x693I used that "dcraw -4 -E" a lot when working on the focus pixel project so I'm familiar with the raw buffer. So if the raw buffer is an indicator of whether we're in mv1080 mode then it looks like the EOSM does go into mv1080crop mode but doesn't go into mv1080 mode even when using the Crop Mode module. Also note that if the frame rate is an indicator of mv720p mode, the EOSM doesn't really go there either.
[EDIT] Added ZOOM mode just for completeness.
Good news everyone. a1ex made a commit on the crop_rec branch (https://bitbucket.org/hudson/magic-lantern/commits/3428fea3c4164ceb9c2f73807e67d12139b36db9) so you no longer need to press PLAY twice to get into crop_rec mode. I built a new version with rbrune's modifications and put it in my bitbucket download area (https://bitbucket.org/daniel_fort/magic-lantern/downloads) for testers.
Thanks. I kept forgetting to push the playback button twice. Very helpful.
is it hard to get it centered?
I just pushed some new code to the pull request. The recorded area is now centered.
Here's a build to play around with, it includes the auto update code from a1ex:
https://drive.google.com/open?id=0B5FJDsqHlkJ6RmVrcUVXYk1kTHc (https://drive.google.com/open?id=0B5FJDsqHlkJ6RmVrcUVXYk1kTHc)
Rbrune love you:)
rbrune, marvellous work.
@rbrune - Quick test on your latest binary and centering looks good. I only checked the 3x3 720p (1x wide) setting.
Noticed you have dot_tune in your modules. It shouldn't be in there for the EOSM. I see DeafEyeJedi had problems with it on your previous binary when he turned it on.
The focus pixel removal tools still aren't working. I'll need more time to figure it out but the way it works is that it uses the raw buffer size for that camera model to determine what focus pixel map file (fpm) to use. It looks like crop_rec (3x3) has the same 1734x693 raw buffer size as "regular" 720p mode (3x5) so this is going to be a challenge.
The focus pixel removal function also looks at the Crop and Pan values to figure out what portion of the fpm file to use. mlv_dump -v is reporting these values:
Crop: 304x28
Pan: 298x28
Quote from: dfort on November 01, 2016, 06:36:13 AM
@rbrune - Quick test on your latest binary and centering looks good. I only checked the 3x3 720p (1x wide) setting.
Noticed you have dot_tune in your modules. It shouldn't be in there for the EOSM. I see DeafEyeJedi had problems with it on your previous binary when he turned it on.
The focus pixel removal tools still aren't working. I'll need more time to figure it out but the way it works is that it uses the raw buffer size for that camera model to determine what focus pixel map file (fpm) to use. It looks like crop_rec (3x3) has the same 1734x693 raw buffer size as "regular" 720p mode (3x5) so this is going to be a challenge.
The focus pixel removal function also looks at the Crop and Pan values to figure out what portion of the fpm file to use. mlv_dump -v is reporting these values:
Crop: 304x28
Pan: 298x28
Yes, the focus pixel removal needs to be adapted. A1ex suggested a new internal structure that will be saved out in the mlv file that would make fixing focus pixels easier and compatible with the crop_rec module.
Quote from: rbrune on November 01, 2016, 10:57:47 PM
Yes, the focus pixel removal needs to be adapted. A1ex suggested a new internal structure that will be saved out in the mlv file that would make fixing focus pixels easier and compatible with the crop_rec module.
Up until now the full raw buffer sizes of each raw video mode were different. Now on the EOSM we have the current mv720 and the new crop_rec mv720 both a full raw buffer size of 1808x727 and active area of 1734x693 but of course with different focus pixel patterns. a1ex pointed out this problem a while back on the Dealing with Focus Pixels in raw video (http://www.magiclantern.fm/forum/index.php?topic=16054.msg160371#msg160371) topic a while back.
I'll work on getting a new pixel map for this crop_rec mode and post my notes on the focus pixel discussion.
so is this the same as mv1080 on the other cameras? or are we getting a little more moire and aliasing than the others? this is mv720 3x3 right not mv1080? just wondering...
@dfort while we are at it can you take a look over here please:
https://bitbucket.org/hudson/magic-lantern/pull-requests/746/700d-rawc-finetune-raw-photo-raw-liveview/diff
Then afterwards compare with my PR:
https://bitbucket.org/hudson/magic-lantern/pull-requests/757/adding-support-for-the-eos-100d-sl1/diff#Lsrc/raw.cF579
My gut feeling says we should unify the offsets for EOSM/650D/700D/100D as seen on Matthew's declined PR. On the other side this looks like to cause some troubles. As far as I recall MLVProducer implemented your dots map :P
Just wanted to shoot out some kind of warning for you to be prepared for a unifying scenario :P
Back to topic
Right, if the focus maps can be updated, I also think it's best to unify the offsets.
I've been working on making a focus pixel map file for this @rbrune pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/762/crop_rec-add-limited-eosm-support-to/diff). I almost forgot what a PITA it is finding all the focus pixels. This new map appears to match the mv1080 map which makes sense because it is a 3x3 pattern though the full raw buffer size is the same as the mv720 (3x5). The focus pixel map files are based on the full raw buffer so if the Crop and Pan offsets are changed it shouldn't affect the map file.
I thought I had it all figured out last night but it failed the lens cap test. Basically, shooting a dark frame and adjusting the develop settings to reveal the noise will also reveal the focus pixels.
@Teamsleepkid - to answer your questions, the focus pixels in this new video mode are more subtle, the aliasing is less of a problem and the frame doesn't need to be stretched in post to correct the aspect ratio. For all practical purposed this is just like mv1080 mode except that the vertical resolution is limited to a maximum of 692 pixels instead of 1158.
Ok--got the pixel map for the @rbrune crop_rec EOSM module. If anyone is interest in how the pixel map file was made:
http://www.magiclantern.fm/forum/index.php?topic=16054.msg174322#msg174322
The focus pixel map file lives here (https://bitbucket.org/daniel_fort/ml-focus-pixels/src/fdbf1caeb8560c5066e90fa58b895ff37c416b2d/fpm/EOSM_crop_rec_map_file/?at=default) until we can figure out how to deal with two video modes that share the same raw buffer size.
If anyone wants to test this I made a binary along with the focus pixel map file that can be used with MLVFS. But wait, there's more! I also included a special version of MLVFS for Mac that has the fpm file installed and has a slight tweak Danne made to deal with with aspect ratio of the crop_rec video format. Windows and Linux users can install the fpm file but I can't help you there. Finally, I also included a dcraw badpixels formatted text file for 1280x692 if anyone wants to use dcraw. It is in my bitbucket download area:
https://bitbucket.org/daniel_fort/magic-lantern/downloads
I do have some comments now that I've played around with this crop_rec module. Of course the most obvious anomaly is that the LiveView is stretched vertically so things look a bit weird. Another thing that is more of a problem with me is that the frame rate runs at 29.97 instead of 23.98. Sure, you can use FPS override but then you can't record sync audio with mlv_snd. The higher frame rate also reduces the largest frame size that you can record continuously. Not to mention that being an old film guy I prefer the look of 24fps video. Of course if the EOSM could run mv720 at 59.94 like the other cameras then I'd be all over that for slow motion even if it means having to use a smaller image size.
tried it but i still have focus pixels.. maybe i missed somethin? reinstalled the version of magic lantern in the download and installed the new tweaked mlvfs for mac..focus pixels on my footage after trying normal and aggressive settings. are we supposed to do something with the fpm file that in missing?
Are you using both the crop_rec and mlv_rec modules? crop_rec needs to be set to the 3x3 setting and on the bottom of the LV screen if you see a warning icon then try going into and out of the ML menu, had that happen to me once. Other than that it should be working with the MLVFS in the package, the new focus pixel map file is installed.
If you're still having issues, upload a small MLV file so we can take a look at it.
https://vimeo.com/190420176 (https://vimeo.com/190420176)
in the bookshelf area focus pixels they are really hard to see but if you add sharpness and contrast they come out. easier to see if you watch in 1080p on vimeo.
https://drive.google.com/open?id=0B_4dpBYObDvEZHZGekdMdUlBX28 (https://drive.google.com/open?id=0B_4dpBYObDvEZHZGekdMdUlBX28)
mlv file with focus dots made sure i was in 3x3 mode no warning sign. i am using the 1.66 aspect ratio does that make a difference? does this only work in 16x9? used mlvfs tweeked version from your post.
Yes, I see them. Darn it. Some of these focus pixels are hard to find and they only show up in certain situations. I'm able to find some of the ones I missed by shooting a dark frame. For some reason the focus pixels in crop_rec are more subtle than other video modes and it also seems to have a different pattern.
Here are some of the suspected focus pixels that show up in your sample footage:
(https://c6.staticflickr.com/6/5671/30817499565_70a6babfed_z.jpg)
(Talk about pixel peeping!)
If you want to help out, here's how to make a focus pixel map file:
http://www.magiclantern.fm/forum/index.php?topic=16054.msg174322#msg174322
Just wanted to let you know. Not trying to be a jerk. Still wondering if true mv1080 would have less aliasing..I've been taping plastic in front of the sensor to try to make a fake vaf. . Funny thing is the best one so far was taping a piece of 35mm film in front. Not practical at all though cause I lose a stop or more of light. Does help with the aliasing though..
Quote from: Teamsleepkid on November 06, 2016, 07:45:03 PM
Just wanted to let you know. Not trying to be a jerk. Still wondering if true mv1080 would have less aliasing..I've been taping plastic in front of the sensor to try to make a fake vaf. . Funny thing is the best one so far was taping a piece of 35mm film in front. Not practical at all though cause I lose a stop or more of light. Does help with the aliasing though..
This mv720 mode with 3x3 binning has exactly the same amount of aliasing as mv1080 would have. It is 3 horizontal columns binned while recording every third line. The EOSM won't ever have the quality of 5D3 binning where pixels are binned in both horizontal and vertical direction - at least as far as we know this is part of the digic processing in the 5D3.
Here's a discussion on how pixel binning works, or at least how we believe it works:
http://www.magiclantern.fm/forum/index.php?topic=16516.0
That's only applicable to the 5D3 right now though maybe the 7D2 and the 5D4 might also do pixel binning, we don't know yet. Pixel binning on the EOSM is probably in the impossible to implement category.
There is no VAF for the EOSM sensor. Well, you could order a special filter (http://store.mosaicengineering.com/Custom-Optical-Anti-Aliasing-Filter-Price-Point-400-USD_p_37.html) but it costs more than the camera. A cheaper solution is a do it yourself filter that can fit in front of the lens (https://recordingsofnature.wordpress.com/2016/04/14/variable-antialiasing-filter-approach-for-dslr-digital-video-diy/). Though I've got to admit that using a piece of 35mm film is a clever solution.
Aliasing isn't as bad when in 1x1 (crop mode) because the pixels are physically closer together. What should theoretically help is increasing the resolution but we're limited there because of the write speed to the SD card. Reducing the frame per second rate and/or the bit depth (http://www.magiclantern.fm/forum/index.php?topic=5601.0) are both doable in order to use a larger image size.
Anyway, so much for theory--I should get back to focus pixel hunting.
Quote from: rbrune on November 06, 2016, 08:04:06 PM
The EOSM won't ever have the quality of 5D3 binning where pixels are binned in both horizontal and vertical direction - at least as far as we know this is part of the digic processing in the 5D3.
I think the binning is done in hardware, directly on the sensor. Why?
- noise analysis: read noise appears to be added after the binning process [1] (http://www.magiclantern.fm/forum/index.php?topic=16516.0). Therefore, binning must be done early in analog domain, not in software.
- Chipworks states that 1DX (also likely to use 3x3 binning) "uses an additional transistor (T5) that can be used to sum the FD nodes of three rows of even (or odd) pixels of the same color" [2] (http://www.chipworks.com/about-chipworks/overview/blog/full-frame-dslr-cameras-part-ii-canon-stays-course) (which matches my findings about the binning pattern)
Anyone ever thought about high resolution black and white? Something with good dynamic range but no color and no aliasing? Just throwing it out there..
Sure, but it is a hardware hack (http://www.maxmax.com/maincamerapage/monochrome-cameras) or for just $7,500 you can get a Leica M Monochrome (https://us.leica-camera.com/Photography/Leica-M/LEICA-M-MONOCHROM2).
Back on topic, between your file and some dark frame's I shot with the lens cap on I think I found all the focus pixels. More info on the focus pixel topic (http://www.magiclantern.fm/forum/index.php?topic=16054.msg174456#msg174456).
Updated package with the latest focus pixel map file now available for testers. This time I eliminated the dcraw "badpixels" file because it takes too much time to create and test it and I doubt anyone is actually using it. Also took the old build offline so I (hopefully) won't see any more focus pixels on testers' videos.
https://bitbucket.org/daniel_fort/magic-lantern/downloads/magiclantern-crop_rec.2016Nov06.EOSM202.zip
nice work
Quote from: Teamsleepkid on November 08, 2016, 10:09:21 AM
nice work
Hey @Teamsleepkid want to do some more testing? Of course this applies to anyone else who would like to pitch in.
I've updated the focus pixel map file for crop_rec on EOSM (http://www.magiclantern.fm/forum/index.php?topic=16054.msg174827#msg174827) and @rbrune created a version that includes 10bit_12bit raw video using the new MLV Lite raw video format so there's plenty to test.
As usual, I put the test builds on my bitbucket download area:
https://bitbucket.org/daniel_fort/magic-lantern/downloads
looks really good so far. 3x3 720 crop mode at 10 bit crashed the camera and i had to pull the battery. i was able to load the footage and after running it through mlvfs i didn't see any focus pixels! awesome. also seemed like it had less aliasing and moire but maybe I'm wrong. looked nice though. gotta get it stable so it doesn't crash and require battery pull. i'll keep testing. thank you for the build.
The freezing camera/battery pull issue with 10-bit 12-bit along with a corrupt first frame has been reported by other EOSM users. It should work at 14-bits.
By the way, if you have a scene where there's high contrast detail you might see some focus pixels pop up every now and then even with the new focus pixel map file that's in the test package. Play around with the chroma smoothing settings. That usually takes care of it.
Getting this thread back on topic. I still try to get true mv1080 and mv720 with 60fps working.
Using the dm-spy-experiments branch I tried to see if I can isolate a function that changes the camera into mv1080 mode when normal h264 recording starts.
So far I came up with the following:
Normal operation to recording 1080
Evf:00014b0c:ad:03: PATH_Select S:0 Z:10000 R:0 DZ:0 SM:1 SV:0 DT:0
Evf:00014ca0:ad:03: PathDriveMode Change: 11->6
Evf:ff37b5e8:ad:03: GetPathDriveInfo[6]
Evf:ff50478c:ad:03: Rec_HD_SelectPath LCD
Normal operation to HDMI output
Evf:00014b0c:ad:03: PATH_Select S:1 Z:10000 R:1 DZ:0 SM:1 SV:0 DT:0
Evf:ff37b5e8:ad:03: GetPathDriveInfo[11]
Evf:ff509db8:ad:03: RecStandby_x1_60fps_SelectPath HDMI(480)
HDMI output back to normal operation
Evf:00014b0c:ad:03: PATH_Select S:1 Z:10000 R:1 DZ:0 SM:1 SV:0 DT:0
Evf:ff37b5e8:ad:03: GetPathDriveInfo[11]
Evf:ff509d90:ad:03: RecStandby_x1_60fps_SelectPath LCD
So it seems the PATH_Select function decides if the camera should change the Evf mode (from mode 11 to mode 6). I hooked the PATH_Select function and overwrote the struct pointed to by reg0 to with the values that evaluate to the Rec_HD mode 11 inside PATH_Select like this:
PATH_SELECT = 0x14ac4;
MEM_PATH_SELECT = 0xe92d4030;
patch_hook_function(PATH_SELECT, MEM_PATH_SELECT, &path_select_hook, "evf_mode: PATH_select hook");
static void path_select_hook(uint32_t* regs, uint32_t* stack, uint32_t pc)
{
uint32_t *mode_struct = (uint32_t *) regs[0];
printf("mode_struct: %x %x %x %x\n %x %x %x %x\n %x %x %x %x\n %x\n",
mode_struct[0], mode_struct[1], mode_struct[2], mode_struct[3],
mode_struct[4], mode_struct[5], mode_struct[6], mode_struct[7],
mode_struct[8], mode_struct[9], mode_struct[10], mode_struct[11],
mode_struct[17]);
if (is_EOSM)
{
switch(evf_mode_index)
{
case LV_REC_MV1080_24FPS:
mode_struct[0] = 1;
mode_struct[2] = 0;
mode_struct[3] = 0;
mode_struct[5] = 1;
break;
}
}
}
Sadly when I activate the hook the live view freezes as soon as the PATH_Select function is called, e.g. by switching to info view and back or by half pressing shutter to focus. Deactivating the hook returns everything to normal.
Interestingly though is that under Debug Image buffers and EDMAC channel 1 change:
Before Image buffers: 720x480, 960x639
After Image buffers: 720x480, 1728x1151
Before EDMAC[1]: 1920x639
After EDMAC[1]: 3456x1151
So I'm not sure what's going on, if my hook is working as intended (need to log debug msgs while hooked) and certainly not sure if I'm actually looking in the right place.
Wow, you're really getting into the nitty gritty of this.
You probably already know this but the EOSM will always show a frame rate of 29.97 when you are in standby mode unless you have FPS override active. Even when the camera is setup for PAL it will show 29.97. When it starts recording H.264 it switches to the frame rate that matches the Canon settings. However, with MLV and raw video it will always record at 29.97--unless FPS override is active. Apparently the camera needs to be tricked into going into H.264 record, though preferably without actually recording an MOV file simultaneously with the MLV or raw video. I believe there were some other issues that required getting into H.264 recording mode. zebra.c has references to H264 all over it.
Don't know if this helps.
propvalues.h
#define RECORDING_H264 (__recording > 0)
#define RECORDING_H264_STARTING (__recording == 1) // 1 is preparing for recording
#define RECORDING_H264_STARTED (__recording == 2) //2 is actually recording
You can use adtg_gui to record the register configurations of all video modes.
Steps:
- enable Debug->ADTG registers
- enable Advanced->DIGIC registers
- if the image gets damaged, try Force low FPS
- go to PLAY and back to LiveView to refresh the video mode and let adtg_gui capture all the registers
- select Advanced->Log Registers Now
- repeat for other video modes
Note: the contents are appended in the log file, so you can simply log everything during one test session.
(side note: a while ago I tried to enable the digital zoom mode on 60D, using the same functions, also without success)
@A1ex:
so I looked at your raw fixes branch and tried if I can also retrieve the LV RAW resolution by reading 0xC0F06800/4:
#ifdef CONFIG_EOSM
uint32_t top_left = shamem_read(0xC0F06800);
uint32_t bot_right = shamem_read(0xC0F06804);
*width = ((bot_right & 0xFFFF) - (top_left & 0xFFFF)) * 4; // Column is /4 instead of /8
*height = (bot_right >> 16) - (top_left >> 16) - 1;
printf("res %d %d\n", *width, *height);
//*width = video_mode_crop ? 1872 : 1808;
//*height = video_mode_crop ? 1060 : 727;
return 1;
#endif
The resolution I get from 600D crop mode is 1872 x 1058 which seems to be fine.
During normal EOSM liveview mode I get 1808 x 1188 but when I do a half-shutter button press to autofocus it goes to 1808 x 725 and then quickly goes back to 1808 x 1188. Needless to say that raw recording does only work in crop mode with these width/height values.
Something really weird is going on in this standy mv720 liveview mode.
mv1080 on the EOSM should be:
*width = 1808;
*height = 1190;
That is if it behaves the same as the 650D/700D/100D.
Quote from: rbrune on December 05, 2016, 11:26:54 PM
During normal EOSM liveview mode I get 1808 x 1188 but when I do a half-shutter button press to autofocus it goes to 1808 x 725 and then quickly goes back to 1808 x 1188.
That's interesting. So, as long as you don't press the shutter halfway, you get a valid 1808 x 1188 stream?
And I guess you have to be careful not to press half-shutter while recording.
Quote from: a1ex on December 06, 2016, 03:04:16 PM
That's interesting. So, as long as you don't press the shutter halfway, you get a valid 1808 x 1188 stream?
This is how it looks like when recorded with raw_rec at 1728x1154, it usually stops after a couple frames or doesn't even start.
First couple frames are with activated crop_rec 3x3 and the latter frames from a recording without (aka standard EOSM liveview) that I did immediately afterwards (one can see that the image parts, bottom left and right, are still leftovers from the 3x3 recording).
By looking at the individual DNG files it seems one could get a couple more usable lines when using crop_rec 3x3 instead of standard mv720, maybe 698 instead of 692.
That looks very familiar. Looks like some of the PREFERRED_RAW_TYPE settings I was trying out searching for something that showed no focus pixels.
http://www.magiclantern.fm/forum/index.php?topic=5601.msg175416#msg175416
@rbrune -- could you push your work branch to bitbucket so we can check it out?
Quote from: dfort on December 07, 2016, 02:39:23 AM
That looks very familiar. Looks like some of the PREFERRED_RAW_TYPE settings I was trying out searching for something that showed no focus pixels.
http://www.magiclantern.fm/forum/index.php?topic=5601.msg175416#msg175416
@rbrune -- could you push your work branch to bitbucket so we can check it out?
Sure, here you go:
https://bitbucket.org/rbrune/magic-lantern/branch/crop_rec_raw_video_10bit_12bit
It's a mix of the 10/12bit, crop_rec and raw_fixes branches.
Crazy stuff going on in that build @rbrune ! It clearly shows that the mv1080 buffer is active and a half-shutter press switches to the mv720 buffer.
(https://c2.staticflickr.com/1/153/30690439553_bc685ea014_z.jpg)
I turned on the RAW_DEBUG_TYPE but it seems that all settings I tried so far are causing a "Raw detect error." Also captured a full raw buffer but to do that I used silent picture and a half-shutter press captures the image it only captured the mv720 buffer. The raw buffer capture was clean though.
@rbrune
I turned on the RAW_DEBUG_TYPE and tried every PREFERRED_RAW_TYPE possible from 1 to 128 and the following were the only ones that didn't cause a "Raw detect error."
7, 11, 48, 50, 75, 80, 87
Unfortunately none of these produced a useable picture. It was worth running this test anyway because I keep finding useable settings above the 64 ".max" limit that is in the debug menu.
So you're getting an mv1080 frame size but the image is still obviously mv720 with the bottom 1/3 of the image filled in with what looks like remnants of whatever was in the buffer when the image was written to the card. The every other frame difference on the bottom 1/3 might be because the buffer empties out every other frame?
If I understand the problem it is because raw video is taken from a video stream that feds the LiveView screen and on the EOSM that stream is always running in mv720 mode but at 29.97fps (mv720 should be 59.94fps). It only changes when it is recording H.264 video. In lv-img-engio.c there is a function called digic_find_lv_buffer - maybe there is a different lv buffer that needs to be used on the EOSM? Just speculating because there seems to be several raw buffer streams to choose from.
Don't know if I'm spouting out useful suggestions or just adding to the noise but I just thought of something that might help.
The only setting where the EOSM shows the "proper" frame rate is in "Movie crop mode" a.k.a. crop mode hack. The source code for crop-mode-hack.c is relatively small and maybe there is some hint in there that we can use?
unsigned int movie_crop_hack_enable()
{
if(!is_crop_hack_supported() || video_mode_crop)
{
return 0;
}
video_mode[0] = 0xc;
video_mode[4] = 2;
prop_request_change(PROP_VIDEO_MODE, video_mode, 0);
return 1;
}
Thanks for all the testing dfort!
The thing I want to try next is to log the register settings for real mv1080 mode and see if setting those like we set 3x3 mode results in a filled up buffer. Or even try to do that while in the 600D crop hack mode buffer.
Another approach would be to just use the current 3x3 crop mode and try to only modify the registers that set start read line and end read line.
Higher fps is another story. It could even be that the timing in the liveview pseudo mv720 mode is prohibitive to reading out more lines. I have the feeling that Canon might have done all of this on the EOS M just to save some power since live view is basically always on.
When dealing with this sort of semi-corrupted frames, here's a change I often make in raw.c:
static int autodetect_black_level(int* black_mean, int* black_stdev_x100)
{
*black_mean = 2048;
*black_stdev_x100 = 1;
return 1;
...
}
Reason: this function checks for a valid raw buffer, with a clean optical black area. There are some situations where the raw buffer is not clean (such as when changing resolution, or when a frame gets corrupted for some reason); in this case, the black level computed from such an image would be invalid and can affect entire recordings.
However, in this case, where you expect incomplete image data, the above check will do more harm than good, so for testing, you can simply hardcode some value (which will bypass the checks).
The behavior is quite puzzling for me. Was the video captured with mlv_lite? Do you get the same artifacts if you take silent pictures? (simple or burst)
Any clues to why mlv_rec(10_bit, 12_bit) almost works(every other dng corrupted)? Can it be related to dbg functions? I see #undef RAW_DEBUG is set so it can,t be this right? From a noob C standpoint. Is there some functions I can disable to test in raw.c for the 5D mark III?
Quote from: Danne on December 11, 2016, 04:11:09 PM
...Is there some functions I can disable to test in raw.c for the 5D mark III?
A bit off topic--this might offer some hints: lv_set_raw / lv_select_raw (http://www.magiclantern.fm/forum/index.php?topic=18393.msg176315#msg176315)
Of course we're trying to find a way to turn the EOSM into a 5D3 so it isn't too far off topic.
Quote from: rbrune on December 11, 2016, 12:14:08 AM
Thanks for all the testing dfort!
The thing I want to try next is to log the register settings for real mv1080 mode and see if setting those like we set 3x3 mode results in a filled up buffer. Or even try to do that while in the 600D crop hack mode buffer.
Another approach would be to just use the current 3x3 crop mode and try to only modify the registers that set start read line and end read line.
Higher fps is another story. It could even be that the timing in the liveview pseudo mv720 mode is prohibitive to reading out more lines. I have the feeling that Canon might have done all of this on the EOS M just to save some power since live view is basically always on.
Anything new to test?
Something that is interesting with the 600D crop hack mode is the the frame rate is 23.98 instead of 29.97 in all other modes. This may be related to the comment I quoted on the first post of this topic:
Quote from: a1exThe EOSM is unique b/c Canon firmware doesn't put it into mv1080 until you start recording (H.264).
The crop hack mode changes the property of the video mode. Maybe it is possible to do something similar to get into mv1080 and true (50p/60p) mv720 mode?
Something I haven't tried yet is to use the dm-spy-experiments branch to try and find out what goes on with PROP_VIDEO_MODE when the camera starts recording H.264.
Quote from: dfort on December 19, 2016, 07:46:40 PM
The crop hack mode changes the property of the video mode. Maybe it is possible to do something similar to get into mv1080 [...]?
Indeed. Enabling crop hack and re-enabling line skipping / column binning in crop_rec should do the trick.
You should be able to get a proof of concept with adtg_gui (no coding needed, just by fiddling with menus). Starting point: ADTG[0x800C] = 2 to set line skipping to 3, CMOS[7] to center vertically. Column binning is usually configured from one of the CMOS registers.
Quote from: a1ex on December 19, 2016, 08:13:36 PM
You should be able to get a proof of concept with adtg_gui (no coding needed, just by fiddling with menus). Starting point: ADTG[0x800C] = 0 to disable line skipping, CMOS[7] to center vertically. Column binning is usually configured from one of the CMOS registers.
I'm really trying hard to follow directions, using this post (http://www.magiclantern.fm/forum/index.php?topic=17021.msg165204#msg165204) as an example of how to use adtg_gui and I'm able to fiddle with menus but can't find ADTG[0x800C]:
(https://c5.staticflickr.com/1/604/31716930676_4deac0490d.jpg)
It is clearly in the adtg_gui.mo source code:
{DST_ADTG, 0x800C, 0, "Line skipping factor (2 = 1080p, 4 = 720p, 0 = zoom)"},
Quote from: a1ex on December 19, 2016, 08:13:36 PM
Enabling crop hack and disabling line skipping / column binning in crop_rec should do the trick.
So changing this to 0x800C, 0 is what I should try?
crop_rec.c
/* 3x3 binning in 720p (in 1080p it's already 3x3) */
case CROP_PRESET_3x3_1X:
/* ADTG2/4[0x800C] = 2: vertical binning factor = 3 */
adtg_new[0] = (struct adtg_new) {6, 0x800C, 2};
break;
I just want to make sure I'm understanding this.
I believe that's just about right @dfort!
Yes, you can try. For adtg_gui, try using this (http://www.magiclantern.fm/forum/index.php?topic=10111.msg145036#msg145036) as a guide.
Some registers are only updated when switching video modes (so you need to do that).
Quote from: a1ex on December 20, 2016, 07:46:02 AM
Some registers are only updated when switching video modes (so you need to do that).
Getting closer. Recording a short H.264 will expose more registers. Found it:
(https://c8.staticflickr.com/1/766/30949642223_4eeb513b8f.jpg)
Figured out that the PLAY button allows editing the hex value but only in one direction. Fortunately it was the direction that I wanted to go.
(https://c8.staticflickr.com/1/671/31759329735_ce1bed1d30.jpg)
Now how do you get the edited value to take?
(https://c4.staticflickr.com/1/469/31759329235_6d8f9fded4.jpg)
Doesn't seem like anything is happening. Shouldn't 0x2 put us into 1080p mode?
Looks like both battery and operator needs a recharge.
Quote from: dfort on December 20, 2016, 09:11:15 AM
Now how do you get the edited value to take?
If the register appeared when starting H.264, you need to perform the same action in order to apply the changes.
However, I expect it to appear also when exiting LiveView (e.g. to PLAY mode or Canon menu) and going back.
Quote from: dfort on December 20, 2016, 09:11:15 AM
Figured out that the PLAY button allows editing the hex value but only in one direction.
Should work with SET followed by up/down/left/right arrows (or top scrollwheel, but you don't have one on the M).
Am I doing this right? Changing CMOS[7] shows a definite change:
(https://c3.staticflickr.com/1/779/31730737346_176589112d.jpg)
And looking at ADTG2[800c] while recording H.264 shows a value of 0x2 and a frame rate of 23.9xx indicating that it is in 1080p mode:
(https://c8.staticflickr.com/1/704/31767803735_a0c2d08e55.jpg)
But modifying that register doesn't seem to have any affect on the frame rate. Also, shooting a silent still and looking at the full raw buffer indicates that it is still in 720p mode.
(https://c5.staticflickr.com/1/464/31394041620_1d16b333d0.jpg)
Shooting a silent still while recording H.264 does capture the mv1080 raw buffer (full raw buffer size=1808x1190) however that just puts us back to where we started--trying to figure out how to get into mv1080 mode without having to simultaneously record H.264 video.
I also tried changing ADTG2[800c] in crop_rec.c and turning on Movie crop mode but that didn't work either.
Seems to me that there should be a way to switch into mv1080 mode like crop-mode-hack.c switches into mv1080crop mode. Movie crop mode (crop hack) seems to be the only mode that is working properly on the EOSM. All the other modes default to 29.97fps including the @rbrune crop_rec which is the closest we got to mv1080 on the EOSM so far albeit recording 3x3 on the mv720 buffer and looking at a wonky LiveView display.
[EDIT] I'm also trying to compile dm-spy-experiments to see if I can log the properties that change when recording H.264. I used that branch a while back when trying to track down the shutter-bug but now it doesn't compile:
debug.o: In function `run_test':
debug.c:(.text+0x988): undefined reference to `debug_intercept'
debug.c:(.text+0xbb0): undefined reference to `debug_intercept'
make: *** [magiclantern] Error 1
You need to switch the crop mode hack on, then undo the crop from adtg_gui.
The CMOS register is set for every single frame, that's why it takes effect right away.
The line skipping register is different. You need to exit LiveView and then return in oder to apply the changes.
To compile the dm-spy branch, also set CONFIG_DEBUG_INTERCEPT = y in Makefile.user.
Quote from: a1ex on December 20, 2016, 09:38:18 PM
You need to switch the crop mode hack on, then undo the crop from adtg_gui.
...exit LiveView and then return in oder to apply the changes.
Only active module is adrg_gui
Turned crop mode hack on
Turned on adtg_gui
Recorded short H.264
Opened adtg_gui and changed ADTG2[800c] to 0x2
Exited LiveView (opened Canon menu)
Recorded short H.264
Still no change - camera is still in crop hack mode (1x1) and didn't change to 3x3.
(https://c6.staticflickr.com/1/516/31772471765_8c6f622df3_z.jpg)
Got dm-spy branch compiled -- thanks!
On 5D3, I didn't have to change the line skipping register; instead, I had to change ADTG[0x8000]. Does it help if you override that one as well?
A technique I've used:
- find the differences between two video modes (enable the first video mode, then select "Show: Modified from now on", then switch to the second video mode)
- save those differences to a log file, for reference (Log Registers Now)
- override those registers that were different (Lock Displayed Registers)
- switch back to the first video mode; it should behave like the second one, at least in some aspects
- find a minimal subset of registers that have to be changed (trial and error)
I haven't had any luck overriding ADTG[0x8000]. At least not with the values that I've tried so far.
Your technique gave me an idea. Since the main thing we want to see is what is changing in the display registers when the camera is recording H.264 I selected "Show: Modified from now on" then started recording H.264 and while the camera was still recording I selected "Log Registers Now". I did it several times to make sure the results were consistant and they were:
Canon EOS M 2.0.2 - Record H.264 1080p 24 NTSC
00028172: 6fd (was 3b7) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=33be8 addr=40481b28 Line count to sample. same as video resolution (g3gg0)
00028173: d71 (was 47c) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=33be8 addr=40481b2c
00f00008: 400 (was 800) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=33e38 addr=40482526
0002802c: 0 (was 110) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=360fc addr=40481b80
00028047: 550 (was 440) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=360fc addr=40481b84
00028178: 6fd (was 3b7) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=359a8 addr=40481bd0 dwSrFstAdtg1[4], Line count + 1
00028179: d71 (was 26f) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=359a8 addr=40481bd4 dwSrFstAdtg1[5]
000282b6: 6f4 (was 3b4) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=359a8 addr=40481be8
000288c6: 4a4 (was 2d4) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f24
000288ce: 4a4 (was 2d4) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f34
000288d5: 903 (was b03) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f40
000288d6: 1cb (was 11f) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f44
000288d7: 2f6 (was 1d2) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f48
000288da: 201 (was 504) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f54
000288db: 706 (was b0a) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f58
000288df: 0 (was 2) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f60
000288e0: 2 (was 1) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f64
000288e1: 1 (was 0) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f68
000288e7: 6 (was 5) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f70
000288e8: 9 (was a) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f74
00028196: 12e (was 190) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35e74 addr=40481ca0 dwSrFstAdtg1[2], Line count + 1
00028197: 38c (was 13a) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35e74 addr=40481ca4 dwSrFstAdtg1[3]
Then I went the other way. With the camera recording H.264 I selected "Show: Modified from now on" stopped recording and selected "Log Registers Now" and did that several times and again got consistant results:
Canon EOS M 2.0.2 - Stop recording H.264 1080p 24 NTSC
00028172: 3b7 (was 6fd) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=33be8 addr=40481b28 Line count to sample. same as video resolution (g3gg0)
00028173: 47c (was d71) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=33be8 addr=40481b2c
00f00008: 800 (was 400) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=33e38 addr=40482526
0002802c: 0 (was 110) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=360fc addr=40481b80
00028047: 550 (was 110) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=360fc addr=40481b84
000282f3: c (was 30a) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=359a8 addr=40481be0 Line count that gets darker (top optical black related)
00028305: 3 (was 0) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=40481860
00028306: 7 (was 0) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=40481864
00028307: f (was 3) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=40481868
00028308: b (was 7) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=4048186c
0002830f: 3 (was 0) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=40481870
00028310: 7 (was 0) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=40481874
00028311: f (was 3) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=40481878
00028312: 9 (was 7) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=4048187c
00028335: 6f (was 0) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=40481884
00028336: 36f (was 0) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=40481888
0002833d: 6f (was 0) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=40481894
0002833e: 1ef (was 0) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=40481898
0002839c: 3 (was 1) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=404818ac
000283a9: 3 (was 1) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=404818e0
00028820: 303 (was 301) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f9c
00028826: e05 (was 5) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481fa4
00028827: 42 (was 3) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481fa8
00028828: 3010 (was 20) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481fac
00028860: f (was e) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481fbc
00028866: 7 (was 9) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481fc0
"Lock Displayed Registers" while recording H.264 didn't work--the frame rate stayed at 29.97 and shooting a simple silent captured the mv720 buffer. The LV display was also garbled up.
Next I thought I'd see what happens when running the same test this time while recording mlv_rec (image size was at 640x78):
Canon EOS M 2.0.2 - Record mlv_rec Canon menu 1080p 24 NTSC but camera records mv720
0002802c: 0 (was 110) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=360fc addr=40481b80
00028047: 550 (was 440) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=360fc addr=40481b84
00028179: 47c (was 26f) ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=359a8 addr=40481bd4 dwSrFstAdtg1[5]
Interesting how few registers changed. Maybe they changed just from switching over to the ML menu? I tried going the other direction but that just crashed the camera.
One more test I tried was to see if the raw buffer changes right when H.264 recording starts and stops. This was much less precise, I just tried shooting a simple silent still right before and right after pressing the record button. Seeing a 1734x693 capture indicated it was a mv720 buffer and 1734x1156 indicated a mv1080 raw buffer. It seems that there is some sort of a time lag because a silent still captured right after recording starts was still mv720 and right after recording stopped it was still mv1080. In other words it looks like the raw buffer size switches when it is actually writing to the card--does that make sense?
It's not obvious what's going on; I would expect a register to be set to 2 in 1080p and 4 in 720p.
Interesting that some registers got changed when recording MLV; I didn't expect any of them to change, maybe it's the extra hacks option?
FPS registers are in the "DIGIC registers" category in adtg_gui, so they won't change by overriding the above ones. You don't need to change them - they already work with crop mode hack.
Raw buffer size (as autodetected by ML) is also retrieved from two DIGIC registers (reply #86). Most registers cannot be read back directly, so they have a shadow copy that contains the last value written to that register. Well-behaved code sets both the register and its shadow copy, but there is some code that only sets the register. I wonder if the EOS M code overrides the resolution registers directly to 720p (so the change is invisible if we read the shadow copy).
This condition can be detected in QEMU, but only after being able to emulate the LiveView (or some part of it) on EOS M. Not very practical. Although, being able to log all the I/O register accesses would be extremely useful for emulation.
If you ever get bored (such as on a long train ride), you could try fiddling with those registers to figure out their meaning. There are lots of interesting things behind them (custom resolutions, extra highlight detail, binning patterns and so on).
Quote from: a1ex on December 22, 2016, 03:40:00 PM
It's not obvious what's going on; I would expect a register to be set to 2 in 1080p and 4 in 720p.
That's what I was looking for in 0002800C:
{DST_ADTG, 0x800C, 0, "Line skipping factor (2 = 1080p, 4 = 720p, 0 = zoom)"},
or something to be set to 0 in 1080p and 1 in 720p:
int mv720 = mv && video_mode_resolution == 1;
int mv1080 = mv && video_mode_resolution == 0;
but that didn't show up either. The most I got out of reply #86 was:
Quote from: rbrune on December 05, 2016, 11:26:54 PMSomething really weird is going on in this standy mv720 liveview mode.
Looks like I'm going to have to book a long train trip.
I was experimenting with shooting raw video while simultaneously recording H.264 and late one night before going to bed I captured a short MLV of my keyboard in mv1080 mode.
(https://c1.staticflickr.com/1/529/31892790976_4a31d562a1_z.jpg)
I saved just one DNG frame and then tried shooting something worth posting but several days later all I got was pink earthquake video. In any case, this just proves what a1ex said back on the first post about the EOSM switching into 1080p when H.264 movie recording starts.
exiftool
Image Size : 1728x972
interesting. just gotta get it repeatable
I did some testing with dfort experimental EOSM_mv1080 branch here:
https://bitbucket.org/daniel_fort/magic-lantern/branch/EOSM_mv1080_experiments
I actually managed to get some good 10bit mv1080 frames out of the build. Well the files had some various unstable results with white and black level and some files are corrupted but proof is there. mv1080 could be possible. But how.
So the hack is like this.
1 - Start H.264 recording
2 - Enter ML menu and select raw recording and while H.264 is recording start record again for MLV recording.
3 - Stop MLV recording go back to ML menu and deselect raw recording
4 - Go out of the menu and stop H.264 recording
Very convenient ;)
Here are the originals files, MLV, dng files
https://bitbucket.org/Dannephoto/magic-lantern/downloads/EOSM_mv1080.zip
I developed with mlv_dump and chroma smooth cs2x2 through cr2hdr.app then adding back white and black level with exiv2.
exiv2 -M"set Exif.Image.BlackLevel Long 2047" -M"set Exif.Image.WhiteLevel Long 15000" *.dng
Some comparisons
mv1080
(https://s30.postimg.org/uucw7d39t/Screen_Shot_2017_01_04_at_17_06_23.png)
mv720
(https://s30.postimg.org/sr2gzp3gx/Screen_Shot_2017_01_04_at_17_06_44.png)
mv1080
(https://s30.postimg.org/kzlr14zbl/Screen_Shot_2017_01_04_at_17_07_47.png)
mv720
(https://s30.postimg.org/ijjxnah8x/Screen_Shot_2017_01_04_at_17_07_56.png)
Almost there.. moire and aliasing looks a lot more controllable in the punch in shot on mv1080:)
@Danne -- Thanks for posting the pics.
This proves that the EOSM can do mv1080 that matches the 650D and 700D because they are very similar, though a the same time very different. Now how to do it without having to record H.264 and MLV simultaneously, thought that's an interesting "feature"!
It,s not a 5D mark III but mv1080 it,s heaps better than mv720p, even though that is cool as well. The eosm is great for stills, timelapsing and in crop_rec it,s really useful with HDR(H.264). If mv1080 moves in there, now with 10bit it will be a true rebel camera. Amazing really, with the 10bit delivering continouos in crop_rec and mv720p.
Nice findings. :o
Thanks for sharing these results @Danne!
So glad we all stuck with the M through thick and thin, right @dfort?
@danne,
Port it to 5D3, I would love h264 proxy recording ;)
Quote from: Kharak on January 05, 2017, 02:20:09 PM
Port it to 5D3, I would love h264 proxy recording ;)
If you look in mlv_rec.c and raw_rec.c you'll see this:
/* if you somehow managed to start recording H.264, let it stop */
if (RECORDING_H264)
return 1;
Just comment that out and it should work. I don't really want to put test builds out there with this because it is too much of a hack. You have to start recording H.264 then get into the ML menu to turn on the raw recording option then you can start and stop recording raw while H.264 is continuously recording. You need to disable raw in order to stop the H.264 recording.
any test builds for this?
Quote from: Teamsleepkid on January 07, 2017, 02:58:08 AM
any test builds for this?
Reply #122
If you want to play around with this:
http://www.magiclantern.fm/forum/index.php?topic=16012.0
or
http://www.magiclantern.fm/forum/index.php?topic=15894.0
or pick whatever suits you from:
http://www.magiclantern.fm/forum/index.php?board=25.0
Oh ok sorry. Didn't see the part about not wanting test builds of this because it's too much of a hack.
Surprise, I posted a test build of this on my bitbucket download page (https://bitbucket.org/daniel_fort/magic-lantern/downloads). It is highly experimental including a firmware upgrade to 2.0.3.
Not that this is anything useable yet but we made some interesting discoveries playing around with it. Most important is that the EOSM can produce mv1080 that rivals the quality of the other cameras. Don't want to double post but this EOSM mv1080 frame of Koks the cat is too good not to share on this topic.
(https://s24.postimg.org/mvvjqkkbp/mv1080eosm_000000.png)
More information about how this was made on the Canon EOS M topic (http://www.magiclantern.fm/forum/index.php?topic=9741.msg178355#msg178355).
One thing we didn't anticipate was that the full raw buffer size of an EOSM mv1080 is 1808x1189. At least that's what we're getting with the hack build. All the other cameras that show focus pixels in raw video have a size of 1808x1190. I was so sure of this that I prepared a focus pixel map file for the EOSM in anticipation of getting this mv1080 puzzle solved. Fortunately it looks like the one vertical pixel was shaved off the bottom of the frame so simply renaming the map file got it working.
mv1080 tried it out. very cool. can't wait for it to be working looks sweet in between the corrupt frames. also tried out 720 3x3 on the experimental page and it looked great. so mv1080 should be even better once its all working. also on 203 now thank you :)
ps that cat looks awesome and the wire mesh on that air conditioner doesn't look like a moire mess! very excited.
Quote from: dfort on January 14, 2017, 06:32:25 AM
One thing we didn't anticipate was that the full raw buffer size of an EOSM mv1080 is 1808x1189. At least that's what we're getting with the hack build. All the other cameras that show focus pixels in raw video have a size of 1808x1190.
This comes from here (https://bitbucket.org/hudson/magic-lantern/pull-requests/778/raw-fixes-part-3/commits) (autodetecting resolution is not perfect (https://bitbucket.org/hudson/magic-lantern/commits/38e78de08ad9632c9dbd2079cff03ccef8fa124b?at=unified), as some cameras report the EDMAC value + 1, others - 1, others exact, others have mixed behavior, depending on video mode). Not sure why.
Guess we should test all other cameras with focus pixels and adjust the autodetected value...
Quote from: a1ex on January 14, 2017, 09:09:41 AM
Guess we should test all other cameras with focus pixels and adjust the autodetected value...
Please describe the steps we should take for running the test and how you want the results reported.
It turned out that I had a file with the raw buffer height one pixel different from one that Danne recorded.
Here's the mlv_dump -v information from the shot of Koks the cat:
Block: RAWI
Offset: 0x00000034
Size: 180
Time: 0.843000 ms
Res: 1728x972
raw_info:
api_version 0x00000001
height 1190
width 1808
pitch 2260
frame_size 0x003973A8
bits_per_pixel 10
black_level 0
white_level 937
active_area.y1 28
active_area.x1 74
active_area.y2 1184
active_area.x2 1808
exposure_bias 0, 0
cfa_pattern 0x02010100
calibration_ill 1
Here's what I got on my test:
Block: RAWI
Offset: 0x00000034
Size: 180
Time: 0.780000 ms
Res: 1728x972
raw_info:
api_version 0x00000001
height 1189
width 1808
pitch 2260
frame_size 0x0039674C
bits_per_pixel 10
black_level 0
white_level 813
active_area.y1 28
active_area.x1 74
active_area.y2 1183
active_area.x2 1808
exposure_bias 0, 0
cfa_pattern 0x02010100
calibration_ill 1
Danne and I re-worked my focus pixel fixing scripts so that they can work with files that are slightly off in height. So far I have only seen this happened when using the H.264 hack to record mv1080 on the EOSM but from your notes it looks like it can also happen randomly. Does it actually affect the recording? These hack recordings have lots of corrupt frames so we shouldn't be judging from these tests.
Quote from: dfort on January 14, 2017, 09:33:45 PM
Danne and I re-worked my focus pixel fixing scripts so that they can work with files that are slightly off in height.
Even better. Currently, there can be a difference of +/- 1 in heights.
Quote
So far I have only seen this happened when using the H.264 hack to record mv1080 on the EOSM but from your notes it looks like it can also happen randomly.
It's not random, it's just inconsistent across camera models and/or video modes.
Quote
Does it actually affect the recording?
Best guess, based on 10/12-bit issues before implementing RAW_SLURP: one missing or extra line at the end, and possible corruption in the first line (which is optical black anyway). It all depends on how Canon's image processor handles mismatches in image data size (how much you request vs how much it has).
If the focus map files work just by renaming, that's a good sign (it means the height mismatch doesn't result in the image being shifted vertically).
I've interpreted the lack of issues reported during the last week as encouraging (as in, no surprises encountered => nothing to report). But since you mentioned this focus pixel map issue (height mismatch), which I expect to be present on 650D and 700D (though not on 100D), I guess it's more likely that nobody bothered to report it.
Quote from: a1ex on January 15, 2017, 01:03:10 AM
If the focus map files work just by renaming, that's a good sign (it means the height mismatch doesn't result in the image being shifted vertically).
Yes, that's the case. At least with all the samples that we have checked which is only a couple.
In the scripts we now do this:
raw_width=1808
raw_height=11**
raw_buffer=1808x11**
Quote from: a1ex on January 15, 2017, 01:03:10 AM
I've interpreted the lack of issues reported during the last week as encouraging (as in, no surprises encountered => nothing to report). But since you mentioned this focus pixel map issue (height mismatch), which I expect to be present on 650D and 700D (though not on 100D), I guess it's more likely that nobody bothered to report it.
I think that most people probably aren't noticing this. If they are using mlv_dump they are used to seeing focus pixels or maybe using chroma smoothing to deal with them.
Could this also very well be the reason why sometimes the focus map files would often miss the borderlines full of dancing dots from back in the day, remember?
No, that was because there weren't enough pixels to interpolate at the edges of the frame. The developers worked that out so it shouldn't be a problem, though with high contrast straight lines and dual iso it is sometime still a problem. A bit of chroma smoothing finishes off the remaining focus pixels.
Some fun stuff from messing around with mv1080 on EOSM.203 (ml-experimental.2017Jan12 - Thanks @dfort) in 24p 1728x972 MLV 10-bit @ ISO 3200 w a 50L of my several months old kitten, Willoughby...
Adobe Standard:
(https://c1.staticflickr.com/1/506/32362156225_4a7caa2893.jpg) (https://flic.kr/p/RiJvyM)
Cinelog-C:
(https://c1.staticflickr.com/1/585/32211894022_110d4794a1.jpg) (https://flic.kr/p/R5snR9)
Cinelog-REC709:
(https://c1.staticflickr.com/1/484/31984978480_081b06e939.jpg) (https://flic.kr/p/QJpnLd)
Here's what it actually looks like w corruptions (ProRes4444 Rec709 from cr2hdr.app (https://www.magiclantern.fm/forum/index.php?topic=15108.0cr2hdr.app)):
https://vimeo.com/199875968
Original DNG's (w Cinelog-C & Cinelog-REC709) & Original MLV (152.6 MB) -- https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads
All files were processed with latest cr2hdr.app (https://www.magiclantern.fm/forum/index.php?topic=15108.0cr2hdr.app) (enabled cs2x2, Black level - 2047, White level - 15000 within mlv_dump settings) with ease and Thanks @Danne!
Thanks for sharing.
Feels like we're so close to having mv1080 on the EOSM -- yet so far.
@rbrune -- You seem to have gotten deep into this, any progress or comments?
@DeafEyeJedi: Strong the force is in young kitty of yours... full video jamming almost achieved... 8)
Quote from: dfort on January 17, 2017, 09:15:57 PM
@rbrune -- You seem to have gotten deep into this, any progress or comments?
I'm still here and reading about whats going on - but currently lack time to further fiddle around with my EOSM. When I have some more free time I will continue with it :)
Yes!!
Dumb questions from a newbie. If I want to shot continuous RAW video with this Canon EOS M, then what is the highest resolution that I could get currently?
I confuse with the crop mode, 3x3 pixel binding etc.
Is the crop mode will make the crop factor of the Canon EOS M to 3x compared to 35mm full frame or is it 3x1.6 (1.6 is the crop factor of this Canon EOS M sensor)?
From what I read, the crop mode is good to reduce moire and aliasing artefact but more prone to noise at high ISO. Does 3x3 pixel binding could reduce the noise at high ISO?
Thanks
Crop_rec in 10bit gives continuous output. And it looks really great.
@yskunto. Have a look at this: http://rbrune.github.io/mlraw/
Pretty much tells you everything you need to know. 3x3 is missing on there but look at the 650 d pretty much the same thing. The 1:1 produces the least Moore and aliasing but it's a small sensor size.
crop_rec, Movie crop mode, zoom mode, 3x3, 3x1.6, yeah this is confusing. What's worse is that this discussion is about getting mv1080 working on the EOSM so pretty much nothing in your post is on topic.
Quote from: yskunto on January 25, 2017, 12:52:50 PM
From what I read, the crop mode is good to reduce moire and aliasing artefact but more prone to noise at high ISO. Does 3x3 pixel binding could reduce the noise at high ISO?
You're referring to Movie crop mode which is the only video mode that is actually working the way it is intended to work on the EOSM. This mode uses all of the pixels (1x1) and doesn't skip any vertical or horizontal lines like the crop_rec module or mv1080 (3x3). Note that only the 5D3 does pixel binning, all other ML enabled cameras skip pixels which can create aliasing artifacts including moiré patterns.
I'm not sure that Movie crop mode is more prone to noise, could you provide a link to where you read that?
The disadvantage to Movie crop mode is that it uses a tiny part of the sensor. When shooting H.264 it will produce a 1920x1080 image with about a 3x crop of the "normal" APS-C image (not to a 35mm "full frame" sensor) but in raw video the maximum size possible in Movie crop mode is 1792x1024. Continuous recording? It might be possible at 23.98fps using 10bit with the 10/12-bit RAW video experimental build (https://builds.magiclantern.fm/experiments.html) but I haven't tested it. With 14bit I was able to get continuous recording with a 1280x720 image size but that's even a smaller area, about a 5x crop.
@Teamsleepkid - nice reference, graphics explain better than words!
[EDIT] The crop_rec mode that Danne mentioned is as close as can get to a stable mv1080. It is a module for the 5D3 that rbrune got working on the EOSM, he's the guy who did those great graphics Teamsleepkid pointed out. It is 3x3 and uses much of the sensor so I'm not sure why it is called a "crop". The disadvantage is that it is using the raw buffer that is normally used for mv720, a.k.a. 720p, so the vertical size is limited. In raw it maxes out at something like 692 vertical lines so it give a super wide image even if you set the aspect ratio to 16x9. In other words, it is very much of an experimental video mode.
Quote from: dfort on January 26, 2017, 03:09:53 AM
It is 3x3 and uses much of the sensor so I'm not sure why it is called a "crop".
This module was originally written to enable 1:1 crop on cameras without one (e.g. 5D3), and the naming suggestion came from here (http://www.magiclantern.fm/forum/index.php?topic=10111.msg125694#msg125694). The other binning modes were added afterwards, as the implementation was quite similar.
Time for a rename?
I don,t know about the rename. It,s kind of established in my brain now.
bin_crop_rec.mo?
bin_rec.mo
I would however have nothing agains metadata as suggested here :).
http://www.magiclantern.fm/forum/index.php?topic=17021.msg173941;topicseen#msg173941
hack, wreck, it: "I'm against it!" (Professor Quincy Adams Wagstaff).
If "bin" is meant to say binning: Rename to "bnn" and count me in.
Form follows function. In this case form = wording.
"bnn"
Yes.
Horse Feathers! According to Wikipedia (so it must be true) the definition of binning is:
QuoteIn the context of image processing, binning is the procedure of combining a cluster of pixels into a single pixel. As such, in 2x2 binning, an array of 4 pixels becomes a single larger pixel, reducing the overall number of pixels.
Well the EOSM doesn't do binning, it does skipping, yet it has crop_rec. Then again, "What's in a name?" (William Shakespeare)
Name by sensor size? Super 35, super 16, super 8. Lol mark iii "vistavision"
Quotecrop_rec, Movie crop mode, zoom mode, 3x3, 3x1.6, yeah this is confusing. What's worse is that this discussion is about getting mv1080 working on the EOSM so pretty much nothing in your post is on topic.
Hey all, You nailed it on confusing! Im trying to figure out Movie Crop Mode on my EOSM. tried this, tried that but all leads me to 3x3. Couldnt find anything on 2.0.2 so upgraded to 2.0.3, put on the experimental from your repo DFort! and got Crop_Rec.mo
How and where do I find
QuoteMovie crop mode which is the only video mode that is actually working the way it is intended to work on the EOSM
I give up!, Also this mode still gives you 3x zoom ? that is my whole point in wanting it.
Movie crop mode is the only mode giving 3x zoom.
crop_rec is a smart way of binning mv720p footage so it gives same quality as mv1080p. Caveat. You get lesser height.
The confusion seems to be between 3x3 and 3x.
3x3 can mean that the sensor is combining 3 pixels in the horizontal and vertical axis. This is know as binning. Only the 5D mark III does this. All the other cameras, including the EOSM does what is called line skipping.
It is a little more complicated because of the bayer pattern on the sensor this but let's simplify it to illustrate what is going on.
(https://c1.staticflickr.com/1/311/32424981801_65cb7c35ca.jpg)
Binning is making these nine pixels behave like one giant pixel while line skipping only uses the one pixel in the middle. Line skipping is much simpler but it does have aliasing issues. The net result from both of these methods is the same. Start with a sensor that has an active horizontal resolution of 5,760 pixels and a vertical resolution of 3,240 and you end up with a 1920x1080 image after doing 3x3 pixel binning or line skipping. Now the EOSM and most of the other Magic Lantern enabled cameras don't have that high of a resolution. The EOSM has a 5184 x 3456 (18 megapixels) sensor so the maximum resolution, assuming you want to keep the HD aspect ratio of 16:9 is 1728x972. How does it do 1920x1080 H.264 video? We're pretty sure it scales the image. How about 1280x720? It does a 3x5 pattern which means it skips (or in the 5D3 bins) 5 pixels vertically and again resizes in camera for H.264 but in raw you need to adjust the aspect ratio to un-squish it.
3x crop on the other hand uses all of the pixels, a.k.a. a 1x1 pattern. Since the pixels are closer together and there is no line skipping there is also less aliasing but a 1920x1080 image would use just a small area of the total sensor thus the "crop" in this video mode. You don't need a module to do this, it is either the "Movie crop mode" option or the digital zoom option. One thing I personally like about Movie crop mode when shooting H.264 is that it extends the effective focal length of your lens so that a 16mm prime lens can become a 48mm lens or adjusted to full frame if your brain thinks that way it would be like having about a 26mm and a 77mm lens in one. Sort of, not really, but good enough to make a point.
Uh oh, I mentioned zoom mode. This isn't as well known but if you tap the magnifying glass icon on the screen while in 1920x1080 mode in the Canon menu you can also get into a crop mode, confusing because it says 5x or 10x on the screen and you will get the same image no matter which setting you use. The big secret is that you can get an image up to 2496x1072 but it isn't all that practical to use.
The crop_rec module on the EOSM doesn't actually "crop" and it uses the 3x3 pattern. Thing is, it uses the raw buffer that is supposed to be used for mv720 mode which is normally a 3x5 pattern so as Danne pointed out it quickly runs out of vertical resolution, thus most shots you'll see from the crop_rec module on the EOSM are very wide screen aspect ratios.
Finally, to get back on topic, the EOSM can't use the full sensor in 3x3 mode which is known as mv1080 in the ML source code. We're trying but so far the only hack seems to be to record H.264 and raw simultaneously. You'll get lots of corrupt frames and a few beautiful ones that inspires us to keep trying to get this mode working on the EOSM.
So does this make everything about as clear as mud?
Thanks for replays from you all. I understand now. The crop factor compared to full frame in crop mode is 3x1.6. Thanks
these new builds have h.264 proxy so any developments on mv1080 with that? we need h.264 running to kick the eos m into mv1080 as i understand it..
Yes, it goes into mv1080p mode while proxy is on. The proxy files are choppy and the mlv files are partly corrupted and a few ones looking good. Works better with 10bit.
cool i tried it. looks good in between those pink frames. but you already knew that.. #catpic
While filming proxy and raw and lowering fps to something around 10fps it actually records mostly corruptfree files and in correct order.
There is a way to film corrupt free mv1080p footage. It ain,t beautiful but it works.
For continouos do this:
1 - Bit Rate (CBR) 0.1x(H.264 files won,t be used anyway, maybe audio)
2 - FPS override 24
3 - Set proxy on and select 10-bit(mlv-lite.mo) Resolution 1488x692
4 - Set Preview in RAW video menu to Hacked(will freeze liveview or make it choppy but images will be corruption free)
Hit record.
Finally managed to downgrade back to 202 from 203 with ease (not quite actually hadn't realized I was in CA mode and not in M mode until just recently and only god knows how long I was in that mode for ha!) as usually with the EOSM I can easily forget that the Modes are only accessible through the 'Q' menu as oppose to most DSLR's would have access to these outside within the body.
Anyway was this done with the latest experimental (ml-raw_video_10bit_12bit.2017Feb11.EOSM202) or the one before that @Danne? <-- Oops looks below!
Only asking because I notice this build was missing the option that enables H264 Proxy (as it did in previous build afaik) so could this have anything to do with commit #88b2139 (Merged raw-h264-proxy into raw_video_10bit_12bit)?
My wrong ... I realized that the H264 Proxy is only available under 'Advanced...' within the RAW module menu (mlv-lite.mo) and not MLV module. D'oh! After all I will get my hands dirty with this momentarily and report my findings asap.
Shot in 24p 1792x1008p in 10-bit RAW (build used: ml-raw_video_10bit_12bit.2017Feb11 from experimental page) w an Opekta 6.5mm @ ISO 800 while in 'Movie crop mode' during early afternoon from indoors.
https://vimeo.com/204071325
Card used: 32GB Sandisk Extreme Pro 95MB/s SDHC I Class 10 (old as hell and still running strong!) 8)
Post workflow: MLV > cr2hdr.app (spat out ProRes 422 Proxy files w Rec709 applied) while I got 4 seconds in each take and it's 99% close to perfection. Seriously no joke in here!
so were getting there. i'm seeing a pink cast on my footage at lower resolutions(black level?). strange thing is it seems to work at higher resolutions. but i have c mounts so i can't film at higher resolutions without vignetting. also noticed just like when rbrun got 3x3 720 working the recording area isn't centered. still cool loving the progress:)
Here's another shot in 24p 1792x1008p in 10-bit RAW w a 50L @ ISO 200 while in 'Movie Crop Mode' of Willoughby:
FYI the first clip is graded with Cinelog DCP 2017 (after applying Cinelog-C in ACR, in OCIO select Alexa Wide Gamut V3 Log-C EI800 as input with Cinelog Rec709 FM as output) + FilmConvert (FJ 8543 VD) applied and then the second clip is without FilmConvert...
https://vimeo.com/204135855
For those that are curious how I did this in post? Here's the screenflow:
https://vimeo.com/204218877
Finally some original 10-bit mlv-lite samples in 24p 1792x1008p for those that don't own an EOSM but want to play:
Few DNG's zipped (4.3 MB) -- https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads/ml-experimental.2017Feb14EOSM 10-bit 1792 × 1008 DNG's.zip (https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads/ml-experimental.2017Feb14EOSM%2010-bit%201792%C3%971008%20DNG's.zip)
Converted MLV zipped (171.7 MB) -- https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads/M14-1405_1_2017-02-14_0001_C0000.zip (https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads/M14-1405_1_2017-02-14_0001_C0000.zip)
Are you getting 1792x1008? What build are you using?
ml-raw_video_10bit_12bit.2017Feb11 from experimental page
OKi you,re in movie crop mode, didn,t realize.
mv1080 ≠ "movie crop mode"
https://youtu.be/vjUARErSL-w (https://youtu.be/vjUARErSL-w)
mv1080 on eos m filmed with dannes settings with c mount 12.5mm lens. shows that the framing is not centered. also getting purple footage on some clips like the first then regular footage like the second clip. does work though.
Actually maybe it is centered..I'm probably wrong
Hi everyone
I was reading through the MV1080 on EOS-M forum and I was very interested by what Danne said here - https://www.magiclantern.fm/forum/index.php?topic=16608.150
I haven't yet tried his suggested settings, but what I don't understand is that you have said to use the 1488x692 resolution, but how could that be MV1080? Wouldn't MV1080 be 1792x1008?
Please let me know
I love all the work that's going into RAW, it's really amazing!
- Nathan
Breakthrough on this topic--1st sample of mv1080 on the EOSM without having to simultaneously record H.264:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg203530#msg203530