mv1080 on EOSM

[Teamsleepkid]

Oh ok sorry. Didn't see the part about not wanting test builds of this because it's too much of a hack.

[dfort]

Surprise, I posted a test build of this on my bitbucket download page. It is highly experimental including a firmware upgrade to 2.0.3.

Not that this is anything useable yet but we made some interesting discoveries playing around with it. Most important is that the EOSM can produce mv1080 that rivals the quality of the other cameras. Don't want to double post but this EOSM mv1080 frame of Koks the cat is too good not to share on this topic.



More information about how this was made on the Canon EOS M topic.

One thing we didn't anticipate was that the full raw buffer size of an EOSM mv1080 is 1808x1189. At least that's what we're getting with the hack build. All the other cameras that show focus pixels in raw video have a size of 1808x1190. I was so sure of this that I prepared a focus pixel map file for the EOSM in anticipation of getting this mv1080 puzzle solved. Fortunately it looks like the one vertical pixel was shaved off the bottom of the frame so simply renaming the map file got it working.

[Teamsleepkid]

mv1080 tried it out. very cool. can't wait for it to be working looks sweet in between the corrupt frames. also tried out 720 3x3 on the experimental page and it looked great. so mv1080 should be even better once its all working. also on 203 now thank you

ps that cat looks awesome and the wire mesh on that air conditioner doesn't look like a moire mess! very excited.

[a1ex]

One thing we didn't anticipate was that the full raw buffer size of an EOSM mv1080 is 1808x1189. At least that's what we're getting with the hack build. All the other cameras that show focus pixels in raw video have a size of 1808x1190.

This comes from here (autodetecting resolution is not perfect, as some cameras report the EDMAC value + 1, others - 1, others exact, others have mixed behavior, depending on video mode). Not sure why.

Guess we should test all other cameras with focus pixels and adjust the autodetected value...

[dfort]

Guess we should test all other cameras with focus pixels and adjust the autodetected value...

Please describe the steps we should take for running the test and how you want the results reported.

It turned out that I had a file with the raw buffer height one pixel different from one that Danne recorded.

Here's the mlv_dump -v information from the shot of Koks the cat:

Code: [Select]

Block: RAWI
  Offset: 0x00000034
    Size: 180
    Time: 0.843000 ms
    Res:  1728x972
    raw_info:
      api_version      0x00000001
      height           1190
      width            1808
      pitch            2260
      frame_size       0x003973A8
      bits_per_pixel   10
      black_level      0
      white_level      937
      active_area.y1   28
      active_area.x1   74
      active_area.y2   1184
      active_area.x2   1808
      exposure_bias    0, 0
      cfa_pattern      0x02010100
      calibration_ill  1

Here's what I got on my test:

Code: [Select]

Block: RAWI
  Offset: 0x00000034
    Size: 180
    Time: 0.780000 ms
    Res:  1728x972
    raw_info:
      api_version      0x00000001
      height           1189
      width            1808
      pitch            2260
      frame_size       0x0039674C
      bits_per_pixel   10
      black_level      0
      white_level      813
      active_area.y1   28
      active_area.x1   74
      active_area.y2   1183
      active_area.x2   1808
      exposure_bias    0, 0
      cfa_pattern      0x02010100
      calibration_ill  1

Danne and I re-worked my focus pixel fixing scripts so that they can work with files that are slightly off in height. So far I have only seen this happened when using the H.264 hack to record mv1080 on the EOSM but from your notes it looks like it can also happen randomly. Does it actually affect the recording? These hack recordings have lots of corrupt frames so we shouldn't be judging from these tests.

[a1ex]

Danne and I re-worked my focus pixel fixing scripts so that they can work with files that are slightly off in height.

Even better. Currently, there can be a difference of +/- 1 in heights.

So far I have only seen this happened when using the H.264 hack to record mv1080 on the EOSM but from your notes it looks like it can also happen randomly.

It's not random, it's just inconsistent across camera models and/or video modes.

Does it actually affect the recording?

Best guess, based on 10/12-bit issues before implementing RAW_SLURP: one missing or extra line at the end, and possible corruption in the first line (which is optical black anyway). It all depends on how Canon's image processor handles mismatches in image data size (how much you request vs how much it has).

If the focus map files work just by renaming, that's a good sign (it means the height mismatch doesn't result in the image being shifted vertically).

I've interpreted the lack of issues reported during the last week as encouraging (as in, no surprises encountered => nothing to report). But since you mentioned this focus pixel map issue (height mismatch), which I expect to be present on 650D and 700D (though not on 100D), I guess it's more likely that nobody bothered to report it.

[dfort]

If the focus map files work just by renaming, that's a good sign (it means the height mismatch doesn't result in the image being shifted vertically).

Yes, that's the case. At least with all the samples that we have checked which is only a couple.

In the scripts we now do this:

Code: [Select]

    raw_width=1808
    raw_height=11**
    raw_buffer=1808x11**


I've interpreted the lack of issues reported during the last week as encouraging (as in, no surprises encountered => nothing to report). But since you mentioned this focus pixel map issue (height mismatch), which I expect to be present on 650D and 700D (though not on 100D), I guess it's more likely that nobody bothered to report it.

I think that most people probably aren't noticing this. If they are using mlv_dump they are used to seeing focus pixels or maybe using chroma smoothing to deal with them.

[DeafEyeJedi]

Could this also very well be the reason why sometimes the focus map files would often miss the borderlines full of dancing dots from back in the day, remember?

[dfort]

No, that was because there weren't enough pixels to interpolate at the edges of the frame. The developers worked that out so it shouldn't be a problem, though with high contrast straight lines and dual iso it is sometime still a problem. A bit of chroma smoothing finishes off the remaining focus pixels.

[DeafEyeJedi]

Some fun stuff from messing around with mv1080 on EOSM.203 (ml-experimental.2017Jan12 - Thanks @dfort) in 24p 1728x972 MLV 10-bit @ ISO 3200 w a 50L of my several months old kitten, Willoughby...

Adobe Standard:


Cinelog-C:


Cinelog-REC709:


Here's what it actually looks like w corruptions (ProRes4444 Rec709 from cr2hdr.app):

Original DNG's (w Cinelog-C & Cinelog-REC709) & Original MLV (152.6 MB) -- https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads

All files were processed with latest cr2hdr.app (enabled cs2x2, Black level - 2047, White level - 15000 within mlv_dump settings) with ease and Thanks @Danne!

[dfort]

Thanks for sharing.

Feels like we're so close to having mv1080 on the EOSM -- yet so far.

@rbrune -- You seem to have gotten deep into this, any progress or comments?

[Licaon_Kter]

@DeafEyeJedi: Strong the force is in young kitty of yours... full video jamming almost achieved... 

[rbrune]

@rbrune -- You seem to have gotten deep into this, any progress or comments?

I'm still here and reading about whats going on - but currently lack time to further fiddle around with my EOSM. When I have some more free time I will continue with it

[Teamsleepkid]

Yes!!

[yskunto]

Dumb questions from a newbie. If I want to shot continuous RAW video with this Canon EOS M, then what is the highest resolution that I could get currently?

I confuse with the crop mode, 3x3 pixel binding etc.
Is the crop mode will make the crop factor of the Canon EOS M to 3x compared to 35mm full frame or is it 3x1.6 (1.6 is the crop factor of this Canon EOS M sensor)?

From what I read, the crop mode is good to reduce moire and aliasing artefact but more prone to noise at high ISO. Does 3x3 pixel binding could reduce the noise at high ISO?

Thanks

[Danne]

Crop_rec in 10bit gives continuous output. And it looks really great.

[Teamsleepkid]

@yskunto. Have a look at this: http://rbrune.github.io/mlraw/
Pretty much tells you everything you need to know. 3x3 is missing on there but look at the 650 d pretty much the same thing. The 1:1 produces the least Moore and aliasing but it's a small sensor size.

[dfort]

crop_rec, Movie crop mode, zoom mode, 3x3, 3x1.6, yeah this is confusing. What's worse is that this discussion is about getting mv1080 working on the EOSM so pretty much nothing in your post is on topic.

From what I read, the crop mode is good to reduce moire and aliasing artefact but more prone to noise at high ISO. Does 3x3 pixel binding could reduce the noise at high ISO?

You're referring to Movie crop mode which is the only video mode that is actually working the way it is intended to work on the EOSM. This mode uses all of the pixels (1x1) and doesn't skip any vertical or horizontal lines like the crop_rec module or mv1080 (3x3). Note that only the 5D3 does pixel binning, all other ML enabled cameras skip pixels which can create aliasing artifacts including moiré patterns.

I'm not sure that Movie crop mode is more prone to noise, could you provide a link to where you read that?

The disadvantage to Movie crop mode is that it uses a tiny part of the sensor. When shooting H.264 it will produce a 1920x1080 image with about a 3x crop of the "normal" APS-C image (not to a 35mm "full frame" sensor) but in raw video the maximum size possible in Movie crop mode is 1792x1024. Continuous recording? It might be possible at 23.98fps using 10bit with the 10/12-bit RAW video experimental build but I haven't tested it. With 14bit I was able to get continuous recording with a 1280x720 image size but that's even a smaller area, about a 5x crop.

@Teamsleepkid - nice reference, graphics explain better than words!

[EDIT] The crop_rec mode that Danne mentioned is as close as can get to a stable mv1080. It is a module for the 5D3 that rbrune got working on the EOSM, he's the guy who did those great graphics Teamsleepkid pointed out. It is 3x3 and uses much of the sensor so I'm not sure why it is called a "crop". The disadvantage is that it is using the raw buffer that is normally used for mv720, a.k.a. 720p, so the vertical size is limited. In raw it maxes out at something like 692 vertical lines so it give a super wide image even if you set the aspect ratio to 16x9. In other words, it is very much of an experimental video mode.

[a1ex]

It is 3x3 and uses much of the sensor so I'm not sure why it is called a "crop".

This module was originally written to enable 1:1 crop on cameras without one (e.g. 5D3), and the naming suggestion came from here. The other binning modes were added afterwards, as the implementation was quite similar.

Time for a rename?

[Danne]

I don,t know about the rename. It,s kind of established in my brain now.
bin_crop_rec.mo?
bin_rec.mo
I would however have nothing agains metadata as suggested here .
http://www.magiclantern.fm/forum/index.php?topic=17021.msg173941;topicseen#msg173941

[dfort]

Time for a rename?

hack_rec
wreck_rec
rec_it

[Walter Schulz]

hack, wreck, it: "I'm against it!" (Professor Quincy Adams Wagstaff).
If "bin" is meant to say binning: Rename to "bnn" and count me in.
Form follows function. In this case form = wording.

[Danne]

"bnn"
Yes.

[dfort]

Horse Feathers! According to Wikipedia (so it must be true) the definition of binning is:

In the context of image processing, binning is the procedure of combining a cluster of pixels into a single pixel. As such, in 2x2 binning, an array of 4 pixels becomes a single larger pixel, reducing the overall number of pixels.

Well the EOSM doesn't do binning, it does skipping, yet it has crop_rec. Then again, "What's in a name?" (William Shakespeare)

[Teamsleepkid]

Name by sensor size? Super 35, super 16, super 8. Lol mark iii "vistavision"