mv1080 on EOSM

[dfort]

Here's a discussion on how pixel binning works, or at least how we believe it works:

http://www.magiclantern.fm/forum/index.php?topic=16516.0

That's only applicable to the 5D3 right now though maybe the 7D2 and the 5D4 might also do pixel binning, we don't know yet. Pixel binning on the EOSM is probably in the impossible to implement category.

There is no VAF for the EOSM sensor. Well, you could order a special filter but it costs more than the camera. A cheaper solution is a do it yourself filter that can fit in front of the lens. Though I've got to admit that using a piece of 35mm film is a clever solution.

Aliasing isn't as bad when in 1x1 (crop mode) because the pixels are physically closer together. What should theoretically help is increasing the resolution but we're limited there because of the write speed to the SD card. Reducing the frame per second rate and/or the bit depth are both doable in order to use a larger image size.

Anyway, so much for theory--I should get back to focus pixel hunting.

[a1ex]

The EOSM won't ever have the quality of 5D3 binning where pixels are binned in both horizontal and vertical direction - at least as far as we know this is part of the digic processing in the 5D3.

I think the binning is done in hardware, directly on the sensor. Why?

- noise analysis: read noise appears to be added after the binning process [1]. Therefore, binning must be done early in analog domain, not in software.
- Chipworks states that 1DX (also likely to use 3x3 binning) "uses an additional transistor (T5) that can be used to sum the FD nodes of three rows of even (or odd) pixels of the same color" [2] (which matches my findings about the binning pattern)

[Teamsleepkid]

Anyone ever thought about high resolution black and white? Something with good dynamic range but no color and no aliasing? Just throwing it out there..

[dfort]

Sure, but it is a hardware hack or for just $7,500 you can get a Leica M Monochrome.

Back on topic, between your file and some dark frame's I shot with the lens cap on I think I found all the focus pixels. More info on the focus pixel topic.

Updated package with the latest focus pixel map file now available for testers. This time I eliminated the dcraw "badpixels" file because it takes too much time to create and test it and I doubt anyone is actually using it. Also took the old build offline so I (hopefully) won't see any more focus pixels on testers' videos.

https://bitbucket.org/daniel_fort/magic-lantern/downloads/magiclantern-crop_rec.2016Nov06.EOSM202.zip

[Teamsleepkid]

nice work

[dfort]

nice work

Hey @Teamsleepkid want to do some more testing? Of course this applies to anyone else who would like to pitch in.

I've updated the focus pixel map file for crop_rec on EOSM and @rbrune created a version that includes 10bit_12bit raw video using the new MLV Lite raw video format so there's plenty to test.

As usual, I put the test builds on my bitbucket download area:

https://bitbucket.org/daniel_fort/magic-lantern/downloads

[Teamsleepkid]

looks really good so far. 3x3 720 crop mode at 10 bit crashed the camera and i had to pull the battery. i was able to load the footage and after running it through mlvfs i didn't see any focus pixels! awesome. also seemed like it had less aliasing and moire but maybe I'm wrong. looked nice though. gotta get it stable so it doesn't crash and require battery pull. i'll keep testing. thank you for the build.

[dfort]

The freezing camera/battery pull issue with 10-bit 12-bit along with a corrupt first frame has been reported by other EOSM users. It should work at 14-bits.

By the way, if you have a scene where there's high contrast detail you might see some focus pixels pop up every now and then even with the new focus pixel map file that's in the test package. Play around with the chroma smoothing settings. That usually takes care of it.

[rbrune]

Getting this thread back on topic. I still try to get true mv1080 and mv720 with 60fps working.

Using the dm-spy-experiments branch I tried to see if I can isolate a function that changes the camera into mv1080 mode when normal h264 recording starts.
So far I came up with the following:

Normal operation to recording 1080

Code Select Expand


       Evf:00014b0c:ad:03: PATH_Select S:0 Z:10000 R:0 DZ:0 SM:1 SV:0 DT:0
       Evf:00014ca0:ad:03: PathDriveMode Change: 11->6
       Evf:ff37b5e8:ad:03: GetPathDriveInfo[6]
       Evf:ff50478c:ad:03: Rec_HD_SelectPath LCD


Normal operation to HDMI output

Code Select Expand


       Evf:00014b0c:ad:03: PATH_Select S:1 Z:10000 R:1 DZ:0 SM:1 SV:0 DT:0
       Evf:ff37b5e8:ad:03: GetPathDriveInfo[11]
       Evf:ff509db8:ad:03: RecStandby_x1_60fps_SelectPath HDMI(480)


HDMI output back to normal operation

Code Select Expand


       Evf:00014b0c:ad:03: PATH_Select S:1 Z:10000 R:1 DZ:0 SM:1 SV:0 DT:0
       Evf:ff37b5e8:ad:03: GetPathDriveInfo[11]
       Evf:ff509d90:ad:03: RecStandby_x1_60fps_SelectPath LCD


So it seems the PATH_Select function decides if the camera should change the Evf mode (from mode 11 to mode 6). I hooked the PATH_Select function and overwrote the struct pointed to by reg0 to with the values that evaluate to the Rec_HD mode 11 inside PATH_Select like this:

Code Select Expand


PATH_SELECT = 0x14ac4;
MEM_PATH_SELECT = 0xe92d4030;
patch_hook_function(PATH_SELECT, MEM_PATH_SELECT, &path_select_hook, "evf_mode: PATH_select hook");


Code Select Expand


static void path_select_hook(uint32_t* regs, uint32_t* stack, uint32_t pc)
{   
    uint32_t *mode_struct = (uint32_t *) regs[0];   
    printf("mode_struct: %x %x %x %x\n %x %x %x %x\n %x %x %x %x\n %x\n",
                                      mode_struct[0], mode_struct[1], mode_struct[2], mode_struct[3],
                                      mode_struct[4], mode_struct[5], mode_struct[6], mode_struct[7],
                                      mode_struct[8], mode_struct[9], mode_struct[10], mode_struct[11],
                                      mode_struct[17]);
   
    if (is_EOSM)
    {
        switch(evf_mode_index)
        {
            case LV_REC_MV1080_24FPS:
                mode_struct[0] = 1;
                mode_struct[2] = 0;
                mode_struct[3] = 0;
                mode_struct[5] = 1;
                break;
        }
    }   
}


Sadly when I activate the hook the live view freezes as soon as the PATH_Select function is called, e.g. by switching to info view and back or by half pressing shutter to focus. Deactivating the hook returns everything to normal.
Interestingly though is that under Debug Image buffers and EDMAC channel 1 change:

Code Select Expand


Before Image buffers: 720x480, 960x639
After Image buffers: 720x480, 1728x1151

Before EDMAC[1]: 1920x639
After EDMAC[1]: 3456x1151


So I'm not sure what's going on, if my hook is working as intended (need to log debug msgs while hooked) and certainly not sure if I'm actually looking in the right place.

[dfort]

Wow, you're really getting into the nitty gritty of this.

You probably already know this but the EOSM will always show a frame rate of 29.97 when you are in standby mode unless you have FPS override active. Even when the camera is setup for PAL it will show 29.97. When it starts recording H.264 it switches to the frame rate that matches the Canon settings. However, with MLV and raw video it will always record at 29.97--unless FPS override is active. Apparently the camera needs to be tricked into going into H.264 record, though preferably without actually recording an MOV file simultaneously with the MLV or raw video. I believe there were some other issues that required getting into H.264 recording mode. zebra.c has references to H264 all over it.

Don't know if this helps.

propvalues.h

Code Select Expand

#define RECORDING_H264 (__recording > 0)
#define RECORDING_H264_STARTING (__recording == 1) // 1 is preparing for recording
#define RECORDING_H264_STARTED (__recording == 2) //2 is actually recording


[a1ex]

You can use adtg_gui to record the register configurations of all video modes.

Steps:
- enable Debug->ADTG registers
- enable Advanced->DIGIC registers
- if the image gets damaged, try Force low FPS
- go to PLAY and back to LiveView to refresh the video mode and let adtg_gui capture all the registers
- select Advanced->Log Registers Now
- repeat for other video modes

Note: the contents are appended in the log file, so you can simply log everything during one test session.

(side note: a while ago I tried to enable the digital zoom mode on 60D, using the same functions, also without success)

[rbrune]

@A1ex:

so I looked at your raw fixes branch and tried if I can also retrieve the LV RAW resolution by reading 0xC0F06800/4:

Code Select Expand


    #ifdef CONFIG_EOSM
    uint32_t top_left = shamem_read(0xC0F06800);
    uint32_t bot_right = shamem_read(0xC0F06804);
    *width  = ((bot_right & 0xFFFF) - (top_left & 0xFFFF)) * 4;   // Column is /4 instead of /8
    *height = (bot_right >> 16) - (top_left >> 16) - 1;         
    printf("res %d %d\n", *width, *height);
    //*width  = video_mode_crop ? 1872 : 1808;
    //*height = video_mode_crop ? 1060 : 727;
    return 1;
    #endif


The resolution I get from 600D crop mode is 1872 x 1058 which seems to be fine.
During normal EOSM liveview mode I get 1808 x 1188 but when I do a half-shutter button press to autofocus it goes to 1808 x 725 and then quickly goes back to 1808 x 1188. Needless to say that raw recording does only work in crop mode with these width/height values.
Something really weird is going on in this standy mv720 liveview mode.

[dfort]

mv1080 on the EOSM should be:

Code Select Expand

*width  = 1808;
*height = 1190;

That is if it behaves the same as the 650D/700D/100D.

[a1ex]

During normal EOSM liveview mode I get 1808 x 1188 but when I do a half-shutter button press to autofocus it goes to 1808 x 725 and then quickly goes back to 1808 x 1188.

That's interesting. So, as long as you don't press the shutter halfway, you get a valid 1808 x 1188 stream?

And I guess you have to be careful not to press half-shutter while recording.

[rbrune]

That's interesting. So, as long as you don't press the shutter halfway, you get a valid 1808 x 1188 stream?

This is how it looks like when recorded with raw_rec at 1728x1154, it usually stops after a couple frames or doesn't even start.

First couple frames are with activated crop_rec 3x3 and the latter frames from a recording without (aka standard EOSM liveview) that I did immediately afterwards (one can see that the image parts, bottom left and right, are still leftovers from the 3x3 recording).

By looking at the individual DNG files it seems one could get a couple more usable lines when using crop_rec 3x3 instead of standard mv720, maybe 698 instead of 692.

[dfort]

That looks very familiar. Looks like some of the PREFERRED_RAW_TYPE settings I was trying out searching for something that showed no focus pixels.

http://www.magiclantern.fm/forum/index.php?topic=5601.msg175416#msg175416

@rbrune -- could you push your work branch to bitbucket so we can check it out?

[rbrune]

That looks very familiar. Looks like some of the PREFERRED_RAW_TYPE settings I was trying out searching for something that showed no focus pixels.

http://www.magiclantern.fm/forum/index.php?topic=5601.msg175416#msg175416

@rbrune -- could you push your work branch to bitbucket so we can check it out?

Sure, here you go:

https://bitbucket.org/rbrune/magic-lantern/branch/crop_rec_raw_video_10bit_12bit

It's a mix of the 10/12bit, crop_rec and raw_fixes branches.

[dfort]

Crazy stuff going on in that build @rbrune ! It clearly shows that the mv1080 buffer is active and a half-shutter press switches to the mv720 buffer.



I turned on the RAW_DEBUG_TYPE but it seems that all settings I tried so far are causing a "Raw detect error." Also captured a full raw buffer but to do that I used silent picture and a half-shutter press captures the image it only captured the mv720 buffer. The raw buffer capture was clean though.

[dfort]

@rbrune

I turned on the RAW_DEBUG_TYPE and tried every PREFERRED_RAW_TYPE possible from 1 to 128 and the following were the only ones that didn't cause a "Raw detect error."

7, 11, 48, 50, 75, 80, 87

Unfortunately none of these produced a useable picture. It was worth running this test anyway because I keep finding useable settings above the 64 ".max" limit that is in the debug menu.

So you're getting an mv1080 frame size but the image is still obviously mv720 with the bottom 1/3 of the image filled in with what looks like remnants of whatever was in the buffer when the image was written to the card. The every other frame difference on the bottom 1/3 might be because the buffer empties out every other frame?

If I understand the problem it is because raw video is taken from a video stream that feds the LiveView screen and on the EOSM that stream is always running in mv720 mode but at 29.97fps (mv720 should be 59.94fps). It only changes when it is recording H.264 video. In lv-img-engio.c there is a function called digic_find_lv_buffer - maybe there is a different lv buffer that needs to be used on the EOSM? Just speculating because there seems to be several raw buffer streams to choose from.

[dfort]

Don't know if I'm spouting out useful suggestions or just adding to the noise but I just thought of something that might help.

The only setting where the EOSM shows the "proper" frame rate is in "Movie crop mode" a.k.a. crop mode hack. The source code for crop-mode-hack.c is relatively small and maybe there is some hint in there that we can use?

Code Select Expand

unsigned int movie_crop_hack_enable()
{
    if(!is_crop_hack_supported() || video_mode_crop)
    {
        return 0;
    }
    video_mode[0] = 0xc;
    video_mode[4] = 2;
    prop_request_change(PROP_VIDEO_MODE, video_mode, 0);
    return 1;
}


[rbrune]

Thanks for all the testing dfort!

The thing I want to try next is to log the register settings for real mv1080 mode and see if setting those like we set 3x3 mode results in a filled up buffer. Or even try to do that while in the 600D crop hack mode buffer.
Another approach would be to just use the current 3x3 crop mode and try to only modify the registers that set start read line and end read line.

Higher fps is another story. It could even be that the timing in the liveview pseudo mv720 mode is prohibitive to reading out more lines. I have the feeling that Canon might have done all of this on the EOS M just to save some power since live view is basically always on.

[a1ex]

When dealing with this sort of semi-corrupted frames, here's a change I often make in raw.c:

Code Select Expand


static int autodetect_black_level(int* black_mean, int* black_stdev_x100)
{
    *black_mean = 2048;
    *black_stdev_x100 = 1;
    return 1;
    ...
}


Reason: this function checks for a valid raw buffer, with a clean optical black area. There are some situations where the raw buffer is not clean (such as when changing resolution, or when a frame gets corrupted for some reason); in this case, the black level computed from such an image would be invalid and can affect entire recordings.

However, in this case, where you expect incomplete image data, the above check will do more harm than good, so for testing, you can simply hardcode some value (which will bypass the checks).

The behavior is quite puzzling for me. Was the video captured with mlv_lite? Do you get the same artifacts if you take silent pictures? (simple or burst)

[Danne]

Any clues to why mlv_rec(10_bit, 12_bit) almost works(every other dng corrupted)? Can it be related to dbg functions? I see #undef RAW_DEBUG is set so it can,t be this right? From a noob C standpoint. Is there some functions I can disable to test in raw.c for the 5D mark III?

[dfort]

...Is there some functions I can disable to test in raw.c for the 5D mark III?

A bit off topic--this might offer some hints:  lv_set_raw / lv_select_raw

Of course we're trying to find a way to turn the EOSM into a 5D3 so it isn't too far off topic.

[dfort]

Thanks for all the testing dfort!

The thing I want to try next is to log the register settings for real mv1080 mode and see if setting those like we set 3x3 mode results in a filled up buffer. Or even try to do that while in the 600D crop hack mode buffer.
Another approach would be to just use the current 3x3 crop mode and try to only modify the registers that set start read line and end read line.

Higher fps is another story. It could even be that the timing in the liveview pseudo mv720 mode is prohibitive to reading out more lines. I have the feeling that Canon might have done all of this on the EOS M just to save some power since live view is basically always on.

Anything new to test?

Something that is interesting with the 600D crop hack mode is the the frame rate is 23.98 instead of 29.97 in all other modes. This may be related to the comment I quoted on the first post of this topic:

The EOSM is unique b/c Canon firmware doesn't put it into mv1080 until you start recording (H.264).

The crop hack mode changes the property of the video mode. Maybe it is possible to do something similar to get into mv1080 and true (50p/60p) mv720 mode?

Something I haven't tried yet is to use the dm-spy-experiments branch to try and find out what goes on with PROP_VIDEO_MODE when the camera starts recording H.264.