mv1080 on EOSM

Started by dfort, February 06, 2016, 04:56:46 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

a1ex

Quote from: dfort on December 19, 2016, 07:46:40 PM
The crop hack mode changes the property of the video mode. Maybe it is possible to do something similar to get into mv1080 [...]?

Indeed. Enabling crop hack and re-enabling line skipping / column binning in crop_rec should do the trick.

You should be able to get a proof of concept with adtg_gui (no coding needed, just by fiddling with menus). Starting point: ADTG[0x800C] = 2 to set line skipping to 3, CMOS[7] to center vertically. Column binning is usually configured from one of the CMOS registers.

dfort

Quote from: a1ex on December 19, 2016, 08:13:36 PM
You should be able to get a proof of concept with adtg_gui (no coding needed, just by fiddling with menus). Starting point: ADTG[0x800C] = 0 to disable line skipping, CMOS[7] to center vertically. Column binning is usually configured from one of the CMOS registers.

I'm really trying hard to follow directions, using this post as an example of how to use adtg_gui and I'm able to fiddle with menus but can't find ADTG[0x800C]:



It is clearly in the adtg_gui.mo source code:

    {DST_ADTG, 0x800C, 0, "Line skipping factor (2 = 1080p, 4 = 720p, 0 = zoom)"},


Quote from: a1ex on December 19, 2016, 08:13:36 PM
Enabling crop hack and disabling line skipping / column binning in crop_rec should do the trick.

So changing this to 0x800C, 0 is what I should try?

crop_rec.c
            /* 3x3 binning in 720p (in 1080p it's already 3x3) */
            case CROP_PRESET_3x3_1X:
                /* ADTG2/4[0x800C] = 2: vertical binning factor = 3 */
                adtg_new[0] = (struct adtg_new) {6, 0x800C, 2};
                break;


I just want to make sure I'm understanding this.

DeafEyeJedi

I believe that's just about right @dfort!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

a1ex

Yes, you can try. For adtg_gui, try using this as a guide.

Some registers are only updated when switching video modes (so you need to do that).

dfort

Quote from: a1ex on December 20, 2016, 07:46:02 AM
Some registers are only updated when switching video modes (so you need to do that).

Getting closer. Recording a short H.264 will expose more registers. Found it:



Figured out that the PLAY button allows editing the hex value but only in one direction. Fortunately it was the direction that I wanted to go.



Now how do you get the edited value to take?



Doesn't seem like anything is happening. Shouldn't 0x2 put us into 1080p mode?

Looks like both battery and operator needs a recharge.

a1ex

Quote from: dfort on December 20, 2016, 09:11:15 AM
Now how do you get the edited value to take?

If the register appeared when starting H.264, you need to perform the same action in order to apply the changes.

However, I expect it to appear also when exiting LiveView (e.g. to PLAY mode or Canon menu) and going back.

Quote from: dfort on December 20, 2016, 09:11:15 AM
Figured out that the PLAY button allows editing the hex value but only in one direction.

Should work with SET followed by up/down/left/right arrows (or top scrollwheel, but you don't have one on the M).

dfort

Am I doing this right? Changing CMOS[7] shows a definite change:



And looking at ADTG2[800c] while recording H.264 shows a value of 0x2 and a frame rate of 23.9xx indicating that it is in 1080p mode:



But modifying that register doesn't seem to have any affect on the frame rate. Also, shooting a silent still and looking at the full raw buffer indicates that it is still in 720p mode.



Shooting a silent still while recording H.264 does capture the mv1080 raw buffer (full raw buffer size=1808x1190) however that just puts us back to where we started--trying to figure out how to get into mv1080 mode without having to simultaneously record H.264 video.

I also tried changing ADTG2[800c] in crop_rec.c and turning on Movie crop mode but that didn't work either.

Seems to me that there should be a way to switch into mv1080 mode like crop-mode-hack.c switches into mv1080crop mode. Movie crop mode (crop hack) seems to be the only mode that is working properly on the EOSM. All the other modes default to 29.97fps including the @rbrune crop_rec which is the closest we got to mv1080 on the EOSM so far albeit recording 3x3 on the mv720 buffer and looking at a wonky LiveView display.

[EDIT] I'm also trying to compile dm-spy-experiments to see if I can log the properties that change when recording H.264. I used that branch a while back when trying to track down the shutter-bug but now it doesn't compile:

debug.o: In function `run_test':
debug.c:(.text+0x988): undefined reference to `debug_intercept'
debug.c:(.text+0xbb0): undefined reference to `debug_intercept'
make: *** [magiclantern] Error 1

a1ex

You need to switch the crop mode hack on, then undo the crop from adtg_gui.

The CMOS register is set for every single frame, that's why it takes effect right away.

The line skipping register is different. You need to exit LiveView and then return in oder to apply the changes.

To compile the dm-spy branch, also set CONFIG_DEBUG_INTERCEPT = y in Makefile.user.

dfort

Quote from: a1ex on December 20, 2016, 09:38:18 PM
You need to switch the crop mode hack on, then undo the crop from adtg_gui.
...exit LiveView and then return in oder to apply the changes.

Only active module is adrg_gui
Turned crop mode hack on
Turned on adtg_gui
Recorded short H.264
Opened adtg_gui and changed ADTG2[800c] to 0x2
Exited LiveView (opened Canon menu)
Recorded short H.264
Still no change - camera is still in crop hack mode (1x1) and didn't change to 3x3.



Got dm-spy branch compiled -- thanks!

a1ex

On 5D3, I didn't have to change the line skipping register; instead, I had to change ADTG[0x8000]. Does it help if you override that one as well?

A technique I've used:
- find the differences between two video modes (enable the first video mode, then select "Show: Modified from now on", then switch to the second video mode)
- save those differences to a log file, for reference (Log Registers Now)
- override those registers that were different (Lock Displayed Registers)
- switch back to the first video mode; it should behave like the second one, at least in some aspects
- find a minimal subset of registers that have to be changed (trial and error)

dfort

I haven't had any luck overriding ADTG[0x8000]. At least not with the values that I've tried so far.

Your technique gave me an idea. Since the main thing we want to see is what is changing in the display registers when the camera is recording H.264 I selected "Show: Modified from now on" then started recording H.264 and while the camera was still recording I selected "Log Registers Now". I did it several times to make sure the results were consistant and they were:

Canon EOS M 2.0.2 - Record H.264 1080p 24 NTSC
00028172:     6fd (was 3b7)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=33be8 addr=40481b28 Line count to sample. same as video resolution (g3gg0)
00028173:     d71 (was 47c)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=33be8 addr=40481b2c
00f00008:     400 (was 800)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=33e38 addr=40482526
0002802c:       0 (was 110)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=360fc addr=40481b80
00028047:     550 (was 440)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=360fc addr=40481b84
00028178:     6fd (was 3b7)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=359a8 addr=40481bd0 dwSrFstAdtg1[4], Line count + 1
00028179:     d71 (was 26f)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=359a8 addr=40481bd4 dwSrFstAdtg1[5]
000282b6:     6f4 (was 3b4)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=359a8 addr=40481be8
000288c6:     4a4 (was 2d4)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f24
000288ce:     4a4 (was 2d4)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f34
000288d5:     903 (was b03)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f40
000288d6:     1cb (was 11f)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f44
000288d7:     2f6 (was 1d2)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f48
000288da:     201 (was 504)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f54
000288db:     706 (was b0a)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f58
000288df:       0 (was 2)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f60
000288e0:       2 (was 1)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f64
000288e1:       1 (was 0)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f68
000288e7:       6 (was 5)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f70
000288e8:       9 (was a)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f74
00028196:     12e (was 190)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35e74 addr=40481ca0 dwSrFstAdtg1[2], Line count + 1
00028197:     38c (was 13a)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35e74 addr=40481ca4 dwSrFstAdtg1[3]


Then I went the other way. With the camera recording H.264 I selected "Show: Modified from now on" stopped recording and selected "Log Registers Now" and did that several times and again got consistant results:

Canon EOS M 2.0.2 - Stop recording H.264 1080p 24 NTSC
00028172:     3b7 (was 6fd)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=33be8 addr=40481b28 Line count to sample. same as video resolution (g3gg0)
00028173:     47c (was d71)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=33be8 addr=40481b2c
00f00008:     800 (was 400)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=33e38 addr=40482526
0002802c:       0 (was 110)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=360fc addr=40481b80
00028047:     550 (was 110)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=360fc addr=40481b84
000282f3:       c (was 30a)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=359a8 addr=40481be0 Line count that gets darker (top optical black related)
00028305:       3 (was 0)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=40481860
00028306:       7 (was 0)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=40481864
00028307:       f (was 3)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=40481868
00028308:       b (was 7)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=4048186c
0002830f:       3 (was 0)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=40481870
00028310:       7 (was 0)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=40481874
00028311:       f (was 3)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=40481878
00028312:       9 (was 7)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=4048187c
00028335:      6f (was 0)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=40481884
00028336:     36f (was 0)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=40481888
0002833d:      6f (was 0)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=40481894
0002833e:     1ef (was 0)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=40481898
0002839c:       3 (was 1)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=404818ac
000283a9:       3 (was 1)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35dcc addr=404818e0
00028820:     303 (was 301)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481f9c
00028826:     e05 (was 5)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481fa4
00028827:      42 (was 3)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481fa8
00028828:    3010 (was 20)       ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481fac
00028860:       f (was e)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481fbc
00028866:       7 (was 9)        ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=35c44 addr=40481fc0


"Lock Displayed Registers" while recording H.264 didn't work--the frame rate stayed at 29.97 and shooting a simple silent captured the mv720 buffer. The LV display was also garbled up.

Next I thought I'd see what happens when running the same test this time while recording mlv_rec (image size was at 640x78):

Canon EOS M 2.0.2 - Record mlv_rec Canon menu 1080p 24 NTSC but camera records mv720
0002802c:       0 (was 110)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=360fc addr=40481b80
00028047:     550 (was 440)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=360fc addr=40481b84
00028179:     47c (was 26f)      ISO=100 Tv=60 Av=91 lv=1 zoom=1 mv=1 res=0 crop=0 task=Evf pc=359a8 addr=40481bd4 dwSrFstAdtg1[5]


Interesting how few registers changed. Maybe they changed just from switching over to the ML menu? I tried going the other direction but that just crashed the camera.

One more test I tried was to see if the raw buffer changes right when H.264 recording starts and stops. This was much less precise, I just tried shooting a simple silent still right before and right after pressing the record button. Seeing a 1734x693 capture indicated it was a mv720 buffer and 1734x1156 indicated a mv1080 raw buffer. It seems that there is some sort of a time lag because a silent still captured right after recording starts was still mv720 and right after recording stopped it was still mv1080. In other words it looks like the raw buffer size switches when it is actually writing to the card--does that make sense?

a1ex

It's not obvious what's going on; I would expect a register to be set to 2 in 1080p and 4 in 720p.

Interesting that some registers got changed when recording MLV; I didn't expect any of them to change, maybe it's the extra hacks option?

FPS registers are in the "DIGIC registers" category in adtg_gui, so they won't change by overriding the above ones. You don't need to change them - they already work with crop mode hack.

Raw buffer size (as autodetected by ML) is also retrieved from two DIGIC registers (reply #86). Most registers cannot be read back directly, so they have a shadow copy that contains the last value written to that register. Well-behaved code sets both the register and its shadow copy, but there is some code that only sets the register. I wonder if the EOS M code overrides the resolution registers directly to 720p (so the change is invisible if we read the shadow copy).

This condition can be detected in QEMU, but only after being able to emulate the LiveView (or some part of it) on EOS M. Not very practical. Although, being able to log all the I/O register accesses would be extremely useful for emulation.

If you ever get bored (such as on a long train ride), you could try fiddling with those registers to figure out their meaning. There are lots of interesting things behind them (custom resolutions, extra highlight detail, binning patterns and so on).

dfort

Quote from: a1ex on December 22, 2016, 03:40:00 PM
It's not obvious what's going on; I would expect a register to be set to 2 in 1080p and 4 in 720p.

That's what I was looking for in 0002800C:

    {DST_ADTG, 0x800C, 0, "Line skipping factor (2 = 1080p, 4 = 720p, 0 = zoom)"},


or something to be set to  0 in 1080p and 1 in 720p:

    int mv720 = mv && video_mode_resolution == 1;
    int mv1080 = mv && video_mode_resolution == 0;


but that didn't show up either. The most I got out of reply #86 was:

Quote from: rbrune on December 05, 2016, 11:26:54 PMSomething really weird is going on in this standy mv720 liveview mode.

Looks like I'm going to have to book a long train trip.

dfort

I was experimenting with shooting raw video while simultaneously recording H.264 and late one night before going to bed I captured a short MLV of my keyboard in mv1080 mode.



I saved just one DNG frame and then tried shooting something worth posting but several days later all I got was pink earthquake video. In any case, this just proves what a1ex said back on the first post about the EOSM switching into 1080p when H.264 movie recording starts.

exiftool
Image Size                      : 1728x972

Teamsleepkid

interesting. just gotta get it repeatable
EOS M

Danne

I did some testing with dfort experimental EOSM_mv1080 branch here:
https://bitbucket.org/daniel_fort/magic-lantern/branch/EOSM_mv1080_experiments

I actually managed to get some good 10bit mv1080 frames out of the build. Well the files had some various unstable results with white and black level and some files are corrupted but proof is there. mv1080 could be possible. But how.
So the hack is like this.
1 - Start H.264 recording
2 - Enter ML menu and select raw recording and while H.264 is recording start record again for MLV recording.
3 - Stop MLV recording go back to ML menu and deselect raw recording
4 - Go out of the menu and stop H.264 recording
Very convenient ;)

Here are the originals files, MLV, dng files
https://bitbucket.org/Dannephoto/magic-lantern/downloads/EOSM_mv1080.zip

I developed with mlv_dump and chroma smooth cs2x2 through cr2hdr.app then adding back white and black level with exiv2.
exiv2 -M"set Exif.Image.BlackLevel Long 2047" -M"set Exif.Image.WhiteLevel Long 15000" *.dng

Some comparisons
mv1080


mv720


mv1080


mv720


Teamsleepkid

Almost there.. moire and aliasing looks a lot more controllable in the punch in shot on mv1080:)
EOS M

dfort

@Danne -- Thanks for posting the pics.

This proves that the EOSM can do mv1080 that matches the 650D and 700D because they are very similar, though a the same time very different. Now how to do it without having to record H.264 and MLV simultaneously, thought that's an interesting "feature"!

Danne

It,s not a 5D mark III but mv1080 it,s heaps better than mv720p, even though that is cool as well. The eosm is great for stills, timelapsing and in crop_rec it,s really useful with HDR(H.264). If mv1080 moves in there, now with 10bit it will be a true rebel camera. Amazing really, with the 10bit delivering continouos in crop_rec and mv720p.

Licaon_Kter


DeafEyeJedi

Thanks for sharing these results @Danne!

So glad we all stuck with the M through thick and thin, right @dfort?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Kharak

@danne,

Port it to 5D3, I would love h264 proxy recording ;)
once you go raw you never go back

dfort

Quote from: Kharak on January 05, 2017, 02:20:09 PM
Port it to 5D3, I would love h264 proxy recording ;)

If you look in mlv_rec.c and raw_rec.c you'll see this:

    /* if you somehow managed to start recording H.264, let it stop */
    if (RECORDING_H264)
        return 1;


Just comment that out and it should work. I don't really want to put test builds out there with this because it is too much of a hack. You have to start recording H.264 then get into the ML menu to turn on the raw recording option then you can start and stop recording raw while H.264 is continuously recording. You need to disable raw in order to stop the H.264 recording.

Teamsleepkid

any test builds for this?
EOS M