Canon 70D

[yokashin]

Thank you 
It's good that someone supports 70D users.

[shinoda]

In the downwloads section you will find:
https://bitbucket.org/ArcziPL/magic-lantern/downloads/

magiclantern-sduhs-Nightly.2018Jul17.70D112.zip -> original sd_uhs, which is benchmarking several settings, so you can find the best setting for your card but it takes ages to complete
magiclantern-sduhscustom-Nightly.2018Jul17.70D112.zip -> custom build, sd_uhs is fixed to SDR104@160MHz setting, it does test the throughput after patching *once*

On request I could prepare a similar mod like Danne did already for M, 100D, 700D, (*), skipping the testing completely. So far I didn't need it.

Thank you, it's very much appreciated! So you recommend to bench it with the first build? I'll do that!

Edit: I have some hiccups, I did a fresh install with the custom build (it tested it first and I have have the same settings with a SanDisk Extreme PRO): the ML menu freezes when I want to change RAW video settings... The lossless setting was the culprit, so I used the mlv_rec module. It's strange, I have the exact same settings as you (160mhz, more than 60mb/s, 40 before) but when I run the benchmark and I use raw video, it doesn't seem to change anything.

[andy kh]

#shinoda  activate crop_rec, raw, mlv, sd_uhs etc. after you restart your camera, dont turn on the raw. if you do so, your camera wil freeze .turn on only the mlv.
i have another build by Danne. i have no problem turning on the raw with that build

[ArcziPL]

Hi shinoda, sorry forgot to mention: this build is directly the sd_uhs branch, which as I understood you have asked for. I don't know how good it is in RAW video recording, I only take the sd_uhs.mo from it and copy it to another build which I use. Probably the sd_uhs.mo (so the module to do overclocking) is what you really wanted.

Please, use only the ML/modules/sd_uhs.mo file from this build and copy it to your SD card with the ML build which you were using up to now and where RAW video works.

BTW: there is no build so far with lossless compression for 70D.

[ArcziPL]

OK, here I post the crop_rec_4k_mlv_lite_snd with sd_uhs (160MHz fixed) added, so you can directly unpack it to your SD and use:

https://bitbucket.org/ArcziPL/magic-lantern/downloads/magiclantern-croprec4kmlvlitesnd+sd_uhs--Nightly.2018Aug10.70D112.zip


1. Start your camera in photo mode
2. Go to ML settings, Modules tab, enable: mlv_lite, mlv_snd, sd_uhs
3. Reboot
4. Go to ML settings, Movie tab, enable RAW video, in its settings change Data format to 10-bit (the normal one, without compression!)
5. Go to Debug tab, run SD overclock, wait until it's done
6. Switch to Movie mode
7. In Canon menu set 1080p 25 or 24fps
8. Start recording.

You don't need to enable the 5x zoom or anything like that. 70D will record 1832x1024 full-frame RAW in that way, providing normal, full-color live preview. You can also use the 3x zoom mode from Canon menu. Thanks to sd_uhs the recording at this resolution at 10 bit bitdepth will be continuous.

When this works, you can experiment with further settings if you want -- higher resolutions with the 5x zoom enabled etc. But I use only what described above. I don't like the broken preview of 5x zoom mode and it will also be very limited in recording time.

Forget so far about crop_rec, mlv_rec etc. You don't need them to start.

[shinoda]

Thanks a lot guys, everything's good!

[shinoda]

Okay so everything's working well, except mlv_play : I can't watch the MLV files after recording (just on the computer), it stays black with the frame numbers changing.

[andy kh]

u mean you cant play back on your computer? which player do you use?

[shinoda]

No, playback on the 70D, no issue with playing MLV files on the computer.

[shinoda]

The mlv_play module was not updated since 02/17 and the latest one from june seems to have fixed some issues: https://bitbucket.org/hudson/magic-lantern/commits/3f90c436d85aff183af93b9bf6a30224811e5fa2

Anyone has the same issue as me?

[ArcziPL]

The mlv_play module was not updated since 02/17 and the latest one from june seems to have fixed some issues

Either I can't follow you or you've mixed up the dates. (?)

The compiled crop_rec_4k_mlv_lite_snd branch, which I linked some posts before, has the mlv_play module in version from 02.2018. The patch, which you mentions, was commited by g3gg0 in June 2017. As a result, this patch is already included in the mlv_play which I posted. So either it just doesn't work on 70D or there was a regression due to the following commits... Going some revisions back would be needed to find it out. FYI: I haven't used mlv_play at all yet.

[ArcziPL]

Just came home and tested: it works with 14-bit videos recorded with mlv_lite but not with 10-bit. The patch from g3gg0, which you mentioned, was rather addressing another problem.

[shinoda]

Either I can't follow you or you've mixed up the dates. (?)

The compiled crop_rec_4k_mlv_lite_snd branch, which I linked some posts before, has the mlv_play module in version from 02.2018. The patch, which you mentions, was commited by g3gg0 in June 2017. As a result, this patch is already included in the mlv_play which I posted. So either it just doesn't work on 70D or there was a regression due to the following commits... Going some revisions back would be needed to find it out. FYI: I haven't used mlv_play at all yet.

Sorry, i was talking about february 12, my bad for the poorly formatted date, and I see I misred the commit...

Same behaviour here: working in 14-bit but not for 10 or 12-bit. Recording seems continuous in 1600*900 with this bit depth, so it'll do! (i just wanted to have some playback for outside shootings)

[David_Hugh]

Had a little time at hand (havent had too much of that lately unfortunatley), and I am trying to get back into the line skipping/binning experiment here, and I observed something. The task was to change the ADTG2[800c], but are we sure that is the right register? It only appears after I played back a h.264 file, not after I recorded it. I can only see it when all 567 something registers are visible, when I stay in video mode (without playback) and go into 5x or change to 1080p, the maximum I can see are 160 something registers.

Should the ADTG reigster I am looking for change when I go into 5x zoom mode? Because there are some that do. I am trying to wrap my head around where I should start looking. In other words - what is an action that I can implement that should defintely move the register I am looking for?

In post #3111 a1ex said that the CMOS registers are likely different. does that mean the VALUE for ADTG2[800c], or a different register alltogether?

It seems to me that this is a task that could potentially be solved, since lossless compression appears to be too complicated for mere earthlings . It would also mean that with the new sd card overclocking feature the 70d could venture into useful slow-mo terretory (in case anyone else with a little time and limited knowledge needed an incentive - or am I missing something?)

As always, thank you everybody for contributing,
Cheers Dave

[ArcziPL]

Same behaviour here: working in 14-bit but not for 10 or 12-bit. Recording seems continuous in 1600*900 with this bit depth, so it'll do! (i just wanted to have some playback for outside shootings)

Just checked recording with mlv_lite from the crop_rec_4k_mlv_snd build on EOS M and the result is same:
14bpp video: plays with mlv_play, showing 14 bit
12bpp video: black screen, showing 12 bit
14bpp lossless video: plays with mlv_play, showing 14 bit JP92
12bpp lossless video: plays with mlv_play, showing 14 bit JP92 (so it seems to work, as the output of the decompressor is 14 bit)

[David_Hugh]

Hey everyone! I think I made a major step forward with the crop_rec module. I discovered that the weird stripe patterns go away when I change CMOS(b) from 0x800 to 0x600. Oddly enough the value in 1080p mode is 0x400! Video in the link is the one I recorded with both ADTG[800c] and CMOS(b) changed!

https://we.tl/t-XlugLACvNW

Is this as useful as I hope it is  ?

[David_Hugh]

I found a reproducable way to get h264 AND raw recording, without the white borders !!! It still got the pink and green pixel and the lag but its working fine otherwise !!
I had to overwrite CMOS(b) to 200 (from 800, value in 1080p is 400) and CMOS(a) to 1f1 (from 5f1, value in 1080p is also 1f1 !!)
I think in my original attempt I didnt refresh the live screen often enough to see which register had which impact
Here's the Link
https://we.tl/t-Ea1XGM5Rkw

As always, hope this helps !

[a1ex]

Sounds promising; mind summarizing what registers you have changed? A screenshot of adtg_gui, showing the "Overriden regs only" tab, should be good enough.

Register 0x800c is the line skipping factor, and MLV metadata shows 60 FPS. Are you trying to get 50/60 FPS with 3x3 binning, right?


Unrelated: just stumbled upon something interesting in the firmware. Not sure if nikfreak or anyone else tried that.

Code: [Select]

call("lv_daf_mode", 1); // "FACTORY:AB_OUT_LEFT"
call("lv_daf_mode", 2); // "FACTORY:AB_OUT_RIGHT"
call("lv_daf_mode", 0); // guess: default, both "channels"

I'd like to see some DNGs from LiveView (plain 1080p) after calling the above functions, without camera movement between the test shots if possible.

@David_Hugh: are you able to compile from source?

[David_Hugh]

Canon EOS 70D 1.1.2
00f0000a:     5f1 ISO=800 Tv=160 Av=28 lv=1 zoom=1 mv=1 res=1 crop=0 task=Evf pc=49fb4 addr=404e7874
00f0000b:     800 ISO=800 Tv=160 Av=28 lv=1 zoom=1 mv=1 res=1 crop=0 task=Evf pc=49fb4 addr=404e7876
0002800c:       4 ISO=800 Tv=160 Av=28 lv=1 zoom=1 mv=1 res=1 crop=0 task=Evf pc=4b5e8 addr=404e5e1c Line skipping factor (2 = 1080p, 4 = 720p, 0 = zoom)


Is this the right format of the log? Yes I am trying to get the 720p/60fps mode to work with 3x3 binning, trying to follow instructions that dfort gave back in march  . Anyway, what I did was

ADTG2[800c] from 4 -->2
CMOS (b) from 800 --> 200  (this is the only value that differs from 1080p mode, its 400 in 1080p
CMOS (a) from 5f1 --> 1f1

Unfortunately I am still not able to compile, I am trying to get there. My method atm was to find registers that change when switching video modes and trying to find combinations that work. In other words, using brute force

[David_Hugh]

As for the crop_rec experiment, I still have the same problems of stuttering (because fps override doesn't work on the 70D I believe), but I also think I'm really onto something. In the link below I shot a scene which I know produces extreme moire and color artifacts for me when shot in 720p mode, but as you can see, in the crop_rec mode this is greatly reduced. This really is the improvement in image quality in 720/60p mode I was looking for, but I guess in order for it to work properly I have to get more familiar with the registers and above all compiling...  .  https://we.tl/t-14HITuZvg6

[David_Hugh]

Alright,
Nothing quite new regarding 60fps 3x3 on the 70D, but here is the requested screenshot with the overidden registers and a screenshot of the frame it produces. As you can see it still got some artifacts and the frames also freeze from time to time during recording. 





Also, I am able to compile now (yay - thanks @dfort for helping me set up an environment). However, as I already said before, I have ZERO coding experience so with regard to the requested functions you'd have to tell me exactly where to insert them in the code.

Totally unrelated. As I know nothing of the architecture of the cams I'm not sure if that's relevant but the reason why I decided for a 70D in the first place is that it's the only cam that runs ML with dual pixel AF, right? Maybe that's why some things work differently? Just a random thought.

[a1ex]

Yes, dual pixel raw is pretty interesting - I've even got some ideas that might help recovering out of focus images (look at [17], [18], [22] and [23]). Only tested on synthetic dual pixel images, with artificial blur and artificial Gaussian noise (in other words, spherical chickens in a vacuum).

I'd try something along these lines - test code goes in debug.c, "Don't click me" menu entry:

Code: [Select]

diff -r 7fe64903a4de src/debug.c
--- a/src/debug.c
+++ b/src/debug.c
@@ -146,6 +146,8 @@
 }
 
+/* a bit of a hack, but we can just reuse this for dumping the raw buffer */
 static void dump_img_task(void* priv, int unused)
 {
+#if 0   /* we don't need these delays for our experiment */
     for (int i = 5; i > 0; i--)
     {
@@ -154,4 +156,5 @@
     }
     NotifyBox(5000, "Dumping VRAMs...");
+#endif
     
     FILE * f = NULL;
@@ -275,6 +278,15 @@
 #endif
 
+/* run "Don't click me", then don't move the camera until finished */
 static void run_test()
 {
+    msleep(5000);           /* wait for camera vibrations to settle after activating the menu */
+    dump_img_task(0, 0);    /* dump default raw buffer */
+    call("lv_daf_mode", 1); // "FACTORY:AB_OUT_LEFT"
+    dump_img_task(0, 0);    /* expecting image slightly shifted in one direction horizontally */
+    call("lv_daf_mode", 2); // "FACTORY:AB_OUT_RIGHT"
+    dump_img_task(0, 0);    /* expecting image slightly shifted in the other direction */
+    call("lv_daf_mode", 0); // guess: default, both "channels"
+    dump_img_task(0, 0);    /* expecting an image identical to the first one (except for the random noise) */
 }

The artifacts from your screenshot are just bad pixels (all cameras have them). ML renders a very low-res aliased preview, to minimize CPU usage; that makes some image defects more noticeable than they really are. Canon preview is done on their image processor (outside the main CPU), but we don't fully understand it (just slowly making progress).

[David_Hugh]

Here's the test results! Pretty ähm.. interesting  .

But almost like you expected, right?
https://we.tl/t-BykEItcY9C

[David_Hugh]

Also, could somebody point me in a direction with regard to the 720p 60fps 3x3 experiment? dfort advised me to lower the fps, but as far as I know, fps overrider doesn't work on the 70D. So I guess I'd have to manually mess with timer A and B, right? Are the timer registers known on the 70D? Because I cant seem to find C0F0[6000] or [6008], which are the registers for other cams if I'm not mistaken?

Should this measure fix the lagging/temporarily freezing recording and preview?
Seems like I'm not too far off but since I know so little about image processing in general I can't really gauge if I'm close or if there are other substantial parts missing in order for this to work.

[a1ex]

Here's the test results! Pretty ähm.. interesting  .

But almost like you expected, right?
https://we.tl/t-BykEItcY9C

Haha, well... not exactly what I was expecting, but nevertheless, very interesting.

Initial notes: 70D-dual-pixel.html (13MB)

Also, could somebody point me in a direction with regard to the 720p 60fps 3x3 experiment? dfort advised me to lower the fps, but as far as I know, fps overrider doesn't work on the 70D. So I guess I'd have to manually mess with timer A and B, right? Are the timer registers known on the 70D? Because I cant seem to find C0F0[6000] or [6008], which are the registers for other cams if I'm not mistaken?

To reduce the FPS, increase C0F06008 (timer A) or C0F06014 (timer B) from adtg_gui. If these registers do not appear, you may need an updated build, as timer B may go back and forth between two values (so old adtg_gui will dismiss it as "noise"). Nikfreak reported this method to work on 70D back then, and I'm working on a rewrite of the entire FPS override implementation to use this method. Didn't get too far, only confirmed the same method would also fix some issues with 700D.

General idea: increase timer A for more horizontal resolution and timer B for more vertical resolution. There may be some multiplier, i.e. 1 timer A increment = 8 pixels on 5D3 and 4 on 700D. Will do the math for 70D later.

Some numeric examples from 5D2 (although this model has fewer registers, therefore a lot easier). On DIGIC 5 models you'll also have some powersaving registers; for experiments, just set them to 0.