Author Topic: 12-bit (and 10-bit) RAW video development discussion  (Read 488471 times)

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3348
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
12-bit (and 10-bit) RAW video development discussion
« Reply #1750 on: February 21, 2019, 08:57:56 PM »
Good to see a new update on the 10/12 bit build.

Perhaps it's been put there for a reason, right?

I guess I'll try it out.

Hopefully you can help us hunt down some bugs while you're at it.

I can't do 5x zoom for normal use, so I don't consider it. It's not worth it for me, personally, to do so.
I'd rather 900p 14 bit raw footage with my 7D to make magic.

Well if you call that magic then how do we expect @dfort to create even more magic when we refused to force ourselves to dig through puddle of mud?

Nevertheless... I think it is worth to try and fine-tune this mighty 7D into the 4K_crop_rec experimental branch while we have this rare opportunity.

5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

domo94

  • Freshman
  • **
  • Posts: 99
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1751 on: February 21, 2019, 09:02:50 PM »
I guess so.

I meant that more in the sense of me just not using 10/12 bit.

I will most def help in the development because I love my 7D to death.

I'll run a couple tests today.

Let's see if I can even use the glitchiness of it in order to make some cool in camera visual effects for music videos, lol.

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3740
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1752 on: February 21, 2019, 10:01:18 PM »
Ha ha -- let's call this the Max Headroom build.

Actually, I've seen much worse. Try 10bit 12bit for some lovely pink highlights. These issues have been resolved on other cameras. It is just a matter of searching the forum and commits.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

domo94

  • Freshman
  • **
  • Posts: 99
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1753 on: February 21, 2019, 10:05:41 PM »
Running tests right now. The same set up that Jedi just recommended and then some of my own.

Nothing is turning out good.

Best I got was MLV Lite module with 12bit standard zoom video. It was most stable with little jitters and skips and frame corruption.

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3348
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1754 on: February 21, 2019, 10:37:47 PM »
Running tests right now. The same set up that Jedi just recommended and then some of my own.

Nothing is turning out good.

Best I got was MLV Lite module with 12bit standard zoom video. It was most stable with little jitters and skips and frame corruption.

Have you tried resetting Canon menu and custom settings?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

domo94

  • Freshman
  • **
  • Posts: 99
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1755 on: February 21, 2019, 11:59:09 PM »
I will reset my settings tonight and start from scratch.

I also notice every once in a while, my camera gets REALLY REALLY slow and laggy.

I'll want to maneuver through menus and selections and the camera would respond like 2-5 seconds later. It varies.

This happens with 2.0.3 for me a lot, almost rarely with 2.0.6 if I remember correctly, but I hardly ever, if not EVER, experienced it on 2.0.6.

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3740
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1756 on: February 22, 2019, 12:59:54 AM »
My 7D.206 port doesn't use the MASTER processor and the experimental 10/12bit also isn't using it because it won't compile. Maybe that makes a difference?
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

IDA_ML

  • Hero Member
  • *****
  • Posts: 660
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1757 on: February 22, 2019, 12:22:32 PM »
Today is my lucky day!  Opening this thread, I found that Dfort posted a new Build for the EOS 7D.  I am so happy, Dfort, that you didn't give up on the good old 7D!  Thank you so much!

I did not have time to test for more than a few minutes - had to run for work, but I noticed that you have fixed the most nasty 7D bug for the last two years - the freezing preview screen at 10 and 12 bits and 2,5K which I had to unfreeze by changing the preview twice before starting each new recording.  Now, I have to do this only once, after camera turn on.  Fantastic!  Not quite WYSIWYG due to 5x magnification but I can frame now!

What is not working yet is the earthquake shaking at 10 and 12 bits in the normal 1728x972 mode.  But you really are pretty close to solving that problem too.  Every second frame is broken, the rest is OK.

Now I get continuous recording with sound at 2496x1198/10bit/24fps and also at 1728x972/14-bit in the old MLV mode.  As expected, image quality and cinematic look are gorgeous and MLVApp does a hell of a job processing the files.  This is also evident from DeafEyeJedi's video and his supermodel - the cat, beautifully illustrating what the 7D is up to!  I had almost forgotten how it feels like working with this high dynamic range, gorgeous colors and large processing headroom.

I am sure, if the 7D receives the attention and developers' effort it deserves, it will pay off to all of us.  So, just keep going, Dfort !!!


dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3740
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1758 on: February 22, 2019, 07:30:02 PM »
Not much closer than we were before. It looks like we still haven't found the right stubs. At least now the 7D will compile on the raw_video_10bit_12bit_LVState branch, albeit without compiling 7D_MASTER.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6032
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1759 on: February 22, 2019, 07:59:02 PM »
That is very good. Now IDA_ML or some other energetic 7D user could start firing up adtg_gui and get busy with crop rec presets.

IDA_ML

  • Hero Member
  • *****
  • Posts: 660
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1760 on: February 23, 2019, 08:52:26 AM »
I have now also tested the "RAW video" mode of the Feb. 21 build.  Unfortunately, things do not work so well here:

Working:
----------
1)  The 1736x976/14bits/24fps choice.  Camera provides about 35 sec. of recording time in that mode.  If you need continuous recording, just use the 1728x972/14bits/24fps choice in the "RAW video (MLV)" mode.  It works very well, also with sound.  No corrupt frames either.

Not working:
--------------
2)  All other selection choices: 1736x976 at 10/12 bits and the 5x zoom mode at 2520x1192 at 10/12/14 bits provide corrupt frames and earthquake shaking.

3)  Trying to record 2496x1196 at 14 bits in both: the RAW video and RAW video (MLV) modes does not stop recording properly, although the MLV file gets recorded.  You need to turn camera off and pull the battery to continue working.

Note:
------
If you add the raw_twk.mo from the Dec. 1, 2016 build to the modules directory and activate it in the Modules menu, you will be able to playback your MLV videos in the camera.

reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2101
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1761 on: February 25, 2019, 08:18:11 AM »
I think I found something that may help with 7d in 10-12bit .
I being investigating canon LV with adtg_gi.mo on my 5d2 ,
I think I have recreated the every other fame corruption plus as @IDA_ML puts it "earthquake shaking"
c0f08188 (raw.width)aka HIV_H_SIZE , can breaks raw buffer if not set to the same as the raw resolution
and can give this  "earthquake" thing .
So maybe check c0f08188 and see what it doing & maybe do a image dump & check the raw buffer size to see if these match
e.g I know on 5d2 in 3xcrop the raw buffer is 2152(H)  & C0f08188=0x907(2311) so if you add the OB area to the raw buffer
2152+160 = 2312 -- so it matches . The question is , does the 7D raw image buffer match the C0f08188 in 10/12bit ?
6 second H264 clip 22MB
EarthQuake-Shaking_M25-0003.mov
MLV - 500MB 1856x1248_29.97
M25-0003.MLV
What I did to recreate the "earthquake shaking" is set c0f08188 from(default) 4d707f7 -> 4d70907 in 1:1 FHD so 2039 to 2311

IDA_ML

  • Hero Member
  • *****
  • Posts: 660
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1762 on: February 25, 2019, 10:13:40 AM »
Thank you very much, Reddeercity!  Any hint from knowledgeable and experienced people like you could be very helpful. 

scrax

  • Contributor
  • Hero Member
  • *****
  • Posts: 1543
  • Code monkey
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1763 on: February 26, 2019, 12:02:18 PM »
600D: ? (internals like 60D)

I've tried on my 600D, here screen captures for 640x480 and crop modes
I'm using ML2.3 for photography with:
EOS 600DML | EOS 400Dplus | EOS 5D MLbeta5- EF 100mm f/2.8 USM Macro  - EF-S 17-85mm f4-5.6 IS USM - EF 70-200mm f/4 L USM - 580EXII - OsX, PS, LR, RawTherapee, LightZone -no video experience-

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3740
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1764 on: February 28, 2019, 03:29:22 AM »
@scrax - There's more to the test than that:

Here's a test I'd like to run on all EvfState models (those with 10/12-bit raw video builds on the Experiments page). Instructions in the commit message. In addition, I'd like a screenshot of the Free Memory dialog in movie mode. I've ran the test on 60D and 5D3 1.1.3; these are the only EvfState models I own.

@reddeercity - Appreciate the comments about tweaking registers with adtg_gui but that module doesn't build on the raw_video_10bit_12bit_LVState so it will require some more work before we can try that.

I've been pulling out what little hair I have left because I would have sworn that the stubs @aprofiti posted were working on my 7D. Now it is back to only working in 5x zoom mode which was always sort of almost working anyway even without the patch manager doing its magic.

There's a note about the stubs needing to start at 0xFF9 but the 7D disassembly doesn't match the mirroring of the 5D2 and 50D which are so far the only two cameras working on that branch--right?
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2101
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1765 on: February 28, 2019, 07:16:46 AM »
@dfort , what about digital poke ?
Before a1ex fixed adtg_gui.mo for d4's (broken from some d5 code etc... ) waza57 &  myself used digital poke to extended raw resolutions  .
This committed db30a11 is for enabling digic poke on 5d2 , should work on 7D i would think. I can see if i can compile adtg_gui mo. for 7d on my local source.
I Idea ! , did you try the adtg_gui.mo from the https://builds.magiclantern.fm/modules.html download page ?
That's the one I've been using on my 5D2 for the last mouth or so . It should work on top of any build .

Also if you can't get the redirect buffer to work , there's still waza 57 Rom Hack  to extend resolutions
Food for thought .

...... 5D2 and 50D which are so far the only two cameras working on that branch--right?
Yes

aprofiti

  • Contributor
  • Member
  • *****
  • Posts: 185
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1766 on: February 28, 2019, 01:12:34 PM »
There's a note about the subs needing to start at 0xFF9 but the 7D disassembly doesn't match the mirroring of the 5D2 and 50D which are so far the only two cameras working on that branch--right?
Are we sure that there is a mirroring also on the 7D?

If I look in both 50D and 5D2 disassembly I can find 4 references for the string "StartPass_x1 CrawAddr : %lx / KindOfCraw : %d" / "StartImagePass_x1 CrawAddr : %lx / KindOfCraw : %d", but only 2 for 7D; same for the second stub.

Try inserting some debug messages to check patch execution:
Code: [Select]
diff --git a/src/patch.c b/src/patch.c
--- a/src/patch.c
+++ b/src/patch.c
@@ -926,6 +926,7 @@
 
 int patch_hook_function(uintptr_t addr, uint32_t orig_instr, patch_hook_function_cbr logging_function, const char * description)
 {
+    printf("In patch_hook_function()\n");
     int err = 0;
 
     /* ensure thread safety */
@@ -944,6 +945,7 @@
     
     if (logging_slot < 0)
     {
+        printf("patch_hook_function() - Error no logging slot available!\n");
         snprintf(last_error, sizeof(last_error), "Patch error at %x (no logging slot)", addr);
         puts(last_error);
         err = E_PATCH_TOO_MANY_PATCHES;
@@ -957,6 +959,7 @@
     if (!check_jump_range((uint32_t) &hook->reloc_insn, (uint32_t) addr + 4) ||
         !check_jump_range((uint32_t) addr,              (uint32_t) hook))
     {
+        printf("patch_hook_function() - Error jump out of range!\n");
         snprintf(last_error, sizeof(last_error), "Patch error at %x (jump out of range)", addr);
         puts(last_error);
         err = E_PATCH_UNKNOWN_ERROR;
@@ -989,12 +992,14 @@
 
     /* since we have modified some code in RAM, sync the caches */
     sync_caches();
-   
+
+    printf("patch_hook_function() - Patching the original instruction!\n");
     /* patch the original instruction to jump to the logging code */
     err = patch_instruction(addr, orig_instr, B_INSTR(addr, hook), description);
     
     if (err)
     {
+        printf("patch_hook_function() - Error something went wrong?\n");
         /* something went wrong? */
         memset(hook, 0, sizeof(union logging_hook_code));
         goto end;
diff --git a/src/raw.c b/src/raw.c
--- a/src/raw.c
+++ b/src/raw.c
@@ -1755,6 +1755,7 @@
     int ok = raw_lv_get_resolution(&width, &height);
     if (ok)
     {
+      //  printf("raw_lv_setedmac_patch() - Updating EDMAC image size\n");
         /* update EDMAC image size */
         int pitch = width * raw_info.bits_per_pixel / 8;
         static struct edmac_info dst_edmac_info;
@@ -2021,8 +2022,11 @@
 #ifndef CONFIG_EDMAC_RAW_SLURP
     call("lv_save_raw", 1);
 #ifdef CONFIG_EDMAC_RAW_PATCH
+    printf("patch_hook_function() - Trying to patch RAW LV x1\n");
     patch_hook_function((uint32_t) &StartImagePass_x1_SetEDmac, 0xE3A03202, raw_lv_setedmac_patch, "RAW LV x1");
+    printf("patch_hook_function() - Trying to patch RAW LV x5\n");
     patch_hook_function((uint32_t) &StartImagePass_x5_SetEDmac, 0xE3A03202, raw_lv_setedmac_patch, "RAW LV x5");
+    printf("patch_hook_function() - Done\n");
 #endif
 #endif
 
@@ -2066,6 +2070,7 @@
 #ifndef CONFIG_EDMAC_RAW_SLURP
     call("lv_save_raw", 0);
 #ifdef CONFIG_EDMAC_RAW_PATCH
+    printf("Unpatching RAW LV x1/x5\n");
     unpatch_memory((uint32_t) &StartImagePass_x1_SetEDmac);
     unpatch_memory((uint32_t) &StartImagePass_x5_SetEDmac);
 #endif

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3740
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1767 on: February 28, 2019, 08:51:32 PM »
@aprofiti - Thanks for the patch.

@dfort Can you try these ones?
Code: [Select]
NSTUB(0xFF27E674, StartImagePass_x1_SetEDmac)
NSTUB(0xFF27F444, StartImagePass_x5_SetEDmac) 



Looks like the "Jump range error" message was already patch.c but I didn't think of opening the console -- Doh!

Ok--so these aren't the stubs we're looking for.

First one It's a bit different but function called looks likely the same.
Looking into the disassembly I can't find a mirror for both stubs in the 0xFF9 range or similar like on 50D

Right, on the 7D these stubs show up only once. I did take a close look at them and it seems that the second one (StartImagePass_x5_SetEDmac) can be matched perfectly to what works on the 5D2/50D but the stub you came up with is slightly different. Is there a reason for that? Here's the "perfect" match:

platform/7D.203/stubs.S
Code: [Select]
NSTUB(0xFF27F43C, StartImagePass_x5_SetEDmac)

Looking at the pieces of the puzzle that fit, don't fit or are missing:

src/raw.c
Code: [Select]
    printf("patch_hook_function() - Trying to patch RAW LV x5");
    patch_hook_function((uint32_t) &StartImagePass_x5_SetEDmac, 0xE3A03202, raw_lv_setedmac_patch, "RAW LV x5");

I get that "E3A03202" is the instruction, "mov r3, #536870912; 0x20000000" which is what the StartImagePass_x5_SetEDmac stub is pointing at. I also looked at the patch_hook_function definition to try and figure out what is being passed to that function:

src/patch.h
Code: [Select]
int patch_hook_function(uintptr_t addr, uint32_t orig_instr, patch_hook_function_cbr logging_function, const char * description);

So--I guess that one looks good but the first stub, StartImagePass_x1_SetEDmac is definitely different. The closest match I could find is the same as the one you found:

platform/7D.203/stubs.S
Code: [Select]
NSTUB(0xFF27E674, StartImagePass_x1_SetEDmac)

However, the instruction at that address is, "E3A00010" which is "mov r0, #16" so maybe we need to modify the patch_hook_function call to this?

src/raw.c
Code: [Select]
    printf("patch_hook_function() - Trying to patch RAW LV x1");
    patch_hook_function((uint32_t) &StartImagePass_x1_SetEDmac, 0xE3A00010, raw_lv_setedmac_patch, "RAW LV x1");

So how did that work out? Same errors as before. Does this mean we're patching a mirrored copy and not executable code?

That means, you should patch some address that Canon code is going to execute, not a mirrored copy.

Where's the code? Show me the code.

@dfort , what about digital poke ?
Before a1ex fixed adtg_gui.mo for d4's (broken from some d5 code etc... ) waza57 &  myself used digital poke to extended raw resolutions  .

I'm not extending raw resolutions at this point, simply trying to get 10/12-bit working on the 7D.

I Idea ! , did you try the adtg_gui.mo from the https://builds.magiclantern.fm/modules.html download page ?

That module comes from the iso-research branch and the 7D isn't supported. I found the stubs:

modules/adtg_gui/adtg_gui.c
Code: [Select]
    else if (is_camera("7D", "2.0.3"))
    {
        ADTG_WRITE_FUNC = 0xFF2C0944; //"[REG] @@@@@@@@@@@@ Start ADTG[CS:%lx]"
        CMOS_WRITE_FUNC = 0xFF2C0B3C; //"[REG] ############ Start CMOS"
        ENGIO_WRITE_FUNC = 0xFF1F6B20;  // from stubs
        ENG_DRV_OUT_FUNC = 0xFF1F675C;
    }

The 7D_MASTER.203 doesn't build on the iso-research branch so it needs the same Makefile hack I used on the raw_video_10bit_12bit_LVState branch. So now it builds, adtg_gui launches and --- it doesn't work!
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12323
  • Emergencies only
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1768 on: February 28, 2019, 09:17:28 PM »
Jump out of range: see the 1300D thread for some background info. On ARM, you cannot jump more than 32 MB with one instruction; you would have to patch two instructions in the original code (i.e. a use long jump).

On 7D, it's easy to fix - in the dm-spy-experiments branch (where I need to patch stuff for logging purposes) I'm starting ML on this camera with the "classic" method (PR #731). I had some trouble with that PR on 60D, though; need to double-check. The new boot method should fix the patch issues.

[ offtopic: on 1300D, that trick won't work, because of the ROM layout; there we really need to use long jumps; I have a local fix that I need to clean up ]

Unfortunately, adtg_gui is not going to work on 7D after some pattern matching. First, one has to find a way to run code on the master processor starting from autoexec.bin. Next, one has to implement ADTG/CMOS logging hooks on the master processor, and forward all the info about these register writes, back into the slave processor, so they can be displayed on the GUI (and hope it's fast enough to avoid interference with the photo capture process). It's really not trivial; possibly more difficult than porting ML on DIGIC 6/7/8.

Mirroring: all Canon cameras use it, but ROM sizes differ. 5D2 uses 8MB for the main ROM (what we call ROM1); the "slave" side of 7D uses 1 x 16MB for the same ROM. Newer cameras use 32MB for the main ROM. Memory map (including ROM mirrors) is displayed in QEMU at startup.

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3740
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1769 on: February 28, 2019, 10:37:22 PM »
It's really not trivial; possibly more difficult than porting ML on DIGIC 6/7/8.

Not good news. Especially considering that I still haven't finished porting the EOSM2 which should be easy.

Memory map (including ROM mirrors) is displayed in QEMU at startup.

You mean like this?

Code: [Select]
./run_canon_fw.sh 7D,firmware=boot=0 -d debugmsg

DebugMsg=0xFF0776AC (from GDB script)
Lockdown read 1
Lockdown read 1
Lockdown read 0
Lockdown read 0
Lockdown read 2
Lockdown read 2
Lockdown read 3
Lockdown read 3
Lockdown read 4
Lockdown read 4
Lockdown read 5
Lockdown read 5
00000000 - 00000FFF: eos.tcm_code
40000000 - 40000FFF: eos.tcm_data
00001000 - 1FFFFFFF: eos.ram
40001000 - 5FFFFFFF: eos.ram_uncached
F0000000 - F0FFFFFF: eos.rom0
F1000000 - F1FFFFFF: eos.rom0_mirror
F2000000 - F2FFFFFF: eos.rom0_mirror
F3000000 - F3FFFFFF: eos.rom0_mirror
F4000000 - F4FFFFFF: eos.rom0_mirror
F5000000 - F5FFFFFF: eos.rom0_mirror
F6000000 - F6FFFFFF: eos.rom0_mirror
F7000000 - F7FFFFFF: eos.rom0_mirror
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror
FA000000 - FAFFFFFF: eos.rom1_mirror
FB000000 - FBFFFFFF: eos.rom1_mirror
FC000000 - FCFFFFFF: eos.rom1_mirror
FD000000 - FDFFFFFF: eos.rom1_mirror
FE000000 - FEFFFFFF: eos.rom1_mirror
FF000000 - FFFFFFFF: eos.rom1_mirror
C0000000 - CFFFFFFF: eos.mmio
[EOS] enabling code execution logging.
[EOS] loading './7D/ROM0.BIN' to 0xF0000000-0xF0FFFFFF
[EOS] loading './7D/ROM1.BIN' to 0xF8000000-0xF8FFFFFF
[MPU] FIXME: using generic MPU spells for 7D.
...
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12323
  • Emergencies only
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1770 on: February 28, 2019, 10:44:16 PM »
Indeed. On 7D you get:

Code: [Select]
F8000000 - F8FFFFFF: eos.rom1
F9000000 - F9FFFFFF: eos.rom1_mirror
FA000000 - FAFFFFFF: eos.rom1_mirror
FB000000 - FBFFFFFF: eos.rom1_mirror
FC000000 - FCFFFFFF: eos.rom1_mirror
FD000000 - FDFFFFFF: eos.rom1_mirror
FE000000 - FEFFFFFF: eos.rom1_mirror
FF000000 - FFFFFFFF: eos.rom1_mirror

On 5D2/50D, just noticed I need to fix it (.rom1_size = 0x00800000; at least qemu prints a warning):

Code: [Select]
F8000000 - F87FFFFF: eos.rom1
F8800000 - F8FFFFFF: eos.rom1_mirror
F9000000 - F97FFFFF: eos.rom1_mirror
F9800000 - F9FFFFFF: eos.rom1_mirror
FA000000 - FA7FFFFF: eos.rom1_mirror
FA800000 - FAFFFFFF: eos.rom1_mirror
FB000000 - FB7FFFFF: eos.rom1_mirror
FB800000 - FBFFFFFF: eos.rom1_mirror
FC000000 - FC7FFFFF: eos.rom1_mirror
FC800000 - FCFFFFFF: eos.rom1_mirror
FD000000 - FD7FFFFF: eos.rom1_mirror
FD800000 - FDFFFFFF: eos.rom1_mirror
FE000000 - FE7FFFFF: eos.rom1_mirror
FE800000 - FEFFFFFF: eos.rom1_mirror
FF000000 - FF7FFFFF: eos.rom1_mirror
FF800000 - FFFFFFFF: eos.rom1_mirror

The last copy of this ROM is actually used by Canon for code execution, so... that's what we need to patch.

According to g3gg0, the main copy of the ROM (the one used for reflashing and other low-level access) is at F8000000, so I've tried to model the memory map in the same way.

aprofiti

  • Contributor
  • Member
  • *****
  • Posts: 185
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1771 on: February 28, 2019, 10:59:47 PM »
Right, on the 7D these stubs show up only once. I did take a close look at them and it seems that the second one (StartImagePass_x5_SetEDmac) can be matched perfectly to what works on the 5D2/50D but the stub you came up with is slightly different. Is there a reason for that? Here's the "perfect" match:
platform/7D.203/stubs.S
Code: [Select]
NSTUB(0xFF27F43C, StartImagePass_x5_SetEDmac)
Maybe I mixed the the two when looking again at the address before posting. Need to recheck...

To use as reference... this is what is showing on 50D:



Last message in console show up when exiting from LV

Jump out of range: see the 1300D thread for some background info. On ARM, you cannot jump more than 32 MB with one instruction; you would have to patch two instructions in the original code (i.e. a use long jump).

On 7D, it's easy to fix - in the dm-spy-experiments branch (where I need to patch stuff for logging purposes) I'm starting ML on this camera with the "classic" method (PR #731). I had some trouble with that PR on 60D, though; need to double-check. The new boot method should fix the patch issues.
just to have a better pictures:
Do it need two instructions in ARM mode into one, so 64bit length in total for the long jump instruction?
Is it done automatically by the patchmanger backend or need to manually add a new "instruction patch request" to make it works?
Or Do we need to fork the changes from PR #731 and apply there the patches instead?

Memory map (including ROM mirrors) is displayed in QEMU at startup.
Yeah, remembered about this, so tried to look at 50D to understand what address range to use, but found that I can't run it anymore...
Something screwed at yesterday after installing exiv2 using packet manager...
Code: [Select]
./run_canon_fw.sh 50D, firmware=boot=0

DebugMsg=0xFF863B10 (from GDB script)
qemu-system-arm: -M 50D,: drive with bus=0, unit=0 (index=0) exists
Reinstalled QEMU but still there... need to figure out what do do....
Then finally forget to post the message for dfort...

On the PR #731 I read: "enable this on 7D (done, please test), maybe also 50D and 700D"
Is there something I can do?
Testing or looking for something? (in that case need some explanation to understand what to do)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12323
  • Emergencies only
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1772 on: February 28, 2019, 11:29:25 PM »
If you boot the 7D with the current method (i.e. loading ML in RscMgr memory), ML will be loaded far away from the ROM. In this case, you will need long jumps (two ARM instructions for one jump, i.e. 64 bits or two int32's to patch). No, patch manager won't do that for you yet, but I'm working on it (I've got it halfway working for 1300D).

However, you can boot the 7D with the "classic" method (see the PR). In this case, you no longer need long jumps, as ML will be loaded close to the main ROM (close enough to use short jumps). We've already tested that some time ago, on the dm-spy-experiments branch, and IIRC it worked.

50D: the only benefit from that PR would be some extra general-purpose memory (10MB, see here). On 700D, there is an extra 4MB block. This extra memory is not essential; one has to make sure that block is really unused by Canon firmware; it's not uncommon for some memory to appear unused in the RscMgr map, but to be actually used under certain conditions. This was the trouble with 60D (and the reason that PR was stalled), so... let's focus on what's actually broken for now.

50D memory map in QEMU: it's the same as 5D2, but I need to fix the code (as the current repo version will display the one for 7D and other newer models).

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3740
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1773 on: March 01, 2019, 09:12:03 PM »
However, you can boot the 7D with the "classic" method (see the PR).

Just checking -- are you referring to this PR?

Then once I get that working in my raw_video_10bit_12bit_LVState_7D_experiments branch then the stubs I found should work? Looks like they are already pointing to a mirrored copy (0xFF) that is used by Canon for code execution. (or did I misunderstand this?)

Code: [Select]
/** LiveView RAW patches **/
NSTUB(0xFF27E674, StartImagePass_x1_SetEDmac)
NSTUB(0xFF27F444, StartImagePass_x5_SetEDmac)
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3740
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1774 on: March 02, 2019, 04:53:22 AM »
Yeah, I know--stop asking questions and just do as the man says.

It works.



Had to take into account the different instructions on the 7D stubs:

src/raw.c
Code: [Select]
#ifndef CONFIG_EDMAC_RAW_SLURP
    call("lv_save_raw", 1);
#ifdef CONFIG_EDMAC_RAW_PATCH
#ifdef CONFIG_7D
    patch_hook_function((uint32_t) &StartImagePass_x1_SetEDmac, 0xE3A00010, raw_lv_setedmac_patch, "RAW LV x1");
#else
    patch_hook_function((uint32_t) &StartImagePass_x1_SetEDmac, 0xE3A03202, raw_lv_setedmac_patch, "RAW LV x1");
#endif
    patch_hook_function((uint32_t) &StartImagePass_x5_SetEDmac, 0xE3A03202, raw_lv_setedmac_patch, "RAW LV x5");
#endif
#endif

Note that something like this should also work for the 500D and 550D. I haven't found the stubs on the 60D, 600D or 1100D but I didn't look very hard.

The good news is now it will record 10-bit 12-bit without the earthquake effect or corrupted frames -- minimally tested. However, now zoom mode (a.k.a. 2.5K or whatever) doesn't work. In fact it crashes. Don't know if this is a problem on the 5D2/50D as well or just a 7D issue. Ironically, this was the only setting that worked when patching wasn't working.

Next step is to enable CONFIG_DIGIC_POKE since getting adtg_gui to work on the 7D isn't likely at this point. Looks like @waza57 had to do some extra work on it for the 5D2. Then we can continue this discussion on the 3K/UHD 5D2 Raw development and Other Digic IV Cams.

In the meantime -- a new 7D test build is on my Bitbucket downloads page.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102