Author Topic: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)  (Read 93772 times)

names_are_hard

  • Contributor
  • New to the forum
  • *****
  • Posts: 43
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #225 on: July 05, 2019, 04:26:11 AM »
It can boot to Canon menus now, but often hangs afterwards.  One definite problem is initialising fonts; 200D (and other digic7 I think? Can't find this on the forum, maybe it was IRC) uses Opentype, not bitmap.  This makes our init error (it's not possible to give a valid address for BFNT_CHAR_CODES to bfnt_ok()).  Digic6-dumper uses some fixed bitmap font I believe, I'll port that next as the easiest option.

At some point I suppose we want an opentype renderer, or some one time step to convert those to bitmap, dump to ML dir, and ML init could load them to mem.

names_are_hard

  • Contributor
  • New to the forum
  • *****
  • Posts: 43
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #226 on: July 07, 2019, 08:17:02 PM »
Reliable booting to Canon menus.  Okay-for-now workaround for the font problem (conditionally skip the Canon font initialisation and only use the RBF fonts on the card).  Means you can't do early logging to the display, but I've changed the logging to disk code so it makes sequential numbered DBG00000.LOG files.

@kitor pointed me at some vram changes in DIGIC7 that were very useful - hello_world() now prints to display...  but the output is junk.  Need to change some consts to get the offset into display right (maybe just BMP_VRAM_START) and do colour correction since DIGIC7 uses YUV display.  Then see why the font output is garbled.

Anyone know if the RBF font routines require valid Canon bitmap fonts?  I'd guess not, looks like we use Canon fonts as fallback, but I don't really know.

I think this commit is mildly broken and needs an msleep(2000) ish before log_finish() in boot-hack.c (or the menus won't load before hello_world() starts and blocks; this means back screen won't display).  Too lazy to retest and push, local build works, I'll push something else soon enough:
https://bitbucket.org/stephen-e/ml_200d/commits/2fffa3bbda8980fdc3ce1c3fe1a8323ddaee45f4

kitor

  • Contributor
  • Member
  • *****
  • Posts: 119
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #227 on: July 08, 2019, 08:30:51 AM »
Won't bmp_putpixel_fast from bmp.h be a problem here? It's used by font_draw_char from frb_font.c:

Code: [Select]
inline void bmp_putpixel_fast(uint8_t * const bvram, int x, int y, uint8_t color)
{
    /* vxvorks skipped */
    bvram[x + y * BMPPITCH] = color;
    /* 500d skipped ;) */
}

It's definitely not YUV-aware.

What about replacing it with code based on disp_set_pixel from minimal.c in digic6-dumper?
EOS R

names_are_hard

  • Contributor
  • New to the forum
  • *****
  • Posts: 43
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #228 on: July 09, 2019, 02:18:30 AM »
Yes, bmp_putpixel_fast() is part of the problem.  disp_set_pixel() also wants fixing in unified, which isn't as simple as I expected, a bunch of symbols are in different places in unified vs digic6-dumper.

I haven't looked much at how bmp_putpixel_fast() is using its colour param.  It's an enum (eg, COLOR_GREEN1 in bmp.h), but seems to get written direct to vram, so I'm missing something.  Not immediately obvious how to map to YUV (maybe it's not needed, map the YUV to the color params first?).

kitor

  • Contributor
  • Member
  • *****
  • Posts: 119
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #229 on: July 09, 2019, 08:34:29 AM »
I see some discussion about YUV stuff on chdk forum -> https://chdk.setepontos.com/index.php?topic=11316.120

As for "simply replacing" - from what I understand now, YUV buffers have also alpha channel, so replacing rgb values with yuv ones will produce werid results (mentioned in chdk thread too). And Canon seemed to change this twice - once to UYVY + separate opacity data that's in 200D (according to comments in minimal.c - and by "adapted from names_are_hard" i believe you are aware of that ;) ) and then to UYVYAA (5D4, M50, R...)

For now I would just replace bmp_putpixel_fast with implementation based on CONFIG_DIGIC_678 part of disp_set_pixel in digic6-dumper/minimal/hello-world/minimal.c, completly ignoring bvram and converting 8 bit input into 32 by some bitwise operations (?, i was never good at optimizing low-level stuff) since now it's hardcoded to white. This surely will be order of magnitude slower, but if that takes us into ML menu, i guess that's good idea to fix/optimize it after we know that we can do GUI stuff.

As for
Quote
disp_set_pixel() also wants fixing in unified, which isn't as simple as I expected, a bunch of symbols are in different places in unified vs digic6-dumper.

I thought that this is used only from BL context, so not required at this stage:
Code: [Select]
grep -irw disp_set_pixel ./
./src/font_direct.c:    disp_set_pixel(x, y, c);
./src/disp_direct.c:void disp_set_pixel(uint32_t x, uint32_t y, uint32_t color)
./src/minimal.c:void disp_set_pixel(int x, int y, int c)
./src/disp_direct.h:void disp_set_pixel(uint32_t x, uint32_t y, uint32_t color);
EOS R

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12222
  • Maintenance mode
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #230 on: July 09, 2019, 08:44:03 AM »
a bunch of symbols are in different places in unified vs digic6-dumper.

Yeah, that's because I did some changes in digic6-dumper in order to accommodate DIGIC 6/7/8 models, in particular, the refactoring to allow different boot methods, or the one to allow different kind of stubs (such as ARM or Thumb code). Indeed, that branch went a little beyond "just" a simple dumper stage.

The digic6-dumper branch also contains a couple of experimental backends, that are not yet in unified:
- qemu (pretty much ready for merging if you ask me)
- new-dryos-task-hooks (also pretty much ready, depends on qemu, but there may be surprises)
- lua_fix (needs testing on currently supported cameras; depends on qemu, new-dryos-task-hooks, 100D and some others; basically good, but many model-specific changes to review; currently broken on at least 5D2 and 500D)
- patchmgr (breaks 7D, may break some reworked features, needs cleanup, no dependencies)

So, I'd say the logical path to include DIGIC 6/7/8 into unified would be to merge the above branches (and their dependencies) first. There's also calle2010's task rework that needs to be included, as it should be applied to all DIGIC 4/5/6/7/8 models. Unfortunately, a straw broke the camel’s back, so... I'm very sorry these things are not done yet :(

In any case, I see your efforts as major progress - sometimes, reinventing the wheel can be very helpful to understand how things work.

The code is kind of ugly [...]

That's also my issue - I can easily bang out code that does the job, but turning it into something clean and maintainable, with comments and without breaking existing stuff, and with readable history... requires a lot of extra effort from my side. Maybe it's worth it in the long run, I don't know. I don't have any experience with maintaining large codebases, besides ML.

names_are_hard

  • Contributor
  • New to the forum
  • *****
  • Posts: 43
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #231 on: July 10, 2019, 02:27:55 AM »
kitor: I don't think bmp_putpixel_fast() is called with rgb values at all.  Look at the values when it's called, and the constants in bmp.h, eg, COLOR_WHITE is 1, COLOR_BLACK is 2.  Indexed colour maybe.  I need to understand this better before I can change it.  Probably won't have much time to work on it before the weekend.

I don't know what disp_set_pixel() is used for, but it's the easier thing to port, so I may as well do both.  I guess this is quite a long way from a menu :)  This is a CONFIG_HELLO_WORLD build, it bails out very early.  Still, having fonts will make debugging much faster.

Alex: sadly I have taken a horrible mix of your changes as I've noticed them or found them useful!  All the ugliness in my branch is my fault.  Your suggestion on merging the various branches makes sense to me, but I really struggled with hg.  I lost changes multiple times.  That's why I moved the whole thing to git, but that makes clean merges harder.  Again, my fault.  If it ever gets nice enough to merge back into real unified I'll make an hg repo.  And if I get up to say, menus, I'll be able to notice if I broke something trying to merge in other things to my code.  Currently it's all broken, can't tell if I did a bad merge ;)

There's no need to apologise for not having magically created enough hours to do my work for me!  It's nobodies work really, we're all volunteers, mostly because we want cool toys for our own cameras I guess.  Besides, I wouldn't have started working on this if I didn't enjoy these low-level puzzles.  And yes, definitely learning more doing things the hard way, which is fine by me.

I don't have a reliable opinion on the shape of the code yet.  I can't tell which parts are actually ugly, and what is simply my confusion.  Probably mostly the latter!  Generally I'd say make it work first, then make it reliable, then, maybe, pretty.  I've reviewed millions of lines of C and seen much, much worse (in commercial products...).

kitor

  • Contributor
  • Member
  • *****
  • Posts: 119
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #232 on: July 10, 2019, 08:18:40 AM »
kitor: I don't think bmp_putpixel_fast() is called with rgb values at all.  Look at the values when it's called, and the constants in bmp.h, eg, COLOR_WHITE is 1, COLOR_BLACK is 2.  Indexed colour maybe.  I need to understand this better before I can change it.  Probably won't have much time to work on it before the weekend.

You are right! I looked at uint_8t, saw color definitions and just assumed it without looking at the numbers. Lesson learned once again: don't assume things  :)
https://magiclantern.fandom.com/wiki/VRAM/BMP
https://chdk.fandom.com/wiki/Frame_buffers -> "Bitmap (or Overlay)"
https://chdk.fandom.com/wiki/Palette
Indeed that's an indexed palette, i guess that it was more consistent between EOS cameras than on PowerShot models, but this apply:
Code: [Select]
#define COLOR_TRANSPARENT_GRAY  0x14 // not portable, old cameras show it as magenta
Since CHDK is running on D6+ cameras, I checked their code (on dev trunk) and while their functions have a bit different definitions, core/gui_draw.c seems to have all the magic implemented that needs to be ported here.
EOS R

esinem

  • New to the forum
  • *
  • Posts: 2
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #233 on: July 10, 2019, 05:38:52 PM »
As a non-techie user, I assume this geek talk means an 800D/T7i version is on the way? Does anyone have any idea when a download is likely to become available for your average idiot like me? :-)

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 6737
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #234 on: July 10, 2019, 05:47:31 PM »
Answered in
Top of page -> User Guide -> FAQ -> Troll Questions
Photogs and videographers: Assist in proof reading upcoming in-camera help!. Your input is wanted and needed!

totalmichel

  • New to the forum
  • *
  • Posts: 12
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #235 on: July 11, 2019, 01:02:08 AM »
Answered in
Top of page -> User Guide -> FAQ -> Troll Questions

👏👏👏

names_are_hard

  • Contributor
  • New to the forum
  • *****
  • Posts: 43
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #236 on: July 11, 2019, 03:03:58 AM »
@kitor - those CHDK links were very helpful, thank you!  It *used* to be indexed, but now it's not.  That's good, I can ignore the indexed code and simply replace it (ifdef for > Digic 5).

@esinem - I don't know that anyone is working on an 800D port.  It will still be several months, minimum, before I have 200D working.  Each camera needs individual work.  Porting between the various Digic7 cams should be easier, so if 200D gets usable, progress on the others may speed up.  Or I may get bored, or get stuck, and stop and it'll never get done.  Who can say? :)

totalmichel

  • New to the forum
  • *
  • Posts: 12
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #237 on: July 12, 2019, 11:14:25 AM »
@kitor - those CHDK links were very helpful, thank you!  It *used* to be indexed, but now it's not.  That's good, I can ignore the indexed code and simply replace it (ifdef for > Digic 5).

@esinem - I don't know that anyone is working on an 800D port.  It will still be several months, minimum, before I have 200D working.  Each camera needs individual work.  Porting between the various Digic7 cams should be easier, so if 200D gets usable, progress on the others may speed up.  Or I may get bored, or get stuck, and stop and it'll never get done.  Who can say? :)


Dont give up 🙂 (on a more serius tone, good job, im a programer, i know the struggle)
I'm waiting for you to complete the 200D to  *try* adapting it to the 6D2 😝
But im nowhere good enough to figure out the things you're doing.
Maybe it will work, or maybe ill brick my camera, who knows...

names_are_hard

  • Contributor
  • New to the forum
  • *****
  • Posts: 43
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #238 on: July 13, 2019, 03:47:51 AM »
Learning other people's code is hard :)

If you want a probably-low-risk task for porting to 6D2, this commit would be good:
https://bitbucket.org/stephen-e/ml_200d/commits/2fffa3bbda8980fdc3ce1c3fe1a8323ddaee45f4
You would need to find the right values for stubs.S and consts.h (Ghidra, or IDA Pro), if you get enough of them right it will let your cam boot while also loading a small amount of ML code (and let you blink the LED for debugging, search for info_led_blink).  I would recommend getting Alex's digic6-dumper branch working first, if you haven't done that already.

If you want a probably-considered-easy task which is completely safe and will help (force...) you to learn the code, you could get this commit building:
https://bitbucket.org/stephen-e/ml_200d/commits/6ce440ec21564070cdddc7783d50c254aca39093

I'm stuck on that.  You don't need to run any code, only fix the makefiles (or headers?  includes?) so platform/200D.101 will build.  That work is needed to make the bootloader able to display things.  I'm giving up on it for now, it's not very important, the bootloader already finishes.  I'm moving to bmp_putpixel_fast(), if I can get that working I can use output on the display for debugging.

names_are_hard

  • Contributor
  • New to the forum
  • *****
  • Posts: 43
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #239 on: July 14, 2019, 05:51:24 AM »
It's (more) alive! This is quite nice, it proves I don't need to get opentype working to get a usable interface.  Haven't tried to track down the green bars yet, I guess I left some weird debug code in.


nikfreak

  • Developer
  • Hero Member
  • *****
  • Posts: 1125
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #240 on: July 14, 2019, 07:14:56 AM »
congrats! Keep up your good work.
70D.112 & 100D.101

kitor

  • Contributor
  • Member
  • *****
  • Posts: 119
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #241 on: July 14, 2019, 08:10:35 AM »
Now that's what i call progress!
Need to find some time and finally dig into your repository.

Inb4 "so when download be available?" -> answer from a few posts ago apply exactly the same now.
EOS R

names_are_hard

  • Contributor
  • New to the forum
  • *****
  • Posts: 43
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #242 on: July 15, 2019, 05:15:10 AM »
I hope this should mean it's fairly easy to get CONFIG_HELLO_WORLD building on other Digic 7 cams!  Let me know your progress!

I fixed the green bars, it was a stupid ifdef mistake.  Next up, I have found that disabling CONFIG_HELLO_WORLD and enabling CONFIG_EARLY_PORT is broken, it doesn't compile.  Even doing the obvious fix (a required function definition was ifdef'd out) the cam doesn't boot as far.  Don't know what's going on there, haven't investigated.  I have CONFIG_PROP_REQUEST_CHANGE *disabled* so I decided to try without CONFIG_EARLY_PORT, and this is better - obviously doesn't really work but further than a CONFIG_HELLO_WORLD build.  It's currently failing in call_init_funcs().  I can see which function is crashing via debug logs.

VERY IMPORTANT: if you are trying this, ensure CONFIG_PROP_REQUEST_CHANGE is not defined.  This would be controlled in platform/200D.101/internals.h or equivalent.  Trying to write to properties is risky, can brick cam.

In unrelated news I decided to actually use my cam today and got this lucky shot: https://imgur.com/a/zltkJjv
They're skittish critters, I am learning how to sneak up on them.  Tamron 60mm F/2 macro.  Can't post in Share Photos, it's not using ML ;)

OlRivrRat

  • Hero Member
  • *****
  • Posts: 507
Re: DIGIC 7 development (200D/SL2, 800D/T7i, 77D, 6D2)
« Reply #243 on: July 15, 2019, 04:52:19 PM »
                 @ N_A_H

        Nice Shot ~ 1 thing I've noticed about Dragon Flies is that they are very repetitive creatures,

so if @1st They get away just be patient & They soon, most likely, will return ~
ORR~DeanB  ~~  80D-ML  &  SL1+ML  &  5D2+ML  &  5DC+ML  &  70D+ML(AliveAgain)