Author Topic: Canon 40D  (Read 313987 times)

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 7075
Re: Canon 40D
« Reply #500 on: March 22, 2020, 12:40:54 AM »
On the CF card write speed , your saying you only get around 20MB/s ?
Is this on a type 1 or type 2 CF card ?

"Type" = Physical dimension (thickness) of a CF-card. Type II is thicker (5 mm) and was used for rotational media (as in harddisks, see MicroDrive).
Here is some info about CF modes
Quote
Explanations
The CF interface is derived from the ATA Interface. The CF 2.0 specification allowed PIO-4 mode that has a theoretical limit of about 15.9MB/s. CF 3.0 introduced two additional PIO modes that are not part of the original ATA specification: PIO-5 and PIO-6, where PIO-6 has a theoretical limit of about 23.8MB/s. Most CF cards with speed ratings from 80x to 166x use PIO-6 to achieve this speed. However, there is currently no FireWire reader with support for PIO-5 or PIO-6. This means that PIO-6 cards run in the slower PIO-4 mode, and therefore run slower than with current USB 2.0 readers. Support for MDMA modes is most likely not that popular, but you can't really tell the difference between PIO-4 and MDMA-2 when a card reader is used, because PIO only occupies the card reader controller, not the host CPU. CF 3.0 also introduced two additional MDMA modes, MDMA-3 and MDMA-4 using the same timing as PIO-5 and PIO-6.
CF 1.0: Max. PIO-2 (240ns), 7.95MB/s.
CF 2.0: Max. PIO-4 (120ns), 15.89MB/s.
CF 3.0: The new CF PIO modes PIO-5 and PIO-6 and the ATA UDMA modes UDMA-0 to UDMA-4 were introduced, offering a theoretical limit of about 63MB/s for UDMA-4.
CF 4.0: The remaining ATA UDMA modes UDMA-5 and UDMA-6 were added, allowing up to about 127MB/s.
CF 5.0: 48-bit commands were added. This command set allows a transfer block size of up to 32MB compared to 128kB with the previous command set. This will allow actual transfer rates to come closer to the theoretical limit.
CF 6.0: The new CF UDMA mode UDMA-7 (24ns/166M) was added, allowing up to about 158MB/s transfer rate.

In case you have plans on using CF cards as regular ATA device: For all timings of 100ns or faster (UDMA-5, UDMA-6 and UDMA-7), the CompactFlash specification only allows one device per bus with a maximum distance of 0.15m (6in) from the card connector to the host controller, at 3.3V supply voltage! In addition to that, 80-wire cables must be used for all UDMA modes.

Current UDMA CF Cards and UDMA CF readers support UDMA-5 or UDMA-6, with real-world transfer rates of more than 80MB/s. This is already beyond the theoretical transfer rate of UDMA-4. Current USB 2.0 UDMA CF readers are maxed out at about 32MB/s, less than 40% of the actual card performance.
Source: Late and lamented http://www.hjreggel.net/cardspeed/index.html (Current = Years and years ago)
Photogs and videographers: Assist in proof reading upcoming in-camera help!. Your input is wanted and needed!

heder

  • Contributor
  • Freshman
  • *****
  • Posts: 69
  • No time for caution
Re: Canon 40D
« Reply #501 on: March 22, 2020, 10:13:51 AM »
Nice work! How close are you to mjpeg stream? How would that work in practice? mv1080p compressed into mjpeg? Recorded as single images? Curious.

Danne, Atleast the older cameras like 40d,450d,1000d were built with jpeg streaming capabilities, were each images is downsampled, mjpeg compressed and finally tranfered to the pc. We can utilize this in magic lantern and save each mjpeg into a file, which will result in a motion-jpeg stream. A tool like ffmpeg supports this non-standard format. The interessing features is that each frame is a full image.


Great Job @heder , I've been watching your progress with great interest  :D
One of my long term goals in to have Mjpeg compression -> .avi (4.2.2 8bit)
I know the stream is there but was unable to fully understand it to be implemented in d4 cams
this something that All ML cams can use specially the older d4 e.g. 500d etc. ...

On the CF card write speed , your saying you only get around 20MB/s ?
Is this on a type 1 or type 2 CF card ? spec. say it can use either one .
There is a branch that has to do with CF bus clocking or overclocking ,
I've been looking in to this for the 5d2 & 50d .
Some useful link from previous conversation's about CF card speed
https://www.magiclantern.fm/forum/index.php?topic=12862.msg206010#msg206010
https://www.magiclantern.fm/forum/index.php?topic=12862.msg206011#msg206011
https://www.magiclantern.fm/forum/index.php?topic=19336.msg206067#msg206067etc. .....


eddeercity, as far as I can tell the 40d only support pio-6 mode but seems to be underclocked (14 mb/s), I wasted alot of time digging into the firmware code looking for UDMA, without success, I used 4 different new card from sandisk, kingstons, transcend (ranging from 866x down to 166x). I did alot of blackbox testing with the timing and ended with approx 20 mb/s which the pio-6 specification support. These are approx 3mm cards so I guess CF-1. The firmware supports CompactFlash Specification Revision 3.0 (found "CFA3.0" strings in firmware) http://rumkin.com/reference/aquapad/media/cfspc3_0.pdf

I documented some of my finding here https://magiclantern.fandom.com/wiki/Register_Map/40D
---

Theoretically: the sensor was a readout rate of 96Mpixel/sec, thats 4004004 pixels per second at 23.976 or ~9 fps at full sensor size. Both should be doable with the CF limited bandwidth (jpeg compression), but there is along list of issue to be solved. I need to understand the yuv-mjpeg livemode "chain" inorder to beable to change the width and height of the images and many many other issues. The yuv-mjpeg chains seem to be is abit more complex that "raw video", this one really needs alot of cache hijacking.
Embedded SW engineer. Current Cameras: Canon 20d, 40d, 350d

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6273
Re: Canon 40D
« Reply #502 on: March 22, 2020, 12:58:36 PM »
Danne, Atleast the older cameras like 40d,450d,1000d were built with jpeg streaming capabilities, were each images is downsampled, mjpeg compressed and finally tranfered to the pc. We can utilize this in magic lantern and save each mjpeg into a file, which will result in a motion-jpeg stream. A tool like ffmpeg supports this non-standard format. The interessing features is that each frame is a full image.
Sounds like a game changer to me :).

Ant123

  • Contributor
  • Member
  • *****
  • Posts: 126
Re: Canon 40D
« Reply #503 on: March 22, 2020, 07:07:40 PM »
We can utilize this in magic lantern and save each mjpeg into a file, which will result in a motion-jpeg stream.

https://www.magiclantern.fm/forum/index.php?topic=8119.msg212429#msg212429

heder

  • Contributor
  • Freshman
  • *****
  • Posts: 69
  • No time for caution
Re: Canon 40D
« Reply #504 on: March 22, 2020, 07:48:43 PM »
https://www.magiclantern.fm/forum/index.php?topic=8119.msg212429#msg212429
Yes exatly.

The JPEG buffer is is populated *and* freed in lvcdevResourceGet, so... that's pretty much the only place for a hook, if you want to reuse Canon's approach. You could place a hook (patchmgr: patch_hook_function) directly into 0xFFAD5870, but that won't solve much (the hook code will only execute when needed, so it will be a bit cleaner). You should also speed up the copying process with either dma_memcpy (HPCopy) or edmac_memcpy; that should get rid of dropped frames.

On newer models (DIGIC 5, IIRC also 60D and newer DIGIC 4), this function no longer runs automagically while in LiveView, so this approach is only valid for older models.

There is another drawback of this JPEG stream: it's not using the full LiveView resolution. On 450D, CR2 resolution is 4312x2876 (with dcraw), so I'd expect something close to 1436x958 in LiveView. On 5D2, CR2 resolution is 5634x3753, LiveView resolution is 1880x1248, LV-JPEG resolution is 1024x680. I'm pretty sure the hardware can deliver MJPEG at full LiveView resolution; "just" need to figure out how to how to request a full-size YUV buffer (most Canons do this while recording H.264) and how to call the JPEG encoder on an arbitrarily-sized YUV buffer (likely very similar to the lossless encoder).

BTW - disabling this JPEG stream on old models could be useful for reducing power consumption. Nothing measured; just theory.

This is the real challenge
Embedded SW engineer. Current Cameras: Canon 20d, 40d, 350d

heder

  • Contributor
  • Freshman
  • *****
  • Posts: 69
  • No time for caution
Re: Canon 40D
« Reply #505 on: March 24, 2020, 08:46:11 PM »
When running with call("lv_save_raw",1) the raw EDMAC channel has the following sizes

LiveView x1  = raw bayer format = 1336 x 872
LiveView x5  = raw bayer format = 1952 x 814
LiveView x10 = raw bayer format = 1952 x 814
Embedded SW engineer. Current Cameras: Canon 20d, 40d, 350d

Ant123

  • Contributor
  • Member
  • *****
  • Posts: 126
Re: Canon 40D
« Reply #506 on: March 25, 2020, 04:49:50 PM »
Did you manage to get the image using "lv_save_raw" ?

heder

  • Contributor
  • Freshman
  • *****
  • Posts: 69
  • No time for caution
Re: Canon 40D
« Reply #507 on: March 26, 2020, 08:00:09 PM »
No, i didnt borther yet, i just analysed the RAW edmac register values.

Im stuck in gui stuff, menus in liveview isnt working, which is needed for raw video.
Embedded SW engineer. Current Cameras: Canon 20d, 40d, 350d

heder

  • Contributor
  • Freshman
  • *****
  • Posts: 69
  • No time for caution
Re: Canon 40D
« Reply #508 on: March 31, 2020, 11:02:36 AM »
* Menus in livemode is now working  :)

* mlv_lite framework is up and running  :D
  - modules vsync is called from my own 40d vsync task (hmm, should use the commonm vsync func, but ok for now)
  - module version structs "VERS" in output header turns into alot of garbage => disabled for now.
  - small hacks set to 0
  - SRM_job set to 0
  - Maximum memory set to 40MB
  - edmac_copy_rectangle_cbr_start fails for now ...  :(
 
The camera save correctly and mlv_dump accepts the and decodes the output file correctly  8) (if I skips dmac_copy_rectangle_cbr_start and just put in dummy data using memset), so I need to get dmac_copy_rectangle_cbr_start verified.

 
 
Embedded SW engineer. Current Cameras: Canon 20d, 40d, 350d

heder

  • Contributor
  • Freshman
  • *****
  • Posts: 69
  • No time for caution
Re: Canon 40D
« Reply #509 on: April 02, 2020, 03:19:47 PM »
Yay .. finally a nice looking raw recorded frame from mlv_lite ! ..  (without edmac support).

1 fps at 1312x872 (x1 mode) using memcpy(), approx each 20th frame is ok. Had to use memcpy to copy line of line. I need to hack the raw edmac system more before I can use it. My current hijacked vsync function is in the wrong spot, and the 40d contnuesly writes the raw edmac destionation register before performing the raw tranfers, and that pointer changes. mlv_lite overwrites this pointer, but without proper timing the camera just overwrites mlv_lites address again, stupid 40d, surrender.

Embedded SW engineer. Current Cameras: Canon 20d, 40d, 350d

kitor

  • Contributor
  • Member
  • *****
  • Posts: 135
Re: Canon 40D
« Reply #510 on: April 03, 2020, 10:46:33 PM »
MLV on 40D  8) Even if that's single frame, congrats. I'll check it just for fun, after whole covid crap ends (dad has pristine 40D with only ~10k frames).
Wish I had enough knowledge to move R forward, but with time that I can spent, I'll maybe finally upgrade my cam to latest FW and update the stubs...
EOS R

heder

  • Contributor
  • Freshman
  • *****
  • Posts: 69
  • No time for caution
Re: Canon 40D
« Reply #511 on: April 04, 2020, 07:50:57 AM »
Updated open beta to 1.0.2

https://www.magiclantern.fm/forum/index.php?topic=1452.msg224594#msg224594

Fixes =
Menu's are now working in LiveMode (use joystick instead of scrollwheel)

I'll be "out of office" for the next 10 days, corona is driving the kids nuts, easter vaccation  @ home. mlv_lite will take some more time, its abit more complex than I thought.

Embedded SW engineer. Current Cameras: Canon 20d, 40d, 350d