Magic Lantern Forum

Developing Magic Lantern => General Development => Topic started by: dpjpandone on June 21, 2014, 03:14:38 AM

Title: A guide to dumping image buffers / VRAM dumps in all modes
Post by: dpjpandone on June 21, 2014, 03:14:38 AM
Image buffer support has been added to QEMU.

If you are experiencing problems with external monitors, this is important to you!  here is how you can help:

We must dump the image buffers in various display modes on as many cameras as possible.

To dump the image buffer:

1.press trash can to enter magic lantern menu
2.from the "DEBUG" menu, select "Dump image buffers" you will now see a countdown "will dump VRAM in 5 seconds"
3.quickly enter the display mode you wish to dump (for example, if you are trying to dump 5xzoom mode, you must quickly press 5x zoom before the 5 second counter runs out)
4.wait for "Dumping VRAMS.............DONE!)"

Now the dump is stored on your memory card.
You must repeat this process for about 75 possible display states

A logical order for this task:

1.dump live view in 1080p video mode (standby)
2.dump live view in 1080p video mode (recording)
3.dump live view with  5x zoom enabled (standby)
4.dump live view with 10x zoom enabled (standby)
5.dump live view in 1080p 3X crop (600D) (Standby)
6.dump live view in 1080p 3X crop (600D) (Recording)
7.dump live view in 720p video mode (standby)
8.dump live view in 720p video mode (recording)
9.dump live view in 480p video mode (standby)
10.dump live view in 480p video mode (recording)
11.dump live view in 480p crop ( 550D) (Standby)
12.dump live view in 480p crop ( 550D) (Recording)
13.switch back to 1080p (via Canon Q menu) press dump and quickly press play (to review a video)
14.enable raw video recording (via ML menu) press dump and quickly press play (to review a mlv video)
15.disable raw video, switch to photo mode (from mode dial or dedicated switch)
16.dump live view in photo mode
17.press dump then quickly press play (to review a photo)
18.For this last one, Image review must be enabled and set to at least 5 sec. (canon menu) after you press "dump VRAM" you must quickly take a picture, and the dump must be captured during image review (automatically review a photo after taken)

***************************************************************************
19. Plug in HDMI monitor and repeat steps 1-18 (1080i output)
20. Now select "Force HDMI-VGA in ML menu" and repeat steps 1-18 (480p output)
21. Disconnect HDMI monitor, Connect SD monitor (canon RCA cable) and repeat steps 1-18
22. From Canon menu, set your Video Region to the opposite of what it is currently set (if it says NTSC, switch to PAL and vice versa) and repeat 1-18.

Don't forget to set your video mode back to your country's settings when you are done.

Not all cameras support all the listed modes, for example,  5D2 has only two (1080p and 480p), 600D has more.... Try to dump every mode that your camera supports. The goal is to collect a dump of every possible display mode on every supported camera.

After you have collected all these dumps, compress the highest level folder into a zip archive to preserve the file/folder structure and upload it here.

If we can collect dumps for most of the cameras it will be possible to emulate these various display modes for automated testing on the nightly build server. This will allow a developer to fix current issues with external displays, and ensure that future features work with external monitors.

Thanks for your help with this task!
Title: Re: A guide to dumping image buffers
Post by: a1ex on June 21, 2014, 09:47:19 AM
Related discussions on Bitbucket:

https://bitbucket.org/hudson/magic-lantern/commits/32f3a5bd81936f986de904b12a542522cb37102f
https://bitbucket.org/hudson/magic-lantern/issue/1582/crop-bars-for-external-smallhd-oled-7-7#comment-10827408

The video modes you need to cover are listed below:

    char* video_mode =
        PLAY_MODE && is_pure_play_photo_mode()          ? "PLAY-PH"  :      /* Playback, reviewing a picture */
        PLAY_MODE && is_pure_play_movie_mode()          ? "PLAY-MV"  :      /* Playback, reviewing a video */
        PLAY_MODE                                       ? "PLAY-UNK" :
        lv && lv_dispsize==5                            ? "ZOOM-X5"  :      /* Zoom x5 (it's the same in all modes) */
        lv && lv_dispsize==10                           ? "ZOOM-X10" :      /* Zoom x10 (it's the same in all modes) */
        lv && lv_dispsize==1 && !is_native_movie_mode() ? "PH-LV"    :      /* Photo LiveView */
        !is_native_movie_mode() && QR_MODE              ? "PH-QR"    :      /* Photo QuickReview (right after taking a picture) */
        !is_native_movie_mode()                         ? "PH-UNK"   :
        video_mode_resolution == 0 && !video_mode_crop && !RECORDING_H264 ? "MV-1080"  :    /* Movie 1080p, standby */
        video_mode_resolution == 1 && !video_mode_crop && !RECORDING_H264 ? "MV-720"   :    /* Movie 720p, standby */
        video_mode_resolution == 2 && !video_mode_crop && !RECORDING_H264 ? "MV-480"   :    /* Movie 480p, standby */
        video_mode_resolution == 0 &&  video_mode_crop && !RECORDING_H264 ? "MVC-1080" :    /* Movie 1080p crop (3x zoom as with 600D), standby */
        video_mode_resolution == 2 &&  video_mode_crop && !RECORDING_H264 ? "MVC-480"  :    /* Movie 480p crop (as with 550D), standby */
        video_mode_resolution == 0 && !video_mode_crop &&  RECORDING_H264 ? "REC-1080" :    /* Movie 1080p, recording */
        video_mode_resolution == 1 && !video_mode_crop &&  RECORDING_H264 ? "REC-720"  :    /* Movie 720p, recording */
        video_mode_resolution == 2 && !video_mode_crop &&  RECORDING_H264 ? "REC-480"  :    /* Movie 480p, recording */
        video_mode_resolution == 0 &&  video_mode_crop &&  RECORDING_H264 ? "RECC1080" :    /* Movie 1080p crop, recording */
        video_mode_resolution == 2 &&  video_mode_crop &&  RECORDING_H264 ? "RECC-480" :    /* Movie 480p crop, recording */
        "MV-UNK";
   
    char* display_mode =
        !EXT_MONITOR_CONNECTED                          ? "LCD"      :          /* Built-in LCD */
        ext_monitor_hdmi && hdmi_code == 5              ? "HDMI1080" :          /* HDMI 1080p (high resolution) */
        ext_monitor_hdmi && hdmi_code == 2              ? "HDMI480 " :          /* HDMI 480p aka HDMI-VGA (use Force HDMI-VGA from ML menu, Display->Advanced; most cameras drop to this mode while recording); */
        _ext_monitor_rca && pal                         ? "SD-PAL"   :          /* SD monitor (RCA cable), PAL selected in Canon menu */
        _ext_monitor_rca && !pal                        ? "SD-NTSC"  : "UNK";   /* SD monitor (RCA cable), NTSC selected in Canon menu */

    int path_len = snprintf(pattern, sizeof(pattern), "VRAM/%s/%s/%s/", CAMERA_MODEL, video_mode, display_mode);


Therefore, for one camera, you have up to 16 video modes x 5 display types, total 80 modes. For a 5D2 or 50D, which has only 1080p or 640p, you have only 50 video modes.

Ignore the "UNK" modes, these are just for fallback, to make sure the conditions are evaluated correctly.
Title: Re: A guide to dumping image buffers
Post by: stevefal on June 21, 2014, 04:51:12 PM
video_mode_crop (H.264) does not apply to 5D3, correct?  If so I count 12x5 or 60 modes for 5D3.
Title: Re: A guide to dumping image buffers
Post by: a1ex on June 21, 2014, 04:59:23 PM
Right, it doesn't.

Also, would be nice if you could use a tripod. If you don't, aligning the 3 image buffers is going to be hard, since the code does not try to do any kind of temporal sync. If the images are not aligned, the only hint will be the bounding boxes, which may or may not match.

I've just updated the code and the builds to save a log file with the raw variable values; 5D3 123 seems to have some new display modes, besides the ones I already know.
Title: Re: A guide to dumping image buffers
Post by: stevefal on June 21, 2014, 05:35:15 PM
What's required for the SD cases? I never use composite video out. I assume a cable needs to be plugged in, but does it need to actually go somewhere? Or just a 75 ohm load?

[edit] Apparently plugging in the silly SD cable disables the internal LCD, so I guess a composite monitor is required unless the cable is plugged in after triggering the dump. I'll do that for the PAL case at least, since I don't have a PAL monitor.
Title: Re: A guide to dumping image buffers
Post by: Markus on June 21, 2014, 07:44:42 PM
Mirroring modes on 5d3 123?
Title: Re: A guide to dumping image buffers
Post by: a1ex on June 21, 2014, 08:25:44 PM
For SD, I think it's enough to plug the cable right before clicking the screenshot.

I don't know how to identify mirroring modes yet, but the logs saved alongside with the dumps may show some clues (can't try it right now myself). Just mention which picture was taken with which mirroring setting.
Title: Re: A guide to dumping image buffers
Post by: stevefal on June 21, 2014, 10:18:01 PM
Here's a set from 5D3_113. I used three different HDMI monitors, including two with 1280x720 native displays. But all three are coming up as 1080p, so I don't know how to generate the 480p HDMI case. It goes pretty fast once you get the hang of it:

http://popspring.com/mldrop/VRAM_stevefal_5D3_113.zip

[edit] Ah, I missed the instruction about 480p. Will try.
Title: Re: A guide to dumping image buffers
Post by: stevefal on June 22, 2014, 01:19:13 AM
I tried forcing 480p and had some issues:

- I noticed that playing picture/video looked very different on my SmallHD in photo versus movie modes. So I captured both (photo first).
- I captured movie standby and record for all three movie formats, but only folders were created, no dump files.

http://popspring.com/mldrop/VRAM_stevefal_5D3_113_480p.zip
Title: Re: A guide to dumping image buffers
Post by: dpjpandone on June 22, 2014, 03:13:25 AM
Steve, I updated the steps, they should be more clear now.

it doesn't matter which hdmi monitor you use, just as long as you do one pass with 1080hdmi, one with 480hdmi, and one with SD monitor (rca cables)
Title: Re: A guide to dumping image buffers
Post by: LRF on June 22, 2014, 10:17:53 PM
Another set of VRAM dumps, this time without mirroring. I think I might have skipped few modes, because didn't have the list with me and was doing other things at the same time. So if you want me to repeat something, just let me know.
Added zoom modes on external monitor, as a1eks wanted (1.4x, 5x, 10x, Panavision).

https://www.dropbox.com/s/ee4vq9ybaq7isdq/VRAM3.zip
Title: Re: A guide to dumping image buffers
Post by: Akos on June 23, 2014, 05:19:56 AM

ALex,


Here the dumps from my Mark3, with 113 firmware, ML from today, 22 Dumps with monitor , 22 without, following from lists from Joahanan:


wetransfer link :
http://we.tl/MBFWrIYTF8
Title: Re: A guide to dumping image buffers
Post by: Akos on June 23, 2014, 05:42:03 AM
..
Title: Re: A guide to dumping image buffers
Post by: a1ex on June 23, 2014, 07:34:46 AM
Thanks, so now I have the 5D3 pretty well covered, in pretty much all modes.

A screenshot from the emulator, from last night:
(http://a1ex.magiclantern.fm/bleeding-edge/qemu-lv-2.png)

(it's not doing anything useful yet, but it shows the emulation goes far enough to load these dumps, display them, and check the alignment of ML overlays)

Here's one from today (loaded a HDMI dump from 5D3 1.1.3):
(http://a1ex.magiclantern.fm/bleeding-edge/qemu-hdmi.png)

Does it match what you see on the monitor?

Other cameras too, please! Even if you don't have an external monitor, you can still do the built-in LCD dumps. If you have the A/V cable, you can do the SD monitor tests just with that (no need to actually use a monitor, just plug the cable).
Title: Re: A guide to dumping image buffers
Post by: LRF on June 23, 2014, 12:43:49 PM
This is how it looks in my case:
No mirroring.

Screen layout 16:10 HDMI, t/b with external monitor set to Underscan
https://www.dropbox.com/s/bo6nz6ub5lafb8d/HDMI_under.jpg


16:10 HDMI, t/b with external monitor set to Overscan
https://www.dropbox.com/s/62k8z7b3hercwg3/HDMI_over.jpg
Title: Re: A guide to dumping image buffers
Post by: Akos on June 23, 2014, 09:27:04 PM
Thanks Alex,



here is 5 short iphone clips i made ,you would be able to see different variations here clearly:


https://www.icloud.com/photostream/#A4Gg6pDrGZTwaq (https://www.icloud.com/photostream/#A4Gg6pDrGZTwaq)

Here's one from today (loaded a HDMI dump from 5D3 1.1.3):
(http://a1ex.magiclantern.fm/bleeding-edge/qemu-hdmi.png)
Does it match what you see on the monitor?


Other cameras too, please! Even if you don't have an external monitor, you can still do the built-in LCD dumps. If you have the A/V cable, you can do the SD monitor tests just with that (no need to actually use a monitor, just plug the cable).
Title: Re: A guide to dumping image buffers
Post by: dpjpandone on June 24, 2014, 04:41:19 AM
Alex, here are my dumps for the 7D:

http://www.filedropper.com/vram

these are LCD, HDMI 1080, and HDMI 480. I am borrowing an SD monitor that does NTSC and PAL on Wednesday. I noticed that the dumps don't necessarily get sorted to the right folders (all hdmi folders are empty) but I think they're all there. Please let me know if these worked.
Title: Re: A guide to dumping image buffers
Post by: a1ex on June 24, 2014, 07:55:28 AM
Looks like I need to try this one myself; vyskocil also reported this bug on 7D. Might be a bug on the file I/O backend or some other quirk, not sure yet.

edit: FIO_CreateFile fails when called with path names longer than 33 characters (why?)
Title: Re: A guide to dumping image buffers
Post by: dpjpandone on June 24, 2014, 04:01:24 PM
Quote from: a1ex on June 24, 2014, 07:55:28 AM
FIO_CreateFile fails when called with path names longer than 33 characters (why?)

   because it's not a 5D3... I'll do 600D and 550D next, which one would you like first?
Title: Re: A guide to dumping image buffers
Post by: a1ex on June 24, 2014, 04:06:26 PM
600D, because I know for sure it has problems with external monitors.
Title: Re: A guide to dumping image buffers
Post by: dpjpandone on June 25, 2014, 12:18:43 PM
600d crashed every time I tried to capture a dump while recording video, regardless of video mode. These are reproducible, battery must be removed.
Title: Re: A guide to dumping image buffers
Post by: a1ex on June 25, 2014, 03:28:49 PM
No problems here on 60D while recording. Any crash logs, or a video showing how exactly it crashes?

Does it help if you use a very low bitrate?
Title: Re: A guide to dumping image buffers
Post by: dpjpandone on June 25, 2014, 04:42:42 PM
I am uploading the VRAM zip as we speak, none of the record dumps were captured..... so it didn't record the video or the dump. Since you mentioned bitrate, I'm going to try with a faster card first. Is it ok if I upload a seperate VRAM.zip with only the record modes and maybe one plain MV live view for each type of display? Or should I go through the whole procedure again? 
Title: Re: A guide to dumping image buffers
Post by: a1ex on June 25, 2014, 04:47:01 PM
You can upload just the new modes, it's easy to merge them.
Title: Re: A guide to dumping image buffers
Post by: dpjpandone on June 25, 2014, 05:35:37 PM
OK, here is 600D LCD, HDMI 1080, and HDMI 480, Ill upload the record modes later today:

http://www.filedropper.com/vram_1
Title: Re: A guide to dumping image buffers
Post by: a1ex on June 25, 2014, 07:58:05 PM
Reproduced the crash on 550D. FPS override helped (no problems at 10 fps).

It looks like this process interferes somehow with LiveView timings, not sure yet.

edit: not quite... the files are not saved, looks like it has the same path length limitation...
Title: Re: A guide to dumping image buffers
Post by: Akos on June 26, 2014, 02:08:31 AM
...
Title: Re: A guide to dumping image buffers
Post by: dpjpandone on June 26, 2014, 05:45:01 AM
Alex,

I noticed in the commits you made some fixes for cameras that do not allow long filenames, so I tried to get the Record dumps for the 600D again, but continued to crash in the same manner, this time I tried with fps override set to 10fps, same behavior, and neither the movie, nor the VRAM dumps were saved to the card. I made a video to show you this behavior, you can see it on vimeo: https://vimeo.com/99203532
Title: Re: A guide to dumping image buffers
Post by: a1ex on June 26, 2014, 07:32:19 AM
Does the crash happen before the beep, or after? (couldn't tell from the video)
Title: Re: A guide to dumping image buffers
Post by: dpjpandone on June 26, 2014, 02:47:45 PM
After the beep.

Are the record mode dumps/filenames working on 7D now? Should I do those again today? I really want to give you a COMPLETE set of VRAM dumps (with the same scene) I got the SD monitor now too....


Title: Re: A guide to dumping image buffers
Post by: a1ex on June 26, 2014, 02:53:15 PM
I guess it should be working fine on 7D.

On 600D, does it help if you disable the beep?
Title: Re: A guide to dumping image buffers
Post by: dpjpandone on June 26, 2014, 04:17:00 PM
NO, no difference with beep disabled
Title: Re: A guide to dumping image buffers
Post by: dpjpandone on June 27, 2014, 04:03:41 AM
Here is 7D:  LCD, HDMI-1080, HDMI-480, SD-NTSC  (I couldn't get the SD monitor to work in PAL mode, maybe vykoscil can dump SD-PAL and we can have a complete set for the 7D.

http://www.filedropper.com/7d

Please let me know if I missed anything.
Title: Re: A guide to dumping image buffers
Post by: a1ex on June 27, 2014, 07:34:54 AM
Looks fine, thanks.

For 600D, you may also try the lowest FPS you can get. If the writing process takes place between two frames, I think it should work. Try a few times to increase your chances of success.

I could also try to write the files in very small chunks (it will be slower, but might help).
Title: Re: A guide to dumping image buffers
Post by: Akos on June 27, 2014, 07:44:17 AM

Alex,


wouldnt it help to post a twitter out on asking people with these dumps may be?





Quote from: a1ex on June 27, 2014, 07:34:54 AM
Looks fine, thanks.

For 600D, you may also try the lowest FPS you can get. If the writing process takes place between two frames, I think it should work. Try a few times to increase your chances of success.

I could also try to write the files in very small chunks (it will be slower, but might help).
Title: Re: A guide to dumping image buffers
Post by: dpjpandone on June 27, 2014, 06:50:20 PM
Alex, I will try what you suggest with the 600D tonight.

Akos, Thanks for providing dumps for your camera! Do you have access to any other EOS cameras?

Do you guys think it's a good idea to make a spreadsheet so Alex can check off each camera when he has sufficient dumps for a particular model? I think it could have camera models vertically and the 5 display modes horizontally.
Title: Re: A guide to dumping image buffers
Post by: Akos on June 27, 2014, 09:38:52 PM

i only got a mark3... i am not even sure if i made any mistakes, for the dumps happened (or didn't) when my monitor went black through some of the dump-processes sometimes for more than 7seconds.
it would be a shame to waste alex's time if my dumps were partially incorrect....but i have no way of knowing if all or some were fine.


i think twittering this cropbar fixing project on autoexec ML twitter account could get people doing dumps which follow ML more intensively  and those followers might have more cameras to do them with.


seeing the crop bars during recording is one thing, but playing back a video and missing then the cropbars, will that be also looked at here at this process to fix the crop bars?





Quote from: dpjpandone on June 27, 2014, 06:50:20 PM
Alex, I will try what you suggest with the 600D tonight.

Akos, Thanks for providing dumps for your camera! Do you have access to any other EOS cameras?

Do you guys think it's a good idea to make a spreadsheet so Alex can check off each camera when he has sufficient dumps for a particular model? I think it could have camera models vertically and the 5 display modes horizontally.
Title: Re: A guide to dumping image buffers
Post by: dpjpandone on June 27, 2014, 10:52:50 PM
Yes, this is why we are also dumping the VRAM in playback mode.
Title: Re: A guide to dumping image buffers
Post by: dpjpandone on June 27, 2014, 11:00:17 PM
Alex,

I tried several different bitrates and framerates on 600D, I was able to get most of the record mode dumps. I still had some crashes in HDMI 1080, but I think I managed to get it. Here they are:

http://www.filedropper.com/600d

Title: Re: A guide to dumping image buffers
Post by: Akos on June 29, 2014, 03:55:51 AM
ALex,


anything else i can do to help you?


BR


Akos








Quote from: a1ex on June 23, 2014, 07:34:46 AM
Thanks, so now I have the 5D3 pretty well covered, in pretty much all modes.

A screenshot from the emulator, from last night:
(http://a1ex.magiclantern.fm/bleeding-edge/qemu-lv-2.png)

(it's not doing anything useful yet, but it shows the emulation goes far enough to load these dumps, display them, and check the alignment of ML overlays)

Here's one from today (loaded a HDMI dump from 5D3 1.1.3):
(http://a1ex.magiclantern.fm/bleeding-edge/qemu-hdmi.png)

Does it match what you see on the monitor?

Other cameras too, please! Even if you don't have an external monitor, you can still do the built-in LCD dumps. If you have the A/V cable, you can do the SD monitor tests just with that (no need to actually use a monitor, just plug the cable).
Title: Re: A guide to dumping image buffers
Post by: Akos on June 29, 2014, 11:47:00 PM

Here the Dumps with forced VGA here through wetransfer :

Available until
6 July, 2014
Download link :

http://we.tl/LoprcU0RZg (http://we.tl/LoprcU0RZg)

[size=78%]i don't have  a SD monitor, nor will i ever buy one,.... The Canon mark3 LCD internal screen has more resolution than an external SD Monitor.[/size]
then  a Lupe is what anyone would use instead of staring at a bigger screen and really low and really bad out of focus resolution.
[/size][size=78%]i threw away all those RCA cables as well,   its 80's technology.. no need anymore of this low res stuff. [/size]
Title: Re: A guide to dumping image buffers
Post by: Akos on June 29, 2014, 11:54:19 PM

Crashes :


i had several blank screens (do you call that a crash?) when it was taking the dumps, usually its when raw is turned on and in raw recording or on raw playing back.
when i checked the dumps, exactly those screen DNG's from those so crucial dumps in Raw mode were corrupt.
Alex does it matter ?
when "not" in raw mode dumps worked fine.


we all want to shoot video in raw, without raw, ML is not needed on a mark3.


i am not sure, but if that dump would happen not 5 seconds after, but 10 seconds after, may it would work better ?
it seemed the Monitor screen needed 6-9 seconds to get turned on and into preview mode....



Quote from: a1ex on June 26, 2014, 07:32:19 AM
Does the crash happen before the beep, or after? (couldn't tell from the video)
Title: Re: A guide to dumping image buffers
Post by: dpjpandone on June 30, 2014, 02:49:34 AM
I had to lower the framerate on the 600D to dump certain modes.




Alex, I got an interesting message on the 600D that may be responsible for my problems when dumping in record mode!

When an SD monitor is connected,  it says "600D does not support play and monitor at the same time" after almost every dump. I think this message is making it crash when recording 1080p




Here are SD-PAL dumps for 7D:  http://www.filedropper.com/7d_1

if you merge them with these: http://www.filedropper.com/7d

then we should have a COMPLETE set of dumps for the 7D! I'll upload SD-NTSC and SD-PAL for 600D shortly.




Here are SD-NTSC and SD-PAL dumps for 600D: http://www.filedropper.com/600d_1   
Alex,

It seems that some of the PAL dumps were not sorted properly, they ended up in same folder as NTSC, don't know why (they dumped into these folders, I zipped them exactly the same)

if you merge with this: http://www.filedropper.com/vram_1

and this: http://www.filedropper.com/600d

we should now also have a COMPLETE set for 600D

Which one would you like next, 550D? or 5DMKII?
Title: Re: A guide to dumping image buffers
Post by: Akos on July 04, 2014, 10:04:04 PM

so many days of silence, has the Effort not lead to any results?
Does this need more help from others?
is it being stopped now this attempt to fix this?


:o :(
Title: Re: A guide to dumping image buffers
Post by: dpjpandone on July 05, 2014, 08:16:22 AM
So far 7D, 600D, and 5D3 are the only cameras that have had dumps collected, so the devs don't have much to work with.  The 5D3 still needs dumps of the composite video modes, can you do those so we'll have another complete set?

I'm sure the issue report you started on bitbucket will be updated when the fix is implemented. In the meantime, I have found a totally workable solution using the "force VGA" option in the display menu. All the overlays align perfectly and you can even playback raw videos with mlvplay with this setting enabled.

I have been happily shooting a new feature I'm working on with this setting, and I really enjoyed working the steadicam and the jib crane with a proper monitoring solution this week!
Title: Re: A guide to dumping image buffers
Post by: ItsMeLenny on July 05, 2014, 11:01:40 AM
Quote from: dpjpandone on July 05, 2014, 08:16:22 AM
So far 7D, 600D, and 5D3 are the only cameras that have had dumps collected

Does a1ex still have a 550D, or will this camera need to be done by somebody else?
Title: Re: A guide to dumping image buffers
Post by: Akos on July 05, 2014, 07:11:23 PM
..
Title: Re: A guide to dumping image buffers
Post by: pompeiisneaks on July 05, 2014, 08:19:23 PM
So I did the dump here and have a dropbox link to the zip file. 

Here's the file on dropbox:

https://www.dropbox.com/s/oxb9f724utgpojd/MLVRAMDUMP5D3.zip

If you can let me know so I can pull it from dropbox that would be good, as its rather big, 1.7gb.  I've Done the testing, a few thigns to note.

1. This is for my 5D3 on 1.2.3 using the 1.2.3 ML from nightly from 07032014.
2. I realized that my external monitor (A marshall 7") supports 24p and 60i, so I did both, but I'm not sure if the dumps support it, so if they 'overwrote' or something, the second set was 24p, so that would be w hat you got.
3. I did the external cable, but I'm really n ot sure how well it worked, because I didn't have easy access to a monitor that supported it, so I just got ready, plugged the cable in (now blind) and did the actions I'd done many times before.  I hope it worked.  I did this for my native NTSC and then went to PAL.

I also didn't do any MLV tests because I don't have any 1080 video, my cards are too slow and I can't get more than 1 frame at 1080 so I just skipped testing MLV playback.  I think you've probably got a lot of that anyway.  If you need me to test MLV, can you get me an MLV file thats at least 10-15 seconds to allow me to 'dump then hit play' on the differing modes?

Let me know if you have any questions.

BTW that dropbox archive is still syncing, so give it at least 30 mins from my post (Estimated time left now, could go up).

~Phil
Title: Re: A guide to dumping image buffers
Post by: dpjpandone on July 06, 2014, 02:22:48 AM
Quote from: ItsMeLenny on July 05, 2014, 11:01:40 AM
Does a1ex still have a 550D, or will this camera need to be done by somebody else?
He has 550d and so do I, but its a time consuming process so if you have a 550d please do some dumps if u have time.
Title: Re: A guide to dumping image buffers
Post by: dpjpandone on July 06, 2014, 02:44:58 AM
Quote from: Akos on July 05, 2014, 07:11:23 PM


force into VGA mode is completely useless unless you want to film for your family album, and you want to shoot at f16.
its not possible to focus correctly for the resolution is below the minimum needed to focus.The massive pixelation ,makes all being out of focus.
The MARK3 internal LCD monitor has more res than forcing VGA ,and that internal LCD  still is barely enough to correctly focus, and using a lupe on a brushless gimbal or any sort of gimbal is not an option,... as we all know.
so sorry to say, its not a workaround at all, because its not a serious option at all.
composite is the very same thing.
so why would alex need those out of date dumps for? will they fix the issue with HDMI cropbar placements?....i kind of doubt that... for they have nothing to do with each other ..
correct?

This is a totally workable solution. Did you ever shoot on the 5D markii before ML Raw? Many serious productions were done on this camera in h264 and this is exactly the resolution it output over the hdmi port while recording. It is the same exact resolution as the back LCD, (480p) or 720x480, and since magic zoom works in 480p mode (unlike 1080i)so focus is no problem. Your small HD monitor should have a 5Dii mode that will scale the image so that the viewfinder fills the whole monitor, but I prefer to use it in 4:3 mode with the full histogram below. I shot for several hours using this setup on a steadicam again today and I am soooo happy that I have an affordable and reliable solution for raw. You shouldn't be discouraged, you can do wonderful and amazing things with the camera RIGHT NOW, TODAY! and it will only get better in the future, so don't let the monitor resolution prevent you from getting inspired and being creative with the tools you have at your disposal.
Title: Re: A guide to dumping image buffers
Post by: 5DanielMIII on July 06, 2014, 04:35:05 AM
Quote from: dpjpandone on July 06, 2014, 02:44:58 AM
you can do wonderful and amazing things with the camera RIGHT NOW, TODAY! and it will only get better in the future, so don't let the monitor resolution prevent you from getting inspired and being creative with the tools you have at your disposal.

Amen

;)
Title: Re: A guide to dumping image buffers
Post by: Akos on July 06, 2014, 06:33:45 PM

Alex, will you be able to fix the issue for the HDMI signal and the MArk3 "without" having to wait for lower res and older canons dumps?
its been a year now since i started this thread on bitbucket , and it seems over 2000 people have read this thread here , which was only started a month ago.
i think this shows people are watching this issue very closely... and are interested in it ... right?

Quote from: a1ex on June 23, 2014, 07:34:46 AM
Thanks, so now I have the 5D3 pretty well covered, in pretty much all modes.
Title: Re: A guide to dumping image buffers
Post by: Audionut on July 06, 2014, 06:48:32 PM
Do you really have to duplicate post here and at bitbucket!

Quote from: Akos on July 06, 2014, 06:33:45 PM
Alex, will you be able to fix the issue for the HDMI signal and the MArk3 "without" having to wait for lower res and older canons dumps?

a1ex develops for Magic Lantern.  This means supporting all cameras where possible.  This means, you will probably have to wait for the other dumps, before a clean and robust codebase is pushed to the nightly builds.


Quote from: Akos on July 06, 2014, 06:33:45 PM
its been a year now since i started this thread on bitbucket , and it seems over 2000 people have read this thread here , which was only started a month ago.
i think this shows people are watching this issue very closely... and are interested in it ... right?

If so many people were interested in it, you would think all of the data the developer has asked for to fix the issue, would have already been delivered........right?
Title: Re: A guide to dumping image buffers
Post by: dsManning on July 06, 2014, 09:57:22 PM
Got the 50D with everything but the composite cables.  I'll try to track them down later today (I'm sure they are at the bottom of a bag somewhere).

https://app.box.com/50D (https://app.box.com/50D)
Title: Re: A guide to dumping image buffers
Post by: Akos on July 07, 2014, 01:05:25 AM

Thanks for the answers  ;)

between the time i mentioned that 2000 already read this and right now, only a few hours have passed by, and look for yourself the number is at about 2200, ...thas 100 people have read this just in the last few hours.... so yes, i would consider this thread and this issue of huge interest to users, even if most might not understand how this here all works.
(i am definitely also one of them ,who still doesn't quite get it, how this all here ends up with results so far :-\


Title: Re: A guide to dumping image buffers
Post by: Levas on July 18, 2014, 01:29:57 PM
Here are the image buffer dumps for the canon 6d PAL mode and NTSC mode (see link below for the data on my google drive).
Will do the image buffer dumps with SD monitor attached later on.
Won't be able to do the image buffer dumps with HDMI monitor attached (Don't have the right cable to connect it to the HDMI on my TV)

https://drive.google.com/folderview?id=0B1BxGc3dfMDaUjlnaENVZ2drYUU&usp=sharing

https://drive.google.com/folderview?id=0B1BxGc3dfMDaME1LNHlfSW0tQTg&usp=sharing
Title: Re: A guide to dumping image buffers
Post by: a1ex on July 18, 2014, 02:07:13 PM
Thanks for the dumps.

@Levas, can you double-check the link?
Title: Re: A guide to dumping image buffers
Post by: nikfreak on July 18, 2014, 02:46:00 PM
Thanx @Levas for posting this.
Link works here.
Title: Re: A guide to dumping image buffers
Post by: a1ex on July 18, 2014, 02:53:53 PM
It works, but the content is not the expected one.
Title: Re: A guide to dumping image buffers
Post by: nikfreak on July 18, 2014, 03:01:22 PM
Yes, just came beware of it. It's the same as he posted in rawdiag here:
http://www.magiclantern.fm/forum/index.php?topic=10111.msg120656#msg120656 (http://www.magiclantern.fm/forum/index.php?topic=10111.msg120656#msg120656)
Title: Re: A guide to dumping image buffers
Post by: Levas on July 18, 2014, 03:26:31 PM
Posted the wrong link, updated my earlier post  :D

https://drive.google.com/folderview?id=0B1BxGc3dfMDaME1LNHlfSW0tQTg&usp=sharing
Title: Re: A guide to dumping image buffers
Post by: papkee on February 04, 2015, 07:29:02 PM
Not much activity here, but I've got a decent number of combinations for the 650D, which also has issues with external monitors.

https://drive.google.com/file/d/0B0znJE7xmPa8SVpyME1yNTlQSG8/view?usp=sharing (https://drive.google.com/file/d/0B0znJE7xmPa8SVpyME1yNTlQSG8/view?usp=sharing)
Title: Re: A guide to dumping image buffers / VRAM dumps in all modes
Post by: David_Hugh on March 18, 2018, 07:31:53 PM
So... I did the Test for the 70D, although I dont have an HDMI Monitor and no AV cable either... I plugged in the USB cable, in the hope that it would still make a difference if there's something connected to that outlet but it doesnt seem like it. Anyway, I still uploaded the files because maybe NTSC/PAL is still valuable.

https://we.tl/pNOzF20KbY
Title: Re: A guide to dumping image buffers / VRAM dumps in all modes
Post by: a1ex on March 18, 2018, 07:36:45 PM
Thanks (actually thanks to all contributors) - even without external monitor, the dumps are still very useful.

Missing data:
- 700D
- 100D
- 60D
- 5D2
- 500D
- 550D
- 1200D
- M2 (no rush)

Since the old links are no longer available, I'm going to re-host these files after organizing them. Will also tell which samples are missing external monitor dumps, but I'm not there yet with the emulation.

P.S. Sneak preview of what I'm working on (using one of these dumps for initial tests):

(https://www.magiclantern.fm/nav/5D3/lv.png)

(that's a screenshot from the emulator)




BTW, doesn't the 70D have a 3x crop mode from Canon menu? Couldn't locate the VRAM dumps from this mode (these are labeled as MVC-1080 / RECC1080).
Title: Re: A guide to dumping image buffers / VRAM dumps in all modes
Post by: David_Hugh on March 18, 2018, 08:10:09 PM
Right on the money as always. I thought the zoom selected from the canon menu was nothign "special" on the 70D (I knew it was on the 600D)
Here's the 3x Zoom for 70D
https://we.tl/WtVN0366ts
Title: Re: A guide to dumping image buffers / VRAM dumps in all modes
Post by: dfort on March 27, 2018, 03:10:31 AM
Quote from: a1ex on March 18, 2018, 07:36:45 PM
Missing data:
- 700D

Here we go:

https://www.dropbox.com/sh/bcv0r3n7ffg148t/AAAzj-iGeQv4QT5cx3QomCL0a?dl=0

Some of these were tricky to get and the buffers might have been dumped while the camera was still switching modes. That might be where the "PH-UNK" and "PLAY-UNK" come from. The NTSC and PAL were done blind because I don't have a standard definition monitor and cables to check what was going on. Also, I'm sure I tried HDMI-480 on every mode but the image buffers were only saved while recording a movie. Some I tired several times so that's why there are some extra files in there.

Let me know if there's anything missing and I'll give it another try.

BTW--first thing I did was to white balance and turned out I was way off. Well, at least the whole test is consistently off, 2500K, Magenta 9.

I'll also try to get what I can out of the EOSM2 and post it in the same dropbox folder.

[EDIT] Looks like some of the files have a size of 0 so they can't be uploaded to Dropbox. I'll also upload a zipped up archive to preserve these empty files.
Title: Re: A guide to dumping image buffers / VRAM dumps in all modes
Post by: rafaelbf on April 01, 2018, 05:59:31 PM
Quote from: a1ex on March 18, 2018, 07:36:45 PM

Missing data:
- 500D


Here is:
https://drive.google.com/open?id=1xXMdcUwSyhqKHGIaNTvJUw_1bS3TsQyp


Nightly.2017Dec07.500D111