Raw Video framing is off when using 60D's 'Movie Crop Mode'. Help!

[xiga]

Hi guys

I've encountered something strange

[xiga]

Arghhh, that wasn't supposed to post already. I was trying to find out how to attach files (sigh)

Anyways, the issue i've encountered is that when i take a Raw Video on my 60D and compare it to a normal .MOV file taken from exactly the same position without moving the camera, the framing is different between the two. The Raw video seems to capture a little lower and to the left compared to the .MOV file.
I intend on using the camera to take some videos of the moon in Movie Crop Mode (to get 1:1 pixel resolution) and to then do a mosaic to stitch it all together. For this purpose all i need is for the camera to capture 45 or secs in 640x480 ML mode at 50fps and it's able to do this just fine, so all is good in this respect.
I am using the latest nightly build and am using MVLMystic to extract the images and they look fine when doing so, it's just the framing that seems to be the issue.

The reason this is a problem, is that i need to use the camera's screen to visualise exactly what i'm capturing, so that i can move on to capturing the next frame. I've tested it, and the in-built camera .MOV video matches exactly with what is shown on the camera screen, but the Raw video does not, so it makes it much more difficult for me to know what exaclt i've captured and where i need to go next.

Can anyone help me with this please?

ps - I have 2 still images from the .MOV and .DNG (raw) videos i'd like to upload, but i'm not sure how. I feel like a total noob here, can anyone explain how i can upload these please?

Cheers!

[dfort]

First of all did you know that you can edit your own posts? There should be an icon at the top right and bottom right of each of your posts, the one on the top says "Modify." You can also Preview your posts before posting them, useful when adding images and videos.

To post pictures and videos you need to use sites like flickr.com and YouTube.com or Vimeo.com. copy and paste the url of the picture/video. Note that for YouTube you need to change the https to http and for flickr you just need what's between the [img] tags, including the tags. To share larger files like DNG, MLV, RAW, use a service like dropbox.com or Google Drive and post the sharing link.

As far as your dilema with the centering, it looks like the 60D doesn't have the CROP_MODE_HACK so you need to zoom in with the magnifying glass button. Chances are that your zoom box isn't exactly centered. You should find out how to center it. On some cameras the trash button does it, on others it is the set button. I'm not sure what it is on the 60D. However, even with the zoom box centered there's a chance that it isn't quite the same in raw as it is with H.264. The best way to get precise framing is to shoot tests and add your own crop marks. You can do this with Magic Lantern but I prefer the lo-tech method of marking the screen with thin striping tape or even a grease pencil. To get the raw image on your screen load the mlv_play module to playback in camera. It won't play real time but it is good enough to check your framing.

Hope this helps.

[xiga]

First off, thank you very much dfort, that was very helpful! :-)

Here are the two files in question, .MOV first, and Raw second:

https://onedrive.live.com/redir?resid=C14ECDF7700B5618!2472&authkey=!ADBztVRh_vg6pE4&v=3&ithint=photo%2cjpg

https://onedrive.live.com/redir?resid=C14ECDF7700B5618!2471&authkey=!AF0w6LTlMty8UzU&v=3&ithint=photo%2cjpg

(i tried using the Insert Image and Hyperlink buttons, and inserting the links between and hitting Preview, but i still couldn't see the images in the Preview, so hopefully the lins above work ok).

So from what you are saying, it sounds like i need to centre my zoom box. I don't mind if it's not 100% accurate, just as long as it's close enough.
Can you elaborate please on how to go about editing the zoom box setting (permanently) so that it matches up with the H.264 files? The trash can button only brings up the ML menu, and the Set button doesn't seem to do anything while waiting to capture.
Surely there has to be a better way than me having to mark the screen in two corners where the capture actually starts (and consequently have to guess where it ends in the other two). Is this not a common thing others have had to deal with? And what is this CROP_MODE_HACK you mention, does it not exist for the 60D?

ps - sorry for all the questions!

[Walter Schulz]

CROP_MODE_HACK enables some cams to use Canon's H.264 crop mode as implemented in 600D. For those cams you will find an additional menu item "Crop mode" in Movie tab: 650D/700D/EOS M.

[dmilligan]

Have you tried recording raw using mv640crop (640 crop mode setting in the Canon menu)?

[dfort]

I made an animated gif of your images to illustrate the issue.



So these were done in by zooming in and recording an MLV and a H.264. Assuming that you didn't touch anything that might have moved the crop area?

To move the zoom area on the 60D look at page 167 of the manual. The Trash button centers the magnifying frame which is of course the same button that brings up the Magic Lantern menu. This is the same as on the 100D and I have ask the developer for a feature request to make it behave like the EOSM where a quick press centers the frame and a longer press brings up the ML menu. In any case, the work around for this is to remove the card, turn on the camera and press the Trash button to center the magnifying frame. It should remain in that position when you power up with the card. This isn't permanent because pressing the select wheel will move the magnifying frame.

[xiga]

Thank you dfort, i was about to upload the images to Flickr to make them easier to view, but your animated Gif is much better! 

Both of these were taken using the 640x480 Movie Crop Mode in the Canon menu. This is the only mode i am interested in, as it gives me an effective Crop Factor of over x7 and perfect 1:1 pixel resolution, which means i can get in real close for the planets and the moon without losing any detail (as would be the case if i chose any other video mode in the Canon menu). I know not a lot of camera models have this mode, and probably very few people here on the forums actually use it for raw video, so i admit i may be in a minority here, which probably doesn't help me!
For the raw one, i simply chose 640x480 as my resolution in the ML menu, so as to not mess with the pixel scale, however as you can see the Raw video doesn't seem to be centred at the same spot as the Canon video when using the Movie Crop Mode.
I don't have a tripod so when i took these i just made sure i did not move the camera at all between shots. I took another couple of shots today, and i think i did a better job of not moving the camera. When i compared the 2 in Photoshop, it appears that the Raw capture is about 5 pixels lower and 70 pixels to the left.

ps - I now don't think this has anything to do with centering the zoom box. I started the camera without the SD card and checked where the zoom box was and it seemed to be dead centre, as pressing the Trash Can button didn't make it move. Also, when using Movie Crop Mode you can't actually zoom in using the magnifying glass, the camera gives you a message saying it's not possible in this mode, so i don't think the zoom box is the answer. Is it possible it could just be down to the ML code not allowing for the exact sensor position when using Movie Crop Mode?

[dmilligan]

If you try what I said, you're problem will go away.

[xiga]

Hi dmilligan

Perhaps I wasn't being clear. I am already using the 640x480 Crop Mode in the Canon Menu (it's the one at the bottom). For the purpose of what I'm trying to do, this is the only mode I need.
The problem is that when using this mode the Raw video framing doesn't match the H.264 version.

[dmilligan]

Ok, dfort's comments made me think you were recording "crop zoom", but you are not, so his suggestions are irrelevant, there is no zoom box to move around, you're not in "crop zoom" (mv1080crop) mode.

The actual raw buffer is slightly larger than than the area recorded by Canon's H.264. ML probably just records a slightly different portion of this area than does the Canon code (ML probably just centers the crop, or uses some heuristic that matches Canon in other more common video modes, and Canon may have it somewhere else, off center). I doubt a lot of people use this video mode for raw video, esp. considering most cameras don't even have this mode, so it' probably just never been noticed before.

You have a couple of options:
Just record the full raw buffer with ML, it's slightly larger than 640x480, just pick the largest resolution in the ML menu that ML will allow. You'll at least get everything you seen on the screen (plus a little bit more).

Figure out the exact location that Canon is using and update ML to crop from this offset when in mv640crop. The best way to do this would be to shoot a silent DNG in this mode (so you get the full buffer) with a tripod of some sort of grid or resolution chart, and then shoot H.264 of the same setup.

[xiga]

Thanks for this dmilligan.

Yeah i also suspected this might be something that just hasn't been noticed up until now, due to the unusual nature of it :/

The problem with option 1, is that it will lead to shorter captures. At present, if i choose 640x480 at 48fps in the ML menu, then i can get 54 secs of continuous video (note, as high an FPS as possible is critical for what i am doing. I'd ideally like 60, but 48 seems to be the sweet spot). This gives me enough frames to then stack the best 10% for each panel of the mosaic. If i choose a higher resolution, it will only cut down on the length of video i can record.

Option 2 sounds interesting. I know nothing about editing ML code, and you kind of lost me on the whole silent DNG thing lol, but i do reckon the offset when shooting Raw in the Movie Crop Mode is in the region of 5 pixels down and 70 pixels left (give or take), so if there's some way to easily copy these numbers into a line of code somewhere then i'm more than willing to give it a bash, with a little help of course 

[dmilligan]

the relevant code in raw_rec.c:line 268:

Code: [Select]

static void update_cropping_offsets()
{
    int sx = raw_info.active_area.x1 + (max_res_x - res_x) / 2;
    int sy = raw_info.active_area.y1 + (max_res_y - res_y) / 2;

    if (FRAMING_PANNING)
    {
        sx += frame_offset_x;
        sy += frame_offset_y;
    }
    else if (FRAMING_CENTER && lv_dispsize > 1)
    {
        /* try to center the recording window on the YUV frame */
        int delta_x, delta_y;
        int ok = focus_box_get_raw_crop_offset(&delta_x, &delta_y);
        if (ok)
        {
            sx = COERCE(sx - delta_x, raw_info.active_area.x1, raw_info.active_area.x2 - res_x);
            sy = COERCE(sy - delta_y, raw_info.active_area.y1, raw_info.active_area.y2 - res_y);
        }
    }

    skip_x = sx;
    skip_y = sy;
   
    refresh_cropmarks();
}
(there's similar code in mlv_rec)

Basically you'll need to add an extra case in there for mv640 crop and adjust skip_x and skip_y accordingly.

Here's how to figure out if you're in mv640crop:

Code: [Select]

    int mv640crop = is_movie_mode() && video_mode_resolution == 2 && video_mode_crop;


[dfort]

Ok--I was way off, I thought you were using zoom mode. I don't have any experience at all with mv640 or mv640crop video modes. Like dmilligan said, most cameras don't even have this mode. By the way, the various raw video modes are described in the Magic Lantern source code in src/raw.c, lines 433-440.

I made up this chart when working on a project that also involved finding out what was going on with raw buffers. Sort of rough but it did the job.

Silent DNG (or MLV) involves loading the silent.mo module and turning on Silent Picture in the Shoot menu. Now it may sound counter intuitive but if you setup your camera to shoot H.264 or raw video that also affects the settings of the Silent Picture when set in the "Simple" setting mode. Taking a Silent Picture involves doing a half shutter press while in movie mode--of course not at the same time as you're shooting a movie. The silent picture will capture the entire buffer area, the movie will be a cropped area of that raw buffer.

I'm not sure how to capture the full buffer at 48fps in option 1 but to find out what the crop is on option 2 superimpose, in a Photoshop or Gimp layers, a frame of your H.264 movie over the Simple Silent Picture. Find where the upper left corner of the H.264 is at in pixels and that is your panX and panY coordinates. Now take an MLV file with the same settings and run it through mlv_dump using the -m and -v options and you will be able to see the panX and panY information, it looks like this:

Code: [Select]

     Pan: 296x247

I'm not sure where you can adjust it but taking another look at raw.c might give some clues:

Code: [Select]

        /**
         * The RAW file has unused areas, usually black; we need to skip them.
         *
         * To find the skip values, start with 0,
         * load the RAW in your favorite photo editor (e.g. ufraw+gimp),
         * then find the usable area, read the coords and plug the skip values here.
         *
         * Try to use even offsets only, otherwise the colors will be screwed up.
         */
---------------------------
        #ifdef CONFIG_60D
        #warning FIXME: are these values correct for 720p and crop modes?
        skip_top    = 26;
        skip_left   = zoom ? 0 : 152;
        skip_right  = zoom ? 0 : 2;
        #endif

Hope this answered some of your questions--or maybe it created some more questions?

By the way, what is it that you are doing with the moon? Making an image stack to increase resolution? (Click on image to link to the post.)

One last tip--dmilligan is going to push you past your comfort zone but the pay off is worth it.

[xiga]

Thanks for your help with this guys, although i'm going to have to hold my hands up and say that i'm in way over my head here in terms of compiling code. I thought it was just going to be a case of editing a few text files, but it looks like there is a lot more to it than that.

Given i already have an idea of the offset in terms of pixels (the Raw video needs to be about 5 pixels Up and 70 pixels Right) is it possible some kind person could do a quick update for raw_rec and mlv_rec for me to test out?
Or alternatively just bring this issue to the attention of a dev so they could look in to this?

Either way, i'm happy to test out any new builds to see if we can nail the crop mode framing closer than it currently is.

ps - The post above describes pretty well the outcome i'm trying to get to (although i'm hoping for better resolution). The idea is that each individual video will be used to create a stack from which i will only take the best 10%. This shoul dgive me over 200 frames to use for each pane, which should be enough to keep the noise down. I will then need to stitch them all together for the final mosaic, so that when one zooms in on the image it keeps all the detail and resolution, rather than breaking down as is normally the case.

[dfort]

I'm looking at the code dmilligan pointed out in raw_rec.c and mlv_rec.c but I'm not sure the proper way to adjust for your 5 pixels Up and 70 pixels Right. If you could shoot a Simple Silent DNG with your camera in the movie mode you're planning to use we can take a look and see if the values in the code snippet that I posted from raw.c are correct. a1ex posted those warning in this commit: https://bitbucket.org/hudson/magic-lantern/commits/233c07fde193dcce4b616e1eafcbc7543e9d78b6 It won't hurt to check it out.

[xiga]

Okay i took a few more shots today.

First, the Silent DNG:
Silent

And here is the H.264 one:
Mov

And just in case it's needed, here is a still from the Raw video as well:
Raw

I overlaid the H.264 image on top of the Silent DNG one in Photoshop and lined them up as closely as i could, and the top left corner of the H.264 one was at position X=124, Y=51.

Hopefully this information will help, but let me know if i need to post up anything else.

Cheers!

[a1ex]

Can you upload the silent DNG, so we can check the skip offsets? However, since you've got a usable image without black borders, they are probably correct (or almost correct).

If you overlap the H.264 image on the silent DNG, you can see Canon does not record the center on the raw buffer as H.264. Thing is, ML has no way to know about this quirk.

However, the solution is now obvious. Just use the digital dolly feature to move the recorded area to the far right.

[xiga]

Here is a link to the Silent.DNG:

https://onedrive.live.com/redir?resid=C14ECDF7700B5618!2504&authkey=!AF7IRSm17GWjjC8&ithint=file%2cDNG

I have had a play with the Digital Dolly feature, but unfortunately i don't think it's going to help. 
Once i turned it on, i was able to see a rectangle that took up almost all of the screen, and i was able to move it around using the direction buttons. And sure enough, it did actually have an effect on the resulting Raw video. The problems however, are that:

1) there is no fine control. I had to press the direction twice in a row for it to move (and then quickly in the opposite direction to stop it moving), and even then it moves way too fast and also in really large increments.
2) there was definitely enough travel in the up and down directions to match the H.264 file vertically, but nowhere near enough horizontally. I could only move it a very small amount to the right, so not enough to correct for the Raw video being shifted too far to the left.
3) wherever i moved it to, it would always reset back to it's original position on each reboot. So not much good for achieveing consistent, repeatable results.

So does this just leave us with the option of editing the code to allow for offsets in some way?

ps - i will be taking some more Silent.DNG's, one in each of the Canon video modes (including one in 1080p and x5 zoom) and will post them up, in the hope we might be able to get to the bottom of this.

[a1ex]

Sorry, can't try it myself right now, but for repeatable results, all you should have to do is to press the right arrow and let it go to the far right position. That's all.

Can you attach a raw still screenshot with the digital dolly moved to the far right, and compare it with the silent picture screenshot and to the H.264 one?

[xiga]

As per dfort's request, here is a link to some Silent DNGs and their corresponding MLV files:

https://onedrive.live.com/redir?resid=C14ECDF7700B5618!2511&authkey=!AK_HoDSjL9HYWds&ithint=folder%2cDNG

In the folder are Silent DNG's and MLV files for each of the following:

1080p 30fps (Canon menu)___the highest reolution ML would let me choose was 1728 x 1156
720p 60fps (Canon menu)___the closest i could get to 720p in ML was 1280 x 694
640x480 60fps (Canon menu)___640x480 resolution chosen in ML
640x480 Movie Crop Mode (Canon menu)___640x480 resolution chosen in ML
1080p 30fps (Canon menu)___this time with x5 Zoom applied, again the highest resolution ML would let me choose was 1728 x 1156

Hopefully these will prove useful.

a1ex, i don't have any still at hand with the digital dolly moved to the far right, nor the H.264 one. I will try again either later tonight or tomorrow morning, but i can say for sure that there is hardly any 'leg room' to move it right, certainly not enough for the correction.

[a1ex]

Right, there are about 30 pixels on the right side, which are not recorded.

The problem is this offset (raw.c):

Code: [Select]

        skip_right  = zoom ? 0 : 2;

It should be 0 in this crop mode.

[dfort]

Here are my observations:

First of all to get the entire raw buffer, including OB area I ran 'dcraw -4 -E' on that mv640crop Silent DNG (920x624). Then I lightened it up to make it easier to see:

Next I checked it in Photoshop. Looks like the out of bounds area ends at 150x26. Does that mean that the top skip is 26 and the right skip is 150?

I don't see any skipped pixels on the right or the bottom:

Here is what I'm seeing on the files xiga posted:

Mode	Raw Buffer Size	skip_top	skip_left	skip_right
mv640crop	920x624	26	150	0
mv640	1888x720	26	152	2
mv720	1888x720	26	152	2
mv1080	1888x1182	26	152	2
zoom	2520x1106	26	0	0

So going back to raw.c:

Code: [Select]

        #ifdef CONFIG_60D
        #warning FIXME: are these values correct for 720p and crop modes?
        skip_top    = 26;
        skip_left   = zoom ? 0 : 152;
        skip_right  = zoom ? 0 : 2;
        #endif

Should it be changed to this?

Code: [Select]

        #ifdef CONFIG_60D
        skip_top    = 26;
        skip_left   = zoom ? 0 : mv640crop ? 150 : 152;
        skip_right  = zoom ? 0 : mv640crop ? 0 : 2;
        #endif

I can make a test build for xiga though I'm not sure if this will address his image centering issue.

[EDIT: @xiga -- I believe that what a1ex is saying is that there are extra pixels in the right side of the raw buffer that aren't being used so if you digital dolly all the way to the right you should be closer to center.]

[xiga]

Thanks for the test build dfort!

I gave it a quick test, taking a couple of rough and ready shots just in my front room. As far as i could tell, i don't think it made any difference though. I'd need to do a more thorough test to be absolutely sure though.

However, this got me thinking, for what i'm trying to achieve here, i suppose it doesn't really matter if the Raw video doesn't exactly match the H.264 one. For me, what matters the most, is that what is displayed on the camera screen itself is exactly what is captured in the Raw video. With this in mind, i tried changing the 'Preview Options' under the Raw Video (MLV) section from Auto to 'ML Grayscale'. There is a note at the bottom of the screen that says 'Looks ugly, but at least framing is correct'. And indeed, when choosing this option i can see the framing suddenly shift in the required direction! It's true that it does indeed look ugly (the resolution is terrible, everything looks blocky as hell) but i took a quick video (lining up something in the very corner of the frame) and the resulting DNG files seemed to put it right in the corner, so i think i may have stumbled into a solution here by accident! I think i can live with the ugly preview issue, i'm hoping i will still be able to make out the features of the moon well enough to know where i am in terms of putting together the mosaic.

One thing i've noticed though, this means i have to have Global Draw on in Liveview (to take advantage of the ML Grayscale option). Global Draw adds a row of information at the top and bottom of the screen. Normally this information is desirable, but in my case it's simply getting in the way of important screen real estate, as what is underneath is still being captured. As a workaround, I can tap the Q button to get a peak through Canon's semi-transparent bar to see what's under there, but it would obviously be preferable to not have to touch the camera at all. I don't think it's possible, but it would be nice to have the option to turn these info strips off but keep Global Draw on in Liveview. Just a thought i suppose!

As for everything else above that's gone before, if you guys still think the issue with the offsets needs updating then i'm happy to continue to help with taking test shots and testing out any new builds.

Thanks everyone for all your help on this, you're all lovely people.

Cheers,

xiga.

[dfort]

I didn't think that adjusting a pixel here and there would do it for you but looks like we cleared up some long standing doubts about raw video modes on the 60D. Looking at the code it seems that mv640crop is unique to the 60D so this next step should do it.

Add this to raw_rec and mlv_rec:

Code: [Select]

    skip_x = sx;
    skip_y = sy;

    refresh_cropmarks();

    /* mv640crop needs this to center the recorded image */
    if (is_movie_mode() && video_mode_resolution == 2 && video_mode_crop)
    {
        skip_x = skip_x - 5;
        skip_y = skip_y + 70;
    }

I believe that the offset should be done after refreshing the cropmarks but I'm not sure. I'm not far very far along the C tutorials so I'm not sure if this is going to work at all but if it does I'll put in a pull request.