Full-resolution silent pictures (silent.mo)

[Stedda]

Want to try it on your camera heres how.....

- I won't include it in the nightly builds at this stage; early adopters will have to compile it themselves.

How do I install it?

Compile ML (both core and silent.mo) from the fullres-silent-pics branch. For 5D3-123, merge the two branches locally first.

If you get a compile error in raw.c, see the porting notes below, find the missing values for your camera, and submit a pull request to the fullres-silent-pics branch.

Porting notes

- RAW_PHOTO_EDMAC needs to be changed on all cameras (see these comments in raw.c)
- Skip offsets for photo mode will have to be re-calibrated (example). Fortunately, it's much easier these days: download raw_diag.mo, select the "OB zones" option, and adjust the offsets until you get the following picture (note the cyan box being just inside the noisy borders):

[kyrobb]

Could this discovery lead to higher resolutions of Raw video in full sensor mode?

[Midphase]

Could this discovery lead to higher resolutions of Raw video in full sensor mode?

From the first post:

4K video?

Too early to ask. Find a way to speed up the capture process first (see the research section below).

However, I don't really expect more than 10 fps (in LV crop mode on 5D3, 1/3.6 of the total sensor area is sampled at 30fps). If you implement a burst mode from the current code base, you'll get up to 5 fps.

[budafilms]

From the first post:

4K video?

Too early to ask. Find a way to speed up the capture process first (see the research section below).

However, I don't really expect more than 10 fps (in LV crop mode on 5D3, 1/3.6 of the total sensor area is sampled at 30fps). If you implement a burst mode from the current code base, you'll get up to 5 fps.

Maybe 2.5k

[barepixels]

Anyone test on 5D2 yet?  are there EXIF info?

[JoshuOne]

  Nice work, once again a1ex et al!
I am hoping that someone who does end up testing this on a 5d2 would share the module for the less code-savvy of us. I really can't wait to try.

[wyrlyn]

Video tutorial about the install method would be great!

[mk11174]

I have been trying to create a trigger for this Full Res silent picture to help capture day lightning, and so far I have it working to where it actually works to where using a half shutter trigger I can capture once I see a flash and it looks like it captures the flash I saw not what I see when I pressed the button.

I was hoping someone with real code know how could look it over to help me clean it up or improve it???

I was trying to use it with a motion trigger or a luminance trigger, but I guess it is not possible since live view has to be paused to make it work?

I did try to use the code from the bolt_rec module and add that as my trigger, but that does not work either, the calculations stop once the full res code starts up.

But this method would be great for hand trigger to get better results that using the camera with mechanical shutter and its faster then using the Full_Res_Silent picture method because captures are 250 to 300ms which is to slow.

So if someone with experience could look it over quickly and just maybe give some tips to make it better and more stable I would be greatful! 

Code: [Select]

// Full-Res Silent Pictures //

#include <module.h>
#include <dryos.h>
#include <bmp.h>
#include <menu.h>
#include <config.h>
#include <property.h>
#include <raw.h>
#include <shoot.h>
#include <zebra.h>
#include <beep.h>
#include <lens.h>


static int full_res_enabled = 0;

static int save = 0;

static char* silent_pic_get_name()
{
    static char imgname[100];
   
    char *extension   = "DNG";
    int file_number   = get_shooting_card()->file_number;
   
        char pattern[100];
        snprintf(pattern, sizeof(pattern), "%s/%04d%%04d.%s", get_dcim_dir(), file_number, 0, extension);
        get_numbered_file_name(pattern, 9999, imgname, sizeof(imgname));

    bmp_printf(FONT_MED, 0, 35, "%s    ", imgname);
    return imgname;
}

static void silent_pic_save_dng(char* filename, struct raw_info * raw_info)
{
    save_dng(filename, raw_info);
}

static void capture()
{

   while(1)
   {

     if(full_res_enabled)
     {
        PauseLiveView();
        clrscr();
        vram_clear_lv();

        struct JobClass * job1 = (void*) call("FA_CreateTestImage");

        call("FA_CaptureTestImage", job1);
        call("FA_DeleteTestImage", job1);

     }

        if(full_res_enabled && get_halfshutter_pressed())
        {
           save = 1;
           break;
        }
   }     

    if(full_res_enabled && save == 1)
    {     
        save = 0;
        void raw_buffer_intercept_from_stateobj();
        raw_buffer_intercept_from_stateobj();

        int new_gui = GUISTATE_QR;
        prop_request_change_wait(PROP_GUI_STATE, &new_gui, 4, 1000);

        /* preview the raw image */
        raw_set_dirty();
        raw_update_params();
        clrscr();
        raw_preview_fast();

        /* prepare to save the file */
        struct raw_info local_raw_info = raw_info;
   
        /* leave some time for overlays to get displayed */
        /* (fixme: saving the DNG will destroy the contents of the raw buffer) */
        //msleep(2000);

        /* save the raw image as DNG */
        char* fn = silent_pic_get_name();
        bmp_printf(FONT_MED, 0, 60, "Saving %d x %d...", local_raw_info.jpeg.width, local_raw_info.jpeg.height);
        silent_pic_save_dng(fn, &local_raw_info);
        bmp_printf(FONT_MED, 0, 60, "Saved %d x %d.   ", local_raw_info.jpeg.width, local_raw_info.jpeg.height);

        msleep(10);
    }
return;   
}


static void full_res_enable()
{
    /* toggle the lv_save_raw flag from raw.c */
    raw_lv_request();
    capture();
    msleep(50);
}

static void full_res_disable()
{
    raw_lv_release();
}

static void raw_lv_request_update()
{
     static int raw_lv_requested = 0;

    if (full_res_enabled && lv && is_movie_mode())
    {
        if (!raw_lv_requested)
        {
            full_res_enable();
            raw_lv_requested = 1;
        }
    }
    else
    {
        if (raw_lv_requested)
        {
            full_res_disable();
            raw_lv_requested = 0;
        }
    }
}

static unsigned int capture_loop_cbr()
{
    raw_lv_request_update();



    if (!full_res_enabled)

    {

        return 0;

    }



    if (!lv || !is_movie_mode())

    {

        return 0;

    }

return 0;
}

static struct menu_entry res_menu[] = {
    {
        .name = "FullResDNG",
        .priv = &full_res_enabled,
        .max = 1,
        .depends_on = DEP_LIVEVIEW,
    },

};

static unsigned int full_res_init()
{
    menu_add("Shoot", res_menu, COUNT(res_menu));
    return 0;
}

static unsigned int full_res_deinit()
{
    return 0;
}

MODULE_INFO_START()
    MODULE_INIT(full_res_init)
    MODULE_DEINIT(full_res_deinit)
MODULE_INFO_END()

MODULE_CBRS_START()
    MODULE_CBR(CBR_SHOOT_TASK, capture_loop_cbr, 0)
MODULE_CBRS_END()

MODULE_CONFIGS_START()
MODULE_CONFIGS_END()

UPDATE: Sorry about making the code a quote before.

[dmilligan]

I think the idea would be not to use LV for the trigger, but to just continuously take silent pics, do the analysis on that data and then only save it if you detect a strong change in luminance. The only slow part is saving the dng.

[Redapple]

YES!

But how do I compile this? how can i download the magic lantern file with this that will work on the 550d? Could someone send me the compiled file?

[mk11174]

I think the idea would be not to use LV for the trigger, but to just continuously take silent pics, do the analysis on that data and then only save it if you detect a strong change in luminance. The only slow part is saving the dng.

Sounds like an idea! Any ideas on how I can code that? I am not a programmer, just very basic knowledge. And thank you for your input!

[a1ex]

You can try something like this (it's like a full-res LiveView):

Code: [Select]

while(1)
{
    /* capture a raw image */
    struct JobClass * job = (void*) call("FA_CreateTestImage");
    call("FA_CaptureTestImage", job);
   
    /* tell the raw backend to catch the raw buffer address from RAW_PHOTO_EDMAC */
    void raw_buffer_intercept_from_stateobj();
    raw_buffer_intercept_from_stateobj();

    /* preview the raw image */
    raw_set_dirty();
    if (raw_update_params())
    {
        raw_preview_fast();
    }

    call("FA_DeleteTestImage", job);
}

Inside this loop, you can add some luma checks (the easiest way is with raw_hist_get_percentile_level), and once you hit some threshold, save the frame as DNG.

Optimizations:
- do the raw preview in some other task, as in mlv_play. Or just switch to grayscale; the overhead should be very small.
- once you detected a change, save not only that frame, but also the next few frames (you can buffer a small number of them - calling srm_malloc_suite(0) will tell you how many).

[mk11174]

You can try something like this (it's like a full-res LiveView):

Great! will try this, thank you very much, hoping to test on lightning later today, its that time of the year here in Florida.

[a1ex]

Some more rolling shutter analysis (with the same script as before):

5D3:

Test images: regular.CR2 and silent.DNG (warning: large files)
Canon settings: 1/4 vs 1/8000, ISO 100.

Image difference in EV (left), exposure difference per row, in EV (mid-left), in milliseconds (mid-right) and timer register configuration (right):


Exposure range  : 1/33.19 ... 1/6.30
Rolling shutter : 0.13 seconds

6D:

Test images from 1%: regular.CR2 and silent.DNG (warning: large files)
Canon settings: 1/4 vs 1/4000, ISO 100.



Exposure range  : 1/3.09 ... 1/1.87
Rolling shutter : 0.21 seconds

500D:

Test images from Greg: regular.CR2 and silent.DNG (warning: large files)
Canon settings: 1/4 vs 1/4000.



Exposure range  : 1/83.45 ... 1/3.43
Rolling shutter : 0.28 seconds

5D2:

(copied the results from my previous post, just to have everyting in one place)

Test images: regular.CR2 and silent.DNG (warning: large files)
Canon settings: 1/4 vs 1/8000.



Exposure range  : 1/59.97 ... 1/3.86
Rolling shutter : 0.24 seconds


Now, let's look at the numbers from the graph and at the FPS timers configured in adtg_gui, and cross-check them with the theory from FPS override: http://magiclantern.wikia.com/wiki/VideoTimer

Let's take the 5D2 first. The main clock, from FPS override, is 24 MHz. Timer A is configured at 0x5db+1 = 1500 units. In the second part of the exposure, timer B is configured at 0xdf+1 and then at 0xedc+1. Exposure time is 1 / (MainClock / timerA / timerB), so the timers are programmed to do 1/71.42 + 1/4.20 seconds = 0.014 + 0.2378 seconds.

So, the last value for timer B (0xedc => 0.2378 seconds) must be for the ramping part (the lines are read out one by one, but since the shutter is open, bottom rows are receiving more light). It matches the measured rolling shutter value.

The previous value for timer B (0xdf) must be for the constant exposure part (the entire sensor is capturing light). Total exposure for the last row would be 0.014 + 0.2378 = 1/3.971 seconds, and for the first active row, there is an optical bar 51px wide => 0.014 + 0.2378 * 51/3804 = 1/58.18 seconds.

The first part of the exposure (0x2f+0x207) is called DummyReadout. I'm not sure what it does, but it seems to be related to FPN correction.

500D: main clock 32 MHz, timer A 0xaf9+1, timer B 0x71+0xc83 (+1+1). The last number must be the rolling shutter: 0.281 seconds. The previous one must be the constant exposure part: 0.01 seconds. Total exposure: bottom row 0.281+0.01 = 1/3.436 seconds, top row 0.281*26/3204 + 0.01 = 1/81.43 seconds. Notice there's no dummy readout step.

5D3: main clock 24 MHz, timer A 0x317+1 and timer B 0x21d+0x1ce4+0x12e+0xf75 (+1 for each term). Rolling shutter: 24e6 / (0x317+1) / (0xf75+1) = 0.1306 seconds.

The previous one (0x12e) must be constant exposure: 0.01 seconds. Total exposure: bottom row 1/7.11, top row 0.01 + 0.1306*80/3950 = 1/79.08. Unexplained: around 0.018 seconds of exposure for all rows.

6D: main clock 25.6 MHz, timer A 0x597+1, timer B 0xe8d+1, rolling shutter 0.208 seconds.

Constant exposure part: 0x335 => 0.046 seconds. Bottom row exposure 1/3.94, unexplained 0.28s. Top row exposure 1/20.78, unexplained 0.275s. Notice the total exposure time is much higher than on the other cameras - exactly this "unexplained" time.

From the screenshot, adding two more timers (0x218+0x110f) gives 1 / (25.6e6 / (0x597+1) / (0x110f+0x218+2)) = 0.274 seconds. Coincidence? Why did the 5D3 get a shorter exposure? (it also has a large timer in the screenshot)


In all cases, notice the FPS timers are giving accurate values for rolling shutter (they match the measured values) and also notice the FPS timer B matches the vertical resolution of the camera (including top optical black bar, and a few extra pixels in newer cameras).

From this, I think the FPS timer A gives the clock for row readout (at each timer A rollover, a line is read out). If you know the main clock, the timer A, and the vertical resolution, you can compute the amount of rolling shutter with very good accuracy. This result probably applies to LiveView readout too, but needs to be double-checked.

Nitpick: you may want to multiply the rolling shutter, as computed from FPS timers, by (total_height-ob_height) / total_height, which is about 0.97...0.99.


If you want to see the same analysis for your camera, take the two test images with this script, then get the ADTG screenshot by following these steps.

[Audionut]

Is that longer exposure time the cause for the noise in the 6D image?

[a1ex]

No idea, maybe just less light.

[Audionut]

Didn't consider less scene light.

The results for 6D are very different to the other cameras.  Wonder if 1% was using some (FPS?) code from his repo during the test?

[a1ex]

FPS override has no effect on silent pictures. From the discussion on Bitbucket, the settings seem to be alright.

It doesn't hurt if anyone else double-checks it, but I expect the same results.

edit: updated the above post with 6D screenshot and some more numbers.

[1%]

none of these other cameras flip fps, ie 5D3 could use fps override without doing it from EVF state.

[dmilligan]

Just had an idea:
Use the full-res silent pics to meter for ETTR before taking a pic (when not in LV)

Edit: meh, nevermind I forgot you'd still have to raise the shutter anyway, so it's not saving shutter clicks :p

[Levas]

Is this gonna come available for time-lapse use, or is there more research needed to implement this for time-lapses ?
Maybe this can be put somehow in the intervalometer menu, as an option for silent full res pics 

[c2s07]

The intervalometer is already able to capture silent pictures, just enable both and it works. :-)

The large delay between the shots, if not reduced, is going to be a little problematic for timelapses, as it will result in rather high real-time multipliers in the finished videos. For example, 1 frame / 20s = real-time x 480 @ 24 fps.

[Levas]

Didn't know that it is that usable already 
The long delay is no problem, I'm mostly interested in nighttime time lapses, skies full of stars.
Normally I would have 15 or 20 seconds between pictures for nighttime time lapses
Do you have a 6d build with full res silent pictures option, or should I ask somebody else to compile it for me ?

[Greg]

I recommend turn off the "Long exp. Noise reduction". Currently, if this feature is enabled, cause err 70.

[dariSSight]

Didn't know that it is that usable already 
The long delay is no problem, I'm mostly interested in nighttime time lapses, skies full of stars.
Normally I would have 15 or 20 seconds between pictures for nighttime time lapses
Do you have a 6d build with full res silent pictures option, or should I ask somebody else to compile it for me ?

I'm confused about this new develop in the Silent Pictures, so what does it allow for you to do higher DNG VIDEO Then the RAW or MLV?
In laymen terms what is the new development.