Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - a1ex

Pages: [1] 2 3 ... 350
Added links to more ports in progress (actually "stalled" might be a better word for some of them) on the download page.

Also added links to existing ROM dumpers:

The black level is the signal level you get in the absence of light. It's a positive value because:
- the value is not the same for every sensel (hence the need for dark frame subtraction)
- values below black level do appear (because of the additive noise); a noise reduction or a frame stacking step can make use of those sub-black values.

"Pushing" the black level to a lower value would be the same as changing the input curve (rendering deep shadows in a lighter tone, rather than black). You probably liked the look, but there's no additional dynamic range to gain this way.

If the black level is incorrectly set above the real value, you will see additional detail from lowering it. Otherwise, there's little or no point in changing it.

Changing the black level also changes the colors in shadows, because the white balance multipliers are usually not equal. If you set all of them to 1, the color cast introduced by changing black level disappears.

On the other hand, there might be additional dynamic range to gain from pushing white level to a higher value. With recent versions, this additional gain will be less than 0.02 EV.
Code: [Select]
 * White level
 * With PREFERRED_RAW_TYPE set to CCD, most cameras appear to clip above 16300
 * (most of them actually use the full range, until 16382)
 * one size fits all: 16200 may sacrifice up to 0.02 EV of highlights
 * that is, log2((16382-2048) / (16000-2048))
#define WHITE_LEVEL 16200

However, if you find pink highlights with this value, I'd be interested to see a sample MLV.

@Danne: pushing white to 16383 may be a little optimistic; there are exposure settings that clip to this value and there are exposure settings that clip to a slightly lower level (including 5D3). Usually, if white level is above clipping point, you get pink highlights; but in some cases, the result is pretty good anyway, so it's a bit of an edge case.

You can check the white level in the samples in this thread:

e.g. with octave (requires read_raw.m):
Code: [Select]
a = read_raw("frame.dng");
prctile(a(:), [90 95 99 99.9 99.99 99.999])

Note the max value from the image can exceed the clipping point from one sensel (happens with hot pixels), that's why I prefer checking it with percentiles.

Camera-specific discussion / Re: Canon 1200D
« on: Today at 08:44:48 AM »
For menu, it's probably as easy as updating BGMT_AV definition in gui.h.

For expo menu, see replies #426, #427, #438.

Camera-specific discussion / Re: Canon 760D / T6s
« on: Today at 08:41:33 AM »
FYI, octacorn also made some progress on 750D (and DIGIC 6 in general), so you may want to get in touch with us on IRC.

A major difference is that I've replaced the interrupt thread with a QEMUTimer. This solved som iothread lock bug, but I guess it should work better than the thread since it's now synced to the guest clock and not the host system.

Yay! That's what I wanted to do next, hoping it would solve the GUI lock-ups. I/O lock was another issue that I didn't know how to solve.

I've managed to merge the latest changes into qemu v2.9.0-rc1. It's probably not a perfect merge, the camera gui hangs up more often than on v2.5.0 for me.

My attempt to merge with 2.8.0 was not very successful (got stuck at making the serial port work, so ended up disabling it), but otherwise it seemed to run fairly well. Will try to integrate your changes and see how it goes.

I'm currently playing with mlv_lite on 5D3, so I've checked your issues on this configuration:

1. works fine here.

2. works fine if sound recording is turned off from Canon menu (otherwise it breaks sound recording, until you exit LiveView and return); that's a problem on newer models without CONFIG_AUDIO_CONTROLS.

3. works fine.

4. (sorry, extremely bleeding edge, so no test binary)

5. not tested

6. works fine here, zoom button blocked, exiting LiveView ends recording cleanly.

You may get different results with mlv_rec (didn't try).

Updated to address some comments from dfort (see the PR page):

I started out setting the Pre-record for 1 second and the Reg trigger to Half-shut:1 frame. Every half-shutter recorded a second instead of just 1 frame.

Ok--don't really need pre-record for this so I turned that off and now I'm getting 2 frames per half-shutter press. Not sure if I'm doing something wrong but it looks like all the settings are telling me that it should record only 1 frame.

Also added fractional time display while recording.

Camera-specific discussion / Re: Canon 1200D
« on: Yesterday at 10:32:52 AM »
ML menu in video mode: select Debug -> Show GUI events and write down the values that appear when pressing the delete button (or record a video).

Focus peaking and false colors: what exactly is wrong with them?

Image Fine-tuning:
- ML digital ISO should... change brightness
- Black level: should adjust the blacks (deep shadows) in LiveView
- Absolute zero sharpness: should override picture style sharpening (should give lower sharpness than Canon's minimum level). If you don't want to pixel-peep, set picture style sharpness to maximum, then enable this option (so the difference will be obvious).
- Edge emphasis: you may need some pixel peeping for this one; not sure if it's of any use in practice.

- DOF settings: here and here
- LV DIGIC peaking: sounds like OK to me, as the effect applies to the entire image (including menus)

Scripting Corner / Re: Help me with LUA
« on: Yesterday at 09:53:19 AM »
Main page -> Documentation -> Lua API
Main page -> Downloads -> Nightly builds -> Lua API

Suggestion for a better place for this link?

Just noticed the link to IRC chat (#magiclantern) somehow disappeared; added it back.

Camera-specific discussion / Re: Canon 100D / SL1 (Beta-4a)
« on: March 26, 2017, 06:10:21 PM »
Indeed, setting the color to 80 fixed this. Look at clock color between schemes 1 and 2, for example.

Unfortunately, this setting is not portable; 60D, for example, paints the info with color 1 (and has a single color scheme), while 100D uses color 80 on all 5 color schemes.

The problem with 60D flexinfo is that yellow is unreadable on color scheme 2 (minor issue; just a nice touch). 60D only has black background and white fonts, so yellow was fine. Maybe I should take the time to unify these layouts a bit.

Looks like I also need to fix that screenshot palette bug. (edit: done!)

Scripting Corner / Re: Lua Scripting (
« on: March 26, 2017, 09:10:58 AM »
Lots of info on focus_pos here. It's not moving the lens in any way, and right now it's probably only useful for research.

Also found a typo that prevented lens.focus_pos from appearing in the HTML docs.

Camera-specific discussion / Re: Canon 100D / SL1 (Beta-4a)
« on: March 26, 2017, 01:09:19 AM »
One FAIL left when running stubs api test right at the beginning:

This one appears to be a false alarm (hopefully fixed).

After merging lua_fix, 100D is next on the list for including in the nightlies. Meanwhile, you may consider the lua_fix build from the Experiments page as "release candidate", so please give it a try. Not just on Lua, but on all other functionality.

P.S. I'm now able to run the 100D ML (unmodified autoexec.bin) in QEMU (so I could check a couple of things locally). So far, so good; only noticed some minor issues:
- the temperature is not yet in Celsius
- ML overlay colors don't adapt to Canon's color scheme outside LiveView (COLOR_FG_NONLV)
- selecting a Kelvin temperature gives a strange display indicator (K over AWB icon); not sure if only happens in QEMU or also on real camera, as properties are not emulated very well
- continuous AF should probably display a warning on ML focus tools (probably on other models as well; need to find the property that triggers it)
- task list overflows (todo: check how many tasks can be started after ML loads)
- memory levels are OK (quite good actually for a Rebel)
- QEMU crashes when selecting "Don't click me" (emulator bug)

Overall impression is very good - nice job @nikfreak! (and sorry for being so slow with reviewing the new ports)

Scripting Corner / Re: Lua Scripting (
« on: March 26, 2017, 12:22:43 AM »
I've revised the behavior of "complex" scripts a bit: in the initial concept, they were started, and if they could not be unloaded, they were automatically configured to autorun. Now this is no longer necessary (and probably a bit confusing as well), because any script (be it simple or complex) can be set to autorun from menu.

Please give it a try, as I'd like to merge it soon. Not only for Lua itself (which doesn't seem very popular these days), but for the other backend updates made to pass the Lua API tests.

There is no such thing as crop_rec MLVs, as crop_rec simply reconfigures the video mode. The MLVs are still made by either mlv_rec or mlv_lite.

Since crop_rec does not (yet?) alter the resolution of the original buffer (it simply changes the sensor scanning parameters), I don't expect it causing any changes in the file structure.

If you are talking about mlv_lite, NULL blocks are used either after the main headers (to make sure their size is multiple of 512, for write speed reasons), or when the 4GB limit is auto-detected the hard way (after a write failure), to discard the incomplete frame that might have been written.

Frame padding (to ensure EDMAC alignment and block size multiple of 512) is done the old-school way (frameSpace and blockSize slightly larger than actual payload, so there may be unused data both before and after the frame - whose size is xRes * yRes * bpp/8). I agree using NULL blocks would have been nicer (just needs extra care if the skip amounts are smaller than the size of a NULL block).

General Help Q&A / Re: Reading 14 bit RAW
« on: March 25, 2017, 07:39:52 AM »
Most likely yes, but didn't try. Here's what I've used for Apertus:

Code: [Select]
/* two pixels packed as 12-bit (Apertus raw12) */
struct raw12_twopix
    unsigned a_hi: 8;
    unsigned b_hi: 4;
    unsigned a_lo: 4;
    unsigned b_lo: 8;
} __attribute__((packed));

Note the bytes in uncompressed Canon raw must be swapped to get valid DNG, at 10, 12 and 14-bit (see reverse_bytes_order in chdk-dng.c), but not at 16.

DNG spec:
If BitsPerSample is not equal to 8 or 16 or 32, then the bits must be packed into bytes using the TIFF default FillOrder of 1 (big-endian), even if the TIFF file itself uses little-endian byte order.

The 16-bit data provided by Canon (which is really 14-bit padded with zeros) is already little endian.

General Help Q&A / Re: Reading 14 bit RAW
« on: March 24, 2017, 11:09:54 PM »
mlv_dump handles all bit depths with the same routine; you may find raw2dng or raw_get_pixel from raw.c (which are hardcoded for 14-bit) easier to understand. Make sure you look at MLVFS as well.

struct raw_pixblock from raw.h may be useful, too. However, bit packing order is not well-defined in the C standard, so... it just happens to work with gcc (and might break with other compilers) [1] [2].

So, to find what shifts to use, the easiest way is probably by looking at disassembly code created by the compiler.

Tip: in ML, there's a shortcut to look at disassembly, either from ML core or from a module: tag any non-static function with DUMP_ASM, then run "make dump_asm".

Camera-specific discussion / Re: Canon 1200D
« on: March 24, 2017, 09:32:21 AM »
Looks good, thanks.

A few days ago, I've attempted to test the installer myself in QEMU (wanted to find out why it crashed, based on previous reports). The result is this - you can now run the unmodified autoexec.bin under QEMU and navigate the menus (both Canon and ML), on 1200D and a bunch of other models.

Previously, this was only possible on 500D (other models required modified autoexec.bin or did not start the GUI at all).

A small change that unlocked Canon menu navigation on many models:

After some refactoring and porting the 500D MPU messages required for GUI, Canon menu navigation is now also working on...

60D, 550D, 600D, 700D, 100D, 1100D and 1200D!

Screenshots (guess the cam):

Test log.

All screenshots here (click on Expand all).

This is a big breakthrough, as it effectively lets me review ML ports or code changes on cameras I don't own :)

General Help Q&A / Re: Focus peaking in Canon zoom mode
« on: March 23, 2017, 05:52:33 PM »
It should be something easy to compute; increasing marker size will reduce the frame rate.

Can you show a mock-up of what you have in mind?

Camera-specific discussion / Re: Canon 1200D
« on: March 23, 2017, 04:17:29 PM »
This is the old installer; please test the one I've linked a couple of times...

General Help Q&A / Re: Focus peaking in Canon zoom mode
« on: March 23, 2017, 09:54:57 AM »
To try peaking in zoom mode, you do have to switch to movie mode, as the menu warning will tell you.

If you find this useful, I may consider enabling it for photo mode as well; however, I didn't do so because, to keep things simple (in a code base already convoluted), I've just disabled all overlays in zoom mode.

General Chat / Re: resolution interpolation to reach higher iso
« on: March 23, 2017, 09:42:16 AM »
The hints I have are summarized here:

I may not be familiar with the proper terminology, as I don't work in this field, sorry.

However, summing is (a+b+...) and averaging is (a+b+...)/n. When you do this in software on floating point, the SNR does not change (as the noise is scaled by the same amount).

When you do this on integer math, the latter has lower SNR from round-off (quantization) error.

When you do this on analog hardware (which is not my field), I'd expect additional noise from the electronics (whether it's with transistors, capacitors or whatever).

The hints from Chipworks suggest it's not digital electronics (not a FPGA), but analog electronics (an additional transistor on the CMOS chip).

What I can tell you from observations:

The signal coming from a 3x3 bin, that enters the ADC, appears to have roughly the same levels*) as the signal coming from a single sensel. In both cases, black level is 2048 and white level is above 15000. So, if the signals are summed somehow, they are normalized (therefore, averaged).

The black level appears to be clamped by an analog feedback loop, which I believe it's applied before the ADC.

Whether the signal normalization is done on the CMOS chip or a separate analog amplifier (before the gains we have identified), I don't know. However, the noise numbers are clearly higher than with software averaging, and the overall read noise level appears comparable to the level from regular photo mode / 5x zoom.

I see this as a hint that pixel binning happens before the main source of read noise in Canon sensors. This noise is rumoured to be on the way between the CMOS chip and the ADC, which are different chips on DIGIC 5 and earlier models. The design apparently changed with newer models.

Reducing the available gains (ADTG, registers 0xFE, registers 8/9/A/B) gives similar behavior in 3x3 and 1:1 modes (this was expected, since it all happens before these amplifiers, I think).

Binning on lines and columns are independent (see crop_rec, which can do 1x3 and 3x1 binning and works by overriding a small part of Canon's settings that are sent to CMOS and ADTG chips).

In particular, for 3x1 (bin 3 lines, read every column), obtained by modifying 1080p, the patches done are:

Code: [Select]
                cmos_new[2] = 0x10E;    /* read every column, centered crop */

                /* ADTG2/4[0x800C] = 2: vertical binning factor = 3 */
                /* ADTG2[0x8806] = 0x6088 (artifacts worse without it) */
                adtg_new[0] = (struct adtg_new) {6, 0x800C, 2};
                adtg_new[1] = (struct adtg_new) {2, 0x8806, 0x6088};

Register 0x800C is already 2 in 1080p; this change only has effect in 720p. Register 0x8806 appears to reduce artifacts (what's happening here is unknown, but it's not a process that alters overall brightness). Register CMOS[2] is used to enable 3x column binning (a single bit) and to position the scanned area horizontally.

That means, all the configuration that affects column binning is done on the CMOS chip, so it happens before the signal enters ADTG (and before ADC, since ADTG gain was found to be analog).

For 1x3 (read every line, bin 3 columns):

Code: [Select]
                cmos_new[1] = (is_720p())
                    ? PACK12(14,10)     /* 720p,  almost centered */
                    : PACK12(11,11);    /* 1080p, almost centered */

                cmos_new[6] = 0x170;    /* pink highlights without this */

                /* ADTG2/4[0x800C] = 0: read every line */
                adtg_new[0] = (struct adtg_new) {6, 0x800C, 0};

Here, we patch 0x800C to disable line binning (so, read every line). This register configures the number of skipped lines on cameras before 5D3. Here, 0 = read every line, 2 = bin every 3 lines, 4 = bin every 5 lines and so on. Intermediate settings work too, but I didn't run any noise tests on them.

I did run the noise tests (, SNR analysis from 2 images) on 1:1 crop_rec vs 1:1 zoom, and 720p 3x3 crop_rec vs 1080p 3x3 Canon default, and found them to be very similar.

So, the above note at *) was a little incorrect: when binning 3 lines, the signal level that enters in the ADC appears to be slightly higher (in any case, less than 0.5 EV, probably 0.1 or 0.2). So we have to patch an extra CMOS register to fix it (by increasing the level after disabling line binning).

You draw the conclusions.

General Chat / Re: resolution interpolation to reach higher iso
« on: March 23, 2017, 02:20:37 AM »
Binning is simply blurring followed by downsampling (reducing the resolution by dropping pixels).

Example on how 5D3 does 3x3 binning to get 1080p Bayer raw from a full-res Bayer matrix:
Code: [Select]
function im = bin_pixels_3x3(im)

f = 1/9 * ...
    1 0 1 0 1;
    0 0 0 0 0;
    1 0 1 0 1;
    0 0 0 0 0;
    1 0 1 0 1;

imf = imfilter(im, f);
im = imf(1:3:end, 1:3:end);

First step is a 3x3 blur (averaging) on each Bayer channel, second step is keeping the central pixel from each 3x3 block (after filtering).

Also FYI, on Canon DSLRs, pixel binning is done by hardware (analog electronics), not firmware.


Pages: [1] 2 3 ... 350