Rolling shutter measurements from flickering lights

Started by a1ex, November 17, 2018, 02:27:14 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

a1ex

Recently, this blog post came to my attention: https://blog.kasson.com/nikon-z6-7/how-fast-is-the-z7-silent-shutter/

This gives a method for measuring rolling shutter of any camera, by filming some flickering light, as long as you know its frequency.

Examples of flickering lights:

- a plain old incandescent bulb in PAL land will flicker at 100 Hz (caveat: mains frequency is not exactly stable, but still reasonably good)
- some laptop monitors will start to flicker when you reduce their brightness; this frequency might be a little more stable
- many LED lights are also flickering; some at hundreds of Hz (useful for us), others at some kHz (not useful for us)
- if you've got an Arduino and a LED, you can program it to flicker at arbitrary frequency -> pwm.ino (default: 500 Hz)
- or, just move the camera around, looking for something that flickers

Unknown frequency?

What if you've got a flickering light, but you don't know its frequency? You can measure it with any ML-enabled camera, as we already know the sensor readout timings:
- open the FPS override submenu, without actually enabling it (i.e. select FPS override in ML menu, leave it OFF and press Q)
- look for Main Clock => camera-specific constant (5D2 and 5D3 use 24 MHz, 700D/650D/M/100D use 32 MHz and so on)
- look for Timer A => this gives line readout speed. Timer A / Main Clock = line readout time. Example: 5D2 25p => 600 / 24 MHz = 25 microseconds per line.
- look for Timer B => this gives frame rate: FPS = Main Clock / Timer A / Timer B.
- write down these values; the Octave script below will do the math for you.

Octave script

I've prepared a small Octave script to perform this measurement: rolling.m

Requirements:
- Octave 4.x with the following packages: image and signal
- to analyze DNG files, you will also need read_raw.m and dcraw

You can either run the measurements on your own (caveat: the script may require some fiddling), or you can upload test images for me to analyze.

Sample test images

All converted from silent picture DNGs:
5D3-500hz-25p.jpg (using pwm.ino at 500 Hz)
5D3-500hz-24p.jpg (same light source)
5D2-100hz-25p-weak.jpg (mains frequency, very weak light, but still usable)

Sample output


octave rolling.m 5D3-500hz-25p.jpg 24e6 480
Using blue channel.
Pattern repeats every 100 lines (method: pwelch).
Pattern repeats every 100 lines (method: overlap).
Pattern repeats every 100 lines (method: zerocross).
Method: pwelch => 100.00 lines
Line readout clock: 50.00 kHz, i.e. 20.00 μs/line (known).
Light source frequency: 500.01 Hz (measured).

octave rolling.m 5D3-500hz-24p.jpg 24e6 440
Using blue channel.
Pattern repeats every 109 lines (method: pwelch).
Pattern repeats every 109 lines (method: overlap).
Pattern repeats every 109 lines (method: zerocross).
Method: pwelch => 108.98 lines
Line readout clock: 54.55 kHz, i.e. 18.33 μs/line (known).
Light source frequency: 500.52 Hz (measured).

octave rolling.m 5D3-500hz-25p.jpg 500
Using blue channel.
Pattern repeats every 100 lines (method: pwelch).
Pattern repeats every 100 lines (method: overlap).
Pattern repeats every 100 lines (method: zerocross).
Method: pwelch => 100.00 lines
Light source frequency: 500.00 Hz (known).
Line readout clock: 49998.86 kHz, i.e. 20.00 μs/line (measured).
Rolling shutter: 25.80 ms for 1290 lines.

octave rolling.m 5D3-500hz-24p.jpg 500
Using blue channel.
Pattern repeats every 109 lines (method: pwelch).
Pattern repeats every 109 lines (method: overlap).
Pattern repeats every 109 lines (method: zerocross).
Method: pwelch => 108.98 lines
Light source frequency: 500.00 Hz (known).
Line readout clock: 54488.46 kHz, i.e. 18.35 μs/line (measured).
Rolling shutter: 23.67 ms for 1290 lines.

octave rolling.m 5D2-100Hz-25p-weak.jpg 24e6 600
Using red channel.
Pattern repeats every 401 lines (method: pwelch).
Pattern repeats every 20 lines (method: overlap).
Pattern repeats every 21 lines (method: zerocross).
Method: pwelch => 400.68 lines
Line readout clock: 40.00 kHz, i.e. 25.00 μs/line (known).
Light source frequency: 99.83 Hz (measured).

# From the blog post: "So, the scan time is a bit over 60 milliseconds [...]"
wget https://blog.kasson.com/wp-content/uploads/2018/10/Z702693.jpg
mogrify -resize "8256x5504" Z702693.jpg
octave rolling.m -v Z702693.jpg 120
Using red channel.
Vignette fix...
Pattern repeats every 736 lines (method: pwelch).
Pattern repeats every 20 lines (method: overlap).
Pattern repeats every 733 lines (method: zerocross).
Method: pwelch => 736.36 lines
Light source frequency: 120.00 Hz (known).
Line readout clock: 88363.15 kHz, i.e. 11.32 μs/line (measured).
Rolling shutter: 62.29 ms for 5504 lines.


Accuracy?

That depends on:
- how stable your test frequency is (mains frequency is probably +/- 1%, maybe more);
- how accurate our stripe size measurement is (let's say +/- 1 pixel, so it depends on stripe size).

A few tests with 5D3 at 1080p25 with the same 500Hz test light, input files 1 2 3 4:

for f in *.DNG; do octave rolling.m $f 500 | grep "measured"; done
Line readout clock: 50123.14 kHz, i.e. 19.95 μs/line (measured).
Line readout clock: 49898.92 kHz, i.e. 20.04 μs/line (measured).
Line readout clock: 49932.19 kHz, i.e. 20.03 μs/line (measured).
Line readout clock: 50137.52 kHz, i.e. 19.95 μs/line (measured).


Not that bad.

Wanted: test images

I'm looking for some test images from recent models not yet running ML, i.e. all DIGIC 6 and newer (including, but not limited to, 80D, 750D/760D, 7D2, 5D4, 5DS, 200D, 800D, 77D, 6D2, M50, EOS R), cross-checked with images from a camera already running ML.

Test conditions:
- blank wall under flickering light, without stray objects (e.g. please don't include the light bulb)
- focus doesn't matter (the script will average every row of the image)
- lens doesn't matter at all (you can perform the experiment without a lens if you want)
- choose any shutter speed that makes the flicker obvious (usually faster shutter speeds are preferred)
- ISO and aperture are not important; just make sure the image is reasonably clean

Test set should include:
- For the camera already running ML:
    - two simple DNG silent pictures from movie mode, one at 1080p24 and another at 1080p25
    - if you don't want to install ML on the test camera, a frame extracted from H.264 video will also work (in this case I'll have to guess the captured resolution)
    - I'll use these to measure (or double-check) the frequency of the light source.
- For the camera not (yet?) running ML:
    - a video frame (extracted from video) at each resolution x frame rate setting from Canon menu
    - if the camera has an option to take completely silent pictures in LiveView, please include one of these as well (full-res JPG)

I'll use these tests to estimate the sensor readout speed and to verify some hypotheses about LiveView configuration for 80D, 5D4 and other recent models I've looked into.

This method can be used with images from non-Canon cameras, too; feel free to submit them if you are curious, just be aware this won't magically bring ML to your camera :)

a1ex

First test: EOS R from kitor, using the 500Hz LED on Arduino Uno, cross-checked with 5D2. I've analyzed the first 8 frames from each test clip.

Results:

Silent still: 17.47 μs/line, 78.25 ms full readout at 6720x4480 => 385 MPix/second readout speed?
4K24/23.98/25/30: 14.45 +/- 0.019 μs/line, 31.2 ms at 3840x2160
1080p24/25/30/50/60: 14.12 +/- 0.067 μs/line, 15.3 ms at 1920x1080
720p25/30/50/60: 21.20 +/- 0.087 μs/line, 15.3 ms at 1280x720
720p100/120: 9.47 +/- 0.204 μs/line, 6.8 ms at 1280x720

Timings expected on 5D4, from this chart and related notes (some numbers were guessed):
- Silent still: 17.46 μs/line (1:1 readout)
- 4K24/25/30: 14.44 μs/line (readout appears to be 4104 x something; likely cropped to 4096x2160 without resizing, i.e. 1:1 readout)
- 1080p24/25/30/50/60: 12.10 μs/line in RAW, 12.10 * 1260 / 1080 = 14.117 μs/line in 1080p output
  (readout 2256x1266, possibly cropped to 2240x1260 or maybe 2244x1262 or something close)
- 720p50/60 (120p standby): 12.10 μs/line in RAW, 12.10 * 756 / 720 = 12.705 μs/line in hypothetical 720p output
  (guess: 3:2 squeezed readout with 5x3 ratio at 2256x902 cropped to 2240x896)
- 720p50/60 on the R: 12.10 μs/line in RAW, 12.10 * 1260 / 720 = 21.175 μs/line in 720p output
  (guess: same readout size as 1080p on the R)
- 720p120: 9.0 μs/line in RAW; expected readout size 2256x762 (5x3 squeeze), possibly cropped to 2240x756
  => 9.0 * 756/720 = 9.45 μs/line

Full details, including raw data and everything you need to reproduce the measurements:
rolling-eos-r.html

Levas

Quote from: a1ex on November 17, 2018, 10:45:02 PM
Silent still: 17.47 μs/line, 78.25 ms full readout at 6720x4480 => 385 MPix/second readout speed?

~12,5 frames per second in Full resolution  :)
As in ~16 MPix frames (5K/6K) in 24 fps  :D



a1ex

Yeah, I had this hypothesis earlier after looking into 5D4 firmware, but could not believe my own eyes :)

Quote from: a1ex on October 16, 2018, 10:28:42 AM
     - 1746: same as with 2194; 12.57 FPS?! 380 Mp/s?! [take with a grain of salt; not confirmed experimentally]

Looking forward to test images from 5D4 and M50, as these models also appear to have a silent shooting mode.

reddeercity

This is very interesting !  :D
Actually I was going start a thread asking questions on Sensor readout & frequency -- frame rate etc. ...
But it seems you have started it , that great .

Quoteoctave rolling.m 5D2-100Hz-25p-weak.jpg 24e6 600
Line readout clock: 40.00 kHz, i.e. 25.00 μs/line (known).
So compared to 5d3 & 5d4 the 5d2 readout is very very  slow , if I understand correctly .
Need to learn more about this .

reddeercity


a1ex

Quote from: reddeercity on November 18, 2018, 04:13:17 AM
So compared to 5d3 & 5d4 the 5d2 readout is very very  slow , if I understand correctly .

Line readout speed is given by Timer A / Main Clock (from FPS override). The latter is constant, so it all depends on how far you can push timer A (lower values = higher speeds). For the entire image, you would just multiply the microseconds/line with the image height.

Both 5D2 and 5D3 have a 24MHz clock, but use different values of timer A.

5D2: default is 600 in 25p and 572 in 24/30p; that means, with default Canon firmware, 25p is read out slower than 24p/30. There's no good reason for that; you can already reprogram the timers from FPS override to reduce rolling shutter. In plain 1080p, timer A can go down to 552 units according to old notes from fps-engio.c (2012); need to double-check. One timer A unit reads out 4 horizontal pixels; there is some overhead, not sure where it comes from.

5D3: default is 480 in 25p and 440 in 24/30p. Both can be "overclocked" to 398 without introducing image artifacts. Some numbers in this thread. One timer A unit reads out 8 horizontal pixels; the overhead is even larger; it might have something to do with binning.

In other words, in 1080p, 5D2 sensor is about 40% slower than 5D3. In photo mode, 5D2 reads out 5792x3804 at 4.2 FPS (from this post, 24e6 / (0x5db+1) / (0xedc+1) = 4.2), while 5D3 does 5936x3950 at 7.65 FPS, so the former is about twice as slow.

5D4/R appear to have similar configurations (the matched timings are unlikely to be just a coincidence, but let's wait for the measurements). My initial impression is that sensor readout is somehow throttled in video mode, possibly to avoid overheating; the silent mode uses a pretty fast readout, and I'd expect it to match the maximum sensor readout speed.

I've got some graphs over here, some more complete than others:



The red-black diagonal on the 5D3 graph shows exactly when the readout process takes place. Note to self: publish the code to get this graph from other models.

Quote from: reddeercity on November 19, 2018, 03:45:05 AM
Seems others have looked in to this with a EOS R --  https://www.dpreview.com/forums/post/61723181

Very cool. This shows just how weak my google-fu is :)

I came across this measurement before running the test, but wasn't aware of this dpreview post.




Expecting something similar from M50 (possibly 27 MHz for the main clock):


timer A =  772, B = 4061 (6176x4061   8.612 fps?) @ e19ebb78
timer A =  534, B = 4061 (6408x4061  12.451 fps?) @ e19ebbd0


The exact same numbers (at different addresses) can be found in M5, M6, 200D, 800D, 77D; are these using the same sensor?!

First line also matches the 80D, which doesn't have a silent capture option IIRC, but why it has two different readout speeds?!

Quote from: a1ex on April 24, 2018, 04:41:05 PM

6288 x 4057 (timer A = 772, B = 4061, 8.612 FPS) @ feb47c28
6288 x 4057 (timer A = 1064, B = 4061, 6.249 FPS) @ feb47cb8


Let's measure them to find out what is actually used.

reddeercity

Great info , I'm getting a better understanding of the readout & head timers .
This helps me.

core51

I can do any thing to help on 80d. But am noon is there any way to instruct me step by step how to help u. Am ready.

core51

I can do any thing to help on 80d. But am noon is there any way to instruct me step by step how to help u. Am ready.

Walter Schulz


horshack

Derp, never mind. Mistook Hz for 2x freq of light transitions at a given Hz.