Full-resolution silent pictures (silent.mo)

Started by a1ex, July 01, 2014, 05:11:15 PM

Previous topic - Next topic

0 Members and 2 Guests are viewing this topic.

mk11174

Quote from: a1ex on July 25, 2014, 07:52:22 PM
Look for CONFIG_600D in raw.c, and the last block has the offsets from photo mode. You can start from the values I've linked above, and maybe fine-tune a few pixels from there.
Ok, no problem, I know where they are in raw.c and how to adjust them, but was hoping for a starter point, thanks for that from the one post will give them a try and go from there and send you new screen when I get them correct.
500D/T1i  550D/T2i  600D/T3i  700D/T5i

mk11174

I was just curious on what a screen would look like from nightly and here is a screen with a straight build from nightly, does this mean there wrong in the nightly which is before the EDMAC change?

500D/T1i  550D/T2i  600D/T3i  700D/T5i

a1ex

Yep. This one is going to cause black level issues that can be very hard to track down, and ETTR is probably not much better than random exposure on this camera.

And here's how I'll fix it. I'll merge the full-res silent picture branch next week, and I'll break all the nightly builds for which I don't have a raw_diag screenshot.

escho

Quote from: a1ex on July 25, 2014, 06:48:02 PM
It's there, just checked the current nightly.

00cd1644 log2f

If it's not present in your custom build, you can add a call to it in builtin-enforcing.c.

Hmm, I compiled from latest ML-sources and math.h is included in builtin-enforcing.c. But the module isn´t loading. In the nightly the modul is loading. Don´t know, how to add a call in my build.
https://sternenkarten.com/
600D, 6D, openSUSE Tumbleweed

a1ex

Something like: float x = log2f(3.14);

escho

Sorry, don´t understand what to do. I must break this stuff, till I have got more understanding in coding.
https://sternenkarten.com/
600D, 6D, openSUSE Tumbleweed

mk11174

        #ifdef CONFIG_600D
        width = 5344; //From Guess Py
        height = 3465;
        skip_left = 142;
        skip_top = 50;
        #endif

To close to edge?????
500D/T1i  550D/T2i  600D/T3i  700D/T5i

a1ex

Maybe. Check the RAW zebras for borders.

mk11174

Quote from: a1ex on July 25, 2014, 09:36:13 PM
Maybe. Check the RAW zebras for borders.

Was this ok or is that a border at the top?
500D/T1i  550D/T2i  600D/T3i  700D/T5i

mk11174

Here is
        #ifdef CONFIG_600D
        width = 5344; //From Guess Py
        height = 3465;
        skip_left = 142;
        skip_top = 52;
        #endif


500D/T1i  550D/T2i  600D/T3i  700D/T5i

mk11174

Here is the 700d with the current skips.

        width = 5280;
        height = 3528;
        skip_left = 72;
        skip_top = 52;



500D/T1i  550D/T2i  600D/T3i  700D/T5i

mk11174

Oh, both these tests are with fullres-silent-pics build.
500D/T1i  550D/T2i  600D/T3i  700D/T5i

mk11174

Here is unified 700d screens
        width = 5280;
        height = 3528;
        skip_left = 72;
        skip_top = 52;


500D/T1i  550D/T2i  600D/T3i  700D/T5i

mk11174

Here is the 600d Unified build with tweaked skips

        width = 5344; //From Guess Py
        height = 3465;
        skip_left = 158;
        skip_top = 54;





Whats up with the white overlap on the left side???
500D/T1i  550D/T2i  600D/T3i  700D/T5i

ayshih

Quote from: mk11174 on July 25, 2014, 11:07:38 PM
Whats up with the white overlap on the left side???
Don't worry about the current unified behavior.  For many cameras, the old value for RAW_PHOTO_EDMAC would often result in image buffers that needed to be skipped ahead by some amount (usually 16 pixels) to have proper alignment.  Once the fullres-silent-pics branch is merged, this will no longer be the case on most (or all?) cameras.
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

dmilligan

Quote from: a1ex on July 25, 2014, 03:36:55 PM
Everybody, please double-check the skip offsets in raw.c and post the confirmation screenshots. The ones from 600D are wrong.
60D:

mucher

Curiously enough to ask if that means that we will not need to press 5X button to go into full pixel mode RAW recording in the future?

If somebody "accidentally" finds how to port these raws through that jpg pipeline...and, combining with that 1/3 stop highlight reduction, and... wrap it with sound into MLV format...I have dropped down in bed to think more about it. Ciao ;)

a1ex

Quote from: ayshih on July 25, 2014, 11:28:54 PM
Don't worry about the current unified behavior.  For many cameras, the old value for RAW_PHOTO_EDMAC would often result in image buffers that needed to be skipped ahead by some amount (usually 16 pixels) to have proper alignment.  Once the fullres-silent-pics branch is merged, this will no longer be the case on most (or all?) cameras.

Right. Some more background info about this change:

This value for EDMAC is the one used at photo capture step. The previous one was from some other postprocessing stage, and usually different on each camera. From the capture to the CR2 and then to LCD preview, there are a bunch of postprocessing steps, like cropping some borders, FPN correction, bad pixel correction, maybe digital ISO (e.g. on 5D2 at ISO > 1600) and who knows what else.

The fullres silent picture code only calls the photo capture step - but not the processing into CR2, YUV, JPG and so on. So, right now I'm sure I'll get the original RAW buffer, as captured, and the behavior will be consistent on all cameras. Furthermore, since the buffer has the data before any postprocessing, modifying the data in this buffer is very likely to end up in the CR2 and the JPG; before, this happened on some cameras, but not on others.

So, this was a good opportunity to make sure the photo raw behavior is consistent on all cameras.

Bad offsets could cause subtle issues that are unlikely to be reported as bugs, and very hard to track down (ML uses these black borders for noise analysis, so any exposure or postprocessing tool that relies on this would give inaccurate results - from slightly weird to totally broken).

There's still a minor issue with this approach: the DNG might be a few pixels larger than the CR2, so the skip offsets might not match. This will only be an issue if you will take a dark frame from a few hundreds of silent pictures, then find out it's not perfectly aligned with your CR2's. Should be easy to fix in post though, and those shooting astro certainly have the skills to do this.

dmilligan

Quote from: a1ex on July 14, 2014, 11:32:29 PM
If you want to see the same analysis for your camera, take the two test images with this script, then get the ADTG screenshot by following these steps.
60D:
silent.DNG   regular.CR2



I'm having trouble getting the octave script to work though, perhaps it something to do with this:

warning: your version of GraphicsMagick limits images to 8 bits per pixel


I just end up with a flat graph. I know there's some rolling shutter, you can see it visually just looking at the silent.DNG

a1ex

Yeah, you need a custom-compiled version of both octave and GraphicsMagick, and that's because the package maintainers on both Mac and major Linux distros chose to build the 8-bit version only.

http://wiki.octave.org/GraphicsMagick
http://marcelojoeng.blogspot.com/2012/11/compile-octave-using-1632-bits-colour.html

If you find a way to do this on Mac, let me know, because I'm preparing a tutorial on processing RAW images in Octave, and I'm a bit stuck because of this issue (the prebuilt versions will not run the tutorial).




Exposure range  : 1/86.47 ... 1/5.55
Rolling shutter : 0.17 seconds


The timer math is left as an exercise to the reader. The 60D sensor seems fairly fast, probably required for high burst rates.

barepixels

I don't know what am doing but I installed the module, then made full-res silent dng mode active then took a pic from LCD

Here is the result

5D2
5D2 + nightly ML

mgrittani

Wondering if someone could post a link to the compiled module and core.  I'm on a 5D2.

Many thanks!

ayshih

Here are results from the 50D.  For this particular model, I note that TimerB, at 0xc9c=3228, is precisely the number of rows of the raw buffer, and thus is the number of times that TimerA should elapse.  That simplifies my conception of the timer math described earlier in this thread, making intuitive sense why it is that both TimerB and the vertical resolution are linked to the rolling-shutter time.

Exposure range  : 1/85.05 ... 1/6.66
Rolling shutter : 0.14 seconds




Timer math:

  • Rolling shutter

    • TimerA(0x4f5+1) / Clock(28.8 MHz) * Rows(3177) = 0.140 s
  • Constant exposure duration

    • TimerA(0x4f5+1) / Clock(28.8 MHz) * TimerB(0xe1+1) = 0.010 s
  • Shortest exposure for a row

    • Constant exposure + TimerA(0x4f5+1) / Clock(28.8 MHz) * OB rows(50) = 0.012 s
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

a1ex

Another fast sensor - didn't quite expect it.

If you consider the lower resolution...


// TimerA / Clock / Columns
5D3 : (0x317+1) / 24.0 / 5936 =  5.56 nanoseconds per pixel
50D : (0x4f5+1) / 28.8 / 4832 =  9.13 nanoseconds per pixel
60D : (0x57d+1) / 28.8 / 5344 =  9.14 nanoseconds per pixel
6D  : (0x597+1) / 25.6 / 5568 = 10.05 nanoseconds per pixel
5D2 : (0x5db+1) / 24.0 / 5792 = 10.79 nanoseconds per pixel
500d: (0xaf9+1) / 32.0 / 4832 = 18.17 nanoseconds per pixel


the 50D sensor is clocked just as fast as the 60D one (the speed difference is too small). So, the 50D should be a good candidate for higher FPS (60D's LiveView goes up to around 37 fps, but the H.264 encoder can no longer keep up).

LiveView clocks seem to be a little slower:

5D3: 440 / 24.0 / 2080 =  8.81 ns/px, overclocked 7.97
60D: 546 / 28.8 / 1880 = 10.08 ns/px, overclocked 9.90
50D: 696 / 28.8 / 1664 = 14.52 ns/px, overclocked 14.31


So, there must be a way to fine-tune the pixel clock somehow, and that should be found by comparing LV with photo capture code (how the hardware is configured in both cases).

edit: fixed a typo, thanks ayshih

marekk

I've got a stupid question ;) How to merge two branches ??

Now I'm using this commands in my script:

hg clone https://bitbucket.org/hudson/magic-lantern/
cd magic-lantern
hg update 5D3-123
hg merge unified


what should I change to add fullres-silents-pics branch ??