Author Topic: Full-resolution silent pictures (silent.mo)  (Read 789260 times)

mk11174

  • Contributor
  • Hero Member
  • *****
  • Posts: 809
Re: Full-resolution silent pictures
« Reply #125 on: July 25, 2014, 08:00:26 PM »
Look for CONFIG_600D in raw.c, and the last block has the offsets from photo mode. You can start from the values I've linked above, and maybe fine-tune a few pixels from there.
Ok, no problem, I know where they are in raw.c and how to adjust them, but was hoping for a starter point, thanks for that from the one post will give them a try and go from there and send you new screen when I get them correct.
500D/T1i  550D/T2i  600D/T3i  700D/T5i

mk11174

  • Contributor
  • Hero Member
  • *****
  • Posts: 809
Re: Full-resolution silent pictures
« Reply #126 on: July 25, 2014, 08:10:22 PM »
I was just curious on what a screen would look like from nightly and here is a screen with a straight build from nightly, does this mean there wrong in the nightly which is before the EDMAC change?

500D/T1i  550D/T2i  600D/T3i  700D/T5i

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Full-resolution silent pictures
« Reply #127 on: July 25, 2014, 08:12:53 PM »
Yep. This one is going to cause black level issues that can be very hard to track down, and ETTR is probably not much better than random exposure on this camera.

And here's how I'll fix it. I'll merge the full-res silent picture branch next week, and I'll break all the nightly builds for which I don't have a raw_diag screenshot.

escho

  • Contributor
  • Hero Member
  • *****
  • Posts: 563
Re: Full-resolution silent pictures
« Reply #128 on: July 25, 2014, 09:05:19 PM »
It's there, just checked the current nightly.

00cd1644 log2f

If it's not present in your custom build, you can add a call to it in builtin-enforcing.c.

Hmm, I compiled from latest ML-sources and math.h is included in builtin-enforcing.c. But the module isn´t loading. In the nightly the modul is loading. Don´t know, how to add a call in my build.
https://sternenkarten.com/
600D, 6D, openSUSE Tumbleweed

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Full-resolution silent pictures
« Reply #129 on: July 25, 2014, 09:07:40 PM »
Something like: float x = log2f(3.14);

escho

  • Contributor
  • Hero Member
  • *****
  • Posts: 563
Re: Full-resolution silent pictures
« Reply #130 on: July 25, 2014, 09:24:43 PM »
Sorry, don´t understand what to do. I must break this stuff, till I have got more understanding in coding.
https://sternenkarten.com/
600D, 6D, openSUSE Tumbleweed

mk11174

  • Contributor
  • Hero Member
  • *****
  • Posts: 809
Re: Full-resolution silent pictures
« Reply #131 on: July 25, 2014, 09:35:38 PM »
        #ifdef CONFIG_600D
        width = 5344; //From Guess Py
        height = 3465;
        skip_left = 142;
        skip_top = 50;
        #endif

To close to edge?????
500D/T1i  550D/T2i  600D/T3i  700D/T5i

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Full-resolution silent pictures
« Reply #132 on: July 25, 2014, 09:36:13 PM »
Maybe. Check the RAW zebras for borders.

mk11174

  • Contributor
  • Hero Member
  • *****
  • Posts: 809
Re: Full-resolution silent pictures
« Reply #133 on: July 25, 2014, 09:40:34 PM »
Maybe. Check the RAW zebras for borders.

Was this ok or is that a border at the top?
500D/T1i  550D/T2i  600D/T3i  700D/T5i

mk11174

  • Contributor
  • Hero Member
  • *****
  • Posts: 809
Re: Full-resolution silent pictures
« Reply #134 on: July 25, 2014, 10:16:34 PM »
Here is
        #ifdef CONFIG_600D
        width = 5344; //From Guess Py
        height = 3465;
        skip_left = 142;
        skip_top = 52;
        #endif


500D/T1i  550D/T2i  600D/T3i  700D/T5i

mk11174

  • Contributor
  • Hero Member
  • *****
  • Posts: 809
Re: Full-resolution silent pictures
« Reply #135 on: July 25, 2014, 10:28:58 PM »
Here is the 700d with the current skips.

        width = 5280;
        height = 3528;
        skip_left = 72;
        skip_top = 52;



500D/T1i  550D/T2i  600D/T3i  700D/T5i

mk11174

  • Contributor
  • Hero Member
  • *****
  • Posts: 809
Re: Full-resolution silent pictures
« Reply #136 on: July 25, 2014, 10:30:47 PM »
Oh, both these tests are with fullres-silent-pics build.
500D/T1i  550D/T2i  600D/T3i  700D/T5i

mk11174

  • Contributor
  • Hero Member
  • *****
  • Posts: 809
Re: Full-resolution silent pictures
« Reply #137 on: July 25, 2014, 10:40:27 PM »
Here is unified 700d screens
        width = 5280;
        height = 3528;
        skip_left = 72;
        skip_top = 52;


500D/T1i  550D/T2i  600D/T3i  700D/T5i

mk11174

  • Contributor
  • Hero Member
  • *****
  • Posts: 809
Re: Full-resolution silent pictures
« Reply #138 on: July 25, 2014, 11:07:38 PM »
Here is the 600d Unified build with tweaked skips

        width = 5344; //From Guess Py
        height = 3465;
        skip_left = 158;
        skip_top = 54;





Whats up with the white overlap on the left side???
500D/T1i  550D/T2i  600D/T3i  700D/T5i

ayshih

  • Contributor
  • Senior
  • *****
  • Posts: 266
Re: Full-resolution silent pictures
« Reply #139 on: July 25, 2014, 11:28:54 PM »
Whats up with the white overlap on the left side???
Don't worry about the current unified behavior.  For many cameras, the old value for RAW_PHOTO_EDMAC would often result in image buffers that needed to be skipped ahead by some amount (usually 16 pixels) to have proper alignment.  Once the fullres-silent-pics branch is merged, this will no longer be the case on most (or all?) cameras.
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: Full-resolution silent pictures
« Reply #140 on: July 26, 2014, 02:56:20 AM »
Everybody, please double-check the skip offsets in raw.c and post the confirmation screenshots. The ones from 600D are wrong.
60D:

mucher

  • Member
  • ***
  • Posts: 121
Re: Full-resolution silent pictures
« Reply #141 on: July 26, 2014, 05:38:25 AM »
Curiously enough to ask if that means that we will not need to press 5X button to go into full pixel mode RAW recording in the future?

If somebody "accidentally" finds how to port these raws through that jpg pipeline...and, combining with that 1/3 stop highlight reduction, and... wrap it with sound into MLV format...I have dropped down in bed to think more about it. Ciao ;)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Full-resolution silent pictures
« Reply #142 on: July 26, 2014, 07:21:28 AM »
Don't worry about the current unified behavior.  For many cameras, the old value for RAW_PHOTO_EDMAC would often result in image buffers that needed to be skipped ahead by some amount (usually 16 pixels) to have proper alignment.  Once the fullres-silent-pics branch is merged, this will no longer be the case on most (or all?) cameras.

Right. Some more background info about this change:

This value for EDMAC is the one used at photo capture step. The previous one was from some other postprocessing stage, and usually different on each camera. From the capture to the CR2 and then to LCD preview, there are a bunch of postprocessing steps, like cropping some borders, FPN correction, bad pixel correction, maybe digital ISO (e.g. on 5D2 at ISO > 1600) and who knows what else.

The fullres silent picture code only calls the photo capture step - but not the processing into CR2, YUV, JPG and so on. So, right now I'm sure I'll get the original RAW buffer, as captured, and the behavior will be consistent on all cameras. Furthermore, since the buffer has the data before any postprocessing, modifying the data in this buffer is very likely to end up in the CR2 and the JPG; before, this happened on some cameras, but not on others.

So, this was a good opportunity to make sure the photo raw behavior is consistent on all cameras.

Bad offsets could cause subtle issues that are unlikely to be reported as bugs, and very hard to track down (ML uses these black borders for noise analysis, so any exposure or postprocessing tool that relies on this would give inaccurate results - from slightly weird to totally broken).

There's still a minor issue with this approach: the DNG might be a few pixels larger than the CR2, so the skip offsets might not match. This will only be an issue if you will take a dark frame from a few hundreds of silent pictures, then find out it's not perfectly aligned with your CR2's. Should be easy to fix in post though, and those shooting astro certainly have the skills to do this.

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: Full-resolution silent pictures
« Reply #143 on: July 26, 2014, 04:45:33 PM »
If you want to see the same analysis for your camera, take the two test images with this script, then get the ADTG screenshot by following these steps.
60D:
silent.DNG   regular.CR2



I'm having trouble getting the octave script to work though, perhaps it something to do with this:
Code: [Select]
warning: your version of GraphicsMagick limits images to 8 bits per pixel

I just end up with a flat graph. I know there's some rolling shutter, you can see it visually just looking at the silent.DNG

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Full-resolution silent pictures
« Reply #144 on: July 26, 2014, 09:45:31 PM »
Yeah, you need a custom-compiled version of both octave and GraphicsMagick, and that's because the package maintainers on both Mac and major Linux distros chose to build the 8-bit version only.

http://wiki.octave.org/GraphicsMagick
http://marcelojoeng.blogspot.com/2012/11/compile-octave-using-1632-bits-colour.html

If you find a way to do this on Mac, let me know, because I'm preparing a tutorial on processing RAW images in Octave, and I'm a bit stuck because of this issue (the prebuilt versions will not run the tutorial).



Code: [Select]
Exposure range  : 1/86.47 ... 1/5.55
Rolling shutter : 0.17 seconds

The timer math is left as an exercise to the reader. The 60D sensor seems fairly fast, probably required for high burst rates.

barepixels

  • Member
  • ***
  • Posts: 134
Re: Full-resolution silent pictures
« Reply #145 on: July 26, 2014, 10:19:44 PM »
I don't know what am doing but I installed the module, then made full-res silent dng mode active then took a pic from LCD

Here is the result

5D2
5D2 + nightly ML

mgrittani

  • New to the forum
  • *
  • Posts: 6
Re: Full-resolution silent pictures
« Reply #146 on: July 27, 2014, 06:03:50 AM »
Wondering if someone could post a link to the compiled module and core.  I'm on a 5D2.

Many thanks!

ayshih

  • Contributor
  • Senior
  • *****
  • Posts: 266
Re: Full-resolution silent pictures
« Reply #147 on: July 27, 2014, 07:25:06 AM »
Here are results from the 50D.  For this particular model, I note that TimerB, at 0xc9c=3228, is precisely the number of rows of the raw buffer, and thus is the number of times that TimerA should elapse.  That simplifies my conception of the timer math described earlier in this thread, making intuitive sense why it is that both TimerB and the vertical resolution are linked to the rolling-shutter time.

Code: [Select]
Exposure range  : 1/85.05 ... 1/6.66
Rolling shutter : 0.14 seconds



Timer math:
  • Rolling shutter
    • TimerA(0x4f5+1) / Clock(28.8 MHz) * Rows(3177) = 0.140 s
  • Constant exposure duration
    • TimerA(0x4f5+1) / Clock(28.8 MHz) * TimerB(0xe1+1) = 0.010 s
  • Shortest exposure for a row
    • Constant exposure + TimerA(0x4f5+1) / Clock(28.8 MHz) * OB rows(50) = 0.012 s
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Full-resolution silent pictures
« Reply #148 on: July 27, 2014, 09:03:01 AM »
Another fast sensor - didn't quite expect it.

If you consider the lower resolution...

Code: [Select]
// TimerA / Clock / Columns
5D3 : (0x317+1) / 24.0 / 5936 =  5.56 nanoseconds per pixel
50D : (0x4f5+1) / 28.8 / 4832 =  9.13 nanoseconds per pixel
60D : (0x57d+1) / 28.8 / 5344 =  9.14 nanoseconds per pixel
6D  : (0x597+1) / 25.6 / 5568 = 10.05 nanoseconds per pixel
5D2 : (0x5db+1) / 24.0 / 5792 = 10.79 nanoseconds per pixel
500d: (0xaf9+1) / 32.0 / 4832 = 18.17 nanoseconds per pixel

the 50D sensor is clocked just as fast as the 60D one (the speed difference is too small). So, the 50D should be a good candidate for higher FPS (60D's LiveView goes up to around 37 fps, but the H.264 encoder can no longer keep up).

LiveView clocks seem to be a little slower:
Code: [Select]
5D3: 440 / 24.0 / 2080 =  8.81 ns/px, overclocked 7.97
60D: 546 / 28.8 / 1880 = 10.08 ns/px, overclocked 9.90
50D: 696 / 28.8 / 1664 = 14.52 ns/px, overclocked 14.31

So, there must be a way to fine-tune the pixel clock somehow, and that should be found by comparing LV with photo capture code (how the hardware is configured in both cases).

edit: fixed a typo, thanks ayshih

marekk

  • Senior
  • ****
  • Posts: 331
  • www.oddajmisie.com
Re: Full-resolution silent pictures
« Reply #149 on: July 27, 2014, 01:54:23 PM »
I've got a stupid question ;) How to merge two branches ??

Now I'm using this commands in my script:

Code: [Select]
hg clone https://bitbucket.org/hudson/magic-lantern/
cd magic-lantern
hg update 5D3-123
hg merge unified

what should I change to add fullres-silents-pics branch ??