Author Topic: [SOLVED!] Dead pixels - Hot pixels - Banding in RAW recording  (Read 102826 times)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #100 on: May 29, 2013, 07:21:27 PM »
It's a mistery why some users are getting banding and others don't. I've also cleared all Canon settings and I get perfect image without any signs of banding or bad pixels, even with extreme curves.

The artifacts look somewhat like what I've got with vignetting correction. My lens (50/1.8) has no correction data on 5D3. Maybe it's related?

nandoide

  • New to the forum
  • *
  • Posts: 41
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #101 on: May 29, 2013, 10:58:08 PM »
I've done a lot of experiments changing parameters of the camera, in canon menus, trying different lenses. All over 6D and hot pixels are there with low fps (6 fps for example) with fps override and low light optimization for example. Fast TV also has the problem but less noticeable.

Well, finally we observe a subtle transition on lv image when I go from 1x to 5x or 5x to 1x. For a lapse of 1/2 second or so, the image looks OK after change zoom mode, and suddenly the display view adquires some kind of digital gain and the hot pixels appear (and perhaps the banding issue and other image issues).  If it where possible to deactivate the second behaviour ...

On 550D there are no trace of these issues.















 

squig

  • Hero Member
  • *****
  • Posts: 518
  • Crash test dummy MK3
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #102 on: May 29, 2013, 11:02:44 PM »
If people don't start providing us with more info we won't be able to get to the bottom of this. I need to know all your shot settings, lens, CF card type, battery type, ML build number, and camera serial number.  Also upload a DNG of the banding because some people confuse the stripes/banding with FPN and I need to be sure it's stripes.

A1ex and I haven't been able to produce the stripes/banding on our cameras which suggests something is interacting with ML and triggering the stripes. Once a number of people provide the info requested hopefully a pattern will emerge.

A1ex somebody mentioned they tried an auto and a manual lens.  I don't have any auto glass. The stripes do look like some kind of interference and AF lens was my first suspect. Are you guys anywhere near a phone tower? Just humour me and go shoot in a forest.  ???

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #103 on: May 29, 2013, 11:29:38 PM »
the image is clear when switching zoom modes because override is reset. same thing as hitting play to reset bad fps override.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #104 on: May 30, 2013, 12:09:08 AM »
FPS override has nothing to do with this problem; these stripes are not noise, but a fixed pattern with 8-pixel periodicity.

A closer look reveals that banding is present only in blue and one of the green channels, at least with this dng: http://webpage-design.org/000379.dng

dcraw -4 -d 000379-1.dng

Bayer pattern:
RG
GB

Left column is OK, right is bad. The difference seems to be an EV factor (gaps are higher in bright areas). Probably easy to fix if we know exactly when it happens.

Correction factors for top-right green channel: 1, 1.0087, 1.0015, 0.9889, repeat.
Correction factors for blue channel: 1, 1.0080, 1.0017, 0.9882, repeat.

Multiply the bad channels with these numbers and you get perfect image.


1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #105 on: May 30, 2013, 12:17:21 AM »
we got similar patterns from fps override. same lines at regular intervals.


try lv_md_visible,0 i guess and then maybe will have defc + sprinkles off?

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #106 on: May 30, 2013, 12:22:28 AM »
FPS override patterns contain noise and increases over time; this one is only a luma multiplier.

I'd like a few RAW files to take a closer look. They should be from different users, with different settings, but they all should show banding. Don't upload large stuff, 1-2 seconds of video is enough.

jasondhines

  • New to the forum
  • *
  • Posts: 15
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #107 on: May 30, 2013, 08:04:47 AM »
5D3 + 24-70- 2.8 II + 64GB 1000X Lexar

F/4 1/50th ISO 1600  Files were both set to WB 2900, +22 Magenta and +1.0 Exposure

Banding is present on the May 28th build. Also, not sure why I can only record 50 frames or so until it frame skips with the May28th build, which wasn't even usable.

May 22nd Build  link - https://drive.google.com/folderview?id=0B3TpGf3-Q35ETl9PczVFTFl3cU0&usp=sharing

May 28th Build link - https://drive.google.com/folderview?id=0B3TpGf3-Q35EeldRY1JFYTdtS0k&usp=sharing


a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #108 on: May 30, 2013, 08:18:07 AM »
I did not ask for DNGs or for comparison between builds; I've asked for raw files that show banding.

squig

  • Hero Member
  • *****
  • Posts: 518
  • Crash test dummy MK3
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #109 on: May 30, 2013, 08:31:39 AM »
I did not ask for DNGs or for comparison between builds; I've asked for raw files that show banding.

I asked for DNGs.

Jason that's just stock standard FPN, the 28th shots are a bit noisier because they're a bit more underexposed. I don't see any of the stripes/banding in those shots. Set the luma denoising slider to 50 in ACR and like magic it disappears.

squig

  • Hero Member
  • *****
  • Posts: 518
  • Crash test dummy MK3
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #110 on: May 30, 2013, 08:37:08 AM »
a1ex this is all your fault, you upped the res so much now we can see all the crap on the sensor!  ::)

jasondhines

  • New to the forum
  • *
  • Posts: 15
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #111 on: May 30, 2013, 08:39:01 AM »
I asked for DNGs.

Jason that's just stock standard FPN, the 28th shots are a bit noisier because they're a bit more underexposed. I don't see any of the stripes/banding in those shots. Set the luma denoising slider to 50 in ACR and like magic it disappears.

Well I'm not really understanding why the exposures should be any different, identical settings, identical controlled lighting. The 28th build clearly is worse and I cannot record more than 70 frames with it for some reason.

jasondhines

  • New to the forum
  • *
  • Posts: 15
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #112 on: May 30, 2013, 08:41:37 AM »
I did not ask for DNGs or for comparison between builds; I've asked for raw files that show banding.


https://docs.google.com/file/d/0B3TpGf3-Q35EODd2ZFVMVllpaUE/edit?usp=sharing

LucaBrasi

  • Freshman
  • **
  • Posts: 70
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #113 on: May 30, 2013, 09:19:41 AM »
I'd like a few RAW files to take a closer look. They should be from different users, with different settings, but they all should show banding. Don't upload large stuff, 1-2 seconds of video is enough.

Canon 5D Mark III
Canon EF 16-35 2.8L USM II
(The tint is due to Genus ND filter)

ML May 29th(5419e04)
ISO 100
Aparture 5.6
fps override Off
fps 23.976

DNG https://www.dropbox.com/s/d2yb2ycg20vtpbo/M30-0910000000.dng
RAW https://www.dropbox.com/s/36e4284k5jarzx5/M30-0910.RAW

Did a test with fps override 24 for 24 Exact but no change.


ML Magic Lantern May 22nd(807981e)
ISO 100
Aparture 5.6
fps override Off
fps 23.976

DNG https://www.dropbox.com/s/5bnirpd2heexuug/M0000000000000.dng
RAW https://www.dropbox.com/s/yf0hggsrqg6x2v0/M0000000.RAW

Vertical banding gone in 22may!

bumkicho

  • Member
  • ***
  • Posts: 102
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #114 on: May 30, 2013, 09:37:02 AM »
@a1ex,
Are you using the same ML files that lourenco kindly compiles for us every day? same source file and same camera, but not the same result. I am wondering if there is a possibility of this issue entering during compilation. I appreciate what lourenco does, I really do. I am just thinking out loud.

squig

  • Hero Member
  • *****
  • Posts: 518
  • Crash test dummy MK3
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #115 on: May 30, 2013, 09:39:56 AM »
Luca your sample has the stripes/banding but the interesting thing is it responds identically to the same level of luma denoising as Jasons sample. Do you have a manual focus lens you can try the same test with?

I'm using the  lourenco builds and I've never had the sitripes/banding.

bumkicho

  • Member
  • ***
  • Posts: 102
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #116 on: May 30, 2013, 09:50:42 AM »
I'm using the  lourenco builds and I've never had the sitripes/banding.

That could very well be a poor speculation out of my ignorance.

LucaBrasi

  • Freshman
  • **
  • Posts: 70
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #117 on: May 30, 2013, 09:50:50 AM »
Do you have a manual focus lens you can try the same test with?

Sorry I wish I had a nice Zeiss prime, but no. my autofocus lens set to manual focus tho :D

bumkicho

  • Member
  • ***
  • Posts: 102
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #118 on: May 30, 2013, 10:05:03 AM »
Just tested with my tamron 24-70 and with my cheap nikon 35mm manual lens. There isn't much difference.

@squig, are you looking at your dng files without noise reduction?

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #119 on: May 30, 2013, 10:07:44 AM »
I think I have a fix. Download latest raw2dng from the sticky topic, try it and post some conversion logs (copy/paste the output from the terminal).

squig

  • Hero Member
  • *****
  • Posts: 518
  • Crash test dummy MK3
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #120 on: May 30, 2013, 10:10:45 AM »
Just tested with my tamron 24-70 and with my cheap nikon 35mm manual lens. There isn't much difference.

@squig, are you looking at your dng files without noise reduction?

Affirmative.

squig

  • Hero Member
  • *****
  • Posts: 518
  • Crash test dummy MK3
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #121 on: May 30, 2013, 10:12:24 AM »
Sorry I wish I had a nice Zeiss prime, but no. my autofocus lens set to manual focus tho :D

(If a1ex's fix doesn't work) shoot a few frames @ 100 ISO with your lens cap on and then remove the lens and shoot a few more with the body cap on. Identical settings both times. Upload the first dng from both.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dead pixels - Hot pixels - Banding in RAW recording
« Reply #122 on: May 30, 2013, 10:47:41 AM »
Jason that's just stock standard FPN, the 28th shots are a bit noisier because they're a bit more underexposed.

Try the new converter and you'll have a little surprise ;)

info@actionstudio.re

  • New to the forum
  • *
  • Posts: 9
Re: [SOLVED?] Dead pixels - Hot pixels - Banding in RAW recording
« Reply #123 on: May 30, 2013, 11:12:00 AM »
May 28, 2013, i test the Magic Lantern May 28th (7fdfc08) : Perfect for me !!!
Vertical banding : FIXED
Hot pixels : FIXED

And finally, i test the Magic Lantern May 28th (8ba641e) : I have Vertical Banding ... grrrr !

Today i reset my 5D MKIII "clear all camera settings" and i run Magic Lantern May 28th(8ba641e) without global draw (Global Draw OFF), without FPS (set to OFF). And vertical banding is fixed ...

Now, i should test with Global Draw ...

Lens : Tamron 24-70 2.8 VC

EDIT : OK, i active GLOBAL DRAW, and the image is perfect ... I think that the "RESET" fixe my Vertical Banding ...

fascina

  • New to the forum
  • *
  • Posts: 27
  • 5D Mark III
Re: [SOLVED?] Dead pixels - Hot pixels - Banding in RAW recording
« Reply #124 on: May 30, 2013, 11:36:04 AM »
I reshot today with newest build (5419e04). I used manual lens (Nikon 50 mm). ISO 100, 1/50th shutter. 8-Pixel Vertical Banding is still there:

http://www.webpage-design.org/M30-0948.RAW

Thanks Alex so much for updating the RAW2Dng, but I just tried and 8-Pix-Vertical-Banding is still there. Please also try with the RAW file above.

8-Pixel-Vertical-Banding is not FPN and is not coming from the sensor, because at RAW Type 19 it completely disappears (but color pixels appear).

Those who don't see the 8-Pixel-Vertical-Banding in their footage maybe don't shoot the right subjects in order to see it. One has to slightly underexpose an even background (like sky or even wall) to see it.

Luma denoising slider to 50 indeed makes the banding less visible, but it also takes away a lot of grain and shadow details, making the footage much more look like compressed.