[SOLVED!] Dead pixels - Hot pixels - Banding in RAW recording

[a1ex]

It's a mistery why some users are getting banding and others don't. I've also cleared all Canon settings and I get perfect image without any signs of banding or bad pixels, even with extreme curves.

The artifacts look somewhat like what I've got with vignetting correction. My lens (50/1. has no correction data on 5D3. Maybe it's related?

[nandoide]

I've done a lot of experiments changing parameters of the camera, in canon menus, trying different lenses. All over 6D and hot pixels are there with low fps (6 fps for example) with fps override and low light optimization for example. Fast TV also has the problem but less noticeable.

Well, finally we observe a subtle transition on lv image when I go from 1x to 5x or 5x to 1x. For a lapse of 1/2 second or so, the image looks OK after change zoom mode, and suddenly the display view adquires some kind of digital gain and the hot pixels appear (and perhaps the banding issue and other image issues).  If it where possible to deactivate the second behaviour ...

On 550D there are no trace of these issues.















 

[squig]

If people don't start providing us with more info we won't be able to get to the bottom of this. I need to know all your shot settings, lens, CF card type, battery type, ML build number, and camera serial number.  Also upload a DNG of the banding because some people confuse the stripes/banding with FPN and I need to be sure it's stripes.

A1ex and I haven't been able to produce the stripes/banding on our cameras which suggests something is interacting with ML and triggering the stripes. Once a number of people provide the info requested hopefully a pattern will emerge.

A1ex somebody mentioned they tried an auto and a manual lens.  I don't have any auto glass. The stripes do look like some kind of interference and AF lens was my first suspect. Are you guys anywhere near a phone tower? Just humour me and go shoot in a forest. 

[1%]

the image is clear when switching zoom modes because override is reset. same thing as hitting play to reset bad fps override.

[a1ex]

FPS override has nothing to do with this problem; these stripes are not noise, but a fixed pattern with 8-pixel periodicity.

A closer look reveals that banding is present only in blue and one of the green channels, at least with this dng: http://webpage-design.org/000379.dng

dcraw -4 -d 000379-1.dng

Bayer pattern:
RG
GB

Left column is OK, right is bad. The difference seems to be an EV factor (gaps are higher in bright areas). Probably easy to fix if we know exactly when it happens.

Correction factors for top-right green channel: 1, 1.0087, 1.0015, 0.9889, repeat.
Correction factors for blue channel: 1, 1.0080, 1.0017, 0.9882, repeat.

Multiply the bad channels with these numbers and you get perfect image.

[1%]

we got similar patterns from fps override. same lines at regular intervals.


try lv_md_visible,0 i guess and then maybe will have defc + sprinkles off?

[a1ex]

FPS override patterns contain noise and increases over time; this one is only a luma multiplier.

I'd like a few RAW files to take a closer look. They should be from different users, with different settings, but they all should show banding. Don't upload large stuff, 1-2 seconds of video is enough.

[jasondhines]

5D3 + 24-70- 2.8 II + 64GB 1000X Lexar

F/4 1/50th ISO 1600  Files were both set to WB 2900, +22 Magenta and +1.0 Exposure

Banding is present on the May 28th build. Also, not sure why I can only record 50 frames or so until it frame skips with the May28th build, which wasn't even usable.

May 22nd Build  link - https://drive.google.com/folderview?id=0B3TpGf3-Q35ETl9PczVFTFl3cU0&usp=sharing

May 28th Build link - https://drive.google.com/folderview?id=0B3TpGf3-Q35EeldRY1JFYTdtS0k&usp=sharing

[a1ex]

I did not ask for DNGs or for comparison between builds; I've asked for raw files that show banding.

[squig]

I did not ask for DNGs or for comparison between builds; I've asked for raw files that show banding.

I asked for DNGs.

Jason that's just stock standard FPN, the 28th shots are a bit noisier because they're a bit more underexposed. I don't see any of the stripes/banding in those shots. Set the luma denoising slider to 50 in ACR and like magic it disappears.

[squig]

a1ex this is all your fault, you upped the res so much now we can see all the crap on the sensor! 

[jasondhines]

I asked for DNGs.

Jason that's just stock standard FPN, the 28th shots are a bit noisier because they're a bit more underexposed. I don't see any of the stripes/banding in those shots. Set the luma denoising slider to 50 in ACR and like magic it disappears.

Well I'm not really understanding why the exposures should be any different, identical settings, identical controlled lighting. The 28th build clearly is worse and I cannot record more than 70 frames with it for some reason.

[jasondhines]

I did not ask for DNGs or for comparison between builds; I've asked for raw files that show banding.

https://docs.google.com/file/d/0B3TpGf3-Q35EODd2ZFVMVllpaUE/edit?usp=sharing

[LucaBrasi]

I'd like a few RAW files to take a closer look. They should be from different users, with different settings, but they all should show banding. Don't upload large stuff, 1-2 seconds of video is enough.

Canon 5D Mark III
Canon EF 16-35 2.8L USM II
(The tint is due to Genus ND filter)

ML May 29th(5419e04)
ISO 100
Aparture 5.6
fps override Off
fps 23.976

DNG https://www.dropbox.com/s/d2yb2ycg20vtpbo/M30-0910000000.dng
RAW https://www.dropbox.com/s/36e4284k5jarzx5/M30-0910.RAW

Did a test with fps override 24 for 24 Exact but no change.


ML Magic Lantern May 22nd(807981e)
ISO 100
Aparture 5.6
fps override Off
fps 23.976

DNG https://www.dropbox.com/s/5bnirpd2heexuug/M0000000000000.dng
RAW https://www.dropbox.com/s/yf0hggsrqg6x2v0/M0000000.RAW

Vertical banding gone in 22may!

[bumkicho]

@a1ex,
Are you using the same ML files that lourenco kindly compiles for us every day? same source file and same camera, but not the same result. I am wondering if there is a possibility of this issue entering during compilation. I appreciate what lourenco does, I really do. I am just thinking out loud.

[squig]

Luca your sample has the stripes/banding but the interesting thing is it responds identically to the same level of luma denoising as Jasons sample. Do you have a manual focus lens you can try the same test with?

I'm using the  lourenco builds and I've never had the sitripes/banding.

[bumkicho]

I'm using the  lourenco builds and I've never had the sitripes/banding.

That could very well be a poor speculation out of my ignorance.

[LucaBrasi]

Do you have a manual focus lens you can try the same test with?

Sorry I wish I had a nice Zeiss prime, but no. my autofocus lens set to manual focus tho

[bumkicho]

Just tested with my tamron 24-70 and with my cheap nikon 35mm manual lens. There isn't much difference.

@squig, are you looking at your dng files without noise reduction?

[a1ex]

I think I have a fix. Download latest raw2dng from the sticky topic, try it and post some conversion logs (copy/paste the output from the terminal).

[squig]

Just tested with my tamron 24-70 and with my cheap nikon 35mm manual lens. There isn't much difference.

@squig, are you looking at your dng files without noise reduction?

Affirmative.

[squig]

Sorry I wish I had a nice Zeiss prime, but no. my autofocus lens set to manual focus tho

(If a1ex's fix doesn't work) shoot a few frames @ 100 ISO with your lens cap on and then remove the lens and shoot a few more with the body cap on. Identical settings both times. Upload the first dng from both.

[a1ex]

Jason that's just stock standard FPN, the 28th shots are a bit noisier because they're a bit more underexposed.

Try the new converter and you'll have a little surprise

[info@actionstudio.re]

May 28, 2013, i test the Magic Lantern May 28th (7fdfc08) : Perfect for me !!!
Vertical banding : FIXED
Hot pixels : FIXED

And finally, i test the Magic Lantern May 28th (8ba641e) : I have Vertical Banding ... grrrr !

Today i reset my 5D MKIII "clear all camera settings" and i run Magic Lantern May 28th(8ba641e) without global draw (Global Draw OFF), without FPS (set to OFF). And vertical banding is fixed ...

Now, i should test with Global Draw ...

Lens : Tamron 24-70 2.8 VC

EDIT : OK, i active GLOBAL DRAW, and the image is perfect ... I think that the "RESET" fixe my Vertical Banding ...

[fascina]

I reshot today with newest build (5419e04). I used manual lens (Nikon 50 mm). ISO 100, 1/50th shutter. 8-Pixel Vertical Banding is still there:

http://www.webpage-design.org/M30-0948.RAW

Thanks Alex so much for updating the RAW2Dng, but I just tried and 8-Pix-Vertical-Banding is still there. Please also try with the RAW file above.

8-Pixel-Vertical-Banding is not FPN and is not coming from the sensor, because at RAW Type 19 it completely disappears (but color pixels appear).

Those who don't see the 8-Pixel-Vertical-Banding in their footage maybe don't shoot the right subjects in order to see it. One has to slightly underexpose an even background (like sky or even wall) to see it.

Luma denoising slider to 50 indeed makes the banding less visible, but it also takes away a lot of grain and shadow details, making the footage much more look like compressed.