Dead pixel help raw video

[ericravern35]

Hi

  I am using the raw video for the first time on my 60d, and the dng's have lots of little red/blue dots in them, which from my reading on here looks to be dead pixels, and I have never seen them before cause the camera gets rids of them but obviously they are in the raw, I hunted on her for a answer on how to get rid of them, either in the camera or in post, I found a post which talked about chdk, but could not understand where I would need to put that on the card director, any help would be good. Any programs that can do the same thing the camera does when shooting stills of h2.64 video to hide these little bastards?
Cheers

Eric

[painya]

http://www.magiclantern.fm/forum/index.php?topic=6658.0;topicseen
I hope this helps 

[ericravern35]

I tried a whole bunch of things from that link, I had no luck with any, I got no idea how to post a screengrab on here, but I will upload some video on vimeo tomorrow in case I am talking about something else. certainly looks like dead pixels to me, but none of those programs seem to help much.
cheers

[iamoui]

I tried a whole bunch of things from that link, I had no luck with any, I got no idea how to post a screengrab on here, but I will upload some video on vimeo tomorrow in case I am talking about something else. certainly looks like dead pixels to me, but none of those programs seem to help much.
cheers

Search the 60D raw thread. Turn lv_af_raw on to avoid dead/hot/stuck pixels if you're not using ACR in your post workflow. ACR will automatically remove the dead pixels.

[Danne]

The hard way
https://www.youtube.com/watch?v=fBxyx-1b188&feature=youtube_gdata_player

[ericravern35]

Hi again
    Thanks for responding, so excuse my ignorance how do I switch on lv_af_raw, is in the menu? or am I changing some text somewhere?
Cheers in advance

[iamoui]

Hi again
    Thanks for responding, so excuse my ignorance how do I switch on lv_af_raw, is in the menu? or am I changing some text somewhere?
Cheers in advance

If you have any of the latest builds from marekk it should be in the raw video menu. Just a heads up, though, if you keep lv_af_raw on all the time, I would switch it to "off" then back "on" again every time you turn your camera on. Bit of a pain but even though it shows that it's "on" when you turn the camera on, it isn't.

[maxotics]

Also, ufRAW has a feature that will interpolate around hot/dead/bad pixels so you can try that for your dng to tiff conversion.  I write about it at the end of that thread. I have seen those blue dots on my 50D in clipped areas.

[hjfilmspeed]

I have a question about this. So i have been shooting with fps override at like .750 fps and i have been noticing a ton of dead pixels or stuck pixels like i would say 20 plus. I dont remember ever seeing this many. Is it that im using a combo of high iso and long exposure that its causing pixels to stick or die or is this a bad sign that im killing pixels or that there allot of dust on my sensor. Any thoughts? i noticed at lower ISOs and normal frame rates this is not and issue. 3200 and fps override at .750 like 20+ dead pixels

[1%]

As the sensor gets hot, more pixels get stuck... every camera at the higher ISO I've picked up has a bunch. Double with low fps override. Nothing seems to interpolate them away though as they aren't marked in raw video. H264 runs the defc engine and they are gone but not the case for raw.

ACR will automatically remove the dead pixels.

Doesn't seem to remove jack or shit... it makes them affect other pixels unless you turn the NR way up. What started as 1 turns into 4 at least.

[maxotics]

@hjfilmspeed, can you post a video of your clip and one of the DNG files?  I developed software that deals with focus pixels, so may be able to modify it for this problem.  Also, you can use ufRAW or dfRAW with hotpixellists, though it has a learning curve. 

[hjfilmspeed]

I will happily post some clips when home! thank you for speedy response! I suppose i could run em all through acr and just use spot remover tool at a small size since most shots the cam does not move and just save that as a preset for other shots provided stuck pix are identical. I was hoping to avoid this since i enjoy the fast workflow of davinci. i have to say that the stuck pixels are usually in the same spot

[maxotics]

I have a similar problem with the focus pixels in that I would like them interpolated around in the RAW file so that I can continue any post-processing path from there.  So that's what the app does, creates an exact duplicate of the RAW file and then loads and processes pixels in each frame.  So I could see this being used in that way, I'd just need for it to take a bad pixel list, like dcraw, and use that.  This is a Windows app by the way.  More important than the video file, is the DNG or RAW file.  If the RAW file is less then 100MBs and you can put it up, I could download that too. 

[hjfilmspeed]

ok give me a sec im can upload a clip that was slightly underexposed using fps override at only iso 800 or so but the dead pix are every where. i can upload like the first couple of frames if that helps. Whats the best way to get the file to you

[maxotics]

I'd get an account at box.com, or something like that, and I can download from there.

[hjfilmspeed]

ok working on it. boy i dont have much to offer in the way of dev but when i have a chance to help i get gitty!!!

im on oct 22 build for 5d3

Ok i made a gmail with drive if that helps. In the drive this 1 folder with 3 folders of underexposed raw files plus there bachelor 3 converted dngs. which if push a little bit exposure wise will rear the ugly specs

iso 3200 fps override 0.750 fps shutter fine tune +1.13 the worst offender

iso 3200 fps override 23.976 fps shutter fine tune +1.13 not as bad but kinda offensive

iso 200 fps override 23.976 fps shutter fine tune +1.13 none at all from what i saw

There all in the gmail drive

Il pm you the log in info. At some point I can send you a fps override of iso 200 fps override 0.750 fps shutter fine tune +1.13

Hope this helps!

[mvejerslev]

>ACR will automatically remove the dead pixels.
>Doesn't seem to remove jack or shit... it makes them affect other pixels unless you turn the NR way up. What started as 1 turns into 4 at least.

Another case of missing EXIF info in the RAW files, I believe...

[maxotics]

Hi hjfilmspeed, so I can motivate myself on this.  WHAT are you trying to do?   I see the pixels.  Since they seem to be split among the 3 channels I assume it is an issue with those sensels at long exposure.  I don't believe RAW video outputs the dead pixel list from the camera.  So we'd need to make one up...and, urgh, where are we trying to go?  Thanks!

[hjfilmspeed]

Well those shots were just for testing purposes.

Please dont do anything in my behalf either. I was hoping i was helping your endeavors. At first i was just nerves that i was killing my sensor with high iso and longer exposures.

Im in the process of making another time lapse ish video and for this i have been using FPS override with higher ISO to avoid shutter clicks. I also noticed that the dead pixels would sometimes go away then come back in the same clip.

also im not sure but it might be wise to see what MLV codec has to offer in the way of exif. I assume hot pixel info is stored there because on normal cr2 files you dont ever see those hot pixels even at 30 sec exposures at 3200 unless the camera misses one. is that how it works? cam writes dead pix data, then acr uses data. I assume so. if not acr would just start erasing all your stars in long exposure of the nebulous.

[maxotics]

Hi hjfilmspeed, AFAIK, when a camera gets hot, and it does if the sensor is dumping images continuously, the heat causes sensels to get "hot" or act in strange ways, the same way heat debilitates everything.  The BMPCC camera, by the way, goes through batteries because it uses powered Peltier cooling on the sensor--or so I've heard.  In short, there's a good chance you're going to get hot pixels if you keep the camera on, shooting video.  The bigger question is can you be certain, even if you have a hot pixel list, that more pixels won't get hot as the camera continues shooting?

In normal video, the hot pixels are mostly smoothed out, but in astronomy, if that's what you're doing, it's not so simple.

You can just open a DNG in Photivo (for example), load the image at 1:1 in "unbayered" mode, get the position of each "dead/hot" pixel, and feed it into any DNG to TIFF conversion routines through a badpixellist.txt file where each pixel is in x,y, time (you can just put 0) coordinates.

Anyway, if there is a concrete need to create some sort of fix, let's talk, otherwise, let's just keep it on the list of things to do?

[hjfilmspeed]

Very interesting. See i think you helped me more then i have helped you. The shots i gave you were very short and i didnt have the cam on very long so the heat must be generated by the high iso and long exposure even after a couple of frames. it must heat up quick. I guess other then post processing and putting your cam on dry ice theres no real way of avoiding this with high iso and low fps (at least if you going for the 180 degree shutter). Making a quick spot remover preset tailored to each sequence wouldnt be to difficult either i guess. probably could pre process in acr then grade in davinci as a tiff sequence.

Thank you so much maxotics!

[mityazabuben]

If you figure out how to make a spot brush pattern in ACR and save it to use all over other footages - pls, share)
I have a concept, but dont know how to realise it - make a matte layer with all hot pixels, than use it in AE with WarpStabilizer + Synthesize edges to remake - http://blogs.adobe.com/aftereffects/2011/06/warp-stabilizre-redux.html. Maybe someone can help and say how to make such universal matte mask?

[maxotics]

If you figure out how to make a spot brush pattern in ACR and save it to use all over other footages - pls, share)
I have a concept, but dont know how to realise it - make a matte layer with all hot pixels, than use it in AE with WarpStabilizer + Synthesize edges to remake - http://blogs.adobe.com/aftereffects/2011/06/warp-stabilizre-redux.html. Maybe someone can help and say how to make such universal matte mask?

Hi Guys, sorry if you already know this.  For hot/dead pixel work you really want to, probably have to, worked with image data straight from the sensor.  On any software has de-bayered each pixel, you're basically now working with 4-pixel blocks, because each pixel channel, red, green or blue, has borrowed the information from the neighbor pixels and there are 2 greens to ever 1 red and 1 blue.

Unfortunately, only PhotoShop and Photivo (free) allow you to effectively work with images in this fashion.  And they are meant for still photography.  I don't know where to begin.  The bottom line is that there are few tools to work with RAW video data in the specialized way you're looking to do.

That's part of the reason why I created stuff to do this.  But even I have a bunch of things I wish I had time to do.  Like work on the moire issue in non-crop mode. 

Here is an article you both might read that is very informative and will help us work together if we continue on.

http://www.stark-labs.com/craig/resources/Articles-&-Reviews/Debayering_API.pdf

[Defkaunta]

Hello guys! Instead of starting a new thread I figured it would be best to reanimate this one as it is pretty much about the same topic and even camera I own(60D).

So i was taking my camera out on a very hot Sunday (32°C) and was shooting quite many RAW Footage clips at highest resolution with my 60D.  During the shoot whenever I recorded many clips after another the temperature of my sensor raised up to 70°C which made me worry as it was also highlighted red instead of orange. After checking the footage on my computer i noticed that at some point a green and purple pixel made its way on my images and seem to stay there, also on every further shot from that time on. I should probably add that  I also swapped lenses a lot that day so I thought that maybe its dust on the sensor. But normally dust doesnt make colored pixels right? So at the moment I am kind of ruling out the possibility of a software issue or luckily I am wrong...

Today I also made a test shot to see if it is still there and sadly it is. You can download a test dng here and you will understand what I mean:

https://www.dropbox.com/s/n1e2grrwbnhaelg/Green_Purple_Pixels.dng

A quick preview can be seen here where i marked the dots :

http://i.imgur.com/VTasxvL.jpg

Did I just kill some pixels on my sensor? Or what do you think? Thanks!

[Defkaunta]

Never mind! I am sorry.

Apparently the converter tool "raw2cdng" was causing issues with converting. I used MlRawViewer this time and the pixels do not show up anymore. That is quite a relief! I remember actually having to switch to another converting tool at some point and this probably caused the issue. Hope this helps some of you.

In short: See if the problem still occurs when using another converting tool. In some cases it might help.